Tactile-auditory shape learning engages the lateral occipital complex

Seeing the piles of the velvet bending under our finger sliding over a tactile stimulator improves the feeling of the fabric

Using friction modulation to simulate fabrics with a tactile stimulator (i.e. virtual surface) is not sufficient to render fabric touch and even more so for hairy fabrics. We hypothesized that seeing the pile of the velvet darken or lighten depending on changes in the finger movement direction on the virtual surface should improve the velvet fabric rendering. Participants actively rubbed a tactile device or a velvet fabric looking at a screen that showed a synthesized image of a velvet that either remained static (V-static) or darkening/lightening with the direction of touch (V-moving). We showed that in V-moving condition, the touched surface was always perceived rougher, which is a descriptor of a real velvet (Experiment 1). Using electroencephalography and sources localization analyses, we found increased activity in the occipital and inferior parietal lobes (Experiment 2) when seeing dark and shining traces during back-and-forth finger movements over the virtual surface. This suggests that these two posterior cortical regions work together to evaluate visuo-tactile congruence between the seen and the felt (tactile). The visuo-tactile binding, evidenced by neural synchronization (specifically, theta band (5-7 Hz) oscillation) in the left inferior posterior parietal lobule, is consistent with enhanced integration of information and probably contributed to the emergence of a more realistic velvet representation.

Distinct but related abilities for visual and haptic object recognition

People vary in their ability to recognize objects visually. Individual differences for matching and recognizing objects visually is supported by a domain-general ability capturing common variance across different tasks (e.g., Richler et al., Psychological Review, 126, 226-251, 2019). Behavioral (e.g., Cooke et al., Neuropsychologia, 45, 484-495, 2007) and neural evidence (e.g., Amedi, Cerebral Cortex, 12, 1202-1212, 2002) suggest overlapping mechanisms in the processing of visual and haptic information in the service of object recognition, but it is unclear whether such group-average results generalize to individual differences. Psychometrically validated measures are required, which have been lacking in the haptic modality. We investigate whether object recognition ability is specific to vision or extends to haptics using psychometric measures we have developed. We use multiple visual and haptic tests with different objects and different formats to measure domain-general visual and haptic abilities and to test for relations across them. We measured object recognition abilities using two visual tests and four haptic tests (two each for two kinds of haptic exploration) in 97 participants. Partial correlation and confirmatory factor analyses converge to support the existence of a domain-general haptic object recognition ability that is moderately correlated with domain-general visual object recognition ability. Visual and haptic abilities share about 25% of their variance, supporting the existence of a multisensory domain-general ability while leaving a substantial amount of residual variance for modality-specific abilities. These results extend our understanding of the structure of object recognition abilities; while there are mechanisms that may generalize across categories, tasks, and modalities, there are still other mechanisms that are distinct between modalities.

Visual Mental Imagery and Neural Dynamics of Sensory Substitution in the Blindfolded Subjects

Although one can recognize the environment by soundscape substituting vision to auditory signal, whether subjects could perceive the soundscape as visual or visual-like sensation has been questioned. In this study, we investigated hierarchical process to elucidate the recruitment mechanism of visual areas by soundscape stimuli in blindfolded subjects. Twenty-two healthy subjects were repeatedly trained to recognize soundscape stimuli converted by visual shape information of letters. An effective connectivity method called dynamic causal modeling (DCM) was employed to reveal how the brain was hierarchically organized to recognize soundscape stimuli. The visual mental imagery model generated cortical source signals of five regions of interest better than auditory bottom-up, cross-modal perception, and mixed models. Spectral couplings between brain areas in the visual mental imagery model were analyzed. While within-frequency coupling is apparent in bottom-up processing where sensory information is transmitted, cross-frequency coupling is prominent in top-down processing, corresponding to the expectation and interpretation of information. Sensory substitution in the brain of blindfolded subjects derived visual mental imagery by combining bottom-up and top-down processing.

Forming cognitive maps for abstract spaces: the roles of the human hippocampus and orbitofrontal cortex

How does the human brain construct cognitive maps for decision-making and inference? Here, we conduct an fMRI study on a navigation task in multidimensional abstract spaces. Using a deep neural network model, we assess learning levels and categorized paths into exploration and exploitation stages. Univariate analyses show higher activation in the bilateral hippocampus and lateral prefrontal cortex during exploration, positively associated with learning level and response accuracy. Conversely, the bilateral orbitofrontal cortex (OFC) and retrosplenial cortex show higher activation during exploitation, negatively associated with learning level and response accuracy. Representational similarity analysis show that the hippocampus, entorhinal cortex, and OFC more accurately represent destinations in exploitation than exploration stages. These findings highlight the collaboration between the medial temporal lobe and prefrontal cortex in learning abstract space structures. The hippocampus may be involved in spatial memory formation and representation, while the OFC integrates sensory information for decision-making in multidimensional abstract spaces.

Learning and navigating digitally rendered haptic spatial layouts

Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.

Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Superior visual rhythm discrimination in expert musicians is most likely not related to cross-modal recruitment of the auditory cortex

Training can influence behavioral performance and lead to brain reorganization. In particular, training in one modality, for example, auditory, can improve performance in another modality, for example, visual. Previous research suggests that one of the mechanisms behind this phenomenon could be the cross-modal recruitment of the sensory areas, for example, the auditory cortex. Studying expert musicians offers a chance to explore this process. Rhythm is an aspect of music that can be presented in various modalities. We designed an fMRI experiment in which professional pianists and non-musicians discriminated between two sequences of rhythms presented auditorily (series of sounds) or visually (series of flashes). Behavioral results showed that musicians performed in both visual and auditory rhythmic tasks better than non-musicians. We found no significant between-group differences in fMRI activations within the auditory cortex. However, we observed that musicians had increased activation in the right Inferior Parietal Lobe when compared to non-musicians. We conclude that the musicians’ superior visual rhythm discrimination is not related to cross-modal recruitment of the auditory cortex; instead, it could be related to activation in higher-level, multimodal areas in the cortex.

Grey Matter Hypertrophy and Atrophy in Early-Blind Adolescents: A Surface-Based Morphometric Study

Objective

This study is aimed at exploring the regional changes in brain cortical morphology (thickness, volume, and surface area) in the early-blind adolescents (EBAs) by using the surface-based morphometric (SBM) method.

Methods

High-resolution structural T1-weighted images (T1WI) of 23 early-blind adolescents (EBAs) and 21 age- and gender-matched normal-sighted controls (NSCs) were acquired. Structural indices, including cortical thickness (CT), cortical volume (CV), and surface area (SA), were analyzed by using FreeSurfer software, and the correlations between structural indices and the blindness duration were computed by Pearson correlation analysis.

Results

Compared to controls, EBAs had significantly reduced CV and SA mainly in the primary visual cortex (V1) and decreased CV in the left vision-related cortices (r-MFC). There were no regions that EBAs had a significantly larger CV or SA than NSCs. EBAs had significantly increased CT in the V1 and strongly involved the visual cortex (right lateral occipital gyrus, LOG.R) and the left superior temporal gyrus (STG.L), while it had decreased CT in the left superior parietal lobule (SPL.L) and the right lingual gyrus (LING.R). Additionally, no correlation was found between cortical morphometric measures and clinical variables in the EBA group.

Conclusions

SBM is a useful method for detecting human brain structural abnormalities in blindness. The results showed that these structural abnormalities in the visual cortex and visual-related areas outside the occipital cortex in the EBAs not only may be influenced by neurodevelopment, degeneration, plasticity, and so on but also involved the interaction of these factors after the early visual deprivation.

Mid-Air Tactile Sensations Evoked by Laser-Induced Plasma: A Neurophysiological Study

This study demonstrates the feasibility of a mid-air means of haptic stimulation at a long distance using the plasma effect induced by laser. We hypothesize that the stress wave generated by laser-induced plasma in the air can propagate through the air to reach the nearby human skin and evoke tactile sensation. To validate this hypothesis, we investigated somatosensory responses in the human brain to laser plasma stimuli by analyzing electroencephalography (EEG) in 14 participants. Three types of stimuli were provided to the index finger: a plasma stimulus induced from the laser, a mechanical stimulus transferred through Styrofoam stick, and a sham stimulus providing only the sound of the plasma and mechanical stimuli at the same time. The event-related desynchronization/synchronization (ERD/S) of sensorimotor rhythms (SMRs) in EEG was analyzed. Every participant verbally reported that they could feel a soft tap on the finger in response to the laser stimulus, but not to the sham stimulus. The spectrogram of EEG evoked by laser stimulation was similar to that evoked by mechanical stimulation; alpha ERD and beta ERS were present over the sensorimotor area in response to laser as well as mechanical stimuli. A decoding analysis revealed that classification error increased when discriminating ERD/S patterns between laser and mechanical stimuli, compared to the case of discriminating between laser and sham, or mechanical and sham stimuli. Our neurophysiological results confirm that tactile sensation can be evoked by the plasma effect induced by laser in the air, which may provide a mid-air haptic stimulation method.

Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations

How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.

Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device

Sensory substitution devices aim at restoring visual functions by converting visual information into auditory or tactile stimuli. Although these devices show promise in the range of behavioral abilities they allow, the processes underlying their use remain underspecified. In particular, while an initial debate focused on the visual versus auditory or tactile nature of sensory substitution, since over a decade, the idea that it reflects a mixture of both has emerged. In order to investigate behaviorally the extent to which visual and auditory processes are involved, participants completed a Stroop-like crossmodal interference paradigm before and after being trained with a conversion device which translates visual images into sounds. In addition, participants' auditory abilities and their phenomenologies were measured. Our study revealed that, after training, when asked to identify sounds, processes shared with vision were involved, as participants’ performance in sound identification was influenced by the simultaneously presented visual distractors. In addition, participants’ performance during training and their associated phenomenology depended on their auditory abilities, revealing that processing finds its roots in the input sensory modality. Our results pave the way for improving the design and learning of these devices by taking into account inter-individual differences in auditory and visual perceptual strategies.

Touching and hearing the shapes: How auditory angular and curved sounds influence proficiency in recognising tactile angle and curve shapes when experienced and inexperienced in using haptic touch

This study investigated whether adding auditory angular and curved sounds to tactile angle and curve shapes – one unspecified sound to one unspecified shape – positively influences the accuracy and exploration time in recognising tactile angles and curves when experienced and inexperienced in using haptic touch. A within-participant experiment was conducted, with two groups of participants: experienced and inexperienced in using haptic touch, and with two conditions: congruous (e.g., angle shape and angular sound) and incongruous (e.g., angle shape and curved sound) tactile and auditory shape information. Adding congruous auditory angular and curved sounds to tactile angle and curve shapes positively influences the accuracy in recognising tactile angles and curves both when experienced and inexperienced in using haptic touch, and the exploration time on correct recognitions when experienced. People integrate tactile and auditory (angle; curve) shape information and this improves their proficiency in recognising tactile angles and curves.

Efficiency of Sensory Substitution Devices Alone and in Combination With Self-Motion for Spatial Navigation in Sighted and Visually Impaired

Human adults can optimally combine vision with self-motion to facilitate navigation. In the absence of visual input (e.g., dark environments and visual impairments), sensory substitution devices (SSDs), such as The vOICe or BrainPort, which translate visual information into auditory or tactile information, could be used to increase navigation precision when integrated together or with self-motion. In Experiment 1, we compared and assessed together The vOICe and BrainPort in aerial maps task performed by a group of sighted participants. In Experiment 2, we examined whether sighted individuals and a group of visually impaired (VI) individuals could benefit from using The vOICe, with and without self-motion, to accurately navigate a three-dimensional (3D) environment. In both studies, 3D motion tracking data were used to determine the level of precision with which participants performed two different tasks (an egocentric and an allocentric task) and three different conditions (two unisensory conditions and one multisensory condition). In Experiment 1, we found no benefit of using the devices together. In Experiment 2, the sighted performance during The vOICe was almost as good as that for self-motion despite a short training period, although we found no benefit (reduction in variability) of using The vOICe and self-motion in combination compared to the two in isolation. In contrast, the group of VI participants did benefit from combining The vOICe and self-motion despite the low number of trials. Finally, while both groups became more accurate in their use of The vOICe with increased trials, only the VI group showed an increased level of accuracy in the combined condition. Our findings highlight how exploiting non-visual multisensory integration to develop new assistive technologies could be key to help blind and VI persons, especially due to their difficulty in attaining allocentric information.

Properties of cross-modal occipital responses in early blindness: An ALE meta-analysis

•

ALE meta-analysis reveals distributed brain networks for object and spatial functions in individuals with early blindness.

•

ALE contrast analysis reveals specific activations in the left cuneus and lingual gyrus for language function, suggesting a reverse hierarchical organization of the visual cortex for early blind individuals.

•

The findings contribute to visual rehabilitation in blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.

Cross-modal occipital responses appear to be essential for nonvisual processing in individuals with early blindness. However, it is not clear whether the recruitment of occipital regions depends on functional domain or sensory modality. The current study utilized a coordinate-based meta-analysis to identify the distinct brain regions involved in the functional domains of object, spatial/motion, and language processing and the common brain regions involved in both auditory and tactile modalities in individuals with early blindness. Following the PRISMA guidelines, a total of 55 studies were included in the meta-analysis. The specific analyses revealed the brain regions that are consistently recruited for each function, such as the dorsal fronto-parietal network for spatial function and ventral occipito-temporal network for object function. This is consistent with the literature, suggesting that the two visual streams are preserved in early blind individuals. The contrast analyses found specific activations in the left cuneus and lingual gyrus for language function. This finding is novel and suggests a reverse hierarchical organization of the visual cortex for early blind individuals. The conjunction analyses found common activations in the right middle temporal gyrus, right precuneus and a left parieto-occipital region. Clinically, this work contributes to visual rehabilitation in early blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.

Altered Functional Connectivity in Patients With Sloping Sensorineural Hearing Loss

Background

Sensory deprivation, such as hearing loss, has been demonstrated to change the intrinsic functional connectivity (FC) of the brain, as measured with resting-state functional magnetic resonance imaging (rs-fMRI). Patients with sloping sensorineural hearing loss (SNHL) are a unique population among the hearing impaired, as they have all been exposed to some auditory input throughout their lifespan and all use spoken language.

Materials and Methods

Twenty patients with SNHL and 21 control subjects participated in a rs-fMRI study. Whole-brain seed-driven FC maps were obtained, with audiological scores of patients, including hearing loss severity and speech performance, used as covariates.

Results

Most profound differences in FC were found between patients with prelingual (before language development, PRE) vs. postlingual onset (after language development, POST) of SNHL. An early onset was related to enhancement in long-range network connections, including the default-mode network, the dorsal-attention network and the fronto-parietal network, as well as in local sensory networks, the visual and the sensorimotor. A number of multisensory brain regions in frontal and parietal cortices, as well as the cerebellum, were also more internally connected. We interpret these effects as top-down mechanisms serving optimization of multisensory experience in SNHL with a prelingual onset. At the same time, POST patients showed enhanced FC between the salience network and multisensory parietal areas, as well as with the hippocampus, when they were compared to those with PRE hearing loss. Signal in several cortex regions subserving visual processing was also more intra-correlated in POST vs. PRE patients. This outcome might point to more attention resources directed to multisensory as well as memory experience. Finally, audiological scores correlated with FC in several sensory and high-order brain regions in all patients.

Conclusion

The results show that a sloping hearing loss is related to altered resting-state brain organization. Effects were shown in attention and cognitive control networks, as well as visual and sensorimotor regions. Specifically, we found that even in a partial hearing deficit (affecting only some of the hearing frequency ranges), the age at the onset affects the brain function differently, pointing to the role of sensitive periods in brain development.

Functional hierarchy for tactile processing in the visual cortex of sighted adults

Perception via different sensory modalities was traditionally believed to be supported by largely separate brain systems. However, a growing number of studies demonstrate that the visual cortices of typical, sighted adults are involved in tactile and auditory perceptual processing. Here, we investigated the spatiotemporal dynamics of the visual cortex's involvement in a complex tactile task: Braille letter recognition. Sighted subjects underwent Braille training and then participated in a transcranial magnetic stimulation (TMS) study in which they tactually identified single Braille letters. During this task, TMS was applied to their left early visual cortex, visual word form area (VWFA), and left early somatosensory cortex at five time windows from 20 to 520 ms following the Braille letter presentation's onset. The subjects' response accuracy decreased when TMS was applied to the early visual cortex at the 120-220 ms time window and when TMS was applied to the VWFA at the 320-420 ms time window. Stimulation of the early somatosensory cortex did not have a time-specific effect on the accuracy of the subjects' Braille letter recognition, but rather caused a general slowdown during this task. Our results indicate that the involvement of sighted people's visual cortices in tactile perception respects the canonical visual hierarchy-the early tactile processing stages involve the early visual cortex, whereas more advanced tactile computations involve high-level visual areas. Our findings are compatible with the metamodal account of brain organization and suggest that the whole visual cortex may potentially support spatial perception in a task-specific, sensory-independent manner.

Windows into the Visual Brain: New Discoveries about the Visual System, Its Functions, and Implications for Practitioners

Introduction

In recent years, major progress has been made in understanding the human visual system because of new investigative techniques. These developments often contradict older concepts about visual function.

Methods

A detailed literature search and interprofessional discussions.

Results

Recent innovative neurological tests are described that are able to show much more accurately the visual pathways, the process of vision, and the close relationships among sensory modalities. These tests also reveal the remarkable neuroplasticity of the human brain and disorders of connectivity that frequently involve visual function.

Discussion

How these recent neurological advances may benefit service providers is discussed.

Implications for practitioners

It is important that from time to time new neurological and ophthalmic developments are summarized for professionals who are involved in the clinical management of individuals with visual disorders and how the newly acquired knowledge affects the diagnosis and intervention strategies. Visual rehabilitation must be based on up-to-date science, which continually changes and grows with research.

Greater Cortical Thickness Is Associated With Enhanced Sensory Function After Arm Rehabilitation in Chronic Stroke

Objective. Somatosensory function is critical to normal motor control. After stroke, dysfunction of the sensory systems prevents normal motor function and degrades quality of life. Structural neuroplasticity underpinnings of sensory recovery after stroke are not fully understood. The objective of this study was to identify changes in bilateral cortical thickness (CT) that may drive recovery of sensory acuity. Methods. Chronic stroke survivors (n = 20) were treated with 12 weeks of rehabilitation. Measures were sensory acuity (monofilament), Fugl-Meyer upper limb and CT change. Permutation-based general linear regression modeling identified cortical regions in which change in CT was associated with change in sensory acuity. Results. For the ipsilesional hemisphere in response to treatment, CT increase was significantly associated with sensory improvement in the area encompassing the occipital pole, lateral occipital cortex (inferior and superior divisions), intracalcarine cortex, cuneal cortex, precuneus cortex, inferior temporal gyrus, occipital fusiform gyrus, supracalcarine cortex, and temporal occipital fusiform cortex. For the contralesional hemisphere, increased CT was associated with improved sensory acuity within the posterior parietal cortex that included supramarginal and angular gyri. Following upper limb therapy, monofilament test score changed from 45.0 ± 13.3 to 42.6 ± 12.9 mm ( P = .063) and Fugl-Meyer score changed from 22.1 ± 7.8 to 32.3 ± 10.1 ( P < .001). Conclusions. Rehabilitation in the chronic stage after stroke produced structural brain changes that were strongly associated with enhanced sensory acuity. Improved sensory perception was associated with increased CT in bilateral high-order association sensory cortices reflecting the complex nature of sensory function and recovery in response to rehabilitation.

Hearing Shapes: Event-related Potentials Reveal the Time Course of Auditory-Visual Sensory Substitution

In auditory-visual sensory substitution, visual information (e.g., shape) can be extracted through strictly auditory input (e.g., soundscapes). Previous studies have shown that image-to-sound conversions that follow simple rules [such as the Meijer algorithm; Meijer, P. B. L. An experimental system for auditory image representation. Transactions on Biomedical Engineering, 39, 111-121, 1992] are highly intuitive and rapidly learned by both blind and sighted individuals. A number of recent fMRI studies have begun to explore the neuroplastic changes that result from sensory substitution training. However, the time course of cross-sensory information transfer in sensory substitution is largely unexplored and may offer insights into the underlying neural mechanisms. In this study, we recorded ERPs to soundscapes before and after sighted participants were trained with the Meijer algorithm. We compared these posttraining versus pretraining ERP differences with those of a control group who received the same set of 80 auditory/visual stimuli but with arbitrary pairings during training. Our behavioral results confirmed the rapid acquisition of cross-sensory mappings, and the group trained with the Meijer algorithm was able to generalize their learning to novel soundscapes at impressive levels of accuracy. The ERP results revealed an early cross-sensory learning effect (150-210 msec) that was significantly enhanced in the algorithm-trained group compared with the control group as well as a later difference (420-480 msec) that was unique to the algorithm-trained group. These ERP modulations are consistent with previous fMRI results and provide additional insight into the time course of cross-sensory information transfer in sensory substitution.

Prior Visual Experience Modulates Learning of Sound Localization Among Blind Individuals

Cross-modal learning requires the use of information from different sensory modalities. This study investigated how the prior visual experience of late blind individuals could modulate neural processes associated with learning of sound localization. Learning was realized by standardized training on sound localization processing, and experience was investigated by comparing brain activations elicited from a sound localization task in individuals with (late blind, LB) and without (early blind, EB) prior visual experience. After the training, EB showed decreased activation in the precuneus, which was functionally connected to a limbic-multisensory network. In contrast, LB showed the increased activation of the precuneus. A subgroup of LB participants who demonstrated higher visuospatial working memory capabilities (LB-HVM) exhibited an enhanced precuneus-lingual gyrus network. This differential connectivity suggests that visuospatial working memory due to the prior visual experience gained via LB-HVM enhanced learning of sound localization. Active visuospatial navigation processes could have occurred in LB-HVM compared to the retrieval of previously bound information from long-term memory for EB. The precuneus appears to play a crucial role in learning of sound localization, disregarding prior visual experience. Prior visual experience, however, could enhance cross-modal learning by extending binding to the integration of unprocessed information, mediated by the cognitive functions that these experiences develop.

Task-specific reorganization of the auditory cortex in deaf humans

The principles that guide large-scale cortical reorganization remain unclear. In the blind, several visual regions preserve their task specificity; ventral visual areas, for example, become engaged in auditory and tactile object-recognition tasks. It remains open whether task-specific reorganization is unique to the visual cortex or, alternatively, whether this kind of plasticity is a general principle applying to other cortical areas. Auditory areas can become recruited for visual and tactile input in the deaf. Although nonhuman data suggest that this reorganization might be task specific, human evidence has been lacking. Here we enrolled 15 deaf and 15 hearing adults into an functional MRI experiment during which they discriminated between temporally complex sequences of stimuli (rhythms). Both deaf and hearing subjects performed the task visually, in the central visual field. In addition, hearing subjects performed the same task in the auditory modality. We found that the visual task robustly activated the auditory cortex in deaf subjects, peaking in the posterior-lateral part of high-level auditory areas. This activation pattern was strikingly similar to the pattern found in hearing subjects performing the auditory version of the task. Although performing the visual task in deaf subjects induced an increase in functional connectivity between the auditory cortex and the dorsal visual cortex, no such effect was found in hearing subjects. We conclude that in deaf humans the high-level auditory cortex switches its input modality from sound to vision but preserves its task-specific activation pattern independent of input modality. Task-specific reorganization thus might be a general principle that guides cortical plasticity in the brain.

Functional and Structural Neuroplasticity Induced by Short-Term Tactile Training Based on Braille Reading

Neuroplastic changes induced by sensory learning have been recognized within the cortices of specific modalities as well as within higher ordered multimodal areas. The interplay between these areas is not fully understood, particularly in the case of somatosensory learning. Here we examined functional and structural changes induced by short-term tactile training based of Braille reading, a task that requires both significant tactile expertise and mapping of tactile input onto multimodal representations. Subjects with normal vision were trained for 3 weeks to read Braille exclusively by touch and scanned before and after training, while performing a same-different discrimination task on Braille characters and meaningless characters. Functional and diffusion-weighted magnetic resonance imaging sequences were used to assess resulting changes. The strongest training-induced effect was found in the primary somatosensory cortex (SI), where we observed bilateral augmentation in activity accompanied by an increase in fractional anisotropy (FA) within the contralateral SI. Increases of white matter fractional anisotropy were also observed in the secondary somatosensory area (SII) and the thalamus. Outside of somatosensory system, changes in both structure and function were found in i.e., the fusiform gyrus, the medial frontal gyri and the inferior parietal lobule. Our results provide evidence for functional remodeling of the somatosensory pathway and higher ordered multimodal brain areas occurring as a result of short-lasting tactile learning, and add to them a novel picture of extensive white matter plasticity.

Other ways of seeing: From behavior to neural mechanisms in the online "visual" control of action with sensory substitution

Vision is the dominant sense for perception-for-action in humans and other higher primates. Advances in sight restoration now utilize the other intact senses to provide information that is normally sensed visually through sensory substitution to replace missing visual information. Sensory substitution devices translate visual information from a sensor, such as a camera or ultrasound device, into a format that the auditory or tactile systems can detect and process, so the visually impaired can see through hearing or touch. Online control of action is essential for many daily tasks such as pointing, grasping and navigating, and adapting to a sensory substitution device successfully requires extensive learning. Here we review the research on sensory substitution for vision restoration in the context of providing the means of online control for action in the blind or blindfolded. It appears that the use of sensory substitution devices utilizes the neural visual system; this suggests the hypothesis that sensory substitution draws on the same underlying mechanisms as unimpaired visual control of action. Here we review the current state of the art for sensory substitution approaches to object recognition, localization, and navigation, and the potential these approaches have for revealing a metamodal behavioral and neural basis for the online control of action.

Braille in the Sighted: Teaching Tactile Reading to Sighted Adults

Blind people are known to have superior perceptual abilities in their remaining senses. Several studies suggest that these enhancements are dependent on the specific experience of blind individuals, who use those remaining senses more than sighted subjects. In line with this view, sighted subjects, when trained, are able to significantly progress in relatively simple tactile tasks. However, the case of complex tactile tasks is less obvious, as some studies suggest that visual deprivation itself could confer large advantages in learning them. It remains unclear to what extent those complex skills, such as braille reading, can be learnt by sighted subjects. Here we enrolled twenty-nine sighted adults, mostly braille teachers and educators, in a 9-month braille reading course. At the beginning of the course, all subjects were naive in tactile braille reading. After the course, almost all were able to read whole braille words at a mean speed of 6 words-per-minute. Subjects with low tactile acuity did not differ significantly in braille reading speed from the rest of the group, indicating that low tactile acuity is not a limiting factor for learning braille, at least at this early stage of learning. Our study shows that most sighted adults can learn whole-word braille reading, given the right method and a considerable amount of motivation. The adult sensorimotor system can thus adapt, to some level, to very complex tactile tasks without visual deprivation. The pace of learning in our group was comparable to congenitally and early blind children learning braille in primary school, which suggests that the blind’s mastery of complex tactile tasks can, to a large extent, be explained by experience-dependent mechanisms.

How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge

How conceptual knowledge is represented in the human brain remains to be determined. To address the differential role of low-level sensory-based and high-level abstract features in semantic processing, we combined behavioral studies of linguistic production and brain activity measures by functional magnetic resonance imaging in sighted and congenitally blind individuals while they performed a property-generation task with concrete nouns from eight categories, presented through visual and/or auditory modalities. Patterns of neural activity within a large semantic cortical network that comprised parahippocampal, lateral occipital, temporo-parieto-occipital and inferior parietal cortices correlated with linguistic production and were independent both from the modality of stimulus presentation (either visual or auditory) and the (lack of) visual experience. In contrast, selected modality-dependent differences were observed only when the analysis was limited to the individual regions within the semantic cortical network. We conclude that conceptual knowledge in the human brain relies on a distributed, modality-independent cortical representation that integrates the partial category and modality specific information retained at a regional level.

Massive cortical reorganization in sighted Braille readers

The brain is capable of large-scale reorganization in blindness or after massive injury. Such reorganization crosses the division into separate sensory cortices (visual, somatosensory...). As its result, the visual cortex of the blind becomes active during tactile Braille reading. Although the possibility of such reorganization in the normal, adult brain has been raised, definitive evidence has been lacking. Here, we demonstrate such extensive reorganization in normal, sighted adults who learned Braille while their brain activity was investigated with fMRI and transcranial magnetic stimulation (TMS). Subjects showed enhanced activity for tactile reading in the visual cortex, including the visual word form area (VWFA) that was modulated by their Braille reading speed and strengthened resting-state connectivity between visual and somatosensory cortices. Moreover, TMS disruption of VWFA activity decreased their tactile reading accuracy. Our results indicate that large-scale reorganization is a viable mechanism recruited when learning complex skills.

DOI:http://dx.doi.org/10.7554/eLife.10762.001

According to most textbooks, our brain is divided into separate areas that are dedicated to specific senses. We have a visual cortex for vision, a tactile cortex for touch, and so on. However, researchers suspect that this division might not be as fixed as the textbooks say. For example, blind people can switch their 'leftover' visual cortex to non-visual purposes, such as reading Braille – a tactile alphabet.

Can this switch in functional organization also happen in healthy people with normal vision? To investigate this, Siuda-Krzywicka, Bola et al. taught a group of healthy, sighted people to read Braille by touch, and monitored the changes in brain activity that this caused using a technique called functional magnetic resonance imaging. According to textbooks, tactile reading should engage the tactile cortex. Yet, the experiment revealed that the brain activity critical for reading Braille by touch did not occur in the volunteers’ tactile cortex, but in their visual cortex.

Further experiments used a technique called transcranial magnetic stimulation to suppress the activity of the visual cortex of the volunteers. This impaired their ability to read Braille by touch. This is a clear-cut proof that sighted adults can re-program their visual cortex for non-visual, tactile purposes.

These results show that intensive training in a complex task can overcome the sensory division-of-labor of our brain. This indicates that our brain is much more flexible than previously thought, and that such flexibility might occur when we learn everyday, complex skills such as driving a car or playing a musical instrument.

The next question that follows from this work is: what enables the brain’s activity to change after learning to read Braille? To understand this, Siuda-Krzywicka, Bola et al. are currently exploring how the physical structure of the brain changes as a result of a person acquiring the ability to read Braille by touch.

DOI:http://dx.doi.org/10.7554/eLife.10762.002

'Visual' parsing can be taught quickly without visual experience during critical periods

Cases of invasive sight-restoration in congenital blind adults demonstrated that acquiring visual abilities is extremely challenging, presumably because visual-experience during critical-periods is crucial for learning visual-unique concepts (e.g. size constancy). Visual rehabilitation can also be achieved using sensory-substitution-devices (SSDs) which convey visual information non-invasively through sounds. We tested whether one critical concept – visual parsing, which is highly-impaired in sight-restored patients – can be learned using SSD. To this end, congenitally blind adults participated in a unique, relatively short (~70 hours), SSD-‘vision’ training. Following this, participants successfully parsed 2D and 3D visual objects. Control individuals naïve to SSDs demonstrated that while some aspects of parsing with SSD are intuitive, the blind’s success could not be attributed to auditory processing alone. Furthermore, we had a unique opportunity to compare the SSD-users’ abilities to those reported for sight-restored patients who performed similar tasks visually, and who had months of eyesight. Intriguingly, the SSD-users outperformed the patients on most criteria tested. These suggest that with adequate training and technologies, key high-order visual features can be quickly acquired in adulthood, and lack of visual-experience during critical-periods can be somewhat compensated for. Practically, these highlight the potential of SSDs as standalone-aids or combined with invasive restoration approaches.

The impact of the stimulation frequency on closed-loop control with electrotactile feedback

BACKGROUND

Electrocutaneous stimulation can restore the missing sensory information to prosthetic users. In electrotactile feedback, the information about the prosthesis state is transmitted in the form of pulse trains. The stimulation frequency is an important parameter since it influences the data transmission rate over the feedback channel as well as the form of the elicited tactile sensations.

METHODS

We evaluated the influence of the stimulation frequency on the subject's ability to utilize the feedback information during electrotactile closed-loop control. Ten healthy subjects performed a real-time compensatory tracking (standard test bench) of sinusoids and pseudorandom signals using either visual feedback (benchmark) or electrocutaneous feedback in seven conditions characterized by different combinations of the stimulation frequency (FSTIM) and tracking error sampling rate (FTE). The tracking error was transmitted using two concentric electrodes placed on the forearm. The quality of tracking was assessed using the Squared Pearson Correlation Coefficient (SPCC), the Normalized Root Mean Square Tracking Error (NRMSTE) and the time delay between the reference and generated trajectories (TDIO).

RESULTS

The results demonstrated that FSTIM was more important for the control performance than FTE. The quality of tracking deteriorated with a decrease in the stimulation frequency, SPCC and NRMSTE (mean) were 87.5% and 9.4% in the condition 100/100 (FTE/FSTIM), respectively, and deteriorated to 61.1% and 15.3% in 5/5, respectively, while the TDIO increased from 359.8 ms in 100/100 to 1009 ms in 5/5. However, the performance recovered when the tracking error sampled at a low rate was delivered using a high stimulation frequency (SPCC = 83.6%, NRMSTE = 10.3%, TDIO = 415.6 ms, in 5/100).

CONCLUSIONS

The likely reason for the performance decrease and recovery was that the stimulation frequency critically influenced the tactile perception quality and thereby the effective rate of information transfer through the feedback channel. The outcome of this study can facilitate the selection of optimal system parameters for somatosensory feedback in upper limb prostheses. The results imply that the feedback variables (e.g., grasping force) should be transmitted at relatively high frequencies of stimulation (>25 Hz), but that they can be sampled at much lower rates (e.g., 5 Hz).

A number-form area in the blind

Distinct preference for visual number symbols was recently discovered in the human right inferior temporal gyrus (rITG). It remains unclear how this preference emerges, what is the contribution of shape biases to its formation and whether visual processing underlies it. Here we use congenital blindness as a model for brain development without visual experience. During fMRI, we present blind subjects with shapes encoded using a novel visual-to-music sensory-substitution device (The EyeMusic). Greater activation is observed in the rITG when subjects process symbols as numbers compared with control tasks on the same symbols. Using resting-state fMRI in the blind and sighted, we further show that the areas with preference for numerals and letters exhibit distinct patterns of functional connectivity with quantity and language-processing areas, respectively. Our findings suggest that specificity in the ventral ‘visual’ stream can emerge independently of sensory modality and visual experience, under the influence of distinct connectivity patterns.

The human visual cortex includes areas with preference for various object categories. Here, Abboud et al. demonstrate using visual-to-music substitution, that the congenitally blind show a similar preference for numerals in the right inferior temporal cortex as sighted individuals, despite having no visual experience.

Functional Connectivity Density in Congenitally and Late Blind Subjects

Visual deprivation during different developmental periods leads to different structural and functional alterations in the brain; however, the effects of visual deprivation on the spontaneous functional organization of the brain remain largely unknown. In this study, we used voxel-based functional connectivity density (FCD) analyses to investigate the effects of visual deprivation during different developmental periods on the spontaneous functional organization of the brain. Compared with the sighted controls (SC), both the congenitally blind (CB) and the late blind (LB) exhibited decreased short- and long-range FCDs in the primary visual cortex (V1) and decreased long-range FCDs in the primary somatosensory and auditory cortices. Although both the CB and LB exhibited increased short-range FCD in the dorsal visual stream, the CB exhibited greater increases in the short- and long-range FCDs in the ventral visual stream and hippocampal complex compared with the LB. Moreover, the short-range FCD of the left V1 exhibited a significant positive correlation with the duration of blindness in the LB. Our findings suggest that visual deprivation before the developmental sensitive period can induce more extensive brain functional reorganization than does visual deprivation after the sensitive period, which may underlie an enhanced capacity for processing nonvisual information in the CB.

Flexibility and Stability in Sensory Processing Revealed Using Visual-to-Auditory Sensory Substitution

The classical view of sensory processing involves independent processing in sensory cortices and multisensory integration in associative areas. This hierarchical structure has been challenged by evidence of multisensory responses in sensory areas, and dynamic weighting of sensory inputs in associative areas, thus far reported independently. Here, we used a visual-to-auditory sensory substitution algorithm (SSA) to manipulate the information conveyed by sensory inputs while keeping the stimuli intact. During scan sessions before and after SSA learning, subjects were presented with visual images and auditory soundscapes. The findings reveal 2 dynamic processes. First, crossmodal attenuation of sensory cortices changed direction after SSA learning from visual attenuations of the auditory cortex to auditory attenuations of the visual cortex. Secondly, associative areas changed their sensory response profile from strongest response for visual to that for auditory. The interaction between these phenomena may play an important role in multisensory processing. Consistent features were also found in the sensory dominance in sensory areas and audiovisual convergence in associative area Middle Temporal Gyrus. These 2 factors allow for both stability and a fast, dynamic tuning of the system when required.

Haptic shape processing in visual cortex

Humans typically rely upon vision to identify object shape, but we can also recognize shape via touch (haptics). Our haptic shape recognition ability raises an intriguing question: To what extent do visual cortical shape recognition mechanisms support haptic object recognition? We addressed this question using a haptic fMRI repetition design, which allowed us to identify neuronal populations sensitive to the shape of objects that were touched but not seen. In addition to the expected shape-selective fMRI responses in dorsal frontoparietal areas, we observed widespread shape-selective responses in the ventral visual cortical pathway, including primary visual cortex. Our results indicate that shape processing via touch engages many of the same neural mechanisms as visual object recognition. The shape-specific repetition effects we observed in primary visual cortex show that visual sensory areas are engaged during the haptic exploration of object shape, even in the absence of concurrent shape-related visual input. Our results complement related findings in visually deprived individuals and highlight the fundamental role of the visual system in the processing of object shape.

Neural pathways conveying novisual information to the visual cortex

The visual cortex has been traditionally considered as a stimulus-driven, unimodal system with a hierarchical organization. However, recent animal and human studies have shown that the visual cortex responds to non-visual stimuli, especially in individuals with visual deprivation congenitally, indicating the supramodal nature of the functional representation in the visual cortex. To understand the neural substrates of the cross-modal processing of the non-visual signals in the visual cortex, we firstly showed the supramodal nature of the visual cortex. We then reviewed how the nonvisual signals reach the visual cortex. Moreover, we discussed if these non-visual pathways are reshaped by early visual deprivation. Finally, the open question about the nature (stimulus-driven or top-down) of non-visual signals is also discussed.

The development of visual areas depends differently on visual experience

Visual experience plays an important role in the development of the visual cortex; however, recent functional imaging studies have shown that the functional organization is preserved in several higher-tier visual areas in congenitally blind subjects, indicating that maturation of visual areas depend unequally on visual experience. In this study, we aim to validate this hypothesis using a multimodality MRI approach. We found increased cortical thickness in the congenitally blind was present in the early visual areas and absent in the higher-tier ones, suggesting that the structural development of the visual cortex depends hierarchically on visual experience. In congenitally blind subjects, the decreased resting-state functional connectivity with the primary somatosensory cortex was more prominent in the early visual areas than in the higher-tier ones and were more pronounced in the ventral stream than in the dorsal one, suggesting that the development of functional organization of the visual cortex also depends differently on visual experience. Moreover, congenitally blind subjects showed normal or increased functional connectivity between ipsilateral higher-tier and early visual areas, suggesting an indirect corticocortical pathway through which somatosenroy information can reach the early visual areas. These findings support our hypothesis that the development of visual areas depends differently on visual experience.

Multisensory interactions between auditory and haptic object recognition

Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.

Language networks in anophthalmia: maintained hierarchy of processing in 'visual' cortex

Imaging studies in blind subjects have consistently shown that sensory and cognitive tasks evoke activity in the occipital cortex, which is normally visual. The precise areas involved and degree of activation are dependent upon the cause and age of onset of blindness. Here, we investigated the cortical language network at rest and during an auditory covert naming task in five bilaterally anophthalmic subjects, who have never received visual input. When listening to auditory definitions and covertly retrieving words, these subjects activated lateral occipital cortex bilaterally in addition to the language areas activated in sighted controls. This activity was significantly greater than that present in a control condition of listening to reversed speech. The lateral occipital cortex was also recruited into a left-lateralized resting-state network that usually comprises anterior and posterior language areas. Levels of activation to the auditory naming and reversed speech conditions did not differ in the calcarine (striate) cortex. This primary 'visual' cortex was not recruited to the left-lateralized resting-state network and showed high interhemispheric correlation of activity at rest, as is typically seen in unimodal cortical areas. In contrast, the interhemispheric correlation of resting activity in extrastriate areas was reduced in anophthalmia to the level of cortical areas that are heteromodal, such as the inferior frontal gyrus. Previous imaging studies in the congenitally blind show that primary visual cortex is activated in higher-order tasks, such as language and memory to a greater extent than during more basic sensory processing, resulting in a reversal of the normal hierarchy of functional organization across 'visual' areas. Our data do not support such a pattern of organization in anophthalmia. Instead, the patterns of activity during task and the functional connectivity at rest are consistent with the known hierarchy of processing in these areas normally seen for vision. The differences in cortical organization between bilateral anophthalmia and other forms of congenital blindness are considered to be due to the total absence of stimulation in 'visual' cortex by light or retinal activity in the former condition, and suggests development of subcortical auditory input to the geniculo-striate pathway.