Recruitment of occipital cortex during sensory substitution training linked to subjective experience of seeing in people with blindness

Cortical activity involved in perception and imagery of visual stimuli in a subject with aphantasia. An EEG case report

Aphantasia has been described as the inability to voluntarily evoke mental images using the "mind's eye." We studied a congenital aphantasic subject using neuropsychological testsand 64 channel EEG recordings, in order to studycortical activity involved in perception and imagery evaluating event-related potentials(N170, P200, N250). The subject is in the normal range of the neuropsychological tests performed, except for specific imagery tests. The EEG results show that when he evokes the same mental image, he starts the evoking process from left temporal instead of frontal areas, he does not activate occipital visual nor left anterior parietal areas.

A self-training program for sensory substitution devices

Sensory Substitution Devices (SSDs) convey visual information through audition or touch, targeting blind and visually impaired individuals. One bottleneck towards adopting SSDs in everyday life by blind users, is the constant dependency on sighted instructors throughout the learning process. Here, we present a proof-of-concept for the efficacy of an online self-training program developed for learning the basics of the EyeMusic visual-to-auditory SSD tested on sighted blindfolded participants. Additionally, aiming to identify the best training strategy to be later re-adapted for the blind, we compared multisensory vs. unisensory as well as perceptual vs. descriptive feedback approaches. To these aims, sighted participants performed identical SSD-stimuli identification tests before and after ~75 minutes of self-training on the EyeMusic algorithm. Participants were divided into five groups, differing by the feedback delivered during training: auditory-descriptive, audio-visual textual description, audio-visual perceptual simultaneous and interleaved, and a control group which had no training. At baseline, before any EyeMusic training, participants SSD objects’ identification was significantly above chance, highlighting the algorithm’s intuitiveness. Furthermore, self-training led to a significant improvement in accuracy between pre- and post-training tests in each of the four feedback groups versus control, though no significant difference emerged among those groups. Nonetheless, significant correlations between individual post-training success rates and various learning measures acquired during training, suggest a trend for an advantage of multisensory vs. unisensory feedback strategies, while no trend emerged for perceptual vs. descriptive strategies. The success at baseline strengthens the conclusion that cross-modal correspondences facilitate learning, given SSD algorithms are based on such correspondences. Additionally, and crucially, the results highlight the feasibility of self-training for the first stages of SSD learning, and suggest that for these initial stages, unisensory training, easily implemented also for blind and visually impaired individuals, may suffice. Together, these findings will potentially boost the use of SSDs for rehabilitation.

Multisensory inclusive design with sensory substitution

Sensory substitution techniques are perceptual and cognitive phenomena used to represent one sensory form with an alternative. Current applications of sensory substitution techniques are typically focused on the development of assistive technologies whereby visually impaired users can acquire visual information via auditory and tactile cross-modal feedback. But despite their evident success in scientific research and furthering theory development in cognition, sensory substitution techniques have not yet gained widespread adoption within sensory-impaired populations. Here we argue that shifting the focus from assistive to mainstream applications may resolve some of the current issues regarding the use of sensory substitution devices to improve outcomes for those with disabilities. This article provides a tutorial guide on how to use research into multisensory processing and sensory substitution techniques from the cognitive sciences to design new inclusive cross-modal displays. A greater focus on developing inclusive mainstream applications could lead to innovative technologies that could be enjoyed by every person.

A N400 ERP Study in letter recognition after passive tactile stimulation training in blind children and sighted controls

BACKGROUND

We previously demonstrated that using a sensory substitution device (SSD) for one week, tactile stimulation results in faster activation of lateral occipital complex in blind children than in seeing controls.

OBJECTIVE

We used long-term haptic tactile stimulation training with an SSD to test if it results in stable cross-modal reassignment of visual pathways after six months, to provide high level processing of tactile semantic content.

METHODS

We enrolled 12 blind and 12 sighted children. The SSD transforms images to a stimulation matrix in contact with the dominant hand. Subjects underwent twice-daily training sessions, 5 days/week for six months. Children were asked to describe line orientation, name letters, and read words. ERP sessions were performed at baseline and 6 months to analyze the N400 ERP component and reaction times (RT). N400 sources were estimated with Low Resolution Electromagnetic Tomography (LORETA). SPM8 was used to make population-level inferences.

RESULTS

We found no group differences in RTs, accuracy of identifications, N400 latencies or distributions with the line task at 1 week or at 6 months. RTs on the letter recognition task were also similar. After 6 months, behavioral training increased accurate letter identification in both seeing and blind children (Chi 2 = 11906.934, p = 0.000), but the increase was larger in blind children (Chi 2 = 8.272, p = 0.004). Behavioral training shifted peak N400 amplitude to left occipital and bilateral parietal cortices in blind children, but to left precentral and postcentral and bilateral occipital cortices in sighted controls.

CONCLUSIONS

Blind children learn to recognize SSD-delivered letters better than seeing controls and had greater N400 amplitude in the occipital region. To the best of our knowledge, our results provide the first published example of standard letter recognition (not Braille) by children with blindness using a tactile delivery system.

How Much Spatial Information Is Lost in the Sensory Substitution Process? Comparing Visual, Tactile, and Auditory Approaches

Sensory substitution devices (SSDs) can convey visuospatial information through spatialised auditory or tactile stimulation using wearable technology. However, the level of information loss associated with this transformation is unknown. In this study, novice users discriminated the location of two objects at 1.2 m using devices that transformed a 16 × 8-depth map into spatially distributed patterns of light, sound, or touch on the abdomen. Results showed that through active sensing, participants could discriminate the vertical position of objects to a visual angle of 1°, 14°, and 21°, and their distance to 2 cm, 8 cm, and 29 cm using these visual, auditory, and haptic SSDs, respectively. Visual SSDs significantly outperformed auditory and tactile SSDs on vertical localisation, whereas for depth perception, all devices significantly differed from one another (visual > auditory > haptic). Our findings highlight the high level of acuity possible for SSDs even with low spatial resolutions (e.g., 16 × 8) and quantify the level of information loss attributable to this transformation for the SSD user. Finally, we discuss ways of closing this “modality gap” found in SSDs and conclude that this process is best benchmarked against performance with SSDs that return to their primary modality (e.g., visuospatial into visual).

Enhancing general spatial skills of young visually impaired people with a programmable distance discrimination training: a case control study

BACKGROUND

The estimation of relative distance is a perceptual task used extensively in everyday life. This important skill suffers from biases that may be more pronounced when estimation is based on haptics. This is especially true for the blind and visually impaired, for which haptic estimation of distances is paramount but not systematically trained. We investigated whether a programmable tactile display, used autonomously, can improve distance discrimination ability in blind and severely visually impaired youngsters between 7 and 22 years-old.

METHODS

Training consisted of four weekly sessions in which participants were asked to haptically find, on the programmable tactile display, the pairs of squares which were separated by the shortest and longest distance in tactile images with multiple squares. A battery of haptic tests with raised-line drawings was administered before and after training, and scores were compared to those of a control group that did only the haptic battery, without doing the distance discrimination training on the tactile display.

RESULTS

Both blind and severely impaired youngsters became more accurate and faster at the task during training. In haptic battery results, blind and severely impaired youngsters who used the programmable display improved in three and two tests, respectively. In contrast, in the control groups, the blind control group improved in only one test, and the severely visually impaired in no tests.

CONCLUSIONS

Distance discrimination skills can be trained equally well in both blind and severely impaired participants. More importantly, autonomous training with the programmable tactile display had generalized effects beyond the trained task. Participants improved not only in the size discrimination test but also in memory span tests. Our study shows that tactile stimulation training that requires minimal human assistance can effectively improve generic spatial skills.

Improving spatial working memory in blind and sighted youngsters using programmable tactile displays

OBJECTIVE

To investigate whether training with tactile matrices displayed with a programmable tactile display improves recalling performance of spatial images in blind, low-vision and sighted youngsters. To code and understand the behavioral underpinnings of learning two-dimensional tactile dispositions, in terms of spontaneous exploration strategies.

METHODS

Three groups of blind, low-vision and sighted youngsters between 6 and 18 years old performed four training sessions with a weekly schedule in which they were asked to memorize single or double spatial layouts, featured as two-dimensional matrices.

RESULTS

Results showed that all groups of participants significantly improved their recall performance compared to the first session baseline in the single-matrix task. No statistical difference in performance between groups emerged in this task. Instead, the learning effect in visually impaired participants is reduced in the double-matrix task, whereas it is still robust in blindfolded sighted controls. We also coded tactile exploration strategies in both tasks and their correlation with performance. Sighted youngsters, in particular, favored a proprioceptive exploration strategy. Finally, performance in the double-matrix task negatively correlated with using one hand and positively correlated with a proprioceptive strategy.

CONCLUSION

The results of our study indicate that blind persons do not easily process two separate spatial layouts. However, rehabilitation programs promoting bi-manual and proprioceptive approaches to tactile exploration might help improve spatial abilities. Finally, programmable tactile displays are an effective way to make spatial and graphical configurations accessible to visually impaired youngsters and they can be profitably exploited in rehabilitation.

A novel paradigm to study spatial memory skills in blind individuals through the auditory modality

Spatial memory is a multimodal representation of the environment, which can be mediated by different sensory signals. Here we investigate how the auditory modality influences memorization, contributing to the mental representation of a scene. We designed an audio test inspired by a validated spatial memory test, the Corsi-Block test for blind individuals. The test was carried out in two different conditions, with non-semantic and semantic stimuli, presented in different sessions and displaced on an audio-tactile device. Furthermore, the semantic sounds were spatially displaced in order to reproduce an audio scene, explored by participants during the test. Thus, we verified if semantic rather than non-semantic sounds are better recalled and whether exposure to an auditory scene can enhance memorization skills. Our results show that sighted subjects performed better than blind participants after the exploration of the semantic scene. This suggests that blind participants focus on the perceived sound positions and do not use items’ locations learned during the exploration. We discuss these results in terms of the role of visual experience on spatial memorization skills and the ability to take advantage of semantic information stored in the memory.

Word recognition: re-thinking prosthetic vision evaluation

OBJECTIVE

Evaluations of vision prostheses and sensory substitution devices have frequently relied on repeated training and then testing with the same small set of items. These multiple forced-choice tasks produced above chance performance in blind users, but it is unclear if the observed performance represents restoration of vision that transfers to novel, untrained items.

APPROACH

Here, we tested the generalizability of the forced-choice paradigm on discrimination of low-resolution word images. Extensive visual training was conducted with the same 10 words used in previous BrainPort tongue stimulation studies. The performance on these 10 words and an additional 50 words was measured before and after the training sessions.

MAIN RESULTS

The results revealed minimal performance improvement with the untrained words, demonstrating instead pattern discrimination limited mostly to the trained words.

SIGNIFICANCE

These findings highlight the need to reconsider current evaluation practices, in particular, the use of forced-choice paradigms with a few highly trained items. While appropriate for measuring the performance thresholds in acuity or contrast sensitivity of a functioning visual system, performance on such tasks cannot be taken to indicate restored spatial pattern vision.

Hearing Shapes: Event-related Potentials Reveal the Time Course of Auditory-Visual Sensory Substitution

In auditory-visual sensory substitution, visual information (e.g., shape) can be extracted through strictly auditory input (e.g., soundscapes). Previous studies have shown that image-to-sound conversions that follow simple rules [such as the Meijer algorithm; Meijer, P. B. L. An experimental system for auditory image representation. Transactions on Biomedical Engineering, 39, 111-121, 1992] are highly intuitive and rapidly learned by both blind and sighted individuals. A number of recent fMRI studies have begun to explore the neuroplastic changes that result from sensory substitution training. However, the time course of cross-sensory information transfer in sensory substitution is largely unexplored and may offer insights into the underlying neural mechanisms. In this study, we recorded ERPs to soundscapes before and after sighted participants were trained with the Meijer algorithm. We compared these posttraining versus pretraining ERP differences with those of a control group who received the same set of 80 auditory/visual stimuli but with arbitrary pairings during training. Our behavioral results confirmed the rapid acquisition of cross-sensory mappings, and the group trained with the Meijer algorithm was able to generalize their learning to novel soundscapes at impressive levels of accuracy. The ERP results revealed an early cross-sensory learning effect (150-210 msec) that was significantly enhanced in the algorithm-trained group compared with the control group as well as a later difference (420-480 msec) that was unique to the algorithm-trained group. These ERP modulations are consistent with previous fMRI results and provide additional insight into the time course of cross-sensory information transfer in sensory substitution.

Individual Differences in Sensory Substitution

Sensory substitution devices were developed in the context of perceptual rehabilitation and they aim at compensating one or several functions of a deficient sensory modality by converting stimuli that are normally accessed through this deficient sensory modality into stimuli accessible by another sensory modality. For instance, they can convert visual information into sounds or tactile stimuli. In this article, we review those studies that investigated the individual differences at the behavioural, neural, and phenomenological levels when using a sensory substitution device. We highlight how taking into account individual differences has consequences for the optimization and learning of sensory substitution devices. We also discuss the extent to which these studies allow a better understanding of the experience with sensory substitution devices, and in particular how the resulting experience is not akin to a single sensory modality. Rather, it should be conceived as a multisensory experience, involving both perceptual and cognitive processes, and emerging on each user’s pre-existing sensory and cognitive capacities.

Designing sensory-substitution devices: Principles, pitfalls and potential1

An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.

Brain Plasticity in Blind Subjects Centralizes Beyond the Modal Cortices

It is well established that the human brain reorganizes following sensory deprivations. In blind individuals, visual processing regions including the lateral occipital cortex (LOC) are activated by auditory and tactile stimuli as demonstrated by neurophysiological and neuroimaging investigations. The mechanisms for such plasticity remain unclear, but shifts in connectivity across existing neural networks appear to play a critical role. The majority of research efforts to date have focused on neuroplastic changes within visual unimodal regions, however we hypothesized that neuroplastic alterations may also occur in brain networks beyond the visual cortices including involvement of multimodal integration regions and heteromodal cortices. In this study, two recently developed graph-theory based functional connectivity analyses, interconnector analyses and local and distant connectivity, were applied to investigate functional reorganization in regional and distributed neural-systems in late-onset blind (LB) and congenitally blind (CB) cohorts each compared to their own group of sighted controls. While functional network alterations as measured by the degree of differential links (DDL) occurred in sensory cortices, neuroplastic changes were most prominent within multimodal and association cortices. Subjects with LB showed enhanced multimodal integration connections in the parieto-opercular, temporoparietal junction (TPJ) and ventral premotor (vPM) regions, while CB individuals exhibited increased superior parietal cortex (SPC) connections. This study reveals the critical role of recipient multi-sensory integration areas in network reorganization and cross-modal plasticity in blind individuals. These findings suggest that aspects of cross-modal neuroplasticity and adaptive sensory-motor and auditory functions may potentially occur through reorganization in multimodal integration regions.

Non-optic vision: Beyond synesthesia?

Patient NS is a 28year-old female who went blind in her early twenties as a result of S-cone syndrome, a degenerative retinal disorder. A few years after losing her vision, she started experiencing visual perceptions of her hands as she moved them and objects that came into contact with her hands. Over the course of a year, these cross-modal sensations evolved to become veridical visual experiences accurately representative of her hands, objects she touched, and to some degree, objects she could infer from her immediate surroundings. We argue that these experiences are distinct from mental imagery as they occurred automatically, remained consistent over time, and were proprioceptively mediated by her head position much like normal optical vision. Moreover, she could neither consciously force these visual experiences to occur without sensory inference nor prevent them from happening when haptically exploring an object. Her previous visual experiences contributed to a strong influence of top-down processing in her perceptions. Though individuals have previously been able to develop limited veridical acquired synesthesia following extensive practice over many years with the use of a special sensory device, none reported experiencing the richness of complexity or degree of top-down processing exhibited by NS. Thus, we posit that NS's case may represent a phenomenon beyond synesthesia altogether.

Improving training for sensory augmentation using the science of expertise

Sensory substitution and augmentation devices (SSADs) allow users to perceive information about their environment that is usually beyond their sensory capabilities. Despite an extensive history, SSADs are arguably not used to their fullest, both as assistive technology for people with sensory impairment or as research tools in the psychology and neuroscience of sensory perception. Studies of the non-use of other assistive technologies suggest one factor is the balance of benefits gained against the costs incurred. We argue that improving the learning experience would improve this balance, suggest three ways in which it can be improved by leveraging existing cognitive science findings on expertise and skill development, and acknowledge limitations and relevant concerns. We encourage the systematic evaluation of learning programs, and suggest that a more effective learning process for SSADs could reduce the barrier to uptake and allow users to reach higher levels of overall capacity.

Reading in the dark: neural correlates and cross-modal plasticity for learning to read entire words without visual experience

Cognitive neuroscience has long attempted to determine the ways in which cortical selectivity develops, and the impact of nature vs. nurture on it. Congenital blindness (CB) offers a unique opportunity to test this question as the brains of blind individuals develop without visual experience. Here we approach this question through the reading network. Several areas in the visual cortex have been implicated as part of the reading network, and one of the main ones among them is the VWFA, which is selective to the form of letters and words. But what happens in the CB brain? On the one hand, it has been shown that cross-modal plasticity leads to the recruitment of occipital areas, including the VWFA, for linguistic tasks. On the other hand, we have recently demonstrated VWFA activity for letters in contrast to other visual categories when the information is provided via other senses such as touch or audition. Which of these tasks is more dominant? By which mechanism does the CB brain process reading? Using fMRI and visual-to-auditory sensory substitution which transfers the topographical features of the letters we compare reading with semantic and scrambled conditions in a group of CB. We found activation in early auditory and visual cortices during the early processing phase (letter), while the later phase (word) showed VWFA and bilateral dorsal-intraparietal activations for words. This further supports the notion that many visual regions in general, even early visual areas, also maintain a predilection for task processing even when the modality is variable and in spite of putative lifelong linguistic cross-modal plasticity. Furthermore, we find that the VWFA is recruited preferentially for letter and word form, while it was not recruited, and even exhibited deactivation, for an immediately subsequent semantic task suggesting that despite only short sensory substitution experience orthographic task processing can dominate semantic processing in the VWFA. On a wider scope, this implies that at least in some cases cross-modal plasticity which enables the recruitment of areas for new tasks may be dominated by sensory independent task specific activation.

Differences in Early Stages of Tactile ERP Temporal Sequence (P100) in Cortical Organization during Passive Tactile Stimulation in Children with Blindness and Controls

Compared to their seeing counterparts, people with blindness have a greater tactile capacity. Differences in the physiology of object recognition between people with blindness and seeing people have been well documented, but not when tactile stimuli require semantic processing. We used a passive vibrotactile device to focus on the differences in spatial brain processing evaluated with event related potentials (ERP) in children with blindness (n = 12) vs. normally seeing children (n = 12), when learning a simple spatial task (lines with different orientations) or a task involving recognition of letters, to describe the early stages of its temporal sequence (from 80 to 220 msec) and to search for evidence of multi-modal cortical organization. We analysed the P100 of the ERP. Children with blindness showed earlier latencies for cognitive (perceptual) event related potentials, shorter reaction times, and (paradoxically) worse ability to identify the spatial direction of the stimulus. On the other hand, they are equally proficient in recognizing stimuli with semantic content (letters). The last observation is consistent with the role of P100 on somatosensory-based recognition of complex forms. The cortical differences between seeing control and blind groups, during spatial tactile discrimination, are associated with activation in visual pathway (occipital) and task-related association (temporal and frontal) areas. The present results show that early processing of tactile stimulation conveying cross modal information differs in children with blindness or with normal vision.

Acquisition of Visual Perception in Blind Adults Using the BrainPort Artificial Vision Device

OBJECTIVE

We sought to determine whether intensive low vision rehabilitation would confer any functional improvement in a sample of blind adults using the BrainPort artificial vision device.

METHOD

Eighteen adults ages 28-69 yr (n=10 men and n=8 women) who had light perception only or worse vision bilaterally spent up to 6 hr per day for 1 wk undergoing structured rehabilitation interventions. The functional outcomes of object identification and word recognition were tested at baseline and after rehabilitation training.

RESULTS

At baseline, participants were unable to complete the two functional assessments. After participation in the 1-wk training protocol, participants were able to use the BrainPort device to complete the two tasks with moderate success.

CONCLUSION

Without training, participants were not able to perform above chance level using the BrainPort device. As artificial vision technologies become available, occupational therapy practitioners can play a key role in clients' success or failure in using these devices.

Color synesthesia. Insight into perception, emotion, and consciousness

Purpose of review

Synesthesia is an extraordinary perceptual phenomenon, in which individuals experience unusual percepts elicited by the activation of an unrelated sensory modality or by a cognitive process. Emotional reactions are commonly associated. The condition prompted philosophical debates on the nature of perception and impacted the course of art history. It recently generated a considerable interest among neuroscientists, but its clinical significance apparently remains underevaluated. This review focuses on the recent studies regarding variants of color synesthesia, the commonest form of the condition.

Recent findings

Synesthesia is commonly classified as developmental and acquired. Developmental forms predispose to changes in primary sensory processing and cognitive functions, usually with better performances in certain aspects and worse in others, and to heightened creativity. Acquired forms of synesthesia commonly arise from drug ingestion or neurological disorders, including thalamic lesions and sensory deprivation (e.g., blindness). Cerebral exploration using structural and functional imaging has demonstrated distinct patterns in cortical activation and brain connectivity for controls and synesthetes. Artworks of affected painters are most illustrative of the nature of synesthetic experiences.

Summary

Results of the recent investigations on synesthesia offered a remarkable insight into the mechanisms of perception, emotion and consciousness, and deserve attention both from neuroscientists and from clinicians.

Occipital cortex activation by long-term repetitive tactile stimulation is necessary for object recognition in blinds: a case report

Tactile vision has been approached from a variety of angles using different techniques. So far, a certain kind of object (and text) recognition has been shown, though seeing as such has not been achieved yet, and it remains unclear. Trough repetitive passive tactile stimulation perceptual processing is transferred from temporo-parietal to occipital areas, which affects object recognition. We report the results of passive tactile stimulation, as well as rTMS, applied to a 50 year old left handed blind male with over 97% loss of vision, who suffers from Peter's anomaly and microphthalmia. After 15 weeks of passive tactile stimulation, the subject showed increased activity in occipital areas associated with the development of visual-like perception which remained unchanged after three months without passive tactile stimulation. Inhibitory rTMS over the visual cortex led to noticeable reduction of spatial recognition performance and visual sensations in this subject. Stable changes in occipital cortical activity can be associated with subjective sensations of seeing. Once occipital activation has been achieved, it is necessary for spatial object recognition. Both facts highlight the implication of occipital areas in tactile vision and the cortical plasticity of passive tactile long-term stimulation in people with blindness.

Successful tactile based visual sensory substitution use functions independently of visual pathway integrity

PURPOSE

Neuronal reorganization after blindness is of critical interest because it has implications for the rational prescription of artificial vision devices. The purpose of this study was to distinguish the microstructural differences between perinatally blind (PB), acquired blind (AB), and normally sighted controls (SCs) and relate these differences to performance on functional tasks using a sensory substitution device (BrainPort).

METHODS

We enrolled 52 subjects (PB n = 11; AB n = 35; SC n = 6). All subjects spent 15 h undergoing BrainPort device training. Outcomes of light perception, motion, direction, temporal resolution, grating, and acuity were tested at baseline and after training. Twenty-six of the subjects were scanned with a three Tesla MRI scanner for diffusion tensor imaging (DTI), and with a positron emission tomography (PET) scanner for mapping regional brain glucose consumption during sensory substitution function. Non-parametric models were used to analyze fractional anisotropy (FA; a DTI measure of microstructural integrity) of the brain via region-of-interest (ROI) analysis and tract-based spatial statistics (TBSS).

RESULTS

At baseline, all subjects performed all tasks at chance level. After training, light perception, time resolution, location and grating acuity tasks improved significantly for all subject groups. ROI and TBSS analyses of FA maps show areas of statistically significant differences (p ≤ 0.025) in the bilateral optic radiations and some visual association connections between all three groups. No relationship was found between FA and functional performance with the BrainPort.

DISCUSSION

All subjects showed performance improvements using the BrainPort irrespective of nature and duration of blindness. Definite brain areas with significant microstructural integrity changes exist among PB, AB, and NC, and these variations are most pronounced in the visual pathways. However, the use of sensory substitution devices is feasible irrespective of microstructural integrity of the primary visual pathways between the eye and the brain. Therefore, tongue based devices devices may be usable for a broad array of non-sighted patients.

A predictive processing theory of sensorimotor contingencies: Explaining the puzzle of perceptual presence and its absence in synesthesia

Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of "perceptual presence" has motivated "sensorimotor theories" which understand perception to involve the mastery of sensorimotor contingencies. However, the mechanistic basis of sensorimotor contingencies and their mastery has remained unclear. Sensorimotor theory also struggles to explain instances of perception, such as synesthesia, that appear to lack perceptual presence and for which relevant sensorimotor contingencies are difficult to identify. On alternative "predictive processing" theories, perceptual content emerges from probabilistic inference on the external causes of sensory signals, however, this view has addressed neither the problem of perceptual presence nor synesthesia. Here, I describe a theory of predictive perception of sensorimotor contingencies which (1) accounts for perceptual presence in normal perception, as well as its absence in synesthesia, and (2) operationalizes the notion of sensorimotor contingencies and their mastery. The core idea is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if those actions are not performed. These "counterfactually-rich" generative models encode sensorimotor contingencies related to repertoires of sensorimotor dependencies, with counterfactual richness determining the degree of perceptual presence associated with a stimulus. While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synesthetic concurrents are hypothesized to be counterfactually poor. In addition to accounting for the phenomenology of synesthesia, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. It may also lead to a new view of the (in)determinacy of normal perception.

The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

Sensory substitution is a promising technique for mitigating the loss of a sensory modality. Sensory substitution devices (SSDs) work by converting information from the impaired sense (e.g., vision) into another, intact sense (e.g., audition). However, there are a potentially infinite number of ways of converting images into sounds, and it is important that the conversion takes into account the limits of human perception and other user-related factors (e.g., whether the sounds are pleasant to listen to). The device explored here is termed “polyglot” because it generates a very large set of solutions. Specifically, we adapt a procedure that has been in widespread use in the design of technology but has rarely been used as a tool to explore perception—namely, interactive genetic algorithms. In this procedure, a very large range of potential sensory substitution devices can be explored by creating a set of “genes” with different allelic variants (e.g., different ways of translating luminance into loudness). The most successful devices are then “bred” together, and we statistically explore the characteristics of the selected-for traits after multiple generations. The aim of the present study is to produce design guidelines for a better SSD. In three experiments, we vary the way that the fitness of the device is computed: by asking the user to rate the auditory aesthetics of different devices (Experiment 1), and by measuring the ability of participants to match sounds to images (Experiment 2) and the ability to perceptually discriminate between two sounds derived from similar images (Experiment 3). In each case, the traits selected for by the genetic algorithm represent the ideal SSD for that task. Taken together, these traits can guide the design of a better SSD.

Perceiving invisible light through a somatosensory cortical prosthesis

Sensory neuroprostheses show great potential for alleviating major sensory deficits. It is not known, however, whether such devices can augment the subject's normal perceptual range. Here we show that adult rats can learn to perceive otherwise invisible infrared light through a neuroprosthesis that couples the output of a head-mounted infrared sensor to their somatosensory cortex (S1) via intracortical microstimulation. Rats readily learn to use this new information source, and generate active exploratory strategies to discriminate among infrared signals in their environment. S1 neurons in these infrared-perceiving rats respond to both whisker deflection and intracortical microstimulation, suggesting that the infrared representation does not displace the original tactile representation. Hence, sensory cortical prostheses, in addition to restoring normal neurological functions, may serve to expand natural perceptual capabilities in mammals.

Cognitive and neuroplasticity mechanisms by which congenital or early blindness may confer a protective effect against schizophrenia

Several authors have noted that there are no reported cases of people with schizophrenia who were born blind or who developed blindness shortly after birth, suggesting that congenital or early (C/E) blindness may serve as a protective factor against schizophrenia. By what mechanisms might this effect operate? Here, we hypothesize that C/E blindness offers protection by strengthening cognitive functions whose impairment characterizes schizophrenia, and by constraining cognitive processes that exhibit excessive flexibility in schizophrenia. After briefly summarizing evidence that schizophrenia is fundamentally a cognitive disorder, we review areas of perceptual and cognitive function that are both impaired in the illness and augmented in C/E blindness, as compared to healthy sighted individuals. We next discuss: (1) the role of neuroplasticity in driving these cognitive changes in C/E blindness; (2) evidence that C/E blindness does not confer protective effects against other mental disorders; and (3) evidence that other forms of C/E sensory loss (e.g., deafness) do not reduce the risk of schizophrenia. We conclude by discussing implications of these data for designing cognitive training interventions to reduce schizophrenia-related cognitive impairment, and perhaps to reduce the likelihood of the development of the disorder itself.

Reading the World through the Skin and Ears: A New Perspective on Sensory Substitution

Sensory substitution devices aim at replacing or assisting one or several functions of a deficient sensory modality by means of another sensory modality. Despite the numerous studies and research programs devoted to their development and integration, sensory substitution devices have failed to live up to their goal of allowing one to "see with the skin" (White et al., 1970) or to "see with the brain" (Bach-y-Rita et al., 2003). These somewhat peremptory claims, as well as the research conducted so far, are based on an implicit perceptual paradigm. Such perceptual assumption accepts the equivalence between using a sensory substitution device and perceiving through a particular sensory modality. Our aim is to provide an alternative model, which defines sensory substitution as being closer to culturally implemented cognitive extensions of existing perceptual skills such as reading. In this article, we will show why the analogy with reading provides a better explanation of the actual findings, that is, both of the positive results achieved and of the limitations noticed across the field of research on sensory substitution. The parallel with the most recent two-route and interactive models of reading (e.g., Dehaene et al., 2005) generates a radically new way of approaching these results, by stressing the dependence of integration on the existing perceptual-semantic route. In addition, the present perspective enables us to generate innovative research questions and specific predictions which set the stage for future work.

Enriched and deprived sensory experience induces structural changes and rewires connectivity during the postnatal development of the brain

During postnatal development, sensory experience modulates cortical development, inducing numerous changes in all of the components of the cortex. Most of the cortical changes thus induced occur during the critical period, when the functional and structural properties of cortical neurons are particularly susceptible to alterations. Although the time course for experience-mediated sensory development is specific for each system, postnatal development acts as a whole, and if one cortical area is deprived of its normal sensory inputs during early stages, it will be reorganized by the nondeprived senses in a process of cross-modal plasticity that not only increases performance in the remaining senses when one is deprived, but also rewires the brain allowing the deprived cortex to process inputs from other senses and cortices, maintaining the modular configuration. This paper summarizes our current understanding of sensory systems development, focused specially in the visual system. It delineates sensory enhancement and sensory deprivation effects at both physiological and anatomical levels and describes the use of enriched environment as a tool to rewire loss of brain areas to enhance other active senses. Finally, strategies to apply restorative features in human-deprived senses are studied, discussing the beneficial and detrimental effects of cross-modal plasticity in prostheses and sensory substitution devices implantation.

Tactile exploration of virtual objects for blind and sighted people: the role of beta 1 EEG band in sensory substitution and supramodal mental mapping

The neural correlates of exploration and cognitive mapping in blindness remain elusive. The role of visuo-spatial pathways in blind vs. sighted subjects is still under debate. In this preliminary study, we investigate, as a possible estimation of the activity in the visuo-spatial pathways, the EEG patterns of blind and blindfolded-sighted subjects during the active tactile construction of cognitive maps from virtual objects compared with rest and passive tactile stimulation. Ten blind and ten matched, blindfolded-sighted subjects participated in the study. Events were defined as moments when the finger was only stimulated (passive stimulation) or the contour of a virtual object was touched (during active exploration). Event-related spectral power and coherence perturbations were evaluated within the beta 1 band (14-18 Hz). They were then related to a subjective cognitive-load estimation required by the explorations [namely, perceived levels of difficulty (PLD)]. We found complementary cues for sensory substitution and spatial processing in both groups: both blind and sighted subjects showed, while exploring, late power decreases and early power increases, potentially associated with motor programming and touch, respectively. The latter involved occipital areas only for blind subjects (long-term plasticity) and only during active exploration, thus supporting tactile-to-visual sensory substitution. In both groups, coherences emerged among the fronto-central, centro-parietal, and occipito-temporal derivations associated with visuo-spatial processing. This seems in accordance with mental map construction involving spatial processing, sensory-motor processing, and working memory. The observed involvement of the occipital regions suggests that a substitution process also occurs in sighted subjects. Only during explorations did coherence correlate positively with PLD for both groups and in derivations, which can be related to visuo-spatial processing, supporting the existence of supramodal spatial processing independently of vision capabilities.