What blindness can tell us about seeing again: merging neuroplasticity and neuroprostheses

Multisensory Calibration: A Variety of Slow and Fast Brain Processes Throughout the Lifespan

From before we are born, throughout development, adulthood, and aging, we are immersed in a multisensory world. At each of these stages, our sensory cues are constantly changing, due to body, brain, and environmental changes. While integration of information from our different sensory cues improves precision, this only improves accuracy if the underlying cues are unbiased. Thus, multisensory calibration is a vital and ongoing process. To meet this grand challenge, our brains have evolved a variety of mechanisms. First, in response to a systematic discrepancy between sensory cues (without external feedback) the cues calibrate one another (unsupervised calibration). Second, multisensory function is calibrated to external feedback (supervised calibration). These two mechanisms superimpose. While the former likely reflects a lower level mechanism, the latter likely reflects a higher level cognitive mechanism. Indeed, neural correlates of supervised multisensory calibration in monkeys were found in higher level multisensory cortical area VIP, but not in the relatively lower level multisensory area MSTd. In addition, even without a cue discrepancy (e.g., when experiencing stimuli from different sensory cues in series) the brain monitors supra-modal statistics of events in the environment and adapts perception cross-modally. This too comprises a variety of mechanisms, including confirmation bias to prior choices, and lower level cross-sensory adaptation. Further research into the neuronal underpinnings of the broad and diverse functions of multisensory calibration, with improved synthesis of theories is needed to attain a more comprehensive understanding of multisensory brain function.

Learning and navigating digitally rendered haptic spatial layouts

Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.

Brain structural changes in blindness: a systematic review and an anatomical likelihood estimation (ALE) meta-analysis

In recent decades, numerous structural brain imaging studies investigated purported morphometric changes in early (EB) and late onset blindness (LB). The results of these studies have not yielded very consistent results, neither with respect to the type, nor to the anatomical locations of the brain morphometric alterations. To better characterize the effects of blindness on brain morphometry, we performed a systematic review and an Anatomical-Likelihood-Estimation (ALE) coordinate-based-meta-analysis of 65 eligible studies on brain structural changes in EB and LB, including 890 EB, 466 LB and 1257 sighted controls. Results revealed atrophic changes throughout the whole extent of the retino-geniculo-striate system in both EB and LB, whereas changes in areas beyond the occipital lobe occurred in EB only. We discuss the nature of some of the contradictory findings with respect to the used brain imaging methodologies and characteristics of the blind populations such as the onset, duration and cause of blindness. Future studies should aim for much larger sample sizes, eventually by merging data from different brain imaging centers using the same imaging sequences, opt for multimodal structural brain imaging, and go beyond a purely structural approach by combining functional with structural connectivity network analyses.

The Topo-Speech sensory substitution system as a method of conveying spatial information to the blind and vision impaired

Humans, like most animals, integrate sensory input in the brain from different sensory modalities. Yet humans are distinct in their ability to grasp symbolic input, which is interpreted into a cognitive mental representation of the world. This representation merges with external sensory input, providing modality integration of a different sort. This study evaluates the Topo-Speech algorithm in the blind and visually impaired. The system provides spatial information about the external world by applying sensory substitution alongside symbolic representations in a manner that corresponds with the unique way our brains acquire and process information. This is done by conveying spatial information, customarily acquired through vision, through the auditory channel, in a combination of sensory (auditory) features and symbolic language (named/spoken) features. The Topo-Speech sweeps the visual scene or image and represents objects' identity by employing naming in a spoken word and simultaneously conveying the objects' location by mapping the x-axis of the visual scene or image to the time it is announced and the y-axis by mapping the location to the pitch of the voice. This proof of concept study primarily explores the practical applicability of this approach in 22 visually impaired and blind individuals. The findings showed that individuals from both populations could effectively interpret and use the algorithm after a single training session. The blind showed an accuracy of 74.45%, while the visually impaired had an average accuracy of 72.74%. These results are comparable to those of the sighted, as shown in previous research, with all participants above chance level. As such, we demonstrate practically how aspects of spatial information can be transmitted through non-visual channels. To complement the findings, we weigh in on debates concerning models of spatial knowledge (the persistent, cumulative, or convergent models) and the capacity for spatial representation in the blind. We suggest the present study's findings support the convergence model and the scenario that posits the blind are capable of some aspects of spatial representation as depicted by the algorithm comparable to those of the sighted. Finally, we present possible future developments, implementations, and use cases for the system as an aid for the blind and visually impaired.

Bioelectronic medicine: Preclinical insights and clinical advances

The nervous system maintains homeostasis and health. Homeostatic disruptions underlying the pathobiology of many diseases can be controlled by bioelectronic devices targeting CNS and peripheral neural circuits. New insights into the regulatory functions of the nervous system and technological developments in bioelectronics drive progress in the emerging field of bioelectronic medicine. Here, we provide an overview of key aspects of preclinical research, translation, and clinical advances in bioelectronic medicine.

Chinese Children with Congenital and Acquired Blindness Represent Concrete Concepts in Vertical Space through Tactile Perception

Many studies have tested perceptual symbols in conceptual processing and found that perceptual symbols contain experiences from multisensory channels. However, whether the disability of one sensory channel affects the processing of the perceptual symbols and then affects conceptual processing is still unknown. This line of research would extend the perceptual symbol theory and have implications for language rehabilitation and mental health for people with disabilities. Therefore, the present study filled in this gap and tested whether Chinese children with congenital and acquired blindness have difficulty in recruiting perceptual symbols in the processing of concrete concepts. Experiment 1 used the word-pair-matching paradigm to test whether blind children used vertical space information in understanding concrete word pairs. Experiment 2 used the word-card-pairing paradigm to test the role of tactile experiences in the processing of concrete concepts for blind children. Results found that blind children automatically activated the spatial information of referents in the processing of concepts through the tactile sensory channel even when the visual sensory channel was disabled. This finding supported the compensatory phenomenon of other sensory channels in conceptual representation. In addition, the difference between elementary school blind children and middle school blind children in judging the spatial position of concrete words also indicated the vital influence of perceptual experiences on perceptual symbols in conceptual representation. Interestingly, there were no significant differences between children with congenital or acquired blindness. This might suggest that the compensatory of other sensory channels did not have a sensitive period. This study not only provided new evidence for the perceptual symbol theory but also found that perceptual symbols could be developed by a compensatory mechanism. This compensatory mechanism can be used to develop a rehabilitation program for improving language learning in blind children. Improved language ability in blind children will also improve their mental health problems caused by difficulties in social interaction (e.g., social anxiety).

A case study in phenomenology of visual experience with retinal prosthesis versus visual-to-auditory sensory substitution

The phenomenology of the blind has provided an age-old, unparalleled means of exploring the enigmatic link between the brain and mind. This paper delves into the unique phenomenological experience of a man who became blind in adulthood. He subsequently underwent both an Argus II retinal prosthesis implant and training, and extensive training on the EyeMusic visual to auditory sensory substitution device (SSD), thereby becoming the first reported case to date of dual proficiency with both devices. He offers a firsthand account into what he considers the great potential of combining sensory substitution devices with visual prostheses as part of a complete visual restoration protocol. While the Argus II retinal prosthesis alone provided him with immediate visual percepts by way of electrically stimulated phosphenes elicited by the device, the EyeMusic SSD requires extensive training from the onset. Yet following the extensive training program with the EyeMusic sensory substitution device, our subject reports that the sensory substitution device allowed him to experience a richer, more complex perceptual experience, that felt more "second nature" to him, while the Argus II prosthesis (which also requires training) did not allow him to achieve the same levels of automaticity and transparency. Following long-term use of the EyeMusic SSD, our subject reported that visual percepts representing mainly, but not limited to, colors portrayed by the EyeMusic SSD are elicited in association with auditory stimuli, indicating the acquisition of a high level of automaticity. Finally, the case study indicates an additive benefit to the combination of both devices on the user's subjective phenomenological visual experience.

Supervised Multisensory Calibration Signals Are Evident in VIP But Not MSTd

Multisensory plasticity enables our senses to dynamically adapt to each other and the external environment, a fundamental operation that our brain performs continuously. We searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) in 2 male rhesus macaques using a paradigm of supervised calibration. We report little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. In contrast, neural correlates of plasticity are found in higher-level multisensory VIP, an area with strong decision-related activity. Accordingly, we observed systematic shifts of VIP tuning curves, which were reflected in the choice-related component of the population response. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions. These results lay the foundation for understanding multisensory neural plasticity, applicable broadly to maintaining accuracy for sensorimotor tasks.SIGNIFICANCE STATEMENT Multisensory plasticity is a fundamental and continual function of the brain that enables our senses to adapt dynamically to each other and to the external environment. Yet, very little is known about the neuronal mechanisms of multisensory plasticity. In this study, we searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) using a paradigm of supervised calibration. We found little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. By contrast, neural correlates of plasticity were found in VIP, a higher-level multisensory area with strong decision-related activity. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions.

Perceptual Learning of Object Recognition in Simulated Retinal Implant Perception - The Effect of Video Training

Purpose

Retinal implants (RIs) provide new vision for patients suffering from photoreceptor degeneration in the retina. The limited vision gained by RI, however, leaves room for improvement by training regimes.

Methods

Two groups of normal-sighted participants were respectively trained with videos or still images of daily objects in a labeling task. Object appearance was simulated to resemble RI perception. In Experiment 1, the training effect was measured as the change in performance during the training, and the same labeling task was conducted after 1 week to test the retention. In Experiment 2 with a different pool of participants, a reverse labeling task was included before (pre-test) and after the training (post-test) to show if the training effect could be generalized into a different task context.

Results

Both groups showed improved object recognition through training that was maintained for a week, and the video group showed better improvement (Experiment 1). Both groups showed improved object recognition in a different task that was maintained for a week, but the video group did not show better retention than the image group (Experiment 2).

Conclusions

Training with video materials leads to more improvement than training with still images in simulated RI perception, but this better improvement was specific to the trained task.

Translational Relevance

We recommend videos as better training materials than still images for patients with RIs to improve object recognition when the task-goal is highly specific. We also propose here that achieving highly specific training goals runs the risk of limiting the generalization of the training effects.

Early deafness leads to re-shaping of functional connectivity beyond the auditory cortex

Early sensory deprivation, such as deafness, shapes brain development in multiple ways. Deprived auditory areas become engaged in the processing of stimuli from the remaining modalities and in high-level cognitive tasks. Yet, structural and functional changes were also observed in non-deprived brain areas, which may suggest the whole-brain network changes in deaf individuals. To explore this possibility, we compared the resting-state functional network organization of the brain in early deaf adults and hearing controls and examined global network segregation and integration. Relative to hearing controls, deaf adults exhibited decreased network segregation and an altered modular structure. In the deaf, regions of the salience network were coupled with the fronto-parietal network, while in the hearing controls, they were coupled with other large-scale networks. Deaf adults showed weaker connections between auditory and somatomotor regions, stronger coupling between the fronto-parietal network and several other large-scale networks (visual, memory, cingulo-opercular and somatomotor), and an enlargement of the default mode network. Our findings suggest that brain plasticity in deaf adults is not limited to changes in the auditory cortex but additionally alters the coupling between other large-scale networks and the development of functional brain modules. These widespread functional connectivity changes may provide a mechanism for the superior behavioral performance of the deaf in visual and attentional tasks.

The Effect of Blindness on Spatial Asymmetries

The human cerebral cortex is asymmetrically organized with hemispheric lateralization pervading nearly all neural systems of the brain. Whether the lack of normal visual development affects hemispheric specialization subserving the deployment of visuospatial attention asymmetries is controversial. In principle, indeed, the lack of early visual experience may affect the lateralization of spatial functions, and the blind may rely on a different sensory input compared to the sighted. In this review article, we thus present a current state-of-the-art synthesis of empirical evidence concerning the effects of visual deprivation on the lateralization of various spatial processes (i.e., including line bisection, mirror symmetry, and localization tasks). Overall, the evidence reviewed indicates that spatial processes are supported by a right hemispheric network in the blind, hence, analogously to the sighted. Such a right-hemisphere dominance, however, seems more accentuated in the blind as compared to the sighted as indexed by the greater leftward bias shown in different spatial tasks. This is possibly the result of the more pronounced involvement of the right parietal cortex during spatial tasks in blind individuals compared to the sighted, as well as of the additional recruitment of the right occipital cortex, which would reflect the cross-modal plastic phenomena that largely characterize the blind brain.

Visual experience is not necessary for the development of face-selectivity in the lateral fusiform gyrus

The fusiform face area responds selectively to faces and is causally involved in face perception. How does face-selectivity in the fusiform arise in development, and why does it develop so systematically in the same location across individuals? Preferential cortical responses to faces develop early in infancy, yet evidence is conflicting on the central question of whether visual experience with faces is necessary. Here, we revisit this question by scanning congenitally blind individuals with fMRI while they haptically explored 3D-printed faces and other stimuli. We found robust face-selective responses in the lateral fusiform gyrus of individual blind participants during haptic exploration of stimuli, indicating that neither visual experience with faces nor fovea-biased inputs is necessary for face-selectivity to arise in the lateral fusiform gyrus. Our results instead suggest a role for long-range connectivity in specifying the location of face-selectivity in the human brain.

Mental Rotation of Digitally-Rendered Haptic Objects by the Visually-Impaired

In the event of visual impairment or blindness, information from other intact senses can be used as substitutes to retrain (and in extremis replace) visual functions. Abilities including reading, mental representation of objects and spatial navigation can be performed using tactile information. Current technologies can convey a restricted library of stimuli, either because they depend on real objects or renderings with low resolution layouts. Digital haptic technologies can overcome such limitations. The applicability of this technology was previously demonstrated in sighted participants. Here, we reasoned that visually-impaired and blind participants can create mental representations of letters presented haptically in normal and mirror-reversed form without the use of any visual information, and mentally manipulate such representations. Visually-impaired and blind volunteers were blindfolded and trained on the haptic tablet with two letters (either L and P or F and G). During testing, they haptically explored on any trial one of the four letters presented at 0°, 90°, 180°, or 270° rotation from upright and indicated if the letter was either in a normal or mirror-reversed form. Rotation angle impacted performance; greater deviation from 0° resulted in greater impairment for trained and untrained normal letters, consistent with mental rotation of these haptically-rendered objects. Performance was also generally less accurate with mirror-reversed stimuli, which was not affected by rotation angle. Our findings demonstrate, for the first time, the suitability of a digital haptic technology in the blind and visually-impaired. Classic devices remain limited in their accessibility and in the flexibility of their applications. We show that mental representations can be generated and manipulated using digital haptic technology. This technology may thus offer an innovative solution to the mitigation of impairments in the visually-impaired, and to the training of skills dependent on mental representations and their spatial manipulation.

Bioelectronic Medicine: From Preclinical Studies on the Inflammatory Reflex to New Approaches in Disease Diagnosis and Treatment

Bioelectronic medicine is an evolving field in which new insights into the regulatory role of the nervous system and new developments in bioelectronic technology result in novel approaches in disease diagnosis and treatment. Studies on the immunoregulatory function of the vagus nerve and the inflammatory reflex have a specific place in bioelectronic medicine. These studies recently led to clinical trials with bioelectronic vagus nerve stimulation in inflammatory diseases and other conditions. Here, we outline key findings from this preclinical and clinical research. We also point to other aspects and pillars of interdisciplinary research and technological developments in bioelectronic medicine.

Gold nanostructure microelectrode arrays for in vitro recording and stimulation from neuronal networks

An ideal microelectrode array (MEA) design should include materials and structures which exhibit biocompatibility, low electrode polarization, low impedance/noise, and structural durability. Here, the fabrication of MEAs with indium tin oxide (ITO) electrodes deposited with self-similar gold nanostructures (GNS) is described. We show that fern leaf fractal-like GNS deposited on ITO electrodes are conducive for neural cell attachment and viability while reducing the interfacial impedance more than two orders of magnitude at low frequencies (100-1000 Hz) versus bare ITO. GNS MEAs, with low interfacial impedance, allowed the detection of extracellular action potentials with excellent signal-to-noise ratios (SNR, 20.26 ± 2.14). Additionally, the modified electrodes demonstrated electrochemical and mechanical stability over 29 d in vitro.

Residual vision activation and the brain-eye-vascular triad: Dysregulation, plasticity and restoration in low vision and blindness - a review

Vision loss due to ocular diseases such as glaucoma, optic neuropathy, macular degeneration, or diabetic retinopathy, are generally considered an exclusive affair of the retina and/or optic nerve. However, the brain, through multiple indirect influences, has also a major impact on functional visual impairment. Such indirect influences include intracerebral pressure, eye movements, top-down modulation (attention, cognition), and emotionally triggered stress hormone release affecting blood vessel dysregulation. Therefore, vision loss should be viewed as the result of multiple interactions within a “brain-eye-vascular triad”, and several eye diseases may also be considered as brain diseases in disguise. While the brain is part of the problem, it can also be part of the solution. Neuronal networks of the brain can “amplify” residual vision through neuroplasticity changes of local and global functional connectivity by activating, modulating and strengthening residual visual signals. The activation of residual vision can be achieved by different means such as vision restoration training, non-invasive brain stimulation, or blood flow enhancing medications. Modulating brain functional networks and improving vascular regulation may offer new opportunities to recover or restore low vision by increasing visual field size, visual acuity and overall functional vision. Hence, neuroscience offers new insights to better understand vision loss, and modulating brain and vascular function is a promising source for new opportunities to activate residual vision to achieve restoration and recovery to improve quality of live in patients suffering from low vision.

Mental Rotation of Digitally-Rendered Haptic Objects

Sensory substitution is an effective means to rehabilitate many visual functions after visual impairment or blindness. Tactile information, for example, is particularly useful for functions such as reading, mental rotation, shape recognition, or exploration of space. Extant haptic technologies typically rely on real physical objects or pneumatically driven renderings and thus provide a limited library of stimuli to users. New developments in digital haptic technologies now make it possible to actively simulate an unprecedented range of tactile sensations. We provide a proof-of-concept for a new type of technology (hereafter haptic tablet) that renders haptic feedback by modulating the friction of a flat screen through ultrasonic vibrations of varying shapes to create the sensation of texture when the screen is actively explored. We reasoned that participants should be able to create mental representations of letters presented in normal and mirror-reversed haptic form without the use of any visual information and to manipulate such representations in a mental rotation task. Healthy sighted, blindfolded volunteers were trained to discriminate between two letters (either L and P, or F and G; counterbalanced across participants) on a haptic tablet. They then tactually explored all four letters in normal or mirror-reversed form at different rotations (0°, 90°, 180°, and 270°) and indicated letter form (i.e., normal or mirror-reversed) by pressing one of two mouse buttons. We observed the typical effect of rotation angle on object discrimination performance (i.e., greater deviation from 0° resulted in worse performance) for trained letters, consistent with mental rotation of these haptically-rendered objects. We likewise observed generally slower and less accurate performance with mirror-reversed compared to prototypically oriented stimuli. Our findings extend existing research in multisensory object recognition by indicating that a new technology simulating active haptic feedback can support the generation and spatial manipulation of mental representations of objects. Thus, such haptic tablets can offer a new avenue to mitigate visual impairments and train skills dependent on mental object-based representations and their spatial manipulation.

The Innate Alarm System and Subliminal Threat Presentation in Posttraumatic Stress Disorder: Neuroimaging of the Midbrain and Cerebellum

BACKGROUND

The innate alarm system, a network of interconnected midbrain, other brainstem, and thalamic structures, serves to rapidly detect stimuli in the environment prior to the onset of conscious awareness. This system is sensitive to threatening stimuli and has evolved to process these stimuli subliminally for hastened responding. Despite the conscious unawareness, the presentation of subliminal threat stimuli generates increased activation of limbic structures, including the amygdala and insula, as well as emotionally evaluative structures, including the cerebellum and orbitofrontal cortex. Posttraumatic stress disorder (PTSD) is associated with an increased startle response and decreased extinction learning to conditioned threat. The role of the innate alarm system in the clinical presentation of PTSD, however, remains poorly understood.

METHODS

Here, we compare midbrain, brainstem, and cerebellar activation in persons with PTSD (n = 26) and matched controls (n = 20) during subliminal threat presentation. Subjects were presented with masked trauma-related and neutral stimuli below conscious threshold. Contrasts of subliminal brain activation for the presentation of neutral stimuli were subtracted from trauma-related brain activation. Group differences in activation, as well as correlations between clinical scores and PTSD activation, were examined. Imaging data were preprocessed utilizing the spatially unbiased infratentorial template toolbox within SPM12.

RESULTS

Analyses revealed increased midbrain activation in PTSD as compared to controls in the superior colliculus, periaqueductal gray, and midbrain reticular formation during subliminal threat as compared to neutral stimulus presentation. Controls showed increased activation in the right cerebellar lobule V during subliminal threat presentation as compared to PTSD. Finally, a negative correlation emerged between PTSD patient scores on the Multiscale Dissociation Inventory for the Depersonalization/Derealization subscale and activation in the right lobule V of the cerebellum during the presentation of subliminal threat as compared to neutral stimuli.

CONCLUSION

We interpret these findings as evidence of innate alarm system overactivation in PTSD and of the prominent role of the cerebellum in the undermodulation of emotion observed in PTSD.

Improvement in reading performance through training with simulated thalamic visual prostheses

Simulations of artificial vision are used to provide the researcher an opportunity to explore different aspects of visual prosthesis device design by observing subject performance on various tasks viewed through the simulation. Such studies typically use normal, sighted subjects to measure performance at a given point in time. Relatively few studies examine performance changes longitudinally to quantitatively assess the benefits from a training plan that would be akin to post-implantation rehabilitation. Here, we had six normal, sighted subjects use a standard reading task with daily practice over eight weeks to understand the effects of an intensive training schedule on adaptation to artificial sight. Subjects read 40 MNREAD-style sentences per session, with a new set each session, that were presented at five font sizes (logMAR 1.0–1.4) and through three center-weighted phosphene patterns (2,000, 1,000, 500 phosphenes). We found that subjects improved their reading accuracy across sessions, and that the training lead to an increase of reading speed that was equivalent to a doubling of available phosphenes. Most importantly, the hardest condition, while initially illegible, supported functional reading after training. Consistent with experience-driven neuroplastic changes, gaps in the training schedule lead to transient decreases in reading speed, but, surprisingly, not reading accuracy. Our findings contribute to our larger project of developing a thalamic visual prosthesis and to post-implant rehabilitation strategies.

Development of visual Neuroprostheses: trends and challenges

Visual prostheses are implantable medical devices that are able to provide some degree of vision to individuals who are blind. This research field is a challenging subject in both ophthalmology and basic science that has progressed to a point where there are already several commercially available devices. However, at present, these devices are only able to restore a very limited vision, with relatively low spatial resolution. Furthermore, there are still many other open scientific and technical challenges that need to be solved to achieve the therapeutic benefits envisioned by these new technologies. This paper provides a brief overview of significant developments in this field and introduces some of the technical and biological challenges that still need to be overcome to optimize their therapeutic success, including long-term viability and biocompatibility of stimulating electrodes, the selection of appropriate patients for each artificial vision approach, a better understanding of brain plasticity and the development of rehabilitative strategies specifically tailored for each patient.

Visual deprivation modifies oscillatory activity in visual and auditory centers

Loss of vision may enhance the capabilities of auditory perception, but the mechanisms mediating these changes remain elusive. Here, visual deprivation in rats resulted in altered oscillatory activities, which appeared to be the result of a common mechanism underlying neuronal assembly formation in visual and auditory centers. The power of high-frequency β and γ oscillations in V1 (the primary visual cortex) and β oscillations in the LGN (lateral geniculate nucleus) was increased after one week of visual deprivation. Meanwhile, the power of β oscillations in A1 (the primary auditory cortex) and the power of β and γ oscillations in the MGB (medial geniculate body) were also enhanced in the absence of visual input. Furthermore, nerve tracing revealed a bidirectional nerve fiber connection between V1 and A1 cortices, which might be involved in transmitting auditory information to the visual cortex, contributing to enhanced auditory perception after visual deprivation. These results may facilitate the better understanding of multisensory cross-modal plasticity.

Amplitude Modulation-based Electrical Stimulation for Encoding Multipixel Spatiotemporal Visual Information in Retinal Neural Activities

Retinal implants have been developed as a promising way to restore partial vision for the blind. The observation and analysis of neural activities can offer valuable insights for successful prosthetic electrical stimulation. Retinal ganglion cell (RGC) activities have been investigated to provide knowledge on the requirements for electrical stimulation, such as threshold current and the effect of stimulation waveforms. To develop a detailed 'stimulation strategy' for faithful delivery of spatiotemporal visual information to the brain, it is essential to examine both the temporal and spatial characteristics of RGC responses, whereas previous studies were mainly focused on one or the other. In this study, we investigate whether the spatiotemporal visual information can be decoded from the RGC network activity evoked by patterned electrical stimulation. Along with a thorough characterization of spatial spreading of stimulation current and temporal information encoding, we demonstrated that multipixel spatiotemporal visual information can be accurately decoded from the population activities of RGCs stimulated by amplitude-modulated pulse trains. We also found that the details of stimulation, such as pulse amplitude range and pulse rate, were crucial for accurate decoding. Overall, the results suggest that useful visual function may be restored by amplitude modulation-based retinal stimulation.

Perspectives on: Materials Aspects for Retinal Prostheses

Retinitis pigmentosa and age-related macular degeneration are both incurable eye diseases that lead to blindness due to photoreceptor degeneration. Electrically stimulating the remaining intact nerve cells may generate some useful vision for patients afflicted with these diseases. Various types of retinal prostheses, sub- and epi-retinal electrode arrays, as well as subretinal microphotodiode arrays are considered from a materials and biocompatibility point of view. Other, more innovative approaches to restoring vision, such as microfluidic pumps and activated nanosystems that deliver neurotransmitters in a controlled way and photodynamic therapy are being developed. This article discusses materials aspects of retinal prostheses that are currently in use or under development.

A qualitative self-study of Retinitis Pigmentosa

Retinitis Pigmentosa (RP) is a retinal dege nerative disease causing progressive blindness. Most research on RP is biomedical, and mostly from an observer perspective, therefore poorly reflecting the lived experience of having RP. Accordingly, the researcher conducted a retrospective qualitative self-study, to analyse reflections on his own experience of diagnosis and receiving a mobility cane, as contained in emails sent to friends and colleagues. This analysis yielded a number of interesting themes significant to the RP experience, namely diagnosis, impact and dealing with RP. Reference is made to literature encouraging ophthalmologists to be actively involved in the facilitation of adjustment to loss of vision. Other significant issues relevant to visual impairment, such as the stigma associated with the long cane and challenging the dominant social discourse regarding disability, are discussed. In addition, the importance of qualitative research to elicit the stories of blind individuals is highlighted.

The Occipital Cortex in the Blind

Studying the brains of blind individuals provides a unique opportunity to investigate how the brain changes and adapts in response to afferent (input) and efferent (output) demands. We discuss evidence suggesting that regions of the brain normally associated with the processing of visual information undergo remarkable dynamic change in response to blindness. These neuroplastic changes implicate not only processing carried out by the remaining senses but also higher cognitive functions such as language and memory. A strong emphasis is placed on evidence obtained from advanced neuroimaging techniques that allow researchers to identify areas of human brain activity, as well as from lesion approaches (both reversible and irreversible) to address the functional relevance and role of these activated areas. A possible mechanism and conceptual framework for these physiological and behavioral changes is proposed.

What is the lived experience for people with Leber Hereditary Optic Neuropathy?

This study aimed to hear the voices of seven people living with Leber Hereditary Optic Neuropathy (LHON), to examine their lived experience. The participants were recruited through a website and social networking sites designed for this community; they discussed their experiences in semi-structured interviews. The analytical process was informed by Interpretive Phenomenological Analysis (IPA) principles. This process illustrated a lived experience that was initially consumed by psycho-social losses and conflicts around identity. However, through pragmatic and resilient attitudes, the participants developed practical methods and emotional coping mechanisms that permitted their adjustment to sight loss and progression in life. Consistent with other studies, the fundamental finding was that the associated feeling of loss and frustration are chronic if, episodic, throughout life.

Progress in the clinical development and utilization of vision prostheses: an update

Vision prostheses, or "bionic eyes", are implantable medical bionic devices with the potential to restore rudimentary sight to people with profound vision loss or blindness. In the past two decades, this field has rapidly progressed, and there are now two commercially available retinal prostheses in the US and Europe, and a number of next-generation devices in development. This review provides an update on the development of these devices and a discussion on the future directions for the field.

Impact of Early and Late Visual Deprivation on the Structure of the Corpus Callosum: A Study Combining Thickness Profile with Surface Tensor-Based Morphometry

Blindness represents a unique model to study how visual experience may shape the development of brain organization. Exploring how the structure of the corpus callosum (CC) reorganizes ensuing visual deprivation is of particular interest due to its important functional implication in vision (e.g., via the splenium of the CC). Moreover, comparing early versus late visually deprived individuals has the potential to unravel the existence of a sensitive period for reshaping the CC structure. Here, we develop a novel framework to capture a complete set of shape differences in the CC between congenitally blind (CB), late blind (LB) and sighted control (SC) groups. The CCs were manually segmented from T1-weighted brain MRI and modeled by 3D tetrahedral meshes. We statistically compared the combination of local area and thickness at each point between subject groups. Differences in area are found using surface tensor-based morphometry; thickness is estimated by tracing the streamlines in the volumetric harmonic field. Group differences were assessed on this combined measure using Hotelling's T(2) test. Interestingly, we observed that the total callosal volume did not differ between the groups. However, our fine-grained analysis reveals significant differences mostly localized around the splenium areas between both blind groups and the sighted group (general effects of blindness) and, importantly, specific dissimilarities between the LB and CB groups, illustrating the existence of a sensitive period for reorganization. The new multivariate statistics also gave better effect sizes for detecting morphometric differences, relative to other statistics. They may boost statistical power for CC morphometric analyses.

Spatial navigation by congenitally blind individuals

Spatial navigation in the absence of vision has been investigated from a variety of perspectives and disciplines. These different approaches have progressed our understanding of spatial knowledge acquisition by blind individuals, including their abilities, strategies, and corresponding mental representations. In this review, we propose a framework for investigating differences in spatial knowledge acquisition by blind and sighted people consisting of three longitudinal models (i.e., convergent, cumulative, and persistent). Recent advances in neuroscience and technological devices have provided novel insights into the different neural mechanisms underlying spatial navigation by blind and sighted people and the potential for functional reorganization. Despite these advances, there is still a lack of consensus regarding the extent to which locomotion and wayfinding depend on amodal spatial representations. This challenge largely stems from methodological limitations such as heterogeneity in the blind population and terminological ambiguity related to the concept of cognitive maps. Coupled with an over‐reliance on potential technological solutions, the field has diffused into theoretical and applied branches that do not always communicate. Here, we review research on navigation by congenitally blind individuals with an emphasis on behavioral and neuroscientific evidence, as well as the potential of technological assistance. Throughout the article, we emphasize the need to disentangle strategy choice and performance when discussing the navigation abilities of the blind population. WIREs Cogn Sci 2016, 7:37–58. doi: 10.1002/wcs.1375

For further resources related to this article, please visit the WIREs website.

How Visual Is the Visual Cortex? Comparing Connectional and Functional Fingerprints between Congenitally Blind and Sighted Individuals

Classical animal visual deprivation studies and human neuroimaging studies have shown that visual experience plays a critical role in shaping the functionality and connectivity of the visual cortex. Interestingly, recent studies have additionally reported circumscribed regions in the visual cortex in which functional selectivity was remarkably similar in individuals with and without visual experience. Here, by directly comparing resting-state and task-based fMRI data in congenitally blind and sighted human subjects, we obtained large-scale continuous maps of the degree to which connectional and functional "fingerprints" of ventral visual cortex depend on visual experience. We found a close agreement between connectional and functional maps, pointing to a strong interdependence of connectivity and function. Visual experience (or the absence thereof) had a pronounced effect on the resting-state connectivity and functional response profile of occipital cortex and the posterior lateral fusiform gyrus. By contrast, connectional and functional fingerprints in the anterior medial and posterior lateral parts of the ventral visual cortex were statistically indistinguishable between blind and sighted individuals. These results provide a large-scale mapping of the influence of visual experience on the development of both functional and connectivity properties of visual cortex, which serves as a basis for the formulation of new hypotheses regarding the functionality and plasticity of specific subregions. Significance statement: How is the functionality and connectivity of the visual cortex shaped by visual experience? By directly comparing resting-state and task-based fMRI data in congenitally blind and sighted subjects, we obtained large-scale continuous maps of the degree to which connectional and functional "fingerprints" of ventral visual cortex depend on visual experience. In addition to revealing regions that are strongly dependent on visual experience (early visual cortex and posterior fusiform gyrus), our results showed regions in which connectional and functional patterns are highly similar in blind and sighted individuals (anterior medial and posterior lateral ventral occipital temporal cortex). These results serve as a basis for the formulation of new hypotheses regarding the functionality and plasticity of specific subregions of the visual cortex.

Retinal prostheses: progress toward the next generation implants

In the last decade, various clinical trials proved the capability of visual prostheses, in particular retinal implants, to restore a useful form of vision. These encouraging results promoted the emerging of several strategies for neuronal stimulation aiming at the restoration of sight. Besides the traditional approach based on electrical stimulation through metal electrodes in the different areas of the visual path (e.g., the visual cortex, the lateral geniculate nucleus, the optic nerve, and the retina), novel concepts for neuronal stimulation have been mostly exploited as building blocks of the next generation of retinal implants. This review is focused on critically discussing recent major advancements in the field of retinal stimulation with particular attention to the findings in the application of novel concepts and materials. Last, the major challenges in the field and their clinical implications will be outlined.

Is there a future for sensory substitution outside academic laboratories?

Sensory substitution devices (SSDs) have been developed with the ultimate purpose of supporting sensory deprived individuals in their daily activities. However, more than forty years after their first appearance in the scientific literature, SSDs still remain more common in research laboratories than in the daily life of people with sensory deprivation. Here, we seek to identify the reasons behind the limited diffusion of SSDs among the blind community by discussing the ergonomic, neurocognitive and psychosocial issues potentially associated with the use of these systems. We stress that these issues should be considered together when developing future devices or improving existing ones. We provide some examples of how to achieve this by adopting a multidisciplinary and participatory approach. These efforts would contribute not solely to address fundamental theoretical research questions, but also to better understand the everyday needs of blind people and eventually promote the use of SSDs outside laboratories.

Science and technology roadmap for graphene, related two-dimensional crystals, and hybrid systems

We present the science and technology roadmap for graphene, related two-dimensional crystals, and hybrid systems, targeting an evolution in technology, that might lead to impacts and benefits reaching into most areas of society. This roadmap was developed within the framework of the European Graphene Flagship and outlines the main targets and research areas as best understood at the start of this ambitious project. We provide an overview of the key aspects of graphene and related materials (GRMs), ranging from fundamental research challenges to a variety of applications in a large number of sectors, highlighting the steps necessary to take GRMs from a state of raw potential to a point where they might revolutionize multiple industries. We also define an extensive list of acronyms in an effort to standardize the nomenclature in this emerging field.

Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training

In a series of studies we have been investigating how multisensory training affects unisensory perceptual learning with speech stimuli. Previously, we reported that audiovisual (AV) training with speech stimuli can promote auditory-only (AO) perceptual learning in normal-hearing adults but can impede learning in congenitally deaf adults with late-acquired cochlear implants. Here, impeder and promoter effects were sought in normal-hearing adults who participated in lipreading training. In Experiment 1, visual-only (VO) training on paired associations between CVCVC nonsense word videos and nonsense pictures demonstrated that VO words could be learned to a high level of accuracy even by poor lipreaders. In Experiment 2, visual-auditory (VA) training in the same paradigm but with the addition of synchronous vocoded acoustic speech impeded VO learning of the stimuli in the paired-associates paradigm. In Experiment 3, the vocoded AO stimuli were shown to be less informative than the VO speech. Experiment 4 combined vibrotactile speech stimuli with the visual stimuli during training. Vibrotactile stimuli were shown to promote visual perceptual learning. In Experiment 5, no-training controls were used to show that training with visual speech carried over to consonant identification of untrained CVCVC stimuli but not to lipreading words in sentences. Across this and previous studies, multisensory training effects depended on the functional relationship between pathways engaged during training. Two principles are proposed to account for stimulus effects: (1) Stimuli presented to the trainee’s primary perceptual pathway will impede learning by a lower-rank pathway. (2) Stimuli presented to the trainee’s lower rank perceptual pathway will promote learning by a higher-rank pathway. The mechanisms supporting these principles are discussed in light of multisensory reverse hierarchy theory (RHT).

Audiovisual spoken word training can promote or impede auditory-only perceptual learning: prelingually deafened adults with late-acquired cochlear implants versus normal hearing adults

Training with audiovisual (AV) speech has been shown to promote auditory perceptual learning of vocoded acoustic speech by adults with normal hearing. In Experiment 1, we investigated whether AV speech promotes auditory-only (AO) perceptual learning in prelingually deafened adults with late-acquired cochlear implants. Participants were assigned to learn associations between spoken disyllabic C(=consonant)V(=vowel)CVC non-sense words and non-sense pictures (fribbles), under AV and then AO (AV-AO; or counter-balanced AO then AV, AO-AV, during Periods 1 then 2) training conditions. After training on each list of paired-associates (PA), testing was carried out AO. Across all training, AO PA test scores improved (7.2 percentage points) as did identification of consonants in new untrained CVCVC stimuli (3.5 percentage points). However, there was evidence that AV training impeded immediate AO perceptual learning: During Period-1, training scores across AV and AO conditions were not different, but AO test scores were dramatically lower in the AV-trained participants. During Period-2 AO training, the AV-AO participants obtained significantly higher AO test scores, demonstrating their ability to learn the auditory speech. Across both orders of training, whenever training was AV, AO test scores were significantly lower than training scores. Experiment 2 repeated the procedures with vocoded speech and 43 normal-hearing adults. Following AV training, their AO test scores were as high as or higher than following AO training. Also, their CVCVC identification scores patterned differently than those of the cochlear implant users. In Experiment 1, initial consonants were most accurate, and in Experiment 2, medial consonants were most accurate. We suggest that our results are consistent with a multisensory reverse hierarchy theory, which predicts that, whenever possible, perceivers carry out perceptual tasks immediately based on the experience and biases they bring to the task. We point out that while AV training could be an impediment to immediate unisensory perceptual learning in cochlear implant patients, it was also associated with higher scores during training.

Revisiting the adaptive and maladaptive effects of crossmodal plasticity

One of the most striking demonstrations of experience-dependent plasticity comes from studies of sensory-deprived individuals (e.g., blind or deaf), showing that brain regions deprived of their natural inputs change their sensory tuning to support the processing of inputs coming from the spared senses. These mechanisms of crossmodal plasticity have been traditionally conceptualized as having a double-edged sword effect on behavior. On one side, crossmodal plasticity is conceived as adaptive for the development of enhanced behavioral skills in the remaining senses of early-deaf or blind individuals. On the other side, crossmodal plasticity raises crucial challenges for sensory restoration and is typically conceived as maladaptive since its presence may prevent optimal recovery in sensory-re-afferented individuals. In the present review we stress that this dichotomic vision is oversimplified and we emphasize that the notions of the unavoidable adaptive/maladaptive effects of crossmodal reorganization for sensory compensation/restoration may actually be misleading. For this purpose we critically review the findings from the blind and deaf literatures, highlighting the complementary nature of these two fields of research. The integrated framework we propose here has the potential to impact on the way rehabilitation programs for sensory recovery are carried out, with the promising prospect of eventually improving their final outcomes.

Face recognition in simulated prosthetic vision: face detection-based image processing strategies

OBJECTIVE

Given the limited visual percepts elicited by current prosthetic devices, it is essential to optimize image content in order to assist implant wearers to achieve better performance of visual tasks. This study focuses on recognition of familiar faces using simulated prosthetic vision.

APPROACH

Combined with region-of-interest (ROI) magnification, three face extraction strategies based on a face detection technique were used: the Viola-Jones face region, the statistical face region (SFR) and the matting face region.

MAIN RESULTS

These strategies significantly enhanced recognition performance compared to directly lowering resolution (DLR) with Gaussian dots. The inclusion of certain external features, such as hairstyle, was beneficial for face recognition. Given the high recognition accuracy achieved and applicable processing speed, SFR-ROI was the preferred strategy. DLR processing resulted in significant face gender recognition differences (i.e. females were more easily recognized than males), but these differences were not apparent with other strategies.

SIGNIFICANCE

Face detection-based image processing strategies improved visual perception by highlighting useful information. Their use is advisable for face recognition when using low-resolution prosthetic vision. These results provide information for the continued design of image processing modules for use in visual prosthetics, thus maximizing the benefits for future prosthesis wearers.