Simulation of artificial vision: II. Eccentric reading of full-page text and the learning of this task

Gaze-contingent processing improves mobility, scene recognition and visual search in simulated head-steered prosthetic vision

Objective.The enabling technology of visual prosthetics for the blind is making rapid progress. However, there are still uncertainties regarding the functional outcomes, which can depend on many design choices in the development. In visual prostheses with a head-mounted camera, a particularly challenging question is how to deal with the gaze-locked visual percept associated with spatial updating conflicts in the brain. The current study investigates a recently proposed compensation strategy based on gaze-contingent image processing with eye-tracking. Gaze-contingent processing is expected to reinforce natural-like visual scanning and reestablished spatial updating based on eye movements. The beneficial effects remain to be investigated for daily life activities in complex visual environments.Approach.The current study evaluates the benefits of gaze-contingent processing versus gaze-locked and gaze-ignored simulations in the context of mobility, scene recognition and visual search, using a virtual reality simulated prosthetic vision paradigm with sighted subjects.Main results.Compared to gaze-locked vision, gaze-contingent processing was consistently found to improve the speed in all experimental tasks, as well as the subjective quality of vision. Similar or further improvements were found in a control condition that ignores gaze-dependent effects, a simulation that is unattainable in the clinical reality.Significance.Our results suggest that gaze-locked vision and spatial updating conflicts can be debilitating for complex visually-guided activities of daily living such as mobility and orientation. Therefore, for prospective users of head-steered prostheses with an unimpaired oculomotor system, the inclusion of a compensatory eye-tracking system is strongly endorsed.

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

Reading text works better than watching videos to improve acuity in a simulation of artificial vision

Simulated artificial vision is used in visual prosthesis design to answer questions about device usability. We previously reported a striking increase in equivalent visual acuity with daily use of a simulation of artificial vision in an active task, reading sentences, that required high levels of subject engagement, but passive activities are more likely to dominate post-implant experience. Here, we investigated the longitudinal effects of a passive task, watching videos. Eight subjects used a simulation of a thalamic visual prosthesis with 1000 phosphenes to watch 23 episodes of classic American television in daily, 25-min sessions, for a period of 1 month with interspersed reading tests that quantified reading accuracy and reading speed. For reading accuracy, we found similar dynamics to the early part of the learning process in our previous report, here leading to an improvement in visual acuity of 0.15 ± 0.05 logMAR. For reading speed, however, no change was apparent by the end of training. We found that single reading sessions drove about twice the improvement in acuity of single video sessions despite being only half as long. We conclude that while passive viewing tasks may prove useful for post-implant rehabilitation, active tasks are likely to be preferable.

An Infrared Image-Enhancement Algorithm in Simulated Prosthetic Vision: Enlarging Working Environment of Future Retinal Prostheses

BACKGROUND

Most existing retinal prostheses contain a built-in visible-light camera module that captures images of the surrounding environment. Thus, in case of insufficient or lack of visible light, the camera fails to work, and the retinal prostheses enter a dormant or "OFF" state. A simple and effective solution is replacing the visible-light camera with a dual-mode camera. The present research aimed to achieve two main purposes: (1) to explore whether the dual-mode camera in prosthesis recipients works under no visible-light conditions and (2) to assess its performance.

METHODS

To accomplish these aims, we enrolled subjects in a psychophysical experiment under simulated prosthetic vision (SPV) conditions. We found that the subjects could complete some simple visual tasks, but the recognition performance under the infrared mode was significantly inferior to that under the visible-light mode. These results inspired us to develop and propose a feasible infrared image-enhancement processing algorithm. Another psychophysical experiment was performed to verify the feasibility of the algorithm.

RESULTS

The obtained results showed that the average efficiency of the subjects completing visual tasks using our enhancement algorithm (0.014±0.001) was significantly higher (p < 0.001) than that of subjects using direct pixelization (0.007±0.001).

CONCLUSIONS

We concluded that a dual-mode camera could be a feasible solution to improving the performance of retinal prostheses as the camera adapted better to the specific existing ambient light conditions. Dual-mode cameras combined with this infrared image-enhancement algorithm could provide a promising direction for the design of future retinal prostheses.

Vision Restoration by Optogenetic Therapy and Developments Toward Sonogenetic Therapy

After revolutionizing neuroscience, optogenetic therapy has entered successfully in clinical trials for restoring vision to blind people with degenerative eye diseases, such as retinitis pigmentosa. These clinical trials still have to evaluate the visual acuity achieved by patients and to determine if it reaches its theoretical limit extrapolated from ex vivo experiments. Different strategies are developed in parallel to reduce required light levels and improve information processing by targeting various cell types. For patients with vision loss due to optic atrophy, as in the case of glaucoma, optogenetic cortical stimulation is hampered by light absorption and scattering by the brain tissue. By contrast, ultrasound waves can diffuse widely through the dura mater and the brain tissue as indicated by ultrasound imaging. Based on our recent results in rodents, we propose the sonogenetic therapy relying on activation of the mechanosensitive channel as a very promising vision restoration strategy with a suitable spatiotemporal resolution. Genomic approaches may thus provide efficient brain machine interfaces for sight restoration.

Retinal prosthetic vision simulation: temporal aspects

Objective. The perception of individuals fitted with retinal prostheses is not fully understood, although several retinal implants have been tested and commercialized. Realistic simulations of perception with retinal implants would be useful for future development and evaluation of such systems.Approach.We implemented a retinal prosthetic vision simulation, including temporal features, which have not been previously simulated. In particular, the simulation included temporal aspects such as persistence and perceptual fading of phosphenes and the electrode activation rate.Main results.The simulated phosphene persistence showed an effective reduction in flickering at low electrode activation rates. Although persistence has a positive effect on static scenes, it smears dynamic scenes. Perceptual fading following continuous stimulation affects prosthetic vision of both static and dynamic scenes by making them disappear completely or partially. However, we showed that perceptual fading of a static stimulus might be countered by head-scanning motions, which together with the persistence revealed the contours of the faded object. We also showed that changing the image polarity may improve simulated prosthetic vision in the presence of persistence and perceptual fading.Significance.Temporal aspects have important roles in prosthetic vision, as illustrated by the simulations. Considering these aspects may improve the future design, the training with, and evaluation of retinal prostheses.

Full gaze contingency provides better reading performance than head steering alone in a simulation of prosthetic vision

The visual pathway is retinotopically organized and sensitive to gaze position, leading us to hypothesize that subjects using visual prostheses incorporating eye position would perform better on perceptual tasks than with devices that are merely head-steered. We had sighted subjects read sentences from the MNREAD corpus through a simulation of artificial vision under conditions of full gaze compensation, and head-steered viewing. With 2000 simulated phosphenes, subjects (n = 23) were immediately able to read under full gaze compensation and were assessed at an equivalent visual acuity of 1.0 logMAR, but were nearly unable to perform the task under head-steered viewing. At the largest font size tested, 1.4 logMAR, subjects read at 59 WPM (50% of normal speed) with 100% accuracy under the full-gaze condition, but at 0.7 WPM (under 1% of normal) with below 15% accuracy under head-steering. We conclude that gaze-compensated prostheses are likely to produce considerably better patient outcomes than those not incorporating eye movements.

A Three-Dimensional Microelectrode Array to Generate Virtual Electrodes for Epiretinal Prosthesis Based on a Modeling Study

Despite many advances in the development of retinal prostheses, clinical reports show that current retinal prosthesis subjects can only perceive prosthetic vision with poor visual acuity. A possible approach for improving visual acuity is to produce virtual electrodes (VEs) through electric field modulation. Generating controllable and localized VEs is a crucial factor in effectively improving the perceptive resolution of the retinal prostheses. In this paper, we aimed to design a microelectrode array (MEA) that can produce converged and controllable VEs by current steering stimulation strategies. Through computational modeling, we designed a three-dimensional concentric ring–disc MEA and evaluated its performance with different stimulation strategies. Our simulation results showed that electrode–retina distance (ERD) and inter-electrode distance (IED) can dramatically affect the distribution of electric field. Also the converged VEs could be produced when the parameters of the three-dimensional MEA were appropriately set. VE sites can be controlled by manipulating the proportion of current on each adjacent electrode in a current steering group (CSG). In addition, spatial localization of electrical stimulation can be greatly improved under quasi-monopolar (QMP) stimulation. This study may provide support for future application of VEs in epiretinal prosthesis for potentially increasing the visual acuity of prosthetic vision.

Behavioural responses to a photovoltaic subretinal prosthesis implanted in non-human primates

Retinal dystrophies and age-related macular degeneration related to photoreceptor degeneration can cause blindness. In blind patients, although the electrical activation of the residual retinal circuit can provide useful artificial visual perception, the resolutions of current retinal prostheses have been limited either by large electrodes or small numbers of pixels. Here we report the evaluation, in three awake non-human primates, of a previously reported near-infrared-light-sensitive photovoltaic subretinal prosthesis. We show that multipixel stimulation of the prosthesis within radiation safety limits enabled eye tracking in the animals, that they responded to stimulations directed at the implant with repeated saccades and that the implant-induced responses were present two years after device implantation. Our findings pave the way for the clinical evaluation of the prosthesis in patients affected by dry atrophic age-related macular degeneration.

Performance of complex visual tasks using simulated prosthetic vision via augmented-reality glasses

Photovoltaic subretinal prosthesis is designed for restoration of central vision in patients with age-related macular degeneration (AMD). We investigated the utility of prosthetic central vision for complex visual tasks using augmented-reality (AR) glasses simulating reduced acuity, contrast, and visual field. AR glasses with blocked central 20° of visual field included an integrated video camera and software which adjusts the image quality according to three user-defined parameters: resolution, corresponding to the equivalent pixel size of an implant; field of view, corresponding to the implant size; and number of grayscale levels. The real-time processed video was streamed on a screen in front of the right eye. Nineteen healthy participants were recruited to complete visual tasks including vision charts, sentence reading, and face recognition. With vision charts, letter acuity exceeded the pixel-sampling limit by 0.2 logMAR. Reading speed decreased with increasing pixel size and with reduced field of view (7°-12°). In the face recognition task (four-way forced choice, 5° angular size) participants identified faces at >75% accuracy, even with 100 μm pixels and only two grayscale levels. With 60 μm pixels and eight grayscale levels, the accuracy exceeded 97%. Subjects with simulated prosthetic vision performed slightly better than the sampling limit on the letter acuity tasks, and were highly accurate at recognizing faces, even with 100 μm/pixel resolution. These results indicate feasibility of reading and face recognition using prosthetic central vision even with 100 μm pixels, and performance improves further with smaller pixels.

Eye Movement Compensation and Spatial Updating in Visual Prosthetics: Mechanisms, Limitations and Future Directions

Despite appearing automatic and effortless, perceiving the visual world is a highly complex process that depends on intact visual and oculomotor function. Understanding the mechanisms underlying spatial updating (i.e., gaze contingency) represents an important, yet unresolved issue in the fields of visual perception and cognitive neuroscience. Many questions regarding the processes involved in updating visual information as a function of the movements of the eyes are still open for research. Beyond its importance for basic research, gaze contingency represents a challenge for visual prosthetics as well. While most artificial vision studies acknowledge its importance in providing accurate visual percepts to the blind implanted patients, the majority of the current devices do not compensate for gaze position. To-date, artificial percepts to the blind population have been provided either by intraocular light-sensing circuitry or by using external cameras. While the former commonly accounts for gaze shifts, the latter requires the use of eye-tracking or similar technology in order to deliver percepts based on gaze position. Inspired by the need to overcome the hurdle of gaze contingency in artificial vision, we aim to provide a thorough overview of the research addressing the neural underpinnings of eye compensation, as well as its relevance in visual prosthetics. The present review outlines what is currently known about the mechanisms underlying spatial updating and reviews the attempts of current visual prosthetic devices to overcome the hurdle of gaze contingency. We discuss the limitations of the current devices and highlight the need to use eye-tracking methodology in order to introduce gaze-contingent information to visual prosthetics.

Advances in retinal prosthesis systems

Retinal prosthesis systems have undergone significant advances in the past quarter century, resulting in the development of several different novel surgical and engineering approaches. Encouraging results have demonstrated partial visual restoration, with improvement in both coarse objective function and performance of everyday tasks. To date, four systems have received marketing approval for use in Europe or the United States, with numerous others undergoing preclinical and clinical evaluation, reflecting the established safety profile of these devices for chronic implantation. This progress represents the first notion that the field of visual restorative medicine could offer blind patients a hope of real and measurable benefit. However, there are numerous complex engineering and biophysical obstacles still to be overcome, to reconcile the gap that remains between artificial and natural vision. Current developments in the form of enhanced image processing algorithms and data transfer approaches, combined with emerging nanofabrication and conductive polymerization techniques, herald an exciting and innovative future for retinal prosthetics. This review provides an update of retinal prosthetic systems currently undergoing development and clinical trials while also addressing future challenges in the field, such as the assessment of functional outcomes in ultra-low vision and strategies for tackling existing hardware and software constraints.

Improvement in reading performance through training with simulated thalamic visual prostheses

Simulations of artificial vision are used to provide the researcher an opportunity to explore different aspects of visual prosthesis device design by observing subject performance on various tasks viewed through the simulation. Such studies typically use normal, sighted subjects to measure performance at a given point in time. Relatively few studies examine performance changes longitudinally to quantitatively assess the benefits from a training plan that would be akin to post-implantation rehabilitation. Here, we had six normal, sighted subjects use a standard reading task with daily practice over eight weeks to understand the effects of an intensive training schedule on adaptation to artificial sight. Subjects read 40 MNREAD-style sentences per session, with a new set each session, that were presented at five font sizes (logMAR 1.0–1.4) and through three center-weighted phosphene patterns (2,000, 1,000, 500 phosphenes). We found that subjects improved their reading accuracy across sessions, and that the training lead to an increase of reading speed that was equivalent to a doubling of available phosphenes. Most importantly, the hardest condition, while initially illegible, supported functional reading after training. Consistent with experience-driven neuroplastic changes, gaps in the training schedule lead to transient decreases in reading speed, but, surprisingly, not reading accuracy. Our findings contribute to our larger project of developing a thalamic visual prosthesis and to post-implant rehabilitation strategies.

Emerging therapies for inherited retinal degeneration

Inherited retinal degenerative diseases, a genetically and phenotypically heterogeneous group of disorders, affect the function of photoreceptor cells and are among the leading causes of blindness. Recent advances in molecular genetics and cell biology are elucidating the pathophysiological mechanisms underlying these disorders and are helping to identify new therapeutic approaches, such as gene therapy, stem cell therapy, and optogenetics. Several of these approaches have entered the clinical phase of development. Artificial replacement of dying photoreceptor cells using retinal prostheses has received regulatory approval. Precise retinal imaging and testing of visual function are facilitating more efficient clinical trial design. In individual patients, disease stage will determine whether the therapeutic strategy should comprise photoreceptor cell rescue to delay or arrest vision loss or retinal replacement for vision restoration.

An optimized content-aware image retargeting method: toward expanding the perceived visual field of the high-density retinal prosthesis recipients

OBJECTIVE

Retinal prosthesis devices have shown great value in restoring some sight for individuals with profoundly impaired vision, but the visual acuity and visual field provided by prostheses greatly limit recipients' visual experience. In this paper, we employ computer vision approaches to seek to expand the perceptible visual field in patients implanted potentially with a high-density retinal prosthesis while maintaining visual acuity as much as possible.

APPROACH

We propose an optimized content-aware image retargeting method, by introducing salient object detection based on color and intensity-difference contrast, aiming to remap important information of a scene into a small visual field and preserve their original scale as much as possible. It may improve prosthetic recipients' perceived visual field and aid in performing some visual tasks (e.g. object detection and object recognition). To verify our method, psychophysical experiments, detecting object number and recognizing objects, are conducted under simulated prosthetic vision. As control, we use three other image retargeting techniques, including Cropping, Scaling, and seam-assisted shrinkability.

MAIN RESULTS

Results show that our method outperforms in preserving more key features and has significantly higher recognition accuracy in comparison with other three image retargeting methods under the condition of small visual field and low-resolution.

SIGNIFICANCE

The proposed method is beneficial to expand the perceived visual field of prosthesis recipients and improve their object detection and recognition performance. It suggests that our method may provide an effective option for image processing module in future high-density retinal implants.

Spatiotemporal Pixelization to Increase the Recognition Score of Characters for Retinal Prostheses

Most of the retinal prostheses use a head-fixed camera and a video processing unit. Some studies proposed various image processing methods to improve visual perception for patients. However, previous studies only focused on using spatial information. The present study proposes a spatiotemporal pixelization method mimicking fixational eye movements to generate stimulation images for artificial retina arrays by combining spatial and temporal information. Input images were sampled with a resolution that was four times higher than the number of pixel arrays. We subsampled this image and generated four different phosphene images. We then evaluated the recognition scores of characters by sequentially presenting phosphene images with varying pixel array sizes (6 × 6, 8 × 8 and 10 × 10) and stimulus frame rates (10 Hz, 15 Hz, 20 Hz, 30 Hz, and 60 Hz). The proposed method showed the highest recognition score at a stimulus frame rate of approximately 20 Hz. The method also significantly improved the recognition score for complex characters. This method provides a new way to increase practical resolution over restricted spatial resolution by merging the higher resolution image into high-frame time slots.

Towards high-resolution retinal prostheses with direct optical addressing and inductive telemetry

OBJECTIVE

Despite considerable advances in retinal prostheses over the last two decades, the resolution of restored vision has remained severely limited, well below the 20/200 acuity threshold of blindness. Towards drastic improvements in spatial resolution, we present a scalable architecture for retinal prostheses in which each stimulation electrode is directly activated by incident light and powered by a common voltage pulse transferred over a single wireless inductive link.

APPROACH

The hybrid optical addressability and electronic powering scheme provides separate spatial and temporal control over stimulation, and further provides optoelectronic gain for substantially lower light intensity thresholds than other optically addressed retinal prostheses using passive microphotodiode arrays. The architecture permits the use of high-density electrode arrays with ultra-high photosensitive silicon nanowires, obviating the need for excessive wiring and high-throughput data telemetry. Instead, the single inductive link drives the entire array of electrodes through two wires and provides external control over waveform parameters for common voltage stimulation.

MAIN RESULTS

A complete system comprising inductive telemetry link, stimulation pulse demodulator, charge-balancing series capacitor, and nanowire-based electrode device is integrated and validated ex vivo on rat retina tissue.

SIGNIFICANCE

Measurements demonstrate control over retinal neural activity both by light and electrical bias, validating the feasibility of the proposed architecture and its system components as an important first step towards a high-resolution optically addressed retinal prosthesis.

Assistive peripheral phosphene arrays deliver advantages in obstacle avoidance in simulated end-stage retinitis pigmentosa: a virtual-reality study

OBJECTIVE

The prospective efficacy of peripheral retinal prostheses for guiding orientation and mobility in the absence of residual vision, as compared to an implant for the central visual field (VF), was evaluated using simulated prosthetic vision (SPV).

APPROACH

Sighted volunteers wearing a head-mounted display performed an obstacle circumvention task under SPV. Mobility and orientation performance with three layouts of prosthetic vision were compared: peripheral prosthetic vision of higher visual acuity (VA) but limited VF, of wider VF but limited VA, as well as centrally restricted prosthetic vision. Learning curves using these layouts were compared fitting an exponential model to the mobility and orientation measures.

MAIN RESULTS

Using peripheral layouts, performance was superior to the central layout. Walking speed with both higher-acuity and wider-angle layouts was 5.6% higher, and mobility errors reduced by 46.4% and 48.6%, respectively, as compared to the central layout. The wider-angle layout yielded the least number of collisions, 63% less than the higher-acuity and 73% less than the central layout. Using peripheral layouts, the number of visual-scanning related head movements was 54.3% (higher-acuity) and 60.7% (wider-angle) lower, as compared to the central layout, and the ratio of time standing versus time walking was 51.9% and 61.5% lower, respectively. Learning curves did not differ between layouts, except for time standing versus time walking, where both peripheral layouts achieved significantly lower asymptotic values compared to the central layout.

SIGNIFICANCE

Beyond complementing residual vision for an improved performance, peripheral prosthetic vision can effectively guide mobility in the later stages of retinitis pigmentosa (RP) without residual vision. Further, the temporal dynamics of learning peripheral and central prosthetic vision are similar. Therefore, development of a peripheral retinal prosthesis and early implantation to alleviate VF constriction in RP should be considered to extend the target group and the time of benefit for potential retinal prosthesis implantees.

Effect of Pixel’s Spatial Characteristics on Recognition of Isolated Pixelized Chinese Character

The influence of pixel’s spatial characteristics on recognition of isolated Chinese character was investigated using simulated prosthestic vision. The accuracy of Chinese character recognition with 4 kinds of pixel number (6*6, 8*8, 10*10, and 12*12 pixel array) and 3 kinds of pixel shape (Square, Dot and Gaussian) and different pixel spacing were tested through head-mounted display (HMD). A captured image of Chinese characters in font style of Hei were pixelized with Square, Dot and Gaussian pixel. Results showed that pixel number was the most important factor which could affect the recognition of isolated pixelized Chinese Chartars and the accuracy of recognition increased with the addition of pixel number. 10*10 pixel array could provide enough information for people to recognize an isolated Chinese character. At low resolution (6*6 and 8*8 pixel array), there were little difference of recognition accuracy between different pixel shape and different pixel spacing. While as for high resolution (10*10 and 12*12 pixel array), the fluctuation of pixel shape and pixel spacing could not affect the performance of recognition of isolated pixelized Chinese Character.

Synthetic 3D diamond-based electrodes for flexible retinal neuroprostheses: Model, production and in vivo biocompatibility

Two retinal implants have recently received the CE mark and one has obtained FDA approval for the restoration of useful vision in blind patients. Since the spatial resolution of current vision prostheses is not sufficient for most patients to detect faces or perform activities of daily living, more electrodes with less crosstalk are needed to transfer complex images to the retina. In this study, we modelled planar and three-dimensional (3D) implants with a distant ground or a ground grid, to demonstrate greater spatial resolution with 3D structures. Using such flexible 3D implant prototypes, we showed that the degenerated retina could mould itself to the inside of the wells, thereby isolating bipolar neurons for specific, independent stimulation. To investigate the in vivo biocompatibility of diamond as an electrode or an isolating material, we developed a procedure for depositing diamond onto flexible 3D retinal implants. Taking polyimide 3D implants as a reference, we compared the number of neurones integrating the 3D diamond structures and their ratio to the numbers of all cells, including glial cells. Bipolar neurones were increased whereas there was no increase even a decrease in the total cell number. SEM examinations of implants confirmed the stability of the diamond after its implantation in vivo. This study further demonstrates the potential of 3D designs for increasing the resolution of retinal implants and validates the safety of diamond materials for retinal implants and neuroprostheses in general.

Simulated prosthetic vision: improving text accessibility with retinal prostheses

Image processing can improve significantly the every-day life of blind people wearing current and upcoming retinal prostheses relying on an external camera. We propose to use a real-time text localization algorithm to improve text accessibility. An augmented text-specific rendering based on automatic text localization has been developed. It has been evaluated in comparison to the classical rendering through a Simulated Prosthetic Vision (SPV) experiment with 16 subjects. Subjects were able to detect text in natural scenes much faster and further with the augmented rendering compared to the control rendering. Our results show that current and next generation of low resolution retinal prostheses may benefit from real-time text detection algorithms.

The effect of normal aging and age-related macular degeneration on perceptual learning

We investigated whether perceptual learning could be used to improve peripheral word identification speed. The relationship between the magnitude of learning and age was established in normal participants to determine whether perceptual learning effects are age invariant. We then investigated whether training could lead to improvements in patients with age-related macular degeneration (AMD). Twenty-eight participants with normal vision and five participants with AMD trained on a word identification task. They were required to identify three-letter words, presented 10° from fixation. To standardize crowding across each of the letters that made up the word, words were flanked laterally by randomly chosen letters. Word identification performance was measured psychophysically using a staircase procedure. Significant improvements in peripheral word identification speed were demonstrated following training (71% ± 18%). Initial task performance was correlated with age, with older participants having poorer performance. However, older adults learned more rapidly such that, following training, they reached the same level of performance as their younger counterparts. As a function of number of trials completed, patients with AMD learned at an equivalent rate as age-matched participants with normal vision. Improvements in word identification speed were maintained at least 6 months after training. We have demonstrated that temporal aspects of word recognition can be improved in peripheral vision with training across a range of ages and these learned improvements are relatively enduring. However, training targeted at other bottlenecks to peripheral reading ability, such as visual crowding, may need to be incorporated to optimize this approach.

Simulation of thalamic prosthetic vision: reading accuracy, speed, and acuity in sighted humans

The psychophysics of reading with artificial sight has received increasing attention as visual prostheses are becoming a real possibility to restore useful function to the blind through the coarse, pseudo-pixelized vision they generate. Studies to date have focused on simulating retinal and cortical prostheses; here we extend that work to report on thalamic designs. This study examined the reading performance of normally sighted human subjects using a simulation of three thalamic visual prostheses that varied in phosphene count, to help understand the level of functional ability afforded by thalamic designs in a task of daily living. Reading accuracy, reading speed, and reading acuity of 20 subjects were measured as a function of letter size, using a task based on the MNREAD chart. Results showed that fluid reading was feasible with appropriate combinations of letter size and phosphene count, and performance degraded smoothly as font size was decreased, with an approximate doubling of phosphene count resulting in an increase of 0.2 logMAR in acuity. Results here were consistent with previous results from our laboratory. Results were also consistent with those from the literature, despite using naive subjects who were not trained on the simulator, in contrast to other reports.

Boron doped diamond biotechnology: from sensors to neurointerfaces

Boron doped nanocrystalline diamond is known as a remarkable material for the fabrication of sensors, taking advantage of its biocompatibility, electrochemical properties, and stability. Sensors can be fabricated to directly probe physiological species from biofluids (e.g. blood or urine), as will be presented. In collaboration with electrophysiologists and biologists, the technology was adapted to enable structured diamond devices such as microelectrode arrays (MEAs), i.e. common electrophysiology tools, to probe neuronal activity distributed over large populations of neurons or embryonic organs. Specific MEAs can also be used to build neural prostheses or implants to compensate function losses due to lesions or degeneration of parts of the central nervous system, such as retinal implants, which exhibit real promise as biocompatible neuroprostheses for in vivo neuronal stimulations. New electrode geometries enable high performance electrodes to surpass more conventional materials for such applications.

Retinal prostheses: clinical results and future challenges

Retinal prostheses aim at restoring visual perception in blind patients affected by retinal diseases leading to the loss of photoreceptors, such as age-related macular degeneration or retinitis pigmentosa. Recent clinical trials have demonstrated the feasibility of this approach for restoring useful vision. Despite a limited number of electrodes (60), and therefore of pixels, some patients were able to read words and to recognize high-contrast objects. Face recognition and independent locomotion in unknown urban environments imply technological breakthroughs to increase the number and density of electrodes. This review presents recent clinical results and discusses future solutions to answer the major technological challenges.

Vision rehabilitation in the case of blindness

This article examines the various vision rehabilitation procedures that are available for early and late blindness. Depending on the pathology involved, several vision rehabilitation procedures exist, or are in development. Visual aids are available for low vision individuals, as are sensory aids for blind persons. Most noninvasive sensory substitution prostheses as well as implanted visual prostheses in development are reviewed. Issues dealing with vision rehabilitation are also discussed, such as problems of biocompatibility, electrical safety, psychosocial aspects, and ethics. Basic studies devoted to vision rehabilitation such as simulation in mathematical models and simulation of artificial vision are also presented. Finally, the importance of accurate rehabilitation assessment is addressed, and tentative market figures are given.

[Restoring vision in blind patients following photoreceptor degeneration: clinical results and future challenges]

Retinal prostheses aim at restoring vision in patients blind from photoreceptor degeneration by electrically stimulating the residual retinal tissue. Currently, the most efficient implants are either inserted in the subretinal space or on the vitreal side of the retina (epi-retinal). Although the residual tissue can partly degenerate, it was shown that acute stimulation of residual neurones can induce visual percepts. Recently, a clinical trial with the epiretinal Argus2 device (60 electrodes) from the company 2nd Sight enabled most patients to orient and find light targets, some even reading words. This device has received a CE mark. Surprisingly, when the subretinal implant from the company Retina Implant AG displaying many more electrodes (1500 electrodes) was evaluated in clinical trials, the patient visual performances were fairly similar. The restored visual performances of the patients demonstrate that blind patients can recover some visual function when their residual retina is properly stimulated. However, the resolution is not yet sufficient to perform complex tasks such as autonomous locomotion, face identification or text reading. Several challenges remain to generate an increase in pixel density corresponding to the increase in electrode number and density. These challenges include the stimulation modality, the tissue/implant interface design, the electrode materials, and the visual information encoder. This review will discuss these great challenges after introducing the major clinical results.

Recognition of objects in simulated irregular phosphene maps for an epiretinal prosthesis

Visual prostheses offer a possibility of restoring vision to the blind. It is necessary to determine minimum requirements for daily visual tasks. To investigate the recognition of common objects in daily life based on the simulated irregular phosphene maps, the effect of four parameters (resolution, distortion, dropout percentage, and gray scale) on object recognition was investigated. The results showed that object recognition accuracy significantly increased with an increase of resolution. Distortion and dropout percentage had significant impact on the object recognition; with the increase of distortion level and dropout percentage, the recognition decreased considerably. The accuracy decreased significantly only at gray level 2, whereas the other three gray levels showed no obvious difference. The two image processing methods (merging pixels to lower the resolution and edge extraction before lowering resolution) showed significant difference on the object recognition when there was a high degree of distortion level or dot dropout.

Neural stimulation for visual rehabilitation: advances and challenges

Blindness affects tens of million people worldwide and its prevalence constantly increases along with population aging. In some pathologies leading to vision loss, prosthetic approaches are currently the only hope for the patient to recover some visual perception. Here, we review the latest advances in visual prosthetic strategies with their respective strength and weakness. The principle is to electrically stimulate neurons along the visual pathway. Ocular approaches target the remaining retinal cells whereas brain stimulation aims at stimulating higher visual structures directly. Even though ocular approaches are less invasive and easier to implement, brain stimulation can be applied to diseases where the connection between the retina and the brain is lost such as in glaucoma and could therefore benefit to patients with different pathologies. Today, numbers of groups are investigating these strategies and the first devices start being commercialized. However, critical bottlenecks still impair our scientific efforts towards efficient visual implants. These challenges include electrode miniaturization, material optimization, multiplexing of stimulation channels and encoding of visual information into electrical stimuli.

Accurate reading with sequential presentation of single letters

Rapid, accurate reading is possible when isolated, single words from a sentence are sequentially presented at a fixed spatial location. We investigated if reading of words and sentences is possible when single letters are rapidly presented at the fovea under user-controlled or automatically controlled rates. When tested with complete sentences, trained participants achieved reading rates of over 60 wpm and accuracies of over 90% with the single letter reading (SLR) method and naive participants achieved average reading rates over 30 wpm with greater than 90% accuracy. Accuracy declined as individual letters were presented for shorter periods of time, even when the overall reading rate was maintained by increasing the duration of spaces between words. Words in the lexicon that occur more frequently were identified with higher accuracy and more quickly, demonstrating that trained participants have lexical access. In combination, our data strongly suggest that comprehension is possible and that SLR is a practicable form of reading under conditions in which normal scanning of text is not possible, or for scenarios with limited spatial and temporal resolution such as patients with low vision or prostheses.

Retinal prostheses: current clinical results and future needs

Degenerative diseases such as age-related macular degeneration (AMD) and retinitis pigmentosa (RP) primarily affect the photoreceptors, ultimately resulting in significant loss of vision. Retinal prostheses aim to elicit neural activity in the remaining retinal cells by detecting and converting light into electrical stimuli that can then be delivered to the retina. The concept of visual prostheses has existed for more than 50 years and recent progress shows promise, yet much remains to be understood about how the visual system will respond to artificial input after years of blindness that necessitate this type of prosthesis. This review focuses on 3 major areas: the histopathologic features of human retina affected by AMD and RP, current results from clinical trials, and challenges to overcome for continued improvement of retinal prostheses.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosures may be found after the references.

Reading with a simulated 60-channel implant

First generation retinal prostheses containing 50–60 electrodes are currently in clinical trials. The purpose of this study was to evaluate the theoretical upper limit (best possible) reading performance attainable with a state-of-the-art 60-channel retinal implant and to find the optimum viewing conditions for the task. Four normal volunteers performed full-page text reading tasks with a low-resolution, 60-pixel viewing window that was stabilized in the central visual field. Two parameters were systematically varied: (1) spatial resolution (image magnification) and (2) the orientation of the rectangular viewing window. Performance was measured in terms of reading accuracy (% of correctly read words) and reading rates (words/min). Maximum reading performances were reached at spatial resolutions between 3.6 and 6 pixels/char. Performance declined outside this range for all subjects. In optimum viewing conditions (4.5 pixels/char), subjects achieved almost perfect reading accuracy and mean reading rates of 26 words/min for the vertical viewing window and of 34 words/min for the horizontal viewing window. These results suggest that, theoretically, some reading abilities can be restored with actual state-of-the-art retinal implant prototypes if “image magnification” is within an “optimum range.” Future retinal implants providing higher pixel resolutions, thus allowing for a wider visual span might allow faster reading rates.

Gaze-contingent simulation of retinopathy: some potential pitfalls and remedies

Many important results in visual neuroscience rely on the use of gaze-contingent retinal stabilization techniques. Our work focuses on the important fraction of these studies that is concerned with the retinal stabilization of visual filters that degrade some specific portions of the visual field. For instance, macular scotomas, often induced by age related macular degeneration, can be simulated by continuously displaying a gaze-contingent mask in the center of the visual field. The gaze-contingent rules used in most of these studies imply only a very minimal processing of ocular data. By analyzing the relationship between gaze and scotoma locations for different oculo-motor patterns, we show that such a minimal processing might have adverse perceptual and oculomotor consequences due mainly to two potential problems: (a) a transient blink-induced motion of the scotoma while gaze is static, and (b) the intrusion of post-saccadic slow eye movements. We have developed new gaze-contingent rules to solve these two problems. We have also suggested simple ways of tackling two unrecognized problems that are a potential source of mismatch between gaze and scotoma locations. Overall, the present work should help design, describe and test the paradigms used to simulate retinopathy with gaze-contingent displays.

Simulating prosthetic vision: Optimizing the information content of a limited visual display

Visual prostheses for the restoration of functional vision are currently under development. To guide prosthesis research and allow for an accurate prognosis of functional gain, simulating the experience of a retinal prosthesis in healthy individuals is desirable. Current simulation paradigms lack crucial aspects of the prosthetic experience such as realistic head- and eye-position-dependent image presentation. We developed a simulation paradigm that used a head-mounted camera and eye tracker to lock the simulation to the point of fixation. We evaluated visual acuity, object recognition and manipulation, and wayfinding under simulated prosthetic vision. We explored three ways of optimizing the information content of the prosthetic visual image: Full-Field representation (wide visual angle, low sampling frequency), Region of Interest (ROI; narrow visible angle, high sampling frequency), and Fisheye (high sampling frequency in the center, progressively lower resolution toward the edges). Full-Field representation facilitated visual search and navigation, whereas ROI improved visual acuity. The Fisheye representation, designed to incorporate the benefits of both Full-Field representation and ROI, performed similarly to ROI with subjects unable to capitalize on the peripheral data. The observation that different image representation conditions prove advantageous for different tasks should be taken into account in the process of designing and testing new visual prosthesis prototypes.

Estimation of phosphene spatial variability for visual prosthesis applications

Visual prostheses are the focus of intensive research efforts to restore some measure of useful vision to blind or near-blind patients. The development of such technology is being guided to an extent by tools that simulate prosthesis behavior for healthy sighted subjects in order to assess system requirements and configurations. These simulators, however, typically assume purely deterministic phosphene properties and thus do not apply any variability to phosphene size, intensity, or location. We address this issue by presenting data on phosphene variability measured in a blind human subject fitted with an optic nerve prosthesis. In order to correct for normal limitations in human-pointing accuracy, the experimental conditions were repeated with sighted subjects. We conclude that identical optic nerve stimulations can result in phosphenes whose perceived locations vary by up to 5 degrees of deviation angle and 10 degrees of position angle. The consistency of phosphenes presented in the peripheral field of view can vary by an additional 3 degrees.