Simulation of Artificial Vision, III: Do the Spatial or Temporal Characteristics of Stimulus Pixelization Really Matter?

Chronic stability of a neuroprosthesis comprising multiple adjacent Utah arrays in monkeys

Objective. Electrical stimulation of visual cortex via a neuroprosthesis induces the perception of dots of light ('phosphenes'), potentially allowing recognition of simple shapes even after decades of blindness. However, restoration of functional vision requires large numbers of electrodes, and chronic, clinical implantation of intracortical electrodes in the visual cortex has only been achieved using devices of up to 96 channels. We evaluated the efficacy and stability of a 1024-channel neuroprosthesis system in non-human primates (NHPs) over more than 3 years to assess its suitability for long-term vision restoration.Approach.We implanted 16 microelectrode arrays (Utah arrays) consisting of 8 × 8 electrodes with iridium oxide tips in the primary visual cortex (V1) and visual area 4 (V4) of two sighted macaques. We monitored the animals' health and measured electrode impedances and neuronal signal quality by calculating signal-to-noise ratios of visually driven neuronal activity, peak-to-peak voltages of the waveforms of action potentials, and the number of channels with high-amplitude signals. We delivered cortical microstimulation and determined the minimum current that could be perceived, monitoring the number of channels that successfully yielded phosphenes. We also examined the influence of the implant on a visual task after 2-3 years of implantation and determined the integrity of the brain tissue with a histological analysis 3-3.5 years post-implantation.Main results. The monkeys remained healthy throughout the implantation period and the device retained its mechanical integrity and electrical conductivity. However, we observed decreasing signal quality with time, declining numbers of phosphene-evoking electrodes, decreases in electrode impedances, and impaired performance on a visual task at visual field locations corresponding to implanted cortical regions. Current thresholds increased with time in one of the two animals. The histological analysis revealed encapsulation of arrays and cortical degeneration. Scanning electron microscopy on one array revealed degradation of IrOxcoating and higher impedances for electrodes with broken tips.Significance. Long-term implantation of a high-channel-count device in NHP visual cortex was accompanied by deformation of cortical tissue and decreased stimulation efficacy and signal quality over time. We conclude that improvements in device biocompatibility and/or refinement of implantation techniques are needed before future clinical use is feasible.

Intracortical current steering shifts the location of evoked neural activity

Objective.Intracortical visual prostheses are being developed to restore sight in people who are blind. The resolution of artificial vision is dictated by the location, proximity and number of electrodes implanted in the brain. However, increasing electrode count and proximity is traded off against tissue damage. Hence, new stimulation methods are needed that can improve the resolution of artificial vision without increasing the number of electrodes. We investigated whether a technique known as current steering can improve the resolution of artificial vision provided by intracortical prostheses without increasing the number of physical electrodes in the brain.Approach.We explored how the locus of neuronal activation could be steered when low amplitude microstimulation was applied simultaneously to two intracortical electrodes. A 64-channel, four-shank electrode array was implanted into the visual cortex of rats (n= 7). The distribution of charge ranged from single-electrode stimulation (100%:0%) to an equal distribution between the two electrodes (50%:50%), thereby steering the current between the physical electrodes. The stimulating electrode separation varied between 300 and 500μm. The peak of the evoked activity was defined as the 'virtual electrode' location.Main results.Current steering systematically shifted the virtual electrode on average between the stimulating electrodes as the distribution of charge was moved from one stimulating electrode to another. This effect was unclear in single trials due to the limited sampling of neurons. A model that scales the cortical response to each physical electrode when stimulated in isolation predicts the evoked virtual electrode response. Virtual electrodes were found to elicit a neural response as effectively and predictably as physical electrodes within cortical tissue on average.Significance.Current steering could be used to increase the resolution of intracortical electrode arrays without altering the number of physical electrodes which will reduce neural tissue damage, power consumption and potential heat dispersion issues.

Improving the spatial resolution of artificial vision using midget retinal ganglion cell populations modelled at the human fovea

OBJECTIVE

Retinal prostheses seek to create artificial vision by stimulating surviving retinal neurons of patients with profound vision impairment. Notwithstanding tremendous research efforts, the performance of all implants tested to date has remained rudimentary, incapable of overcoming the threshold for legal blindness. To maximize the perceptual efficacy of retinal prostheses, a device must be capable of controlling retinal neurons with greater spatiotemporal precision. Most studies of retinal stimulation were derived from either non-primate species or the peripheral primate retina. We investigated if artificial stimulation could leverage the high spatial resolution afforded by the neural substrates at the primate fovea and surrounding regions to achieve improved percept qualities.

APPROACH

We began by developing a new computational model capable of generating anatomically accurate retinal ganglion cell (RGC) populations within the human central retina. Next, multiple RGC populations across the central retina were stimulated in-silico to compare clinical and recently proposed neurostimulation configurations based on their ability to improve perceptual efficacy and reduce activation thresholds.

MAIN RESULTS

Our model uniquely upholds eccentricity-dependent characteristics such as RGC density and dendritic field diameter, whilst incorporating anatomically accurate features such as axon projection and three-dimensional RGC layering, features often forgone in favor of reduced computational complexity. Following epiretinal stimulation, the RGCs in our model produced response patterns in shapes akin to the complex percepts reported in clinical trials. Our results also demonstrated that even within the neuron-dense central retina, epiretinal stimulation using a multi-return hexapolar electrode arrangement could reliably achieve spatially focused RGC activation and could achieve single-cell excitation in 74% of all tested locations.

SIGNIFICANCE

This study establishes an anatomically accurate three-dimensional model of the human central retina and demonstrates the potential for an epiretinal hexapolar configuration to achieve consistent, spatially confined retinal responses, even within the neuron-dense foveal region. Our results promote the prospect and optimization of higher spatial resolution in future epiretinal implants.

Simulating the perceptual effects of electrode-retina distance in prosthetic vision

OBJECTIVE

Retinal prostheses aim to restore some vision in retinitis pigmentosa and age-related macular degeneration blind patients. Many spatial and temporal aspects have been found to affect prosthetic vision. Our objective is to study the impact of the space-variant distance between the stimulating electrodes and the surface of the retina on prosthetic vision and how to mitigate this impact.

APPROACH

A prosthetic vision simulation was built to demonstrate the perceptual effects of the electrode-retina distance (ERD) with different random spatial variations, such as size, brightness, shape, dropout, and spatial shifts. Three approaches for reducing the ERD effects are demonstrated: electrode grouping (quads), ERD-based input-image enhancement, and object scanning with and without phosphene persistence. A quantitative assessment for the first two approaches was done based on experiments with 20 subjects and three vision-based computational image similarity metrics.

MAIN RESULTS

The effects of various ERDs on phosphenes' size, brightness, and shape were simulated. Quads, chosen according to the ERDs, effectively elicit phosphenes without exceeding the safe charge density limit, whereas single electrodes with large ERD cannot do so. Input-image enhancement reduced the ERD effects effectively. These two approaches significantly improved ERD-affected prosthetic vision according to the experiment and image similarity metrics. A further reduction of the ERD effects was achieved by scanning an object while moving the head.

SIGNIFICANCE

ERD has multiple effects on perception with retinal prostheses. One of them is vision loss caused by the incapability of electrodes with large ERD to evoke phosphenes. The three approaches presented in this study can be used separately or together to mitigate the impact of ERD. A consideration of our approaches in reducing the perceptual effects of the ERD may help improve the perception with current prosthetic technology and influence the design of future prostheses.

Neurophysiological considerations for visual implants

Neural implants have the potential to restore visual capabilities in blind individuals by electrically stimulating the neurons of the visual system. This stimulation can produce visual percepts known as phosphenes. The ideal location of electrical stimulation for achieving vision restoration is widely debated and dependent on the physiological properties of the targeted tissue. Here, the neurophysiology of several potential target structures within the visual system will be explored regarding their benefits and downfalls in producing phosphenes. These regions will include the lateral geniculate nucleus, primary visual cortex, visual area 2, visual area 3, visual area 4 and the middle temporal area. Based on the existing engineering limitations of neural prostheses, we anticipate that electrical stimulation of any singular brain region will be incapable of achieving high-resolution naturalistic perception including color, texture, shape and motion. As improvements in visual acuity facilitate improvements in quality of life, emulating naturalistic vision should be one of the ultimate goals of visual prostheses. To achieve this goal, we propose that multiple brain areas will need to be targeted in unison enabling different aspects of vision to be recreated.

Perceptual Learning of Object Recognition in Simulated Retinal Implant Perception - The Effect of Video Training

Purpose

Retinal implants (RIs) provide new vision for patients suffering from photoreceptor degeneration in the retina. The limited vision gained by RI, however, leaves room for improvement by training regimes.

Methods

Two groups of normal-sighted participants were respectively trained with videos or still images of daily objects in a labeling task. Object appearance was simulated to resemble RI perception. In Experiment 1, the training effect was measured as the change in performance during the training, and the same labeling task was conducted after 1 week to test the retention. In Experiment 2 with a different pool of participants, a reverse labeling task was included before (pre-test) and after the training (post-test) to show if the training effect could be generalized into a different task context.

Results

Both groups showed improved object recognition through training that was maintained for a week, and the video group showed better improvement (Experiment 1). Both groups showed improved object recognition in a different task that was maintained for a week, but the video group did not show better retention than the image group (Experiment 2).

Conclusions

Training with video materials leads to more improvement than training with still images in simulated RI perception, but this better improvement was specific to the trained task.

Translational Relevance

We recommend videos as better training materials than still images for patients with RIs to improve object recognition when the task-goal is highly specific. We also propose here that achieving highly specific training goals runs the risk of limiting the generalization of the training effects.

Retinal prosthetic vision simulation: temporal aspects

Objective. The perception of individuals fitted with retinal prostheses is not fully understood, although several retinal implants have been tested and commercialized. Realistic simulations of perception with retinal implants would be useful for future development and evaluation of such systems.Approach.We implemented a retinal prosthetic vision simulation, including temporal features, which have not been previously simulated. In particular, the simulation included temporal aspects such as persistence and perceptual fading of phosphenes and the electrode activation rate.Main results.The simulated phosphene persistence showed an effective reduction in flickering at low electrode activation rates. Although persistence has a positive effect on static scenes, it smears dynamic scenes. Perceptual fading following continuous stimulation affects prosthetic vision of both static and dynamic scenes by making them disappear completely or partially. However, we showed that perceptual fading of a static stimulus might be countered by head-scanning motions, which together with the persistence revealed the contours of the faded object. We also showed that changing the image polarity may improve simulated prosthetic vision in the presence of persistence and perceptual fading.Significance.Temporal aspects have important roles in prosthetic vision, as illustrated by the simulations. Considering these aspects may improve the future design, the training with, and evaluation of retinal prostheses.

Monitoring Cortical Response and Electrode-Retina Impedance Under Epiretinal Stimulation in Rats

Retinal prosthesis can restore partial vision in patients with retinal degenerative diseases such as retinitis pigmentosa and age-related macular degeneration. Epiretinal prosthesis is one of three therapeutic approaches, which received regulatory approval several years ago. The thresholds of an epiretinal stimulation is partly determined by the size of the physical gap between the electrode and the retina after implantation. Precise positioning of epiretinal stimulating electrode array is still a challenging task. In this study, we demonstrate an approach to positioning epiretinal prostheses for an optimal response at the cortical output by monitoring both the impedance at the electrode-retina interface and the evoked-potential at the cortical level. We implanted a single-channel electrode on the epiretinal surface in adult rats, acutely, guided by both the impedance at the electrode-retina interface and by electrically evoked potentials (EEPs) in the visual cortex during retinal stimulation. We observe that impedance monotonously increases with decreasing electrode-retina distance, but that the strongest cortical responses were achieved at intermediate impedance levels. When the electrode penetrates the retina, the impedance keeps increasing. The effect of stimulation on the retina changes from epiretinal paradigm to intra-retinal paradigm and a decrease in cortical activation is observed. It is found that high impedance is not always favorable to elicit best cortical responses. Histopathological results showed that the electrode was placed at the intra-retinal space at high impedance value. These results show that monitoring impedance at the electrode-retina interface is necessary but not sufficient in obtaining strong evoked-potentials at the cortical level. Monitoring the cortical EEPs together with the impedance can improve the safety of implantation as well as efficacy of stimulation in the next generation of retinal implants.

Influence of field of view in visual prostheses design: Analysis with a VR system

OBJECTIVE

Visual prostheses are designed to restore partial functional vision in patients with total vision loss. Retinal visual prostheses provide limited capabilities as a result of low resolution, limited field of view and poor dynamic range. Understanding the influence of these parameters in the perception results can guide prostheses research and design.

APPROACH

In this work, we evaluate the influence of field of view with respect to spatial resolution in visual prostheses, measuring the accuracy and response time in a search and recognition task. Twenty-four normally sighted participants were asked to find and recognize usual objects, such as furniture and home appliance in indoor room scenes. For the experiment, we use a new simulated prosthetic vision system that allows simple and effective experimentation. Our system uses a virtual-reality environment based on panoramic scenes. The simulator employs a head-mounted display which allows users to feel immersed in the scene by perceiving the entire scene all around. Our experiments use public image datasets and a commercial head-mounted display. We have also released the virtual-reality software for replicating and extending the experimentation.

MAIN RESULTS

Results show that the accuracy and response time decrease when the field of view is increased. Furthermore, performance appears to be correlated with the angular resolution, but showing a diminishing return even with a resolution of less than 2.3 phosphenes per degree.

SIGNIFICANCE

Our results seem to indicate that, for the design of retinal prostheses, it is better to concentrate the phosphenes in a small area, to maximize the angular resolution, even if that implies sacrificing field of view.

Semantic and structural image segmentation for prosthetic vision

Prosthetic vision is being applied to partially recover the retinal stimulation of visually impaired people. However, the phosphenic images produced by the implants have very limited information bandwidth due to the poor resolution and lack of color or contrast. The ability of object recognition and scene understanding in real environments is severely restricted for prosthetic users. Computer vision can play a key role to overcome the limitations and to optimize the visual information in the prosthetic vision, improving the amount of information that is presented. We present a new approach to build a schematic representation of indoor environments for simulated phosphene images. The proposed method combines a variety of convolutional neural networks for extracting and conveying relevant information about the scene such as structural informative edges of the environment and silhouettes of segmented objects. Experiments were conducted with normal sighted subjects with a Simulated Prosthetic Vision system. The results show good accuracy for object recognition and room identification tasks for indoor scenes using the proposed approach, compared to other image processing methods.

Active photonic sensing for super-resolved reading performance in simulated prosthetic vision

In this work, we study the enhancement of simulated prosthetic reading performance through "active photonic sensing" in normally sighted subjects. Three sensing paradigms were implemented: active sensing, in which the subject actively scanned the presented words using the computer mouse, with an option to control text size; passive scanning produced by software-initiated horizontal movements of words; and no scanning. Our findings reveal a 30% increase in word recognition rate with active scanning as compared to no or passive scanning and up to 14-fold increase with zooming. These results highlight the importance of a patient interactive interface and shed light on techniques that can greatly enhance prosthetic vision quality.

Design and validation of a foldable and photovoltaic wide-field epiretinal prosthesis

Retinal prostheses have been developed to fight blindness in people affected by outer retinal layer dystrophies. To date, few hundred patients have received a retinal implant. Inspired by intraocular lenses, we have designed a foldable and photovoltaic wide-field epiretinal prosthesis (named POLYRETINA) capable of stimulating wireless retinal ganglion cells. Here we show that within a visual angle of 46.3 degrees, POLYRETINA embeds 2215 stimulating pixels, of which 967 are in the central area of 5 mm, it is foldable to allow implantation through a small scleral incision, and it has a hemispherical shape to match the curvature of the eye. We demonstrate that it is not cytotoxic and respects optical and thermal safety standards; accelerated ageing shows a lifetime of at least 2 years. POLYRETINA represents significant progress towards the improvement of both visual acuity and visual field with the same device, a current challenging issue in the field.

Spatiotemporal Pixelization to Increase the Recognition Score of Characters for Retinal Prostheses

Most of the retinal prostheses use a head-fixed camera and a video processing unit. Some studies proposed various image processing methods to improve visual perception for patients. However, previous studies only focused on using spatial information. The present study proposes a spatiotemporal pixelization method mimicking fixational eye movements to generate stimulation images for artificial retina arrays by combining spatial and temporal information. Input images were sampled with a resolution that was four times higher than the number of pixel arrays. We subsampled this image and generated four different phosphene images. We then evaluated the recognition scores of characters by sequentially presenting phosphene images with varying pixel array sizes (6 × 6, 8 × 8 and 10 × 10) and stimulus frame rates (10 Hz, 15 Hz, 20 Hz, 30 Hz, and 60 Hz). The proposed method showed the highest recognition score at a stimulus frame rate of approximately 20 Hz. The method also significantly improved the recognition score for complex characters. This method provides a new way to increase practical resolution over restricted spatial resolution by merging the higher resolution image into high-frame time slots.

Effect of Pixel’s Spatial Characteristics on Recognition of Isolated Pixelized Chinese Character

The influence of pixel’s spatial characteristics on recognition of isolated Chinese character was investigated using simulated prosthestic vision. The accuracy of Chinese character recognition with 4 kinds of pixel number (6*6, 8*8, 10*10, and 12*12 pixel array) and 3 kinds of pixel shape (Square, Dot and Gaussian) and different pixel spacing were tested through head-mounted display (HMD). A captured image of Chinese characters in font style of Hei were pixelized with Square, Dot and Gaussian pixel. Results showed that pixel number was the most important factor which could affect the recognition of isolated pixelized Chinese Chartars and the accuracy of recognition increased with the addition of pixel number. 10*10 pixel array could provide enough information for people to recognize an isolated Chinese character. At low resolution (6*6 and 8*8 pixel array), there were little difference of recognition accuracy between different pixel shape and different pixel spacing. While as for high resolution (10*10 and 12*12 pixel array), the fluctuation of pixel shape and pixel spacing could not affect the performance of recognition of isolated pixelized Chinese Character.

Electrical Characterization of 3D Au Microelectrodes for Use in Retinal Prostheses

In order to provide high-quality visual information to patients who have implanted retinal prosthetic devices, the number of microelectrodes should be large. As the number of microelectrodes is increased, the dimensions of each microelectrode must be decreased, which in turn results in an increased microelectrode interface impedance and decreased injection current dynamic range. In order to improve the trade-off envelope between the number of microelectrodes and the current injection characteristics, a 3D microelectrode structure can be used as an alternative. In this paper, the electrical characteristics of 2D and 3D Au microelectrodes were investigated. In order to examine the effects of the structural difference, 2D and 3D Au microelectrodes with different base areas but similar effective surface areas were fabricated and evaluated. Interface impedances were measured and similar dynamic ranges were obtained for both 2D and 3D Au microelectrodes. These results indicate that more electrodes can be implemented in the same area if 3D designs are used. Furthermore, the 3D Au microelectrodes showed substantially enhanced electrical durability characteristics against over-injected stimulation currents, withstanding electrical currents that are much larger than the limit measured for 2D microelectrodes of similar area. This enhanced electrical durability property of 3D Au microelectrodes is a new finding in microelectrode research, and makes 3D microelectrodes very desirable devices.

Simulated prosthetic vision: improving text accessibility with retinal prostheses

Image processing can improve significantly the every-day life of blind people wearing current and upcoming retinal prostheses relying on an external camera. We propose to use a real-time text localization algorithm to improve text accessibility. An augmented text-specific rendering based on automatic text localization has been developed. It has been evaluated in comparison to the classical rendering through a Simulated Prosthetic Vision (SPV) experiment with 16 subjects. Subjects were able to detect text in natural scenes much faster and further with the augmented rendering compared to the control rendering. Our results show that current and next generation of low resolution retinal prostheses may benefit from real-time text detection algorithms.

Simulation of thalamic prosthetic vision: reading accuracy, speed, and acuity in sighted humans

The psychophysics of reading with artificial sight has received increasing attention as visual prostheses are becoming a real possibility to restore useful function to the blind through the coarse, pseudo-pixelized vision they generate. Studies to date have focused on simulating retinal and cortical prostheses; here we extend that work to report on thalamic designs. This study examined the reading performance of normally sighted human subjects using a simulation of three thalamic visual prostheses that varied in phosphene count, to help understand the level of functional ability afforded by thalamic designs in a task of daily living. Reading accuracy, reading speed, and reading acuity of 20 subjects were measured as a function of letter size, using a task based on the MNREAD chart. Results showed that fluid reading was feasible with appropriate combinations of letter size and phosphene count, and performance degraded smoothly as font size was decreased, with an approximate doubling of phosphene count resulting in an increase of 0.2 logMAR in acuity. Results here were consistent with previous results from our laboratory. Results were also consistent with those from the literature, despite using naive subjects who were not trained on the simulator, in contrast to other reports.

Face recognition in simulated prosthetic vision: face detection-based image processing strategies

OBJECTIVE

Given the limited visual percepts elicited by current prosthetic devices, it is essential to optimize image content in order to assist implant wearers to achieve better performance of visual tasks. This study focuses on recognition of familiar faces using simulated prosthetic vision.

APPROACH

Combined with region-of-interest (ROI) magnification, three face extraction strategies based on a face detection technique were used: the Viola-Jones face region, the statistical face region (SFR) and the matting face region.

MAIN RESULTS

These strategies significantly enhanced recognition performance compared to directly lowering resolution (DLR) with Gaussian dots. The inclusion of certain external features, such as hairstyle, was beneficial for face recognition. Given the high recognition accuracy achieved and applicable processing speed, SFR-ROI was the preferred strategy. DLR processing resulted in significant face gender recognition differences (i.e. females were more easily recognized than males), but these differences were not apparent with other strategies.

SIGNIFICANCE

Face detection-based image processing strategies improved visual perception by highlighting useful information. Their use is advisable for face recognition when using low-resolution prosthetic vision. These results provide information for the continued design of image processing modules for use in visual prosthetics, thus maximizing the benefits for future prosthesis wearers.

Recognition of objects in simulated irregular phosphene maps for an epiretinal prosthesis

Visual prostheses offer a possibility of restoring vision to the blind. It is necessary to determine minimum requirements for daily visual tasks. To investigate the recognition of common objects in daily life based on the simulated irregular phosphene maps, the effect of four parameters (resolution, distortion, dropout percentage, and gray scale) on object recognition was investigated. The results showed that object recognition accuracy significantly increased with an increase of resolution. Distortion and dropout percentage had significant impact on the object recognition; with the increase of distortion level and dropout percentage, the recognition decreased considerably. The accuracy decreased significantly only at gray level 2, whereas the other three gray levels showed no obvious difference. The two image processing methods (merging pixels to lower the resolution and edge extraction before lowering resolution) showed significant difference on the object recognition when there was a high degree of distortion level or dot dropout.

Visual acuity of simulated thalamic visual prostheses in normally sighted humans

Simulation in normally sighted individuals is a crucial tool to evaluate the performance of potential visual prosthesis designs prior to human implantation of a device. Here, we investigated the effects of electrode count on visual acuity, learning rate and response time in 16 normally sighted subjects using a simulated thalamic visual prosthesis, providing the first performance reports for thalamic designs. A new letter recognition paradigm using a multiple-optotype two-alternative forced choice task was adapted from the Snellen eye chart, and specifically devised to be readily communicated to both human and non-human primate subjects. Validation of the method against a standard Snellen acuity test in 21 human subjects showed no significant differences between the two tests. The novel task was then used to address three questions about simulations of the center-weighted phosphene patterns typical of thalamic designs: What are the expected Snellen acuities for devices with varying numbers of contacts, do subjects display rapid adaptation to the new visual modality, and can response time in the task provide clues to the mechanisms of perception in low-resolution artificial vision? Population performance (hit rate) was significantly above chance when viewing Snellen 20/200 optotypes (Log MAR 1.0) with 370 phosphenes in the central 10 degrees of vision, ranging to Snellen 20/800 (Log MAR 1.6) with 25 central phosphenes. Furthermore, subjects demonstrated learning within the 1–2 hours of task experience indicating the potential for an effective rehabilitation and possibly better visual performance after a longer period of training. Response time differences suggest that direct letter perception occurred when hit rate was above 75%, whereas a slower strategy like feature-based pattern matching was used in conditions of lower relative resolution. As pattern matching can substantially boost effective acuity, these results suggest post-implant therapy should specifically address feature detection skills.

Electrical characteristics of 2D and 3D microelectrodes for high-resolution retinal prostheses

In order to provide high quality visual information to patients who have implanted retinal prosthetic devices, the number of microelectrode should be large. As the number of microelectrode is increased, the dimension of the microelectrode is decreased, which in turn results in the increased interface impedance of microelectrode and decreased dynamic range of injection current. In addition, the reduced maximum limit of injection current may not be sufficiently large to stimulate the ganglion cells in a retina. In order to improve the trade-off envelope between number of microelectrode and current injection limit, a 3D microelectrode structure can be used as an alternative. From the advancement of microfabrication technology, the fabrication of highly-accurate 3D structures with small dimensions is possible. This paper presents a first comprehensive electrical characterization of 2D and 3D microelectrodes for high-resolution retinal prostheses. Microelectrodes which differ in shapes and diameters are analyzed. Their interface impedances and charge injection limits are quantitatively analyzed. This research can be used to define requirements for further retinal prosthetic device research.

Estimation of simulated phosphene size based on tactile perception

Clinical trials have successfully shown that a visual prosthesis can elicit visual perception (phosphenes) in the visual field. Psychophysical studies based on simulated prosthetic vision offer an effective means to evaluate and refine prosthetic vision. We designed three experiments to examine the effect of phosphene luminance, flicker rate, and eccentricity on the ability to estimate simulated phosphene sizes using tactile perception. Thirty subjects participated in the three experiments. There was a linear increase in reported size as visual stimulus size increased. Judgment was significantly affected by stimulus luminance and eccentricity (P < 0.05) but not by flicker rates. Brighter stimuli were perceived as being larger, and the more eccentric the position, the larger the estimated size. These simulation studies, although idealized, suggested that tactile perception is a potential way to estimate phosphene sizes.

Reading with a simulated 60-channel implant

First generation retinal prostheses containing 50–60 electrodes are currently in clinical trials. The purpose of this study was to evaluate the theoretical upper limit (best possible) reading performance attainable with a state-of-the-art 60-channel retinal implant and to find the optimum viewing conditions for the task. Four normal volunteers performed full-page text reading tasks with a low-resolution, 60-pixel viewing window that was stabilized in the central visual field. Two parameters were systematically varied: (1) spatial resolution (image magnification) and (2) the orientation of the rectangular viewing window. Performance was measured in terms of reading accuracy (% of correctly read words) and reading rates (words/min). Maximum reading performances were reached at spatial resolutions between 3.6 and 6 pixels/char. Performance declined outside this range for all subjects. In optimum viewing conditions (4.5 pixels/char), subjects achieved almost perfect reading accuracy and mean reading rates of 26 words/min for the vertical viewing window and of 34 words/min for the horizontal viewing window. These results suggest that, theoretically, some reading abilities can be restored with actual state-of-the-art retinal implant prototypes if “image magnification” is within an “optimum range.” Future retinal implants providing higher pixel resolutions, thus allowing for a wider visual span might allow faster reading rates.

Gaze-contingent simulation of retinopathy: some potential pitfalls and remedies

Many important results in visual neuroscience rely on the use of gaze-contingent retinal stabilization techniques. Our work focuses on the important fraction of these studies that is concerned with the retinal stabilization of visual filters that degrade some specific portions of the visual field. For instance, macular scotomas, often induced by age related macular degeneration, can be simulated by continuously displaying a gaze-contingent mask in the center of the visual field. The gaze-contingent rules used in most of these studies imply only a very minimal processing of ocular data. By analyzing the relationship between gaze and scotoma locations for different oculo-motor patterns, we show that such a minimal processing might have adverse perceptual and oculomotor consequences due mainly to two potential problems: (a) a transient blink-induced motion of the scotoma while gaze is static, and (b) the intrusion of post-saccadic slow eye movements. We have developed new gaze-contingent rules to solve these two problems. We have also suggested simple ways of tackling two unrecognized problems that are a potential source of mismatch between gaze and scotoma locations. Overall, the present work should help design, describe and test the paradigms used to simulate retinopathy with gaze-contingent displays.

Simulating prosthetic vision: Optimizing the information content of a limited visual display

Visual prostheses for the restoration of functional vision are currently under development. To guide prosthesis research and allow for an accurate prognosis of functional gain, simulating the experience of a retinal prosthesis in healthy individuals is desirable. Current simulation paradigms lack crucial aspects of the prosthetic experience such as realistic head- and eye-position-dependent image presentation. We developed a simulation paradigm that used a head-mounted camera and eye tracker to lock the simulation to the point of fixation. We evaluated visual acuity, object recognition and manipulation, and wayfinding under simulated prosthetic vision. We explored three ways of optimizing the information content of the prosthetic visual image: Full-Field representation (wide visual angle, low sampling frequency), Region of Interest (ROI; narrow visible angle, high sampling frequency), and Fisheye (high sampling frequency in the center, progressively lower resolution toward the edges). Full-Field representation facilitated visual search and navigation, whereas ROI improved visual acuity. The Fisheye representation, designed to incorporate the benefits of both Full-Field representation and ROI, performed similarly to ROI with subjects unable to capitalize on the peripheral data. The observation that different image representation conditions prove advantageous for different tasks should be taken into account in the process of designing and testing new visual prosthesis prototypes.

Correction for Chinese character patterns formed by simulated irregular phosphene map

To reduce the unfavorable influence of phosphene array irregularity on the form of Chinese character pattern so as to improve recognition accuracy in visual prostheses, two correction methods were put forward. One method was to generate phosphene closest to the target point in regular arrays using weighted nearest neighbor search. The other was to generate phosphene whose center located in the region covered by dilated characters. Based on a simulation system, Chinese character recognition tests were given to fifteen normally sighted subjects under five degrees of array irregularity (0.2, 0.4, 0.6, 0.8, 1.0) without correction. The recognition accuracy decreased with the increase of irregularity. When the recognition accuracy dropped below 80%, two correction methods were applied and their effects were evaluated. With the increase of array irregularity, both effects on the accuracy of recognition grew considerably. Comparison between the two methods revealed that the former method afforded higher recognition accuracy and the latter only applied to phosphene map with serious irregularity.

Rehabilitation regimes based upon psychophysical studies of prosthetic vision

Human trials of prototype visual prostheses have successfully elicited visual percepts (phosphenes) in the visual field of implant recipients blinded through retinitis pigmentosa and age-related macular degeneration. Researchers are progressing rapidly towards a device that utilizes individual phosphenes as the elementary building blocks to compose a visual scene. This form of prosthetic vision is expected, in the near term, to have low resolution, large inter-phosphene gaps, distorted spatial distribution of phosphenes, restricted field of view, an eccentrically located phosphene field and limited number of expressible luminance levels. In order to fully realize the potential of these devices, there needs to be a training and rehabilitation program which aims to assist the prosthesis recipients to understand what they are seeing, and also to adapt their viewing habits to optimize the performance of the device. Based on the literature of psychophysical studies in simulated and real prosthetic vision, this paper proposes a comprehensive, theoretical training regime for a prosthesis recipient: visual search, visual acuity, reading, face/object recognition, hand-eye coordination and navigation. The aim of these tasks is to train the recipients to conduct visual scanning, eccentric viewing and reading, discerning low-contrast visual information, and coordinating bodily actions for visual-guided tasks under prosthetic vision. These skills have been identified as playing an important role in making prosthetic vision functional for the daily activities of their recipients.

Simulation of artificial vision: IV. Visual information required to achieve simple pointing and manipulation tasks

Retinal prostheses attempt to restore some amount of vision to totally blind patients. Vision evoked this way will be however severely constrained because of several factors (e.g., size of the implanted device, number of stimulating contacts, etc.). We used simulations of artificial vision to study how such restrictions of the amount of visual information provided would affect performance on simple pointing and manipulation tasks. Five normal subjects participated in the study. Two tasks were used: pointing on random targets (LEDs task) and arranging wooden chips according to a given model (CHIPs task). Both tasks had to be completed while the amount of visual information was limited by reducing the resolution (number of pixels) and modifying the size of the effective field of view. All images were projected on a 10 degrees x 7 degrees viewing area, stabilised at a given position on the retina. In central vision, the time required to accomplish the tasks remained systematically slower than with normal vision. Accuracy was close to normal at high image resolutions and decreased at 500 pixels or below, depending on the field of view used. Subjects adapted quite rapidly (in less than 15 sessions) to performing both tasks in eccentric vision (15 degrees in the lower visual field), achieving after adaptation performances close to those observed in central vision. These results demonstrate that, if vision is restricted to a small visual area stabilised on the retina (as would be the case in a retinal prosthesis), the perception of several hundreds of retinotopically arranged phosphenes is still needed to restore accurate but slow performance on pointing and manipulation tasks. Considering that present prototypes afford less than 100 stimulation contacts and that our simulations represent the most favourable visual input conditions that the user might experience, further development is required to achieve optimal rehabilitation prospects.

Psychophysics of prosthetic vision: II. stochastic sampling, the phosphene image, and noise

Stimulation of the diseased retina via an intraocular electrode array is a proposed means of restoring some vision to the profoundly blind. A prosthetic device to this end would involve post-implantation calibration (analogous to cochlear implant fitting), wherein the subject indicates those discrete positions in the visual field where luminous percepts are elicited. This procedure would be a source of noise, because the indicated positions would only approximate the actual positions in the visual field. Put differently, the procedure introduces sampling jitter, and would therefore affect clinical outcomes such as mobility and reading speeds. The nature of this noise is the concern of the present paper; we derive an expression for the noise power spectrum as it relates to the statistical nature of the sampling jitter. We show that, generally, jitter has greater effect on higher spatial-frequencies, that is, those areas of the implantee's visual perception that represent fine detail are more prone to noise. More specifically, the noise spectrum depends on the characteristic function of the random variable describing the sampling jitter. Our results signal the need for experimental work that characterizes sampling jitter in implantees, plus the need for simulations that allow a better understanding of perception and the noisy phosphene image.

A quantitative analysis of head movement behaviour during visual acuity assessment under prosthetic vision simulation

In most current vision prosthesis designs, head movement is the sole director of visual gaze and scanning due to the head-mounted nature of the camera. Study of this unnatural behaviour may provide insight into improved prosthesis designs and rehabilitation procedures. In this paper, we conducted a psychophysical study to investigate the characteristics of head movements of normally sighted subjects undergoing a visual acuity task in simulated prosthetic vision (SPV). In 12 naïve, untrained subjects, we recorded spontaneous changes in the amount of head movements during SPV sessions compared to control (normal vision) sessions. The observed behaviour continued to be refined until five or six sessions of practice. Increased head movement velocity was shown to be correlated to improved visual acuity performance, up to 0.3 logMAR, an equivalent of detecting details at half the physical size compared to complete deprivation of head movements. We postulate that visual scanning can as much as double the spatial frequency information in prosthetic vision. Increased head movement velocity observed when subjects were attempting smaller test items and for low-pass filtering schemes with higher cut-off frequencies may be further evidence that higher frequency content may be available through visual scanning, unconsciously driving subjects to increase head movement velocity.

Simulating auditory and visual sensorineural prostheses: a comparative review

Microelectronic vision prosthesis proposes to render luminous spots (so-called phosphenes) in the visual field of the otherwise blind subject by way of an implanted array of stimulating electrodes, and in doing so restore some spatial vision. There are now many research teams worldwide working towards a therapeutic device, analogous to the cochlear implant, for the profoundly blind. Despite the similarities between the cochlear implant and vision prostheses, there are few instances in the literature where the two approaches are compared and contrasted with a mind to informing the science and engineering of the latter. This is the focus of the present review; specifically, our interest is psychophysics and signal processing. Firstly, we examine the cochlear implant, and review a handful of psychophysical work: the acoustic simulation of cochlear implants and the method used. We focus on the use of normally hearing subjects (played coloured noise bands or sine waves) as a means of investigating cochlear-implant efficacy and speech processing algorithms. These results provide guidance to vision researchers, for they address the interpretation of simulation data, and flag key areas, such as 'artificial' perception in the presence of noise, that require experimental work in coming years. Secondly, we provide an up-to-date review of the body of analogous psychophysical work: the visual simulation, involving normal observers, of microelectronic vision prosthesis. These simulations allow predictions as to the likely clinical efficacy of the prosthesis; indeed, results to date suggest that a number on the order of 100 implanted electrodes will afford subjects mobility and recognition of faces (and other complex stimuli), while even fewer electrodes facilitate reading printed text and very simple visuomanual tasks. Further, the simulations allow investigations of image and signal processing strategies, plus they provide researchers in the field, and other interested persons, a perceptual experience that approximates what a prosthesis will likely afford implantees.

Visual prosthetics 2006: assessment and expectations

This report provides a brief overview of blinding eye diseases for which prosthetic vision may hold promise as a treatment modality, and of current and near-term technological approaches towards the creation of prosthetic interfaces with the remaining visual system. Principal anatomical, physiological, technological and functional obstacles and possible solutions are outlined, and references are provided to pioneering work by over a dozen groups on four continents.

Psychophysics of prosthetic vision: III. stochastic rendering, the phosphene image, and perception

For pt.I see ibid., p.Z004024-7. For pt.II see ibid., p.Z004558-61. This paper examines the rendering of luminous spots ("phosphenes") in the visual field, and their stochastic positioning as a means of anti-aliasing the resulting spotty image ("phosphene image"). We derive an equation concerning the correlations of pairs of phosphenes comprising the phosphene image, and show the relationship to the statistics governing the stochastic positioning. We present some examples where stochastic rendering assists the veridical perception of textures, and argue for its superiority as cf. ordered rendering. Our preliminary results suggest that it may be perceptually effective to manufacture disordered arrays of stimulating electrodes for intraocular implantation.