Global activity shaping strategies for a retinal implant

Axonal stimulation affects the linear summation of single-point perception in three Argus II users

Objective.Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.Approach.We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ('between-axon') and along axon bundles ('along-axon'). Statistical analyses were conducted using linear regression and partial correlation analysis.Main results.Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants.Significance.The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Nanoparticle-based optical interfaces for retinal neuromodulation: a review

Degeneration of photoreceptors in the retina is a leading cause of blindness, but commonly leaves the retinal ganglion cells (RGCs) and/or bipolar cells extant. Consequently, these cells are an attractive target for the invasive electrical implants colloquially known as "bionic eyes." However, after more than two decades of concerted effort, interfaces based on conventional electrical stimulation approaches have delivered limited efficacy, primarily due to the current spread in retinal tissue, which precludes high-acuity vision. The ideal prosthetic solution would be less invasive, provide single-cell resolution and an ability to differentiate between different cell types. Nanoparticle-mediated approaches can address some of these requirements, with particular attention being directed at light-sensitive nanoparticles that can be accessed via the intrinsic optics of the eye. Here we survey the available known nanoparticle-based optical transduction mechanisms that can be exploited for neuromodulation. We review the rapid progress in the field, together with outstanding challenges that must be addressed to translate these techniques to clinical practice. In particular, successful translation will likely require efficient delivery of nanoparticles to stable and precisely defined locations in the retinal tissues. Therefore, we also emphasize the current literature relating to the pharmacokinetics of nanoparticles in the eye. While considerable challenges remain to be overcome, progress to date shows great potential for nanoparticle-based interfaces to revolutionize the field of visual prostheses.

An actor-model framework for visual sensory encoding

A fundamental challenge in neuroengineering is determining a proper artificial input to a sensory system that yields the desired perception. In neuroprosthetics, this process is known as artificial sensory encoding, and it holds a crucial role in prosthetic devices restoring sensory perception in individuals with disabilities. For example, in visual prostheses, one key aspect of artificial image encoding is to downsample images captured by a camera to a size matching the number of inputs and resolution of the prosthesis. Here, we show that downsampling an image using the inherent computation of the retinal network yields better performance compared to learning-free downsampling methods. We have validated a learning-based approach (actor-model framework) that exploits the signal transformation from photoreceptors to retinal ganglion cells measured in explanted mouse retinas. The actor-model framework generates downsampled images eliciting a neuronal response in-silico and ex-vivo with higher neuronal reliability than the one produced by a learning-free approach. During the learning process, the actor network learns to optimize contrast and the kernel's weights. This methodological approach might guide future artificial image encoding strategies for visual prostheses. Ultimately, this framework could be applicable for encoding strategies in other sensory prostheses such as cochlear or limb.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Learning to see via epiretinal implant stimulationin silicowith model-based deep reinforcement learning

OBJECTIVE

Diseases such as age-related macular degeneration and retinitis pigmentosa cause the degradation of the photoreceptor layer. One approach to restore vision is to electrically stimulate the surviving retinal ganglion cells with a microelectrode array such as epiretinal implants. Epiretinal implants are known to generate visible anisotropic shapes elongated along the axon fascicles of neighboring retinal ganglion cells. Recent work has demonstrated that to obtain isotropic pixel-like shapes, it is possible to map axon fascicles and avoid stimulating them by inactivating electrodes or lowering stimulation current levels. Avoiding axon fascicule stimulation aims to remove brushstroke-like shapes in favor of a more reduced set of pixel-like shapes.

APPROACH

In this study, we propose the use of isotropic and anisotropic shapes to render intelligible images on the retina of a virtual patient in a reinforcement learning environment named rlretina. The environment formalizes the task as using brushstrokes in a stroke-based rendering task.

MAIN RESULTS

We train a deep reinforcement learning agent that learns to assemble isotropic and anisotropic shapes to form an image. We investigate which error-based or perception-based metrics are adequate to reward the agent. The agent is trained in a model-based data generation fashion using the psychophysically validated axon map model to render images as perceived by different virtual patients. We show that the agent can generate more intelligible images compared to the naive method in different virtual patients.

SIGNIFICANCE

This work shares a new way to address epiretinal stimulation that constitutes a first step towards improving visual acuity in artificially-restored vision using anisotropic phosphenes.

SpikeSEE: An energy-efficient dynamic scenes processing framework for retinal prostheses

Intelligent and low-power retinal prostheses are highly demanded in this era, where wearable and implantable devices are used for numerous healthcare applications. In this paper, we propose an energy-efficient dynamic scenes processing framework (SpikeSEE) that combines a spike representation encoding technique and a bio-inspired spiking recurrent neural network (SRNN) model to achieve intelligent processing and extreme low-power computation for retinal prostheses. The spike representation encoding technique could interpret dynamic scenes with sparse spike trains, decreasing the data volume. The SRNN model, inspired by the human retina's special structure and spike processing method, is adopted to predict the response of ganglion cells to dynamic scenes. Experimental results show that the Pearson correlation coefficient of the proposed SRNN model achieves 0.93, which outperforms the state-of-the-art processing framework for retinal prostheses. Thanks to the spike representation and SRNN processing, the model can extract visual features in a multiplication-free fashion. The framework achieves 8 times power reduction compared with the convolutional recurrent neural network (CRNN) processing-based framework. Our proposed SpikeSEE predicts the response of ganglion cells more accurately with lower energy consumption, which alleviates the precision and power issues of retinal prostheses and provides a potential solution for wearable or implantable prostheses.

Restoration of the senses and human communication: Sustainable Development Goals 3 and 9

PURPOSE

This invited commentary addresses the importance of the senses in human communication, outlines advances achieved with cochlear implants, and new research directions to improve neural prostheses.

RESULT

In severely deaf people, cochlear implants restore speech understanding and enable children to achieve spoken language. Research in neural prostheses is advancing the restoration of hearing, vision, tactile senses, movement and the management of epilepsy. Bio-inspired stimulation strategies incorporating temporal and spatial characteristics of neural responses may deliver improved speech, vision and tactile perception using prostheses. To achieve stable long-term stimulation, chronic inflammation at the brain-electrode interface may be reduced using ROCK/Rho signalling pathway inhibitors and materials with brain-mimicking properties.

CONCLUSION

This commentary paper addresses two Sustainable Development Goals: industry, innovation and infrastructure (SDG 9) and good health and well-being (SDG 3).

Quantifying visual acuity for pre-clinical testing of visual prostheses

Objective.Visual prostheses currently restore only limited vision. More research and pre-clinical work are required to improve the devices and stimulation strategies that are used to induce neural activity that results in visual perception. Evaluation of candidate strategies and devices requires an objective way to convert measured and modelled patterns of neural activity into a quantitative measure of visual acuity.Approach.This study presents an approach that compares evoked patterns of neural activation with target and reference patterns. A d-prime measure of discriminability determines whether the evoked neural activation pattern is sufficient to discriminate between the target and reference patterns and thus provides a quantified level of visual perception in the clinical Snellen and MAR scales. The magnitude of the resulting value was demonstrated using scaled standardized 'C' and 'E' optotypes.Main results.The approach was used to assess the visual acuity provided by two alternative stimulation strategies applied to simulated retinal implants with different electrode pitch configurations and differently sized spreads of neural activity. It was found that when there is substantial overlap in neural activity generated by different electrodes, an estimate of acuity based only upon electrode pitch is incorrect; our proposed method gives an accurate result in both circumstances.Significance.Quantification of visual acuity using this approach in pre-clinical development will allow for more rapid and accurate prototyping of improved devices and neural stimulation strategies.

Toward a personalized closed-loop stimulation of the visual cortex: Advances and challenges

Current cortical visual prosthesis approaches are primarily unidirectional and do not consider the feed-back circuits that exist in just about every part of the nervous system. Herein, we provide a brief overview of some recent developments for better controlling brain stimulation and present preliminary human data indicating that closed-loop strategies could considerably enhance the effectiveness, safety, and long-term stability of visual cortex stimulation. We propose that the development of improved closed-loop strategies may help to enhance our capacity to communicate with the brain.

Towards aSmart Bionic Eye: AI-powered artificial vision for the treatment of incurable blindness

Objective.How can we return a functional form of sight to people who are living with incurable blindness? Despite recent advances in the development of visual neuroprostheses, the quality of current prosthetic vision is still rudimentary and does not differ much across different device technologies.Approach.Rather than aiming to represent the visual scene as naturally as possible, aSmart Bionic Eyecould provide visual augmentations through the means of artificial intelligence-based scene understanding, tailored to specific real-world tasks that are known to affect the quality of life of people who are blind, such as face recognition, outdoor navigation, and self-care.Main results.Complementary to existing research aiming to restore natural vision, we propose a patient-centered approach to incorporate deep learning-based visual augmentations into the next generation of devices.Significance.The ability of a visual prosthesis to support everyday tasks might make the difference between abandoned technology and a widely adopted next-generation neuroprosthetic device.

Human-in-the-loop optimization of visual prosthetic stimulation

Retinal prostheses are a promising strategy to restore sight to patients with retinal degenerative diseases. These devices compensate for the loss of photoreceptors by electrically stimulating neurons in the retina. Currently, the visual function that can be recovered with such devices is very limited. This is due, in part, to current spread, unintended axonal activation, and the limited resolution of existing devices. Here we show, using a recent model of prosthetic vision, that optimizing how visual stimuli are encoded by the device can help overcome some of these limitations, leading to dramatic improvements in visual perception.

APPROACH

We propose a strategy to do this in practice, using patients' feedback in a visual task. The main challenge of our approach comes from the fact that, typically, one only has access to a limited number of noisy responses from patients. We propose two ways to deal with this: first, we use a model of prosthetic vision to constrain and simplify the optimization. We show that, if one knew the parameters of this model for a given patient, it would be possible to greatly improve their perceptual performance. Second we propose a preferential Bayesian optimization to efficiently learn these model parameters for each patient, using minimal trials.

MAIN RESULTS

To test our approach, we presented healthy subjects with visual stimuli generated by a recent model of prosthetic vision, to replicate the perceptual experience of patients fitted with an implant. Our optimization procedure led to significant and robust improvements in perceived image quality, that transferred to increased performance in other tasks.

SIGNIFICANCE

Importantly, our strategy is agnostic to the type of prosthesis and thus could readily be implemented in existing implants.

Improving the spatial resolution of artificial vision using midget retinal ganglion cell populations modelled at the human fovea

OBJECTIVE

Retinal prostheses seek to create artificial vision by stimulating surviving retinal neurons of patients with profound vision impairment. Notwithstanding tremendous research efforts, the performance of all implants tested to date has remained rudimentary, incapable of overcoming the threshold for legal blindness. To maximize the perceptual efficacy of retinal prostheses, a device must be capable of controlling retinal neurons with greater spatiotemporal precision. Most studies of retinal stimulation were derived from either non-primate species or the peripheral primate retina. We investigated if artificial stimulation could leverage the high spatial resolution afforded by the neural substrates at the primate fovea and surrounding regions to achieve improved percept qualities.

APPROACH

We began by developing a new computational model capable of generating anatomically accurate retinal ganglion cell (RGC) populations within the human central retina. Next, multiple RGC populations across the central retina were stimulated in-silico to compare clinical and recently proposed neurostimulation configurations based on their ability to improve perceptual efficacy and reduce activation thresholds.

MAIN RESULTS

Our model uniquely upholds eccentricity-dependent characteristics such as RGC density and dendritic field diameter, whilst incorporating anatomically accurate features such as axon projection and three-dimensional RGC layering, features often forgone in favor of reduced computational complexity. Following epiretinal stimulation, the RGCs in our model produced response patterns in shapes akin to the complex percepts reported in clinical trials. Our results also demonstrated that even within the neuron-dense central retina, epiretinal stimulation using a multi-return hexapolar electrode arrangement could reliably achieve spatially focused RGC activation and could achieve single-cell excitation in 74% of all tested locations.

SIGNIFICANCE

This study establishes an anatomically accurate three-dimensional model of the human central retina and demonstrates the potential for an epiretinal hexapolar configuration to achieve consistent, spatially confined retinal responses, even within the neuron-dense foveal region. Our results promote the prospect and optimization of higher spatial resolution in future epiretinal implants.

Advances in Neuroscience, Not Devices, Will Determine the Effectiveness of Visual Prostheses

Background: Innovations in engineering and neuroscience have enabled the development of sophisticated visual prosthetic devices. In clinical trials, these devices have provided visual acuities as high as 20/460, enabled coarse navigation, and even allowed for reading of short words. However, long-term commercial viability arguably rests on attaining even better vision and more definitive improvements in tasks of daily living and quality of life. Purpose: Here we review technological and biological obstacles in the implementation of visual prosthetics. Conclusions: Research in the visual prosthetic field has tackled significant technical challenges, including biocompatibility, signal spread through neural tissue, and inadvertent activation of passing axons; however, significant gaps in knowledge remain in the realm of neuroscience, including the neural code of vision and visual plasticity. We assert that further optimization of prosthetic devices alone will not provide markedly improved visual outcomes without significant advances in our understanding of neuroscience.

Neural activity shaping utilizing a partitioned target pattern

Electrical stimulation of neural tissue is used in both clinical and experimental devices to evoke a desired spatiotemporal pattern of neural activity. These devices induce a local field that drives neural activation, referred to as an activating function or generator signal. In visual prostheses, the spread of generator signal from each electrode within the neural tissue results in a spread of visual perception, referred to as a phosphene. In cases where neighboring phosphenes overlap, it is desirable to use current steering or neural activity shaping strategies to manipulate the generator signal between the electrodes to provide greater control over the total pattern of neural activity. Applying opposite generator signal polarities in neighboring regions of the retina forces the generator signal to pass through zero at an intermediate point, thus inducing low neural activity that may be perceived as a high-contrast line. This approach provides a form of high contrast visual perception, but it requires partitioning of the target pattern into those regions that use positive or negative generator signals. This discrete optimization is an NP-hard problem that is subject to being trapped in detrimental local minima. This investigation proposes a new partitioning method using image segmentation to determine the most beneficial positive and negative generator signal regions. Utilizing a database of 1000 natural images, the method is compared to alternative approaches based upon the mean squared error of the outcome. Under nominal conditions and with a set computation limit, partitioning provided improvement for 32% of these images. This percentage increased to 89% when utilizing image pre-processing to emphasize perceptual features of the images. The percentage of images that were dealt with most effectively with image segmentation increased as lower computation limits were imposed on the algorithms.

Targeted Stimulation of Retinal Ganglion Cells in Epiretinal Prostheses: A Multiscale Computational Study

Retinal prostheses aim at restoring partial sight to patients that are blind due to retinal degenerative diseases by electrically stimulating the surviving healthy retinal neurons. Ideally, the electrical stimulation of the retina is intended to induce localized, focused, percepts only; however, some epiretinal implant subjects have reported seeing elongated phosphenes in a single electrode stimulation due to the axonal activation of retinal ganglion cells (RGCs). This issue can be addressed by properly devising stimulation waveforms so that the possibility of inducing axonal activation of RGCs is minimized. While strategies to devise electrical stimulation waveforms to achieve a focal RGCs response have been reported in literature, the underlying mechanisms are not well understood. This article intends to address this gap; we developed morphologically and biophysically realistic computational models of two classified RGCs: D1-bistratified and A2-monostratified. Computational results suggest that the sodium channel band (SOCB) is less sensitive to modulations in stimulation parameters than the distal axon (DA), and DA stimulus threshold is less sensitive to physiological differences among RGCs. Therefore, over a range of RGCs distal axon diameters, short-pulse symmetric biphasic waveforms can enhance the stimulation threshold difference between the SOCB and the DA. Appropriately designed waveforms can avoid axonal activation of RGCs, implying a consequential reduction of undesired strikes in the visual field.

Creation of virtual channels in the retina using synchronous and asynchronous stimulation - a modelling study

OBJECTIVE

Implantable retinal prostheses aim to provide artificial vision to those suffering from retinal degenerative diseases by electrically stimulating the remaining retinal neurons using a multi-electrode array. The spatial resolution of these devices can be improved by creation of so-called virtual channels (VCs) that are commonly achieved through synchronized stimulation of multiple electrodes. It is largely unclear though if VCs can be created using asynchronous stimulation, which was the primary aim of this study.

APPROACH

A computational model of multi-layered retina and epi-retinal dual-electrode stimulation was developed to simulate the neural activity of populations of retinal ganglion cells (RGCs) using the VC strategy under both synchronous and asynchronous stimulation conditions.

MAIN RESULTS

Our simulation suggests that VCs can be created using asynchronous stimulation. VC performance under both synchronous and asynchronous stimulation conditions can be improved by optimizing stimulation parameters such as current intensity, current ratio (α) between two electrodes, electrode spacing and the stimulation waveform. In particular, two VC performance measures; (1) linear displacement of the centroid of RGC activation, and (2) the RGC activation size consistency as a function of different current ratios α, have comparable performance under asynchronous and synchronous stimulation with appropriately selected stimulation parameters.

SIGNIFICANCE

Our findings support the possibility of creating VCs in the retina under both synchronous and asynchronous stimulation conditions. The results provide theoretical evidence for future retinal prosthesis designs with higher spatial resolution and power efficiency whilst reducing the number of current sources required to achieve these outcomes.

In-Vivo Microsystems: A Review

In-vivo sensors yield valuable medical information by measuring directly on the living tissue of a patient. These devices can be surface or implant devices. Electrical activity in the body, from organs or muscles can be measured using surface electrodes. For short term internal devices, catheters are used. These include cardiac catheter (in blood vessels) and bladder catheters. Due to the size and shape of the catheters, silicon devices provided an excellent solution for sensors. Since many cardiac catheters are disposable, the high volume has led to lower prices of the silicon sensors. Many catheters use a single sensor, but silicon offers the opportunity to have multi sensors in a single catheter, while maintaining small size. The cardiac catheter is usually inserted for a maximum of 72 h. Some devices may be used for a short-to-medium period to monitor parameters after an operation or injury (1-4 weeks). Increasingly, sensing, and actuating, devices are being applied to longer term implants for monitoring a range of parameters for chronic conditions. Devices for longer term implantation presented additional challenges due to the harshness of the environment and the stricter regulations for biocompatibility and safety. This paper will examine the three main areas of application for in-vivo devices: surface devices and short/medium-term and long-term implants. The issues of biocompatibility and safety will be discussed.

Stimulation Strategies for Improving the Resolution of Retinal Prostheses

Electrical stimulation using implantable devices with arrays of stimulating electrodes is an emerging therapy for neurological diseases. The performance of these devices depends greatly on their ability to activate populations of neurons with high spatiotemporal resolution. To study electrical stimulation of populations of neurons, retina serves as a useful model because the neural network is arranged in a planar array that is easy to access. Moreover, retinal prostheses are under development to restore vision by replacing the function of damaged light sensitive photoreceptors, which makes retinal research directly relevant for curing blindness. Here we provide a progress review on stimulation strategies developed in recent years to improve the resolution of electrical stimulation in retinal prostheses. We focus on studies performed with explanted retinas, in which electrophysiological techniques are the most advanced. We summarize achievements in improving the spatial and temporal resolution of electrical stimulation of the retina and methods to selectively stimulate neurons with different visual functions. Future directions for retinal prostheses development are also discussed, which could provide insights for other types of neuromodulatory devices in which high-resolution electrical stimulation is required.