Towards model-based control of Parkinson's disease

Control-theoretic integration of stimulation and electrophysiology for cognitive enhancement

Transcranial electrical stimulation (tES) technology and neuroimaging are increasingly coupled in basic and applied science. This synergy has enabled individualized tES therapy and facilitated causal inferences in functional neuroimaging. However, traditional tES paradigms have been stymied by relatively small changes in neural activity and high inter-subject variability in cognitive effects. In this perspective, we propose a tES framework to treat these issues which is grounded in dynamical systems and control theory. The proposed paradigm involves a tight coupling of tES and neuroimaging in which M/EEG is used to parameterize generative brain models as well as control tES delivery in a hybrid closed-loop fashion. We also present a novel quantitative framework for cognitive enhancement driven by a new computational objective: shaping how the brain reacts to potential "inputs" (e.g., task contexts) rather than enforcing a fixed pattern of brain activity.

Automated pose estimation in primates

Understanding the behavior of primates is important for primatology, for psychology, and for biology more broadly. It is also important for biomedicine, where primates are an important model organism, and whose behavior is often an important variable of interest. Our ability to rigorously quantify behavior has, however, long been limited. On one hand, we can rigorously quantify low-information measures like preference, looking time, and reaction time; on the other, we can use more gestalt measures like behavioral categories tracked via ethogram, but at high cost and with high variability. Recent technological advances have led to a major revolution in behavioral measurement that offers affordable and scalable rigor. Specifically, digital video cameras and automated pose tracking software can provide measures of full-body position (i.e., pose) of primates over time (i.e., behavior) with high spatial and temporal resolution. Pose-tracking technology in turn can be used to infer behavioral states, such as eating, sleeping, and mating. We call this technological approach behavioral imaging. In this review, we situate the behavioral imaging revolution in the history of the study of behavior, argue for investment in and development of analytical and research techniques that can profit from the advent of the era of big behavior, and propose that primate centers and zoos will take on a more central role in relevant fields of research than they have in the past.

State-space optimal feedback control of optogenetically driven neural activity

Objective.The rapid acceleration of tools for recording neuronal populations and targeted optogenetic manipulation has enabled real-time, feedback control of neuronal circuits in the brain. Continuously-graded control of measured neuronal activity poses a wide range of technical challenges, which we address through a combination of optogenetic stimulation and a state-space optimal control framework implemented in the thalamocortical circuit of the awake mouse.Approach.Closed-loop optogenetic control of neurons was performed in real-time via stimulation of channelrhodopsin-2 expressed in the somatosensory thalamus of the head-fixed mouse. A state-space linear dynamical system model structure was used to approximate the light-to-spiking input-output relationship in both single-neuron as well as multi-neuron scenarios when recording from multielectrode arrays. These models were utilized to design state feedback controller gains by way of linear quadratic optimal control and were also used online for estimation of state feedback, where a parameter-adaptive Kalman filter provided robustness to model-mismatch.Main results.This model-based control scheme proved effective for feedback control of single-neuron firing rate in the thalamus of awake animals. Notably, the graded optical actuation utilized here did not synchronize simultaneously recorded neurons, but heterogeneity across the neuronal population resulted in a varied response to stimulation. Simulated multi-output feedback control provided better control of a heterogeneous population and demonstrated how the approach generalizes beyond single-neuron applications.Significance.To our knowledge, this work represents the first experimental application of state space model-based feedback control for optogenetic stimulation. In combination with linear quadratic optimal control, the approaches laid out and tested here should generalize to future problems involving the control of highly complex neural circuits. More generally, feedback control of neuronal circuits opens the door to adaptively interacting with the dynamics underlying sensory, motor, and cognitive signaling, enabling a deeper understanding of circuit function and ultimately the control of function in the face of injury or disease.

Falling off a limit cycle using phase-agnostic stimuli: Applications to clinical oscillopathies

For over a century, physiological studies have shown that precisely timed pulses can switch off a biological oscillator. This empiric finding has shaped our mechanistic understanding of how perturbations start, stop, and reset biological oscillators and has led to treatments that suppress pathological oscillations using electrical pulses given within specified therapeutic phase windows. Here, we present evidence, using numerical simulations of models of epileptic seizures and reentrant tachycardia, that the phase window can be opened to the entire cycle using novel complex stimulus waveforms. Our results reveal that the trajectories are displaced by such phase-agnostic stimuli off the oscillator's limit cycle and corralled into a region where oscillation is suppressed, irrespective of the phase at which the stimulus was applied. Our findings suggest the need for broadening theoretical understanding of how complex perturbing waveforms interact with biological oscillators to access their arrhythmic states. In clinical practice, oscillopathies may be treated more effectively with non-traditional stimulus waveforms that obviate the need for phase specificity.

Model-based analysis and forecast of sleep-wake regulatory dynamics: Tools and applications to data

Extensive clinical and experimental evidence links sleep-wake regulation and state of vigilance (SOV) to neurological disorders including schizophrenia and epilepsy. To understand the bidirectional coupling between disease severity and sleep disturbances, we need to investigate the underlying neurophysiological interactions of the sleep-wake regulatory system (SWRS) in normal and pathological brains. We utilized unscented Kalman filter based data assimilation (DA) and physiologically based mathematical models of a sleep-wake regulatory network synchronized with experimental measurements to reconstruct and predict the state of SWRS in chronically implanted animals. Critical to applying this technique to real biological systems is the need to estimate the underlying model parameters. We have developed an estimation method capable of simultaneously fitting and tracking multiple model parameters to optimize the reconstructed system state. We add to this fixed-lag smoothing to improve reconstruction of random input to the system and those that have a delayed effect on the observed dynamics. To demonstrate application of our DA framework, we have experimentally recorded brain activity from freely behaving rodents and classified discrete SOV continuously for many-day long recordings. These discretized observations were then used as the "noisy observables" in the implemented framework to estimate time-dependent model parameters and then to forecast future state and state transitions from out-of-sample recordings.

Dysregulation of excitatory neural firing replicates physiological and functional changes in aging visual cortex

The mammalian visual system has been the focus of countless experimental and theoretical studies designed to elucidate principles of neural computation and sensory coding. Most theoretical work has focused on networks intended to reflect developing or mature neural circuitry, in both health and disease. Few computational studies have attempted to model changes that occur in neural circuitry as an organism ages non-pathologically. In this work we contribute to closing this gap, studying how physiological changes correlated with advanced age impact the computational performance of a spiking network model of primary visual cortex (V1). Our results demonstrate that deterioration of homeostatic regulation of excitatory firing, coupled with long-term synaptic plasticity, is a sufficient mechanism to reproduce features of observed physiological and functional changes in neural activity data, specifically declines in inhibition and in selectivity to oriented stimuli. This suggests a potential causality between dysregulation of neuron firing and age-induced changes in brain physiology and functional performance. While this does not rule out deeper underlying causes or other mechanisms that could give rise to these changes, our approach opens new avenues for exploring these underlying mechanisms in greater depth and making predictions for future experiments.

Closed-Loop neuromodulation for clustering neuronal populations

Pathological synchronization of neurons is associated with symptoms of movement disorders, such as Parkinson's disease and essential tremor. High-frequency deep brain stimulation (DBS) suppresses symptoms, presumably through the desynchronization of neurons. Coordinated reset (CR) delivers trains of high-frequency stimuli to different regions in the brain through multiple electrodes and may have more persistent therapeutic effects than conventional DBS. As an alternative to CR, we present a closed-loop control setup that desynchronizes neurons in brain slices by inducing clusters using a single electrode. Our setup uses calcium fluorescence imaging to extract carbachol-induced neuronal oscillations in real time. To determine the appropriate stimulation waveform for inducing clusters in a population of neurons, we calculate the phase of the neuronal populations and then estimate the phase response curve (PRC) of those populations to electrical stimulation. The phase and PRC are then fed into a control algorithm called the input of maximal instantaneous efficiency (IMIE). By using IMIE, the synchrony across the slice is decreased by dividing the population of neurons into subpopulations without suppressing the oscillations locally. The desynchronization effect is persistent 10 s after stimulation is stopped. The IMIE control algorithm may be used as a novel closed-loop DBS approach to suppress the symptoms of Parkinson's disease and essential tremor by inducing clusters with a single electrode.NEW & NOTEWORTHY Here, we present a closed-loop controller to desynchronize neurons in brain slices by inducing clusters using a single electrode using calcium imaging feedback. Phase of neurons are estimated in real time, and from the phase response curve stimulation is applied to achieve target phase differences. This method is an alternative to coordinated reset and is a novel therapy that could be used to disrupt synchronous neuronal oscillations thought to be the mechanism underlying Parkinson's disease.

Application of Reinforcement Learning to Deep Brain Stimulation in a Computational Model of Parkinson's Disease

Deep brain stimulation (DBS) has been proven to be an effective treatment to deal with the symptoms of Parkinson's disease (PD). Currently, the DBS is in an open-loop pattern with which the stimulation parameters remain constant regardless of fluctuations in the disease state, and adjustments of parameters rely mostly on trial and error of experienced clinicians. This could bring adverse effects to patients due to possible overstimulation. Thus closed-loop DBS of which stimulation parameters are automatically adjusted based on variations in the ongoing neurophysiological signals is desired. In this paper, we present a closed-loop DBS method based on reinforcement learning (RL) to regulate stimulation parameters based on a computational model. The network model consists of interconnected biophysically-based spiking neurons, and the PD state is described as distorted relay reliability of thalamus (TH). Results show that the RL-based closed-loop control strategy can effectively restore the distorted relay reliability of the TH but with less DBS energy expenditure.

Model-based optimized phase-deviation deep brain stimulation for Parkinson 's disease

High-frequency deep brain stimulation (HF-DBS) of the subthalamic nucleus (STN), globus pallidus interna (GPi) and globus pallidus externa (GPe) are often considered as effective methods for the treatment of Parkinson's disease (PD). However, the stimulation of a single nucleus by HF-DBS can cause specific physical damage, produce side effects and usually consume more electrical energy. Therefore, we use a biophysically-based model of basal ganglia-thalamic circuits to explore more effective stimulation patterns to reduce adverse effects and save energy. In this paper, we computationally investigate the combined DBS of two nuclei with the phase deviation between two stimulation waveforms (CDBS). Three different stimulation combination strategies are proposed, i.e., STN and GPe CDBS (SED), STN and GPi CDBS (SID), as well as GPi and GPe CDBS (GGD). Resultantly, it is found that anti-phase CDBS is more effective in improving parkinsonian dynamical properties, including desynchronization of neurons and the recovery of the thalamus relay ability. Detailed simulation investigation shows that anti-phase SED and GGD are superior to SID. Besides, the energy consumption can be largely reduced by SED and GGD (72.5% and 65.5%), compared to HF-DBS. These results provide new insights into the optimal stimulation parameter and target choice of PD, which may be helpful for the clinical practice.

Noise-Induced Improvement of the Parkinsonian State: A Computational Study

The benefit of noise in improving the basal ganglia (BG) dysfunctions, especially Parkinsonian state, is explored in this paper. High frequency (≥ 100 Hz) deep brain stimulation (DBS), as a clinical effective stimulation method, has compelling and fantastic results in alleviating the motor symptoms of Parkinson's disease (PD). However, the mechanism of DBS is still unclear. And the selection of the DBS waveform parameters faces great challenges to further optimize the stimulation effects and to reduce its energy expenditure. Considering that the desynchronization of the BG neuronal activities is benefited from the forced high frequency regular spikes driven by standard high frequency DBS, we expect to explore a novel stimulation method that has capability of restoring the BG physiological firing patterns without introducing artificial high-frequency fires. In this paper, a colored noise stimulation is used as a neuromodulation method to disrupt the firing patterns of the pathological neuronal activities. A computational model of the BG that exhibits the intrinsic properties of the BG neurons and their interactions with the thalamic (Th) cells is employed. Based on the model, we investigate the effects of noise stimulation and explore the impacts of the noise stimulation parameters on both relay reliability of the Th neurons and energy expenditure of the stimulation. By comparison, it can be found that noise stimulation does not entrain the network to an artificial high-frequency firing state, but induces the pathological increased synchronous activities back to a normal physiological level. Moreover, besides the capability of restoring the neuronal state, the benefits of the noise also include its balanced waveform to avert potential tissue or electrode damage and its ability to reduce the energy expenditure to 50% less than that of the standard DBS, when the noise stimulation has low frequency (≤ 100 Hz) and appropriate intensity. Thus, the exploration of the optimal noise-induced improvement of the BG dysfunction is of great significance in treating symptoms of neurological disorders such as PD.

Robust closed-loop control of spike-and-wave discharges in a thalamocortical computational model of absence epilepsy

In this paper, we investigate the abatement of spike-and-wave discharges in a thalamocortical model using a closed-loop brain stimulation method. We first explore the complex states and various transitions in the thalamocortical computational model of absence epilepsy by using bifurcation analysis. We demonstrate that the Hopf and double cycle bifurcations are the key dynamical mechanisms of the experimental observed bidirectional communications during absence seizures through top-down cortical excitation and thalamic feedforward inhibition. Then, we formulate the abatement of epileptic seizures to a closed-loop tracking control problem. Finally, we propose a neural network based sliding mode feedback control system to drive the dynamics of pathological cortical area to track the desired normal background activities. The control system is robust to uncertainties and disturbances, and its stability is guaranteed by Lyapunov stability theorem. Our results suggest that the seizure abatement can be modeled as a tracking control problem and solved by a robust closed-loop control method, which provides a promising brain stimulation strategy.

The onset mechanism of Parkinson's beta oscillations: A theoretical analysis

In this paper, we build a basal ganglia-cortex-thalamus model to study the oscillatory mechanisms and boundary conditions of the beta frequency band (13-30 Hz) that appears in the subthalamic nucleus. First, a theoretical oscillatory boundary formula is obtained in a simplified model by using the Laplace transform and linearization process of the system at fixed points. Second, we simulate the oscillatory boundary conditions through numerical calculations, which fit with our theoretical results very well, at least in the changing trend. We find that several critical coupling strengths in the model exert great effects on the oscillations, the mechanisms of which differ but can be explained in detail by our model and the oscillatory boundary formula. Specifically, we note that the relatively small or large sizes of the coupling strength from the fast-spiking interneurons to the medium spiny neurons and from the cortex to the fast-spiking interneurons both have obvious maintenance roles on the states. Similar phenomena have been reported in other neurological diseases, such as absence epilepsy. However, some of those interesting mutual regulation mechanisms in the model have rarely been considered in previous studies. In addition to the coupling weight in the pathway, in this work, we show that the delay is a key parameter that affects oscillations. On the one hand, the system needs a minimum delay to generate oscillations; on the other hand, in the appropriate range, a longer delay leads to a higher activation level of the subthalamic nucleus. In this paper, we study the oscillation activities that appear on the subthalamic nucleus. Moreover, all populations in the model show the dynamic behaviour of a synchronous resonance. Therefore, we infer that the mechanisms obtained can be expanded to explore the state of other populations, and that the model provides a unified framework for studying similar problems in the future. Moreover, the oscillatory boundary curves obtained are all critical conditions between the stable state and beta frequency oscillation. The method is also suitable for depicting other common frequency bands during brain oscillations, such as the alpha band (8-12 Hz), theta band (4-7 Hz) and delta band (1-3 Hz). Thus, the results of this work are expected to help us better understand the onset mechanism of parkinson's oscillations and can inspire related experimental research in this field.

Toward adaptive deep brain stimulation in Parkinson's disease: a review

Clinical deep brain stimulation (DBS) is now regarded as the therapeutic intervention of choice at the advanced stages of Parkinson's disease. However, some major challenges of DBS are stimulation induced side effects and limited pacemaker battery life. Side effects and shortening of pacemaker battery life are mainly as a result of continuous stimulation and poor stimulation focus. These drawbacks can be mitigated using adaptive DBS (aDBS) schemes. Side effects resulting from continuous stimulation can be reduced through adaptive control using closed-loop feedback, while those due to poor stimulation focus can be mitigated through spatial adaptation. Other advantages of aDBS include automatic, rather than manual, initial adjustment and programming, and long-term adjustments to maintain stimulation parameters with changes in patient's condition. Both result in improved efficacy. This review focuses on the major areas that are essential in driving technological advances for the various aDBS schemes. Their challenges, prospects and progress so far are analyzed. In addition, important advances and milestones in state-of-the-art aDBS schemes are highlighted – both for closed-loop adaption and spatial adaption. With perspectives and future potentials of DBS provided at the end.

Modulation of Parkinsonian State With Uncertain Disturbance Based on Sliding Mode Control

Parkinson's disease (PD) is a degenerative disorder of central nervous system that endangers the olds' health seriously. The motor symptoms of PD can be attributed to the distorted relay reliability of thalamus to cortical sensorimotor input that results from the increase of inhibitory input from internal segment of the globus pallidum (GPi). Based on this, we construct the GPi-thalamocortical computational model to generate the normal and pathological firing patterns by varying GPi spike train input. A kind of closed-loop deep brain stimulation (DBS) strategy is proposed here. Our control objective is to make the controlled membrane potential of the thalamic neuron return to the normal firing pattern. The control input that directly acts on the thalamus is the DBS waveform, which is adjusted in real time according to the feedback signal. Aimed at a certain system without the change of object parameters or stochastic disturbance, the input-output feedback linearization method is able to eliminate the error between the system output and the desired output. When uncertain elements taken into consideration in the system, the simulation results indicate that sliding mode control scheme provides better effectiveness and higher robustness.

Qualitative changes in phase-response curve and synchronization at the saddle-node-loop bifurcation

Prominent changes in neuronal dynamics have previously been attributed to a specific switch in onset bifurcation, the Bogdanov-Takens (BT) point. This study unveils another, relevant and so far underestimated transition point: the saddle-node-loop bifurcation, which can be reached by several parameters, including capacitance, leak conductance, and temperature. This bifurcation turns out to induce even more drastic changes in synchronization than the BT transition. This result arises from a direct effect of the saddle-node-loop bifurcation on the limit cycle and hence spike dynamics. In contrast, the BT bifurcation exerts its immediate influence upon the subthreshold dynamics and hence only indirectly relates to spiking. We specifically demonstrate that the saddle-node-loop bifurcation (i) ubiquitously occurs in planar neuron models with a saddle node on invariant cycle onset bifurcation, and (ii) results in a symmetry breaking of the system's phase-response curve. The latter entails an increase in synchronization range in pulse-coupled oscillators, such as neurons. The derived bifurcation structure is of interest in any system for which a relaxation limit is admissible, such as Josephson junctions and chemical oscillators.

The Use of Neurocomputational Models as Alternatives to Animal Models in the Development of Electrical Brain Stimulation Treatments

Recent publications call for more animal models to be used and more experiments to be performed, in order to better understand the mechanisms of neurodegenerative disorders, to improve human health, and to develop new brain stimulation treatments. In response to these calls, some limitations of the current animal models are examined by using Deep Brain Stimulation (DBS) in Parkinson's disease as an illustrative example. Without focusing on the arguments for or against animal experimentation, or on the history of DBS, the present paper argues that given recent technological and theoretical advances, the time has come to consider bioinspired computational modelling as a valid alternative to animal models, in order to design the next generation of human brain stimulation treatments. However, before computational neuroscience is fully integrated in the translational process and used as a substitute for animal models, several obstacles need to be overcome. These obstacles are examined in the context of institutional, financial, technological and behavioural lock-in. Recommendations include encouraging agreement to change long-term habitual practices, explaining what alternative models can achieve, considering economic stakes, simplifying administrative and regulatory constraints, and carefully examining possible conflicts of interest.

Closed-Loop Modulation of the Pathological Disorders of the Basal Ganglia Network

A generalized predictive closed-loop control strategy to improve the basal ganglia activity patterns in Parkinson's disease (PD) is explored in this paper. Based on system identification, an input-output model is established to reveal the relationship between external stimulation and neuronal responses. The model contributes to the implementation of the generalized predictive control (GPC) algorithm that generates the optimal stimulation waveform to modulate the activities of neuronal nuclei. By analyzing the roles of two critical control parameters within the GPC law, optimal closed-loop control that has the capability of restoring the normal relay reliability of the thalamus with the least stimulation energy expenditure can be achieved. In comparison with open-loop deep brain stimulation and traditional static control schemes, the generalized predictive closed-loop control strategy can optimize the stimulation waveform without requiring any particular knowledge of the physiological properties of the system. This type of closed-loop control strategy generates an adaptive stimulation waveform with low energy expenditure with the potential to improve the treatments for PD.

Efficient implementation of a real-time estimation system for thalamocortical hidden Parkinsonian properties

Real-time estimation of dynamical characteristics of thalamocortical cells, such as dynamics of ion channels and membrane potentials, is useful and essential in the study of the thalamus in Parkinsonian state. However, measuring the dynamical properties of ion channels is extremely challenging experimentally and even impossible in clinical applications. This paper presents and evaluates a real-time estimation system for thalamocortical hidden properties. For the sake of efficiency, we use a field programmable gate array for strictly hardware-based computation and algorithm optimization. In the proposed system, the FPGA-based unscented Kalman filter is implemented into a conductance-based TC neuron model. Since the complexity of TC neuron model restrains its hardware implementation in parallel structure, a cost efficient model is proposed to reduce the resource cost while retaining the relevant ionic dynamics. Experimental results demonstrate the real-time capability to estimate thalamocortical hidden properties with high precision under both normal and Parkinsonian states. While it is applied to estimate the hidden properties of the thalamus and explore the mechanism of the Parkinsonian state, the proposed method can be useful in the dynamic clamp technique of the electrophysiological experiments, the neural control engineering and brain-machine interface studies.

The role of environmental constraints in walking: Effects of steering and sharp turns on gait dynamics

Stride durations in gait exhibit long-range correlation (LRC) which tends to disappear with certain movement disorders. The loss of LRC has been hypothesized to result from a reduction of functional degrees of freedom of the neuromuscular apparatus. A consequence of this theory is that environmental constraints such as the ones induced during constant steering may also reduce LRC. Furthermore, obstacles may perturb control of the gait cycle and also reduce LRC. To test these predictions, seven healthy participants walked freely overground in three conditions: unconstrained, constrained (constant steering), and perturbed (frequent 90° turns). Both steering and sharp turning reduced LRC with the latter having a stronger effect. Competing theories explain LRC in gait by positing fractal CPGs or a biomechanical process of kinetic energy reuse. Mediation analysis showed that the effect of the experimental manipulation in the current experiment depends partly on a reduction in walking speed. This supports the biomechanical theory. We also found that the local Hurst exponent did not reflect the frequent changes of heading direction. This suggests that the recovery from the sharp turn perturbation, a kind of relaxation time, takes longer than the four to seven meters between successive turns in the present study.

Closed-Loop Control of Tremor-Predominant Parkinsonian State Based on Parameter Estimation

A significant feature of Parkinson's disease (PD) is the inability of the thalamus to respond faithfully to sensorimotor information from the cerebral cortex. This may be the result of abnormal oscillations in the basal ganglia (BG). Deep brain stimulation (DBS) is regarded as an effective method to modulate these pathological brain rhythmic activities. However, the selection of DBS parameters is challenging because the mechanism is not well understood. This work proposes the design of a closed-loop control strategy to automatically adjust the parameters of a DBS waveform based on a computational model. By estimating the synaptic input from BG to the thalamic neuron model as feedback variable, we designed and compared various control algorithms to counteract the effects of pathological oscillatory inputs. We then obtained optimal DBS parameters to modulate the tremor-predominant Parkinsonian state. We showed that even a simple proportional controller provides higher fidelity of thalamic relay of sensorimotor information and lower energy expenditure, as compared with classical open-loop DBS. Integral action further enhances DBS performance. Additionally, a positive bias voltage further improves the relay ability of the thalamus with decreased stimulation energy expenditure. These findings were conducive to the development of a more effective DBS to further improve the treatment of the PD.

Closed-loop firing rate regulation of two interacting excitatory and inhibitory neural populations of the basal ganglia

This paper develops a new closed-loop firing rate regulation strategy for a population of neurons in the subthalamic nucleus, derived using a model-based analysis of the basal ganglia. The system is described using a firing rate model, in order to analyse the generation of beta-band oscillations. On this system, a proportional regulation of the firing rate reduces the gain of the subthalamo-pallidal loop in the parkinsonian case, thus impeding pathological oscillation generation. A filter with a well-chosen frequency is added to this proportional scheme, in order to avoid a potential instability of the feedback loop due to actuation and measurement delays. Our main result is a set of conditions on the parameters of the stimulation strategy that guarantee both its stability and a prescribed delay margin. A discussion on the applicability of the proposed method and a complete set of mathematical proofs is included.

Neurological Diseases from a Systems Medicine Point of View

The difficulty to understand, diagnose, and treat neurological disorders stems from the great complexity of the central nervous system on different levels of physiological granularity. The individual components, their interactions, and dynamics involved in brain development and function can be represented as molecular, cellular, or functional networks, where diseases are perturbations of networks. These networks can become a useful research tool in investigating neurological disorders if they are properly tailored to reflect corresponding mechanisms. Here, we review approaches to construct networks specific for neurological disorders describing disease-related pathology on different scales: the molecular, cellular, and brain level. We also briefly discuss cross-scale network analysis as a necessary integrator of these scales.

Adaptive control of Parkinson's state based on a nonlinear computational model with unknown parameters

The objective here is to explore the use of adaptive input-output feedback linearization method to achieve an improved deep brain stimulation (DBS) algorithm for closed-loop control of Parkinson's state. The control law is based on a highly nonlinear computational model of Parkinson's disease (PD) with unknown parameters. The restoration of thalamic relay reliability is formulated as the desired outcome of the adaptive control methodology, and the DBS waveform is the control input. The control input is adjusted in real time according to estimates of unknown parameters as well as the feedback signal. Simulation results show that the proposed adaptive control algorithm succeeds in restoring the relay reliability of the thalamus, and at the same time achieves accurate estimation of unknown parameters. Our findings point to the potential value of adaptive control approach that could be used to regulate DBS waveform in more effective treatment of PD.

Analysis of Oscillatory Neural Activity in Series Network Models of Parkinson's Disease During Deep Brain Stimulation

Parkinson's disease is a progressive, neurodegenerative disorder, characterized by hallmark motor symptoms. It is associated with pathological, oscillatory neural activity in the basal ganglia. Deep brain stimulation (DBS) is often successfully used to treat medically refractive Parkinson's disease. However, the selection of stimulation parameters is based on qualitative assessment of the patient, which can result in a lengthy tuning period and a suboptimal choice of parameters. This study explores fourth-order, control theory-based models of oscillatory activity in the basal ganglia. Describing function analysis is applied to examine possible mechanisms for the generation of oscillations in interacting nuclei and to investigate the suppression of oscillations with high-frequency stimulation. The theoretical results for the suppression of the oscillatory activity obtained using both the fourth-order model, and a previously described second-order model, are optimized to fit clinically recorded local field potential data obtained from Parkinsonian patients with implanted DBS. Close agreement between the power of oscillations recorded for a range of stimulation amplitudes is observed ( R(2)=0.69-0.99 ). The results suggest that the behavior of the system and the suppression of pathological neural oscillations with DBS is well described by the macroscopic models presented. The results also demonstrate that in this instance, a second-order model is sufficient to model the clinical data, without the need for added complexity. Describing the system behavior with computationally efficient models could aid in the identification of optimal stimulation parameters for patients in a clinical environment.

Cost-efficient FPGA implementation of basal ganglia and their Parkinsonian analysis

The basal ganglia (BG) comprise multiple subcortical nuclei, which are responsible for cognition and other functions. Developing a brain-machine interface (BMI) demands a suitable solution for the real-time implementation of a portable BG. In this study, we used a digital hardware implementation of a BG network containing 256 modified Izhikevich neurons and 2048 synapses to reliably reproduce the biological characteristics of BG on a single field programmable gate array (FPGA) core. We also highlighted the role of Parkinsonian analysis by considering neural dynamics in the design of the hardware-based architecture. Thus, we developed a multi-precision architecture based on a precise analysis using the FPGA-based platform with fixed-point arithmetic. The proposed embedding BG network can be applied to intelligent agents and neurorobotics, as well as in BMI projects with clinical applications. Although we only characterized the BG network with Izhikevich models, the proposed approach can also be extended to more complex neuron models and other types of functional networks.

Computational modeling of neurostimulation in brain diseases

Neurostimulation as a therapeutic tool has been developed and used for a range of different diseases such as Parkinson's disease, epilepsy, and migraine. However, it is not known why the efficacy of the stimulation varies dramatically across patients or why some patients suffer from severe side effects. This is largely due to the lack of mechanistic understanding of neurostimulation. Hence, theoretical computational approaches to address this issue are in demand. This chapter provides a review of mechanistic computational modeling of brain stimulation. In particular, we will focus on brain diseases, where mechanistic models (e.g., neural population models or detailed neuronal models) have been used to bridge the gap between cellular-level processes of affected neural circuits and the symptomatic expression of disease dynamics. We show how such models have been, and can be, used to investigate the effects of neurostimulation in the diseased brain. We argue that these models are crucial for the mechanistic understanding of the effect of stimulation, allowing for a rational design of stimulation protocols. Based on mechanistic models, we argue that the development of closed-loop stimulation is essential in order to avoid inference with healthy ongoing brain activity. Furthermore, patient-specific data, such as neuroanatomic information and connectivity profiles obtainable from neuroimaging, can be readily incorporated to address the clinical issue of variability in efficacy between subjects. We conclude that mechanistic computational models can and should play a key role in the rational design of effective, fully integrated, patient-specific therapeutic brain stimulation.

Closed-loop stimulation of a delayed neural fields model of parkinsonian STN-GPe network: a theoretical and computational study

Several disorders are related to pathological brain oscillations. In the case of Parkinson's disease, sustained low-frequency oscillations (especially in the β-band, 13-30 Hz) correlate with motor symptoms. It is still under debate whether these oscillations are the cause of parkinsonian motor symptoms. The development of techniques enabling selective disruption of these β-oscillations could contribute to the understanding of the underlying mechanisms, and could be exploited for treatments. A particularly appealing technique is Deep Brain Stimulation (DBS). With clinical electrical DBS, electrical currents are delivered at high frequency to a region made of potentially heterogeneous neurons (the subthalamic nucleus (STN) in the case of Parkinson's disease). Even more appealing is DBS with optogenetics, which is until now a preclinical method using both gene transfer and deep brain light delivery and enabling neuromodulation at the scale of one given neural network. In this work, we rely on delayed neural fields models of STN and the external Globus Pallidus (GPe) to develop, theoretically validate and test in silico a closed-loop stimulation strategy to disrupt these sustained oscillations with optogenetics. First, we rely on tools from control theory to provide theoretical conditions under which sustained oscillations can be attenuated by a closed-loop stimulation proportional to the measured activity of STN. Second, based on this theoretical framework, we show numerically that the proposed closed-loop stimulation efficiently attenuates sustained oscillations, even in the case when the photosensitization effectively affects only 50% of STN neurons. We also show through simulations that oscillations disruption can be achieved when the same light source is used for the whole STN population. We finally test the robustness of the proposed strategy to possible acquisition and processing delays, as well as parameters uncertainty.

Optimal entrainment of heterogeneous noisy neurons

We develop a methodology to design a stimulus optimized to entrain nonlinear, noisy limit cycle oscillators with uncertain properties. Conditions are derived which guarantee that the stimulus will entrain the oscillators despite these uncertainties. Using these conditions, we develop an energy optimal control strategy to design an efficient entraining stimulus and apply it to numerical models of noisy phase oscillators and to in vitro hippocampal neurons. In both instances, the optimal stimuli outperform other similar but suboptimal entraining stimuli. Because this control strategy explicitly accounts for both noise and inherent uncertainty of model parameters, it could have experimental relevance to neural circuits where robust spike timing plays an important role.

Improving desynchronization of Parkinsonian neuronal network via triplet-structure coordinated reset stimulation

We investigate how the triplet-structure coordinated reset stimulations (CRS), which acts on the GPe, STN and GPi within the basal ganglia-thalamocortical motor circuit, can destabilize the strong synchronous state and improve the reliability of thalamic relay in the parkinsonian network. It is shown that compared with the permanent (1:0 ON-OFF) CRS or the classic deep brain stimulation paradigm, the periodic m:n ON-OFF CRS (i.e., m ON-cycles stimulation followed by n OFF-cycles stimulation) can significantly desynchronize the neuronal network of Parkinson's disease, and evidently improve the fidelity of thalamic relay. In addition, the CRS-induced desynchronization can be greatly enhanced when the STN subpopulation within the pathologic network is subjected to the synaptic plasticity. Furthermore, the desynchronization and reliability can also be further improved as the closed-loop CRS strategy is introduced. The obtained results can be helpful for us to understand the pathophysiology mechanism of Parkinson's disease, even though the feasibility of CRS still needs to be explored in clinic.

Switching neuronal state: optimal stimuli revealed using a stochastically-seeded gradient algorithm

Inducing a switch in neuronal state using energy optimal stimuli is relevant to a variety of problems in neuroscience. Analytical techniques from optimal control theory can identify such stimuli; however, solutions to the optimization problem using indirect variational approaches can be elusive in models that describe neuronal behavior. Here we develop and apply a direct gradient-based optimization algorithm to find stimulus waveforms that elicit a change in neuronal state while minimizing energy usage. We analyze standard models of neuronal behavior, the Hodgkin-Huxley and FitzHugh-Nagumo models, to show that the gradient-based algorithm: (1) enables automated exploration of a wide solution space, using stochastically generated initial waveforms that converge to multiple locally optimal solutions; and (2) finds optimal stimulus waveforms that achieve a physiological outcome condition, without a priori knowledge of the optimal terminal condition of all state variables. Analysis of biological systems using stochastically-seeded gradient methods can reveal salient dynamical mechanisms underlying the optimal control of system behavior. The gradient algorithm may also have practical applications in future work, for example, finding energy optimal waveforms for therapeutic neural stimulation that minimizes power usage and diminishes off-target effects and damage to neighboring tissue.

A Hamilton-Jacobi-Bellman approach for termination of seizure-like bursting

We use Hamilton-Jacobi-Bellman methods to find minimum-time and energy-optimal control strategies to terminate seizure-like bursting behavior in a conductance-based neural model. Averaging is used to eliminate fast variables from the model, and a target set is defined through bifurcation analysis of the slow variables of the model. This method is illustrated for a single neuron model and for a network model to illustrate its efficacy in terminating bursting once it begins. This work represents a numerical proof-of-concept that a new class of control strategies can be employed to mitigate bursting, and could ultimately be adapted to treat medically intractible epilepsy in patient-specific models.

Locally optimal extracellular stimulation for chaotic desynchronization of neural populations

We use optimal control theory to design a methodology to find locally optimal stimuli for desynchronization of a model of neurons with extracellular stimulation. This methodology yields stimuli which lead to positive Lyapunov exponents, and hence desynchronizes a neural population. We analyze this methodology in the presence of interneuron coupling to make predictions about the strength of stimulation required to overcome synchronizing effects of coupling. This methodology suggests a powerful alternative to pulsatile stimuli for deep brain stimulation as it uses less energy than pulsatile stimuli, and could eliminate the time consuming tuning process.

Closed-loop brain-machine-body interfaces for noninvasive rehabilitation of movement disorders

Traditional approaches for neurological rehabilitation of patients affected with movement disorders, such as Parkinson's disease (PD), dystonia, and essential tremor (ET) consist mainly of oral medication, physical therapy, and botulinum toxin injections. Recently, the more invasive method of deep brain stimulation (DBS) showed significant improvement of the physical symptoms associated with these disorders. In the past several years, the adoption of feedback control theory helped DBS protocols to take into account the progressive and dynamic nature of these neurological movement disorders that had largely been ignored so far. As a result, a more efficient and effective management of PD cardinal symptoms has emerged. In this paper, we review closed-loop systems for rehabilitation of movement disorders, focusing on PD, for which several invasive and noninvasive methods have been developed during the last decade, reducing the complications and side effects associated with traditional rehabilitation approaches and paving the way for tailored individual therapeutics. We then present a novel, transformative, noninvasive closed-loop framework based on force neurofeedback and discuss several future developments of closed-loop systems that might bring us closer to individualized solutions for neurological rehabilitation of movement disorders.

A combined method to estimate parameters of the thalamocortical model from a heavily noise-corrupted time series of action potential

A combined method composing of the unscented Kalman filter (UKF) and the synchronization-based method is proposed for estimating electrophysiological variables and parameters of a thalamocortical (TC) neuron model, which is commonly used for studying Parkinson's disease for its relay role of connecting the basal ganglia and the cortex. In this work, we take into account the condition when only the time series of action potential with heavy noise are available. Numerical results demonstrate that not only this method can estimate model parameters from the extracted time series of action potential successfully but also the effect of its estimation is much better than the only use of the UKF or synchronization-based method, with a higher accuracy and a better robustness against noise, especially under the severe noise conditions. Considering the rather important role of TC neuron in the normal and pathological brain functions, the exploration of the method to estimate the critical parameters could have important implications for the study of its nonlinear dynamics and further treatment of Parkinson's disease.

Migraine generator network and spreading depression dynamics as neuromodulation targets in episodic migraine

Migraine is a common disabling headache disorder characterized by recurrent episodes sometimes preceded or accompanied by focal neurological symptoms called aura. The relation between two subtypes, migraine without aura (MWoA) and migraine with aura (MWA), is explored with the aim to identify targets for neuromodulation techniques. To this end, a dynamically regulated control system is schematically reduced to a network of the trigeminal nerve, which innervates the cranial circulation, an associated descending modulatory network of brainstem nuclei, and parasympathetic vasomotor efferents. This extends the idea of a migraine generator region in the brainstem to a larger network and is still simple and explicit enough to open up possibilities for mathematical modeling in the future. In this study, it is suggested that the migraine generator network (MGN) is driven and may therefore respond differently to different spatio-temporal noxious input in the migraine subtypes MWA and MWoA. The noxious input is caused by a cortical perturbation of homeostasis, known as spreading depression (SD). The MGN might even trigger SD in the first place by a failure in vasomotor control. As a consequence, migraine is considered as an inherently dynamical disease to which a linear course from upstream to downstream events would not do justice. Minimally invasive and noninvasive neuromodulation techniques are briefly reviewed and their rational is discussed in the context of the proposed mechanism.

Towards dynamical network biomarkers in neuromodulation of episodic migraine

Computational methods have complemented experimental and clinical neurosciences and led to improvements in our understanding of the nervous systems in health and disease. In parallel, neuromodulation in form of electric and magnetic stimulation is gaining increasing acceptance in chronic and intractable diseases. In this paper, we firstly explore the relevant state of the art in fusion of both developments towards translational computational neuroscience. Then, we propose a strategy to employ the new theoretical concept of dynamical network biomarkers (DNB) in episodic manifestations of chronic disorders. In particular, as a first example, we introduce the use of computational models in migraine and illustrate on the basis of this example the potential of DNB as early-warning signals for neuromodulation in episodic migraine.

Using a hybrid neuron in physiologically inspired models of the basal ganglia

Our current understanding of the basal ganglia (BG) has facilitated the creation of computational models that have contributed novel theories, explored new functional anatomy and demonstrated results complementing physiological experiments. However, the utility of these models extends beyond these applications. Particularly in neuromorphic engineering, where the basal ganglia's role in computation is important for applications such as power efficient autonomous agents and model-based control strategies. The neurons used in existing computational models of the BG, however, are not amenable for many low-power hardware implementations. Motivated by a need for more hardware accessible networks, we replicate four published models of the BG, spanning single neuron and small networks, replacing the more computationally expensive neuron models with an Izhikevich hybrid neuron. This begins with a network modeling action-selection, where the basal activity levels and the ability to appropriately select the most salient input is reproduced. A Parkinson's disease model is then explored under normal conditions, Parkinsonian conditions and during subthalamic nucleus deep brain stimulation (DBS). The resulting network is capable of replicating the loss of thalamic relay capabilities in the Parkinsonian state and its return under DBS. This is also demonstrated using a network capable of action-selection. Finally, a study of correlation transfer under different patterns of Parkinsonian activity is presented. These networks successfully captured the significant results of the originals studies. This not only creates a foundation for neuromorphic hardware implementations but may also support the development of large-scale biophysical models. The former potentially providing a way of improving the efficacy of DBS and the latter allowing for the efficient simulation of larger more comprehensive networks.

Transient localized wave patterns and their application to migraine

Transient dynamics is pervasive in the human brain and poses challenging problems both in mathematical tractability and clinical observability. We investigate statistical properties of transient cortical wave patterns with characteristic forms (shape, size, duration) in a canonical reaction-diffusion model with mean field inhibition. The patterns are formed by ghost behavior near a saddle-node bifurcation in which a stable traveling wave (node) collides with its critical nucleation mass (saddle). Similar patterns have been observed with fMRI in migraine. Our results support the controversial idea that waves of cortical spreading depression (SD) have a causal relationship with the headache phase in migraine and, therefore, occur not only in migraine with aura (MA), but also in migraine without aura (MO), i.e., in the two major migraine subtypes. We suggest a congruence between the prevalence of MO and MA with the statistical properties of the traveling waves' forms according to which two predictions follow: (i) the activation of nociceptive mechanisms relevant for headache is dependent upon a sufficiently large instantaneous affected cortical area; and (ii) the incidence of MA is reflected in the distance to the saddle-node bifurcation. We also observed that the maximal instantaneous affected cortical area is anticorrelated to both SD duration and total affected cortical area, which can explain why the headache is less severe in MA than in MO. Furthermore, the contested notion of MO attacks with silent aura is resolved. We briefly discuss model-based control and means by which neuromodulation techniques may affect pathways of pain formation.

Closed-loop control of the thalamocortical relay neuron's Parkinsonian state based on slow variable

A novel closed-loop control strategy is proposed to control Parkinsonian state based on a computational model. By modeling thalamocortical relay neurons under external electric field, a slow variable feedback control is applied to restore its relay functionality. Qualitative and quantitative analysis demonstrates the performance of feedback controller based on slow variable is more efficient compared with traditional feedback control based on fast variable. These findings point to the potential value of model-based design of feedback controllers for Parkinson's disease.

Minimum energy control for in vitro neurons

OBJECTIVE

To demonstrate the applicability of optimal control theory for designing minimum energy charge-balanced input waveforms for single periodically-firing in vitro neurons from brain slices of Long-Evans rats.

APPROACH

The method of control uses the phase model of a neuron and does not require prior knowledge of the neuron's biological details. The phase model of a neuron is a one-dimensional model that is characterized by the neuron's phase response curve (PRC), a sensitivity measure of the neuron to a stimulus applied at different points in its firing cycle. The PRC for each neuron is experimentally obtained by measuring the shift in phase due to a short-duration pulse injected into the periodically-firing neuron at various phase values. Based on the measured PRC, continuous-time, charge-balanced, minimum energy control waveforms have been designed to regulate the next firing time of the neuron upon application at the onset of an action potential.

MAIN RESULT

The designed waveforms can achieve the inter-spike-interval regulation for in vitro neurons with energy levels that are lower than those of conventional monophasic pulsatile inputs of past studies by at least an order of magnitude. They also provide the advantage of being charge-balanced. The energy efficiency of these waveforms is also shown by performing several supporting simulations that compare the performance of the designed waveforms against that of phase shuffled surrogate inputs, variants of the minimum energy waveforms obtained from suboptimal PRCs, as well as pulsatile stimuli that are applied at the point of maximum PRC. It was found that the minimum energy waveforms perform better than all other stimuli both in terms of control and in the amount of energy used. Specifically, it was seen that these charge-balanced waveforms use at least an order of magnitude less energy than conventional monophasic pulsatile stimuli.

SIGNIFICANCE

The significance of this work is that it uses concepts from the theory of optimal control and introduces a novel approach in designing minimum energy charge-balanced input waveforms for neurons that are robust to noise and implementable in electrophysiological experiments.

Failure of delayed feedback deep brain stimulation for intermittent pathological synchronization in Parkinson's disease

Suppression of excessively synchronous beta-band oscillatory activity in the brain is believed to suppress hypokinetic motor symptoms of Parkinson's disease. Recently, a lot of interest has been devoted to desynchronizing delayed feedback deep brain stimulation (DBS). This type of synchrony control was shown to destabilize the synchronized state in networks of simple model oscillators as well as in networks of coupled model neurons. However, the dynamics of the neural activity in Parkinson's disease exhibits complex intermittent synchronous patterns, far from the idealized synchronous dynamics used to study the delayed feedback stimulation. This study explores the action of delayed feedback stimulation on partially synchronized oscillatory dynamics, similar to what one observes experimentally in parkinsonian patients. We employ a computational model of the basal ganglia networks which reproduces experimentally observed fine temporal structure of the synchronous dynamics. When the parameters of our model are such that the synchrony is unphysiologically strong, the feedback exerts a desynchronizing action. However, when the network is tuned to reproduce the highly variable temporal patterns observed experimentally, the same kind of delayed feedback may actually increase the synchrony. As network parameters are changed from the range which produces complete synchrony to those favoring less synchronous dynamics, desynchronizing delayed feedback may gradually turn into synchronizing stimulation. This suggests that delayed feedback DBS in Parkinson's disease may boost rather than suppress synchronization and is unlikely to be clinically successful. The study also indicates that delayed feedback stimulation may not necessarily exhibit a desynchronization effect when acting on a physiologically realistic partially synchronous dynamics, and provides an example of how to estimate the stimulation effect.