Deriving neural network controllers from neuro-biological data: implementation of a single-leg stick insect controller

Leg-body coordination strategies for obstacle avoidance and narrow space navigation of multi-segmented, legged robots

Introduction

Millipedes can avoid obstacle while navigating complex environments with their multi-segmented body. Biological evidence indicates that when the millipede navigates around an obstacle, it first bends the anterior segments of its corresponding anterior segment of its body, and then gradually propagates this body bending mechanism from anterior to posterior segments. Simultaneously, the stride length between pairs of legs inside the bending curve decreases to coordinate the leg motions with the bending mechanism of the body segments. In robotics, coordination between multiple legs and body segments during turning for navigating in complex environments, e.g., narrow spaces, has not been fully realized in multi-segmented, multi-legged robots with more than six legs.

Method

To generate the efficient obstacle avoidance turning behavior in a multi-segmented, multi-legged (millipede-like) robot, this study explored three possible strategies of leg and body coordination during turning: including the local leg and body coordination at the segment level in a manner similar to millipedes, global leg amplitude change in response to different turning directions (like insects), and the phase reversal of legs inside of turning curve during obstacle avoidance (typical engineering approach).

Results

Using sensory inputs obtained from the antennae located at the robot head and recurrent neural control, different turning strategies were generated, with gradual body bending propagation from the anterior to posterior body segments.

Discussion

We discovered differences in the performance of each turning strategy, which could guide the future control development of multi-segmented, legged robots.

Adaptive load feedback robustly signals force dynamics in robotic model of Carausius morosus stepping

Animals utilize a number of neuronal systems to produce locomotion. One type of sensory organ that contributes in insects is the campaniform sensillum (CS) that measures the load on their legs. Groups of the receptors are found on high stress regions of the leg exoskeleton and they have significant effects in adapting walking behavior. Recording from these sensors in freely moving animals is limited by technical constraints. To better understand the load feedback signaled by CS to the nervous system, we have constructed a dynamically scaled robotic model of the Carausius morosus stick insect middle leg. The leg steps on a treadmill and supports weight during stance to simulate body weight. Strain gauges were mounted in the same positions and orientations as four key CS groups (Groups 3, 4, 6B, and 6A). Continuous data from the strain gauges were processed through a previously published dynamic computational model of CS discharge. Our experiments suggest that under different stepping conditions (e.g., changing "body" weight, phasic load stimuli, slipping foot), the CS sensory discharge robustly signals increases in force, such as at the beginning of stance, and decreases in force, such as at the end of stance or when the foot slips. Such signals would be crucial for an insect or robot to maintain intra- and inter-leg coordination while walking over extreme terrain.

Network Architecture Producing Swing to Stance Transitions in an Insect Walking System

The walking system of the stick insect is one of the most thoroughly described invertebrate systems. We know a lot about the role of sensory input in the control of stepping of a single leg. However, the neuronal organization and connectivity of the central neural networks underlying the rhythmic activation and coordination of leg muscles still remain elusive. It is assumed that these networks can couple in the absence of phasic sensory input due to the observation of spontaneous recurrent patterns (SRPs) of coordinated motor activity equivalent to fictive stepping-phase transitions. Here we sought to quantify the phase of motor activity within SRPs in the isolated and interconnected meso- and meta-thoracic ganglia. We show that SRPs occur not only in the meso-, but also in the metathoracic ganglia of the stick insect, discovering a qualitative difference between them. We construct a network based on neurophysiological data capable of reproducing the measured SRP phases to investigate this difference. By comparing network output to the biological measurements we confirm the plausibility of the architecture and provide a hypothesis to account for these qualitative differences. The neural architecture we present couples individual central pattern generators to reproduce the fictive stepping-phase transitions observed in deafferented stick insect preparations after pharmacological activation, providing insights into the neural architecture underlying coordinated locomotion.

Insect-Inspired Robots: Bridging Biological and Artificial Systems

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

Information-based centralization of locomotion in animals and robots

The centralization of locomotor control from weak and local coupling to strong and global is hard to assess outside of particular modeling frameworks. We developed an empirical, model-free measure of centralization that compares information between control signals and both global and local states. A second measure, co-information, quantifies the net redundancy in global and local control. We first validate that our measures predict centralization in simulations of phase-coupled oscillators. We then test how centralization changes with speed in freely running cockroaches. Surprisingly, across all speeds centralization is constant and muscle activity is more informative of the global kinematic state (the averages of all legs) than the local state of that muscle's leg. Finally we use a legged robot to show that mechanical coupling alone can change the centralization of legged locomotion. The results of these systems span a design space of centralization and co-information for biological and robotic systems.

A kinematic model of stick-insect walking

Animal, and insect walking (locomotion) in particular, have attracted much attention from scientists over many years up to now. The investigations included behavioral, electrophysiological experiments, as well as modeling studies. Despite the large amount of material collected, there are left many unanswered questions as to how walking and related activities are generated, maintained, and controlled. It is obvious that for them to take place, precise coordination within muscle groups of one leg and between the legs is required: intra- and interleg coordination. The nature, the details, and the interactions of these coordination mechanisms are not entirely clear. To help uncover them, we made use of modeling techniques, and succeeded in developing a six-leg model of stick-insect walking. Our main goal was to prove that the same model can mimic a variety of walking-related behavioral modes, as well as the most common coordination patterns of walking just by changing the values of a few input or internal variables. As a result, the model can reproduce the basic coordination patterns of walking: tetrapod and tripod and the transition between them. It can also mimic stop and restart, change from forward-to-backward walking and back. Finally, it can exhibit so-called search movements of the front legs both while walking or standing still. The mechanisms of the model that enable it to produce the aforementioned behavioral modes can hint at and prove helpful in uncovering further details of the biological mechanisms underlying walking.

Swing Velocity Profiles of Small Limbs Can Arise from Transient Passive Torques of the Antagonist Muscle Alone

In large limbs, changing motor neuron activity typically controls within-movement velocity. For example, sequential agonist-antagonist-agonist motor neuron firing typically underlies the slowing often present at the end of human reaches. In physiological movements of large limbs, antagonistic muscle passive torque is generally negligible. In small limbs, alternatively, passive torques can determine limb rest position, generate restoring movements to it, and decrease agonist-generated movement amplitude and velocity maxima. These observations suggest that, in small limbs, passive forces might also control velocity changes within movements. We investigated this issue in stick insect middle leg femur-tibia (FT) joint. During swing, the FT joint extensor muscle actively shortens and the flexor muscle passively lengthens. As in human reaching, after its initial acceleration, FT joint velocity continuously decreases. We measured flexor passive forces during imposed stretches spanning the ranges of FT joint angles, angular velocities, and movement amplitudes present in leg swings. The viscoelastic "transient" passive force that occurs during and soon after stretch depended on all three variables and could be tens of times larger than the "steady-state" passive force commonly measured long after stretch end. We combined these data, the flexor and extensor moment arms, and an existing extensor model to simulate FT joint swing. To measure only passive (flexor) muscle-dependent effects, we used constant extensor activations in these simulations. In simulations using data from ten flexor muscles, flexor passive torque could always produce swings with, after swing initiation, continuously decreasing velocities. Antagonist muscle passive torques alone can thus control within-movement velocity.

A simple extension of inverted pendulum template to explain features of slow walking✰

Locomotion involves complex interactions between an organism and its environment. Despite these complex interactions, many characteristics of the motion of an animal's center of mass (COM) can be modeled using simple mechanical models such as inverted pendulum (IP) and spring-loaded inverted pendulum (SLIP) which employ a single effective leg to model an animal's COM. However, because these models are simple, they also have many limitations. We show that one limitation of IP and SLIP and many other simple mechanical models of locomotion is that they cannot model many observed features of locomotion at slow speeds. This limitation is due to the fact that the gravitational force is too strong, and, if unopposed, compels the animal to complete its stance in a relatively short time. We propose a new model, AS-IP (Angular Spring modulated Inverted Pendulum), in which the body is attached to the leg using springs which resist the leg's movement away from the vertical plane, and thus provides a means to model forces that effectively counter gravity. We show that AS-IP provides a mechanism by which an animal can tune its stance duration, and provide evidence that AS-IP is an excellent model for the motion of a fly's COM. More generally, we conclude that combining AS-IP with SLIP will greatly expand our ability to model legged locomotion over a range of speeds.

Force dynamics and synergist muscle activation in stick insects: the effects of using joint torques as mechanical stimuli

Many sensory systems are tuned to specific parameters of behaviors and have effects that are task-specific. We have studied how force feedback contributes to activation of synergist muscles in serially homologous legs of stick insects. Forces were applied using conventional half-sine or ramp and hold functions. We also utilized waveforms of joint torques calculated from experiments in freely walking animals. In all legs, forces applied to either the tarsus (foot) or proximal leg segment (trochanter) activated synergist muscles that generate substrate grip and support, but coupling of the depressor muscle to tarsal forces was weak in the front legs. Activation of trochanteral receptors using ramp and hold functions generated positive feedback to the depressor muscle in all legs when animals were induced to seek substrate grip. However, discharges of the synergist flexor muscle showed adaptation at moderate force levels. In contrast, application of forces using torque waveforms, which do not have a static hold phase, produced sustained discharges in muscle synergies with little adaptation. Firing frequencies reflected the magnitude of ground reaction forces, were graded to changes in force amplitude, and could also be modulated by transient force perturbations added to the waveforms. Comparison of synergist activation by torques and ramp and hold functions revealed a strong influence of force dynamics (dF/d t). These studies support the idea that force receptors can act to tune muscle synergies synchronously to the range of force magnitudes and dynamics that occur in each leg according to their specific use in behavior. NEW & NOTEWORTHY The effects of force receptors (campaniform sensilla) on leg muscles and synergies were characterized in stick insects using both ramp and hold functions and waveforms of joint torques calculated by inverse dynamics. Motor responses were sustained and showed reduced adaptation to the more "natural" and nonlinear torque stimuli. Calculation of the first derivative (dF/d t) of the torque waveforms demonstrated that this difference was correlated with the dynamic sensitivities of the system.

Six-legged walking in insects: how CPGs, peripheral feedback, and descending signals generate coordinated and adaptive motor rhythms

Walking is a rhythmic locomotor behavior of legged animals, and its underlying mechanisms have been the subject of neurobiological research for more than 100 years. In this article, we review relevant historical aspects and contemporary studies in this field of research with a particular focus on the role of central pattern generating networks (CPGs) and their contribution to the generation of six-legged walking in insects. Aspects of importance are the generation of single-leg stepping, the generation of interleg coordination, and how descending signals influence walking. We first review how CPGs interact with sensory signals from the leg in the generation of leg stepping. Next, we summarize how these interactions are modified in the generation of motor flexibility for forward and backward walking, curve walking, and speed changes. We then review the present state of knowledge with regard to the role of CPGs in intersegmental coordination and how CPGs might be involved in mediating descending influences from the brain for the initiation, maintenance, modification, and cessation of the motor output for walking. Throughout, we aim to specifically address gaps in knowledge, and we describe potential future avenues and approaches, conceptual and methodological, with the latter emphasizing in particular options arising from the advent of neurogenetic approaches to this field of research and its combination with traditional approaches.

Development and Training of a Neural Controller for Hind Leg Walking in a Dog Robot

Animals dynamically adapt to varying terrain and small perturbations with remarkable ease. These adaptations arise from complex interactions between the environment and biomechanical and neural components of the animal's body and nervous system. Research into mammalian locomotion has resulted in several neural and neuro-mechanical models, some of which have been tested in simulation, but few “synthetic nervous systems” have been implemented in physical hardware models of animal systems. One reason is that the implementation into a physical system is not straightforward. For example, it is difficult to make robotic actuators and sensors that model those in the animal. Therefore, even if the sensorimotor circuits were known in great detail, those parameters would not be applicable and new parameter values must be found for the network in the robotic model of the animal. This manuscript demonstrates an automatic method for setting parameter values in a synthetic nervous system composed of non-spiking leaky integrator neuron models. This method works by first using a model of the system to determine required motor neuron activations to produce stable walking. Parameters in the neural system are then tuned systematically such that it produces similar activations to the desired pattern determined using expected sensory feedback. We demonstrate that the developed method successfully produces adaptive locomotion in the rear legs of a dog-like robot actuated by artificial muscles. Furthermore, the results support the validity of current models of mammalian locomotion. This research will serve as a basis for testing more complex locomotion controllers and for testing specific sensory pathways and biomechanical designs. Additionally, the developed method can be used to automatically adapt the neural controller for different mechanical designs such that it could be used to control different robotic systems.

Neurodynamics in the Sensorimotor Loop: Representing Behavior Relevant External Situations

In the context of the dynamical system approach to cognition and supposing that brains or brain-like systems controlling the behavior of autonomous systems are permanently driven by their sensor signals, the paper approaches the question of neurodynamics in the sensorimotor loop in a purely formal way. This is carefully done by addressing the problem in three steps, using the time-discrete dynamics of standard neural networks and a fiber space representation for better clearness. Furthermore, concepts like meta-transients, parametric stability and dynamical forms are introduced, where meta-transients describe the effect of realistic sensor inputs, parametric stability refers to a class of sensor inputs all generating the “same type” of dynamic behavior, and a dynamical form comprises the corresponding class of parametrized dynamical systems. It is argued that dynamical forms are the essential internal representatives of behavior relevant external situations. Consequently, it is suggested that dynamical forms are the basis for a memory of these situations. Finally, based on the observation that not all brain process have a direct effect on the motor activity, a natural splitting of neurodynamics into vertical (internal) and horizontal (effective) parts is introduced.

Improved system identification using artificial neural networks and analysis of individual differences in responses of an identified neuron

Mathematical modelling is used routinely to understand the coding properties and dynamics of responses of neurons and neural networks. Here we analyse the effectiveness of Artificial Neural Networks (ANNs) as a modelling tool for motor neuron responses. We used ANNs to model the synaptic responses of an identified motor neuron, the fast extensor motor neuron, of the desert locust in response to displacement of a sensory organ, the femoral chordotonal organ, which monitors movements of the tibia relative to the femur of the leg. The aim of the study was threefold: first to determine the potential value of ANNs as tools to model and investigate neural networks, second to understand the generalisation properties of ANNs across individuals and to different input signals and third, to understand individual differences in responses of an identified neuron. A metaheuristic algorithm was developed to design the ANN architectures. The performance of the models generated by the ANNs was compared with those generated through previous mathematical models of the same neuron. The results suggest that ANNs are significantly better than LNL and Wiener models in predicting specific neural responses to Gaussian White Noise, but not significantly different when tested with sinusoidal inputs. They are also able to predict responses of the same neuron in different individuals irrespective of which animal was used to develop the model, although notable differences between some individuals were evident.

A three-leg model producing tetrapod and tripod coordination patterns of ipsilateral legs in the stick insect

Insect locomotion requires the precise coordination of the movement of all six legs. Detailed investigations have revealed that the movement of the legs is controlled by local dedicated neuronal networks, which interact to produce walking of the animal. The stick insect is well suited to experimental investigations aimed at understanding the mechanisms of insect locomotion. Beside the experimental approach, models have also been constructed to elucidate those mechanisms. Here, we describe a model that replicates both the tetrapod and tripod coordination pattern of three ipsilateral legs. The model is based on an earlier insect leg model, which includes the three main leg joints, three antagonistic muscle pairs, and their local neuronal control networks. These networks are coupled via angular signals to establish intraleg coordination of the three neuromuscular systems during locomotion. In the present three-leg model, we coupled three such leg models, representing front, middle, and hind leg, in this way. The coupling was between the levator-depressor local control networks of the three legs. The model could successfully simulate tetrapod and tripod coordination patterns, as well as the transition between them. The simulations showed that for the interleg coordination during tripod, the position signals of the levator-depressor neuromuscular systems sent between the legs were sufficient, while in tetrapod, additional information on the angular velocities in the same system was necessary, and together with the position information also sufficient. We therefore suggest that, during stepping, the connections between the levator-depressor neuromuscular systems of the different legs are of primary importance.

Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot

Walking animals, like insects, with little neural computing can effectively perform complex behaviors. For example, they can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, exteroceptive sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent neural network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors) in the network to generate different turning angles with short-term memory for a walking robot. The turning information is transmitted as descending steering signals to the neural locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations. The adaptation also enables the robot to effectively escape from sharp corners or deadlocks. Using backbone joint control embedded in the the locomotion control allows the robot to climb over small obstacles. Consequently, it can successfully explore and navigate in complex environments. We firstly tested our approach on a physical simulation environment and then applied it to our real biomechanical walking robot AMOSII with 19 DOFs to adaptively avoid obstacles and navigate in the real world.

A Computational Model of a Descending Mechanosensory Pathway Involved in Active Tactile Sensing

Many animals, including humans, rely on active tactile sensing to explore the environment and negotiate obstacles, especially in the dark. Here, we model a descending neural pathway that mediates short-latency proprioceptive information from a tactile sensor on the head to thoracic neural networks. We studied the nocturnal stick insect Carausius morosus, a model organism for the study of adaptive locomotion, including tactually mediated reaching movements. Like mammals, insects need to move their tactile sensors for probing the environment. Cues about sensor position and motion are therefore crucial for the spatial localization of tactile contacts and the coordination of fast, adaptive motor responses. Our model explains how proprioceptive information about motion and position of the antennae, the main tactile sensors in insects, can be encoded by a single type of mechanosensory afferents. Moreover, it explains how this information is integrated and mediated to thoracic neural networks by a diverse population of descending interneurons (DINs). First, we quantified responses of a DIN population to changes in antennal position, motion and direction of movement. Using principal component (PC) analysis, we find that only two PCs account for a large fraction of the variance in the DIN response properties. We call the two-dimensional space spanned by these PCs ‘coding-space’ because it captures essential features of the entire DIN population. Second, we model the mechanoreceptive input elements of this descending pathway, a population of proprioceptive mechanosensory hairs monitoring deflection of the antennal joints. Finally, we propose a computational framework that can model the response properties of all important DIN types, using the hair field model as its only input. This DIN model is validated by comparison of tuning characteristics, and by mapping the modelled neurons into the two-dimensional coding-space of the real DIN population. This reveals the versatility of the framework for modelling a complete descending neural pathway.

Many nocturnal and burrowing animals rely on their tactile sense to explore the surrounding space, and tactile cues are often used to adapt locomotion to a structurally complex environment. Most mammals use facial whiskers for active tactile exploration, while most insects use their antennae. Since whiskers and antennae are long, thin, cylindrical structures, they must be moved to probe the surrounding space. The nervous system therefore has to keep track of tactile sensor movement by encoding sensor position and motion in order to locate tactile contacts. Here, we model a descending neural pathway of the stick insect, which transfers information about tactile sensor movement to thoracic neural networks with short latency. We show that information about sensor position and motion can be derived from a single class of proprioceptors at the antennal joints, and present a computational model that explains the activity of four previously described groups of descending interneurons during antennal stimulation. Our model is validated against electrophysiological data on antennal mechanoreceptors and descending interneurons.

Controlling legs for locomotion-insights from robotics and neurobiology

Walking is the most common terrestrial form of locomotion in animals. Its great versatility and flexibility has led to many attempts at building walking machines with similar capabilities. The control of walking is an active research area both in neurobiology and robotics, with a large and growing body of work. This paper gives an overview of the current knowledge on the control of legged locomotion in animals and machines and attempts to give walking control researchers from biology and robotics an overview of the current knowledge in both fields. We try to summarize the knowledge on the neurobiological basis of walking control in animals, emphasizing common principles seen in different species. In a section on walking robots, we review common approaches to walking controller design with a slight emphasis on biped walking control. We show where parallels between robotic and neurobiological walking controllers exist and how robotics and biology may benefit from each other. Finally, we discuss where research in the two fields diverges and suggest ways to bridge these gaps.

A network model comprising 4 segmental, interconnected ganglia, and its application to simulate multi-legged locomotion in crustaceans

Inter-segmental coordination is crucial for the locomotion of animals. Arthropods show high variability of leg numbers, from 6 in insects up to 750 legs in millipedes. Despite this fact, the anatomical and functional organization of their nervous systems show basic similarities. The main similarities are the segmental organization, and the way the function of the segmental units is coordinated. We set out to construct a model that could describe locomotion (walking) in animals with more than 6 legs, as well as in 6-legged animals (insects). To this end, we extended a network model by Daun-Gruhn and Tóth (Journal of Computational Neuroscience, doi: 10.1007/s10827-010-0300-1 , 2011). This model describes inter-segmental coordination of the ipsilateral legs in the stick insect during walking. Including an additional segment (local network) into the original model, we could simulate coordination patterns that occur in animals walking on eight legs (e.g., crayfish). We could improve the model by modifying its original cyclic connection topology. In all model variants, the phase relations between the afferent segmental excitatory sensory signals and the oscillatory activity of the segmental networks played a crucial role. Our results stress the importance of this sensory input on the generation of different stable coordination patterns. The simulations confirmed that using the modified connection topology, the flexibility of the model behaviour increased, meaning that changing a single phase parameter, i.e., gating properties of just one afferent sensory signal was sufficient to reproduce all coordination patterns seen in the experiments.

Behavior control in the sensorimotor loop with short-term synaptic dynamics induced by self-regulating neurons

The behavior and skills of living systems depend on the distributed control provided by specialized and highly recurrent neural networks. Learning and memory in these systems is mediated by a set of adaptation mechanisms, known collectively as neuronal plasticity. Translating principles of recurrent neural control and plasticity to artificial agents has seen major strides, but is usually hampered by the complex interactions between the agent's body and its environment. One of the important standing issues is for the agent to support multiple stable states of behavior, so that its behavioral repertoire matches the requirements imposed by these interactions. The agent also must have the capacity to switch between these states in time scales that are comparable to those by which sensory stimulation varies. Achieving this requires a mechanism of short-term memory that allows the neurocontroller to keep track of the recent history of its input, which finds its biological counterpart in short-term synaptic plasticity. This issue is approached here by deriving synaptic dynamics in recurrent neural networks. Neurons are introduced as self-regulating units with a rich repertoire of dynamics. They exhibit homeostatic properties for certain parameter domains, which result in a set of stable states and the required short-term memory. They can also operate as oscillators, which allow them to surpass the level of activity imposed by their homeostatic operation conditions. Neural systems endowed with the derived synaptic dynamics can be utilized for the neural behavior control of autonomous mobile agents. The resulting behavior depends also on the underlying network structure, which is either engineered or developed by evolutionary techniques. The effectiveness of these self-regulating units is demonstrated by controlling locomotion of a hexapod with 18 degrees of freedom, and obstacle-avoidance of a wheel-driven robot.

Linear combination of one-step predictive information with an external reward in an episodic policy gradient setting: a critical analysis

One of the main challenges in the field of embodied artificial intelligence is the open-ended autonomous learning of complex behaviors. Our approach is to use task-independent, information-driven intrinsic motivation(s) to support task-dependent learning. The work presented here is a preliminary step in which we investigate the predictive information (the mutual information of the past and future of the sensor stream) as an intrinsic drive, ideally supporting any kind of task acquisition. Previous experiments have shown that the predictive information (PI) is a good candidate to support autonomous, open-ended learning of complex behaviors, because a maximization of the PI corresponds to an exploration of morphology- and environment-dependent behavioral regularities. The idea is that these regularities can then be exploited in order to solve any given task. Three different experiments are presented and their results lead to the conclusion that the linear combination of the one-step PI with an external reward function is not generally recommended in an episodic policy gradient setting. Only for hard tasks a great speed-up can be achieved at the cost of an asymptotic performance lost.

Walknet, a bio-inspired controller for hexapod walking

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.

Using individual-muscle specific instead of across-muscle mean data halves muscle simulation error

Hill-type parameter values measured in experiments on single muscles show large across-muscle variation. Using individual-muscle specific values instead of the more standard approach of across-muscle means might therefore improve muscle model performance. We show here that using mean values increased simulation normalized RMS error in all tested motor nerve stimulation paradigms in both isotonic and isometric conditions, doubling mean simulation error from 9 to 18 (different at p < 0.0001). These data suggest muscle-specific measurement of Hill-type model parameters is necessary in work requiring highly accurate muscle model construction. Maximum muscle force (F (max)) showed large (fourfold) across-muscle variation. To test the role of F (max) in model performance we compared the errors of models using mean F (max) and muscle-specific values for the other model parameters, and models using muscle-specific F (max) values and mean values for the other model parameters. Using muscle-specific F (max) values did not improve model performance compared to using mean values for all parameters, but using muscle-specific values for all parameters but F (max) did (to an error of 14, different from muscle-specific, mean all parameters, and mean only F (max) errors at p ≤ 0.014). Significantly improving model performance thus required muscle-specific values for at least a subset of parameters other than F (max), and best performance required muscle-specific values for this subset and F (max). Detailed consideration of model performance suggested that remaining model error likely stemmed from activation of both fast and slow motor neurons in our experiments and inadequate specification of model activation dynamics.

A neuromechanical model explaining forward and backward stepping in the stick insect

The mechanism underlying the generation of stepping has been the object of intensive studies. Stepping involves the coordinated movement of different leg joints and is, in the case of insects, produced by antagonistic muscle pairs. In the stick insect, the coordinated actions of three such antagonistic muscle pairs produce leg movements and determine the stepping pattern of the limb. The activity of the muscles is controlled by the nervous system as a whole and more specifically by local neuronal networks for each muscle pair. While many basic properties of these control mechanisms have been uncovered, some important details of their interactions in various physiological conditions have so far remained unknown. In this study, we present a neuromechanical model of the coupled protractor-retractor and levator-depressor neuromuscular systems and use it to elucidate details of their coordinated actions during forward and backward walking. The switch from protraction to retraction is evoked at a critical angle of the femur during downward movement. This angle represents a sensory input that integrates load, motion, and ground contact. Using the model, we can make detailed suggestions as to how rhythmic stepping might be generated by the central pattern generators of the local neuronal networks, how this activity might be transmitted to the corresponding motoneurons, and how the latter might control the activity of the related muscles. The entirety of these processes yields the coordinated interaction between neuronal and mechanical parts of the system. Moreover, we put forward a mechanism by which motoneuron activity could be modified by a premotor network and suggest that this mechanism might serve as a basis for fast adaptive behavior, like switches between forward and backward stepping, which occur, for example, during curve walking, and especially sharp turning, of insects.

Lessons for circuit function from large insects: towards understanding the neural basis of motor flexibility

Motor behaviors result from information processing that occurs in multiple neural networks acting at all levels from the initial selection of the behavior to its final generation. A long-standing research interest is how single neural networks can help generate different motor behaviors, that is, the origin of motor flexibility. Modern experimental techniques allow studying neural network activity during the production of multiple motor behaviors. Recent data provide strong evidence that the neural networks controlling insect legs are individually modified in task-dependent and finely tuned fashions. Understanding the mechanistic basis of these neural network modifications will be of particular interest in the upcoming years.

Dominance of local sensory signals over inter-segmental effects in a motor system: experiments

Legged locomotion requires that information local to one leg, and inter-segmental signals coming from the other legs are processed appropriately to establish a coordinated walking pattern.However, very little is known about the relative importance of local and inter-segmental signals when they converge upon the central pattern generators (CPGs) of different leg joints.We investigated this question on the CPG of the middle leg coxa–trochanter (CTr)-joint of the stick insect which is responsible for lifting and lowering the leg.We used a semi-intact preparation with an intact front leg stepping on a treadmill, and simultaneously stimulated load sensors of the middle leg.We found that middle leg load signals induce bursts in the middle leg depressor motoneurons(MNs). The same local load signals could also elicit rhythmic activity in the CPG of the middle leg CTr-joint when the stimulation of middle leg load sensors coincided with front leg stepping. However, the influence of front leg stepping was generally weak such that front leg stepping alone was only rarely accompanied by switching between middle leg levator and depressor MN activity. We therefore conclude that the impact of the local sensory signals on the levator–depressor motor system is stronger than the inter-segmental influence through front leg stepping.