Insect-inspired AI for autonomous robots

Recent advances in insect vision in a 3D world: looming stimuli and escape behaviour

Detecting looming motion directly towards the insect is vital to its survival. Looming detection in two insects, flies and locusts, is described and contrasted. Pathways using looming detectors to trigger action and their topographical layout in the brain is explored in relation to facilitating behavioural selection. Similar visual stimuli, such as looming motion, are processed by nearby glomeruli in the brain. Insect-inspired looming motion detectors are combined to detect and avoid collision in different scenarios by robots, vehicles and unmanned aerial vehicle (UAV)s.

Firing feature-driven neural circuits with scalable memristive neurons for robotic obstacle avoidance

Neural circuits with specific structures and diverse neuronal firing features are the foundation for supporting intelligent tasks in biology and are regarded as the driver for catalyzing next-generation artificial intelligence. Emulating neural circuits in hardware underpins engineering highly efficient neuromorphic chips, however, implementing a firing features-driven functional neural circuit is still an open question. In this work, inspired by avoidance neural circuits of crickets, we construct a spiking feature-driven sensorimotor control neural circuit consisting of three memristive Hodgkin-Huxley neurons. The ascending neurons exhibit mixed tonic spiking and bursting features, which are used for encoding sensing input. Additionally, we innovatively introduce a selective communication scheme in biology to decode mixed firing features using two descending neurons. We proceed to integrate such a neural circuit with a robot for avoidance control and achieve lower latency than conventional platforms. These results provide a foundation for implementing real brain-like systems driven by firing features with memristive neurons and put constructing high-order intelligent machines on the agenda.

Why animals can outrun robots

Animals are much better at running than robots. The difference in performance arises in the important dimensions of agility, range, and robustness. To understand the underlying causes for this performance gap, we compare natural and artificial technologies in the five subsystems critical for running: power, frame, actuation, sensing, and control. With few exceptions, engineering technologies meet or exceed the performance of their biological counterparts. We conclude that biology's advantage over engineering arises from better integration of subsystems, and we identify four fundamental obstacles that roboticists must overcome. Toward this goal, we highlight promising research directions that have outsized potential to help future running robots achieve animal-level performance.

Experimental study and geometrical method to design bio-inspired robotic kinematic chains of inching-locomotion caterpillars

Inching-locomotion caterpillars (ILAR) show impressive environmental adaptation, having high dexterity and flexibility. To design robots that mimic these abilities, a novel bioinspired robotic design (BIROD) method is presented. The method is composed by an algorithm for geometrical kinematic analysis (GEKINS) to standardize the proportional dimensions according to the insect's anatomy and obtain the kinematic chains. The approach is experimentally applied to analyze the locomotion and kinematic chain of these specimens:Geometridae-two pair of prolegs (represents 35 000 species) andPlusiinae-three pair of prolegs (represents 400 species). The obtained data indicate that the application of the proposed method permits to locate the attachment mechanisms, joints, links, and to calculate angular displacement, angular average velocity, number of degrees of freedom, and thus the kinematic chain.Geometridaein contrast toPlusiinae, shows a longer walk-stride length, a lower number of single-rotational joints in 2D (3 DOF versus 4 DOF), and a lower number of dual-rotational joints in 3D (6 DOF versus 8 DOF). The application of BIROD and GEKINS provides the forward kinematics for 35 400 ILAR species and are expected to be useful as a preliminary phase for the design of bio-inspired arthropod robots.

GAHLS: an optimized graph analytics based high level synthesis framework

The urgent need for low latency, high-compute and low power on-board intelligence in autonomous systems, cyber-physical systems, robotics, edge computing, evolvable computing, and complex data science calls for determining the optimal amount and type of specialized hardware together with reconfigurability capabilities. With these goals in mind, we propose a novel comprehensive graph analytics based high level synthesis (GAHLS) framework that efficiently analyzes complex high level programs through a combined compiler-based approach and graph theoretic optimization and synthesizes them into message passing domain-specific accelerators. This GAHLS framework first constructs a compiler-assisted dependency graph (CaDG) from low level virtual machine (LLVM) intermediate representation (IR) of high level programs and converts it into a hardware friendly description representation. Next, the GAHLS framework performs a memory design space exploration while account for the identified computational properties from the CaDG and optimizing the system performance for higher bandwidth. The GAHLS framework also performs a robust optimization to identify the CaDG subgraphs with similar computational structures and aggregate them into intelligent processing clusters in order to optimize the usage of underlying hardware resources. Finally, the GAHLS framework synthesizes this compressed specialized CaDG into processing elements while optimizing the system performance and area metrics. Evaluations of the GAHLS framework on several real-life applications (e.g., deep learning, brain machine interfaces) demonstrate that it provides 14.27× performance improvements compared to state-of-the-art approaches such as LegUp 6.2.

Scalable Electrophysiology of Millimeter-Scale Animals with Electrode Devices

Millimeter-scale animals such as Caenorhabditis elegans, Drosophila larvae, zebrafish, and bees serve as powerful model organisms in the fields of neurobiology and neuroethology. Various methods exist for recording large-scale electrophysiological signals from these animals. Existing approaches often lack, however, real-time, uninterrupted investigations due to their rigid constructs, geometric constraints, and mechanical mismatch in integration with soft organisms. The recent research establishes the foundations for 3-dimensional flexible bioelectronic interfaces that incorporate microfabricated components and nanoelectronic function with adjustable mechanical properties and multidimensional variability, offering unique capabilities for chronic, stable interrogation and stimulation of millimeter-scale animals and miniature tissue constructs. This review summarizes the most advanced technologies for electrophysiological studies, based on methods of 3-dimensional flexible bioelectronics. A concluding section addresses the challenges of these devices in achieving freestanding, robust, and multifunctional biointerfaces.

Neuromorphic sequence learning with an event camera on routes through vegetation

For many robotics applications, it is desirable to have relatively low-power and efficient onboard solutions. We took inspiration from insects, such as ants, that are capable of learning and following routes in complex natural environments using relatively constrained sensory and neural systems. Such capabilities are particularly relevant to applications such as agricultural robotics, where visual navigation through dense vegetation remains a challenging task. In this scenario, a route is likely to have high self-similarity and be subject to changing lighting conditions and motion over uneven terrain, and the effects of wind on leaves increase the variability of the input. We used a bioinspired event camera on a terrestrial robot to collect visual sequences along routes in natural outdoor environments and applied a neural algorithm for spatiotemporal memory that is closely based on a known neural circuit in the insect brain. We show that this method is plausible to support route recognition for visual navigation and more robust than SeqSLAM when evaluated on repeated runs on the same route or routes with small lateral offsets. By encoding memory in a spiking neural network running on a neuromorphic computer, our model can evaluate visual familiarity in real time from event camera footage.

A look into feedback neural computation upon collision selectivity

Physiological studies have shown that a group of locust's lobula giant movement detectors (LGMDs) has a diversity of collision selectivity to approaching objects, relatively darker or brighter than their backgrounds in cluttered environments. Such diversity of collision selectivity can serve locusts to escape from attack by natural enemies, and migrate in swarm free of collision. For computational studies, endeavours have been made to realize the diverse selectivity which, however, is still one of the most challenging tasks especially in complex and dynamic real world scenarios. The existing models are mainly formulated as multi-layered neural networks with merely feed-forward information processing, and do not take into account the effect of re-entrant signals in feedback loop, which is an essential regulatory loop for motion perception, yet never been explored in looming perception. In this paper, we inaugurate feedback neural computation for constructing a new LGMD-based model, named F-LGMD to look into the efficacy upon implementing different collision selectivity. Accordingly, the proposed neural network model features both feed-forward processing and feedback loop. The feedback control propagates output signals of parallel ON/OFF channels back into their starting neurons, thus makes part of the feed-forward neural network, i.e. the ON/OFF channels and the feedback loop form an iterative cycle system. Moreover, the feedback control is instantaneous, which leads to the existence of a fixed point whereby the fixed point theorem is applied to rigorously derive valid range of feedback coefficients. To verify the effectiveness of the proposed method, we conduct systematic experiments covering synthetic and natural collision datasets, and also online robotic tests. The experimental results show that the F-LGMD, with a unified network, can fulfil the diverse collision selectivity revealed in physiology, which not only reduces considerably the handcrafted parameters compared to previous studies, but also offers a both efficient and robust scheme for collision perception through feedback neural computation.

Fluid-structure interactions of bristled wings: the trade-off between weight and drag

The smallest flying insects often have bristled wings resembling feathers or combs. We combined experiments and three-dimensional numerical simulations to investigate the trade-off between wing weight and drag generation. In experiments of bristled strips, a reduced physical model of the bristled wing, we found that the elasto-viscous number indicates when reconfiguration occurs in the bristles. Analysis of existing biological data suggested that bristled wings of miniature insects lie below the reconfiguration threshold, thus avoiding drag reduction. Numerical simulations of bristled strips showed that there exist optimal numbers of bristles that maximize the weighted drag when the additional volume due to the bristles is taken into account. We found a scaling relationship between the rescaled optimal numbers and the dimensionless bristle length. This result agrees qualitatively with and provides an upper bound for the bristled wing morphological data analysed in this study.

Mechanosensory Control of Locomotion in Animals and Robots: Moving Forward

While animals swim, crawl, walk, and fly with apparent ease, building robots capable of robust locomotion remains a significant challenge. In this review, we draw attention to mechanosensation-the sensing of mechanical forces generated within and outside the body-as a key sense that enables robust locomotion in animals. We discuss differences between mechanosensation in animals and current robots with respect to (1) the encoding properties and distribution of mechanosensors and (2) the integration and regulation of mechanosensory feedback. We argue that robotics would benefit greatly from a detailed understanding of these aspects in animals. To that end, we highlight promising experimental and engineering approaches to study mechanosensation, emphasizing the mutual benefits for biologists and engineers that emerge from moving forward together.

Ajna: Generalized deep uncertainty for minimal perception on parsimonious robots

Robots are active agents that operate in dynamic scenarios with noisy sensors. Predictions based on these noisy sensor measurements often lead to errors and can be unreliable. To this end, roboticists have used fusion methods using multiple observations. Lately, neural networks have dominated the accuracy charts for perception-driven predictions for robotic decision-making and often lack uncertainty metrics associated with the predictions. Here, we present a mathematical formulation to obtain the heteroscedastic aleatoric uncertainty of any arbitrary distribution without prior knowledge about the data. The approach has no prior assumptions about the prediction labels and is agnostic to network architecture. Furthermore, our class of networks, Ajna, adds minimal computation and requires only a small change to the loss function while training neural networks to obtain uncertainty of predictions, enabling real-time operation even on resource-constrained robots. In addition, we study the informational cues present in the uncertainties of predicted values and their utility in the unification of common robotics problems. In particular, we present an approach to dodge dynamic obstacles, navigate through a cluttered scene, fly through unknown gaps, and segment an object pile, without computing depth but rather using the uncertainties of optical flow obtained from a monocular camera with onboard sensing and computation. We successfully evaluate and demonstrate the proposed Ajna network on four aforementioned common robotics and computer vision tasks and show comparable results to methods directly using depth. Our work demonstrates a generalized deep uncertainty method and demonstrates its utilization in robotics applications.

FlyDetector-Automated Monitoring Platform for the Visual-Motor Coordination of Honeybees in a Dynamic Obstacle Scene Using Digital Paradigm

Vision plays a crucial role in the ability of compound-eyed insects to perceive the characteristics of their surroundings. Compound-eyed insects (such as the honeybee) can change the optical flow input of the visual system by autonomously controlling their behavior, and this is referred to as visual-motor coordination (VMC). To analyze an insect's VMC mechanism in dynamic scenes, we developed a platform for studying insects that actively shape the optic flow of visual stimuli by adapting their flight behavior. Image-processing technology was applied to detect the posture and direction of insects' movement, and automatic control technology provided dynamic scene stimulation and automatic acquisition of perceptual insect behavior. In addition, a virtual mapping technique was used to reconstruct the visual cues of insects for VMC analysis in a dynamic obstacle scene. A simulation experiment at different target speeds of 1-12 m/s was performed to verify the applicability and accuracy of the platform. Our findings showed that the maximum detection speed was 8 m/s, and triggers were 95% accurate. The outdoor experiments showed that flight speed in the longitudinal axis of honeybees was more stable when facing dynamic barriers than static barriers after analyzing the change in geometric optic flow. Finally, several experiments showed that the platform can automatically and efficiently monitor honeybees' perception behavior, and can be applied to study most insects and their VMC.

Neural Networks for Navigation: From Connections to Computations

Many animals can navigate toward a goal they cannot see based on an internal representation of that goal in the brain's spatial maps. These maps are organized around networks with stable fixed-point dynamics (attractors), anchored to landmarks, and reciprocally connected to motor control. This review summarizes recent progress in understanding these networks, focusing on studies in arthropods. One factor driving recent progress is the availability of the Drosophila connectome; however, it is increasingly clear that navigation depends on ongoing synaptic plasticity in these networks. Functional synapses appear to be continually reselected from the set of anatomical potential synapses based on the interaction of Hebbian learning rules, sensory feedback, attractor dynamics, and neuromodulation. This can explain how the brain's maps of space are rapidly updated; it may also explain how the brain can initialize goals as stable fixed points for navigation.

How honey bees make fast and accurate decisions

Honey bee ecology demands they make both rapid and accurate assessments of which flowers are most likely to offer them nectar or pollen. To understand the mechanisms of honey bee decision-making, we examined their speed and accuracy of both flower acceptance and rejection decisions. We used a controlled flight arena that varied both the likelihood of a stimulus offering reward and punishment and the quality of evidence for stimuli. We found that the sophistication of honey bee decision-making rivalled that reported for primates. Their decisions were sensitive to both the quality and reliability of evidence. Acceptance responses had higher accuracy than rejection responses and were more sensitive to changes in available evidence and reward likelihood. Fast acceptances were more likely to be correct than slower acceptances; a phenomenon also seen in primates and indicative that the evidence threshold for a decision changes dynamically with sampling time. To investigate the minimally sufficient circuitry required for these decision-making capacities, we developed a novel model of decision-making. Our model can be mapped to known pathways in the insect brain and is neurobiologically plausible. Our model proposes a system for robust autonomous decision-making with potential application in robotics.

Robot Programming from Fish Demonstrations

Fish are capable of learning complex relations found in their surroundings, and harnessing their knowledge may help to improve the autonomy and adaptability of robots. Here, we propose a novel learning from demonstration framework to generate fish-inspired robot control programs with as little human intervention as possible. The framework consists of six core modules: (1) task demonstration, (2) fish tracking, (3) analysis of fish trajectories, (4) acquisition of robot training data, (5) generating a perception-action controller, and (6) performance evaluation. We first describe these modules and highlight the key challenges pertaining to each one. We then present an artificial neural network for automatic fish tracking. The network detected fish successfully in 85% of the frames, and in these frames, its average pose estimation error was less than 0.04 body lengths. We finally demonstrate how the framework works through a case study focusing on a cue-based navigation task. Two low-level perception-action controllers were generated through the framework. Their performance was measured using two-dimensional particle simulations and compared against two benchmark controllers, which were programmed manually by a researcher. The fish-inspired controllers had excellent performance when the robot was started from the initial conditions used in fish demonstrations (>96% success rate), outperforming the benchmark controllers by at least 3%. One of them also had an excellent generalisation performance when the robot was started from random initial conditions covering a wider range of starting positions and heading angles (>98% success rate), again outperforming the benchmark controllers by 12%. The positive results highlight the utility of the framework as a research tool to form biological hypotheses on how fish navigate in complex environments and design better robot controllers on the basis of biological findings.

A virtuous cycle between invertebrate and robotics research: perspective on a decade of Living Machines research

Many invertebrates are ideal model systems on which to base robot design principles due to their success in solving seemingly complex tasks across domains while possessing smaller nervous systems than vertebrates. Three areas are particularly relevant for robot designers: Research on flying and crawling invertebrates has inspired new materials and geometries from which robot bodies (their morphologies) can be constructed, enabling a new generation of softer, smaller, and lighter robots. Research on walking insects has informed the design of new systems for controlling robot bodies (their motion control) and adapting their motion to their environment without costly computational methods. And research combining wet and computational neuroscience with robotic validation methods has revealed the structure and function of core circuits in the insect brain responsible for the navigation and swarming capabilities (their mental faculties) displayed by foraging insects. The last decade has seen significant progress in the application of principles extracted from invertebrates, as well as the application of biomimetic robots to model and better understand how animals function. This Perspectives paper on the past 10 years of the Living Machines conference outlines some of the most exciting recent advances in each of these fields before outlining lessons gleaned and the outlook for the next decade of invertebrate robotic research.

Varieties of visual navigation in insects

The behaviours and cognitive mechanisms animals use to orient, navigate, and remember spatial locations exemplify how cognitive abilities have evolved to suit a number of different mobile lifestyles and habitats. While spatial cognition observed in vertebrates has been well characterised in recent decades, of no less interest are the great strides that have also been made in characterizing and understanding the behavioural and cognitive basis of orientation and navigation in invertebrate models and in particular insects. Insects are known to exhibit remarkable spatial cognitive abilities and are able to successfully migrate over long distances or pinpoint known locations relying on multiple navigational strategies similar to those found in vertebrate models-all while operating under the constraint of relatively limited neural architectures. Insect orientation and navigation systems are often tailored to each species' ecology, yet common mechanistic principles can be observed repeatedly. Of these, reliance on visual cues is observed across a wide number of insect groups. In this review, we characterise some of the behavioural strategies used by insects to solve navigational problems, including orientation over short-distances, migratory heading maintenance over long distances, and homing behaviours to known locations. We describe behavioural research using examples from a few well-studied insect species to illustrate how visual cues are used in navigation and how they interact with non-visual cues and strategies.