Optic flow based spatial vision in insects

Path integration and optic flow in flying insects: a review of current evidence

Path integration is a key navigation mechanism used by many animals, involving the integration of direction and distance of path segments to form a goal vector that allows an animal to return directly to its starting point. While well established for animals walking on solid ground, evidence for path integration in animals moving without ground contact, such as flying insects, is less clear. The review focuses on flying Hymenoptera, particularly bees, which are extensively studied. Although bees can use flight distance and direction information, evidence for genuine path integration is limited. Accurately assessing distance travelled is a major challenge for flying animals, because it relies on optic flow-the movement of visual patterns across the eye caused by locomotion. Optic flow depends on both the animal's speed and the spatial layout of the environment, making it ambiguous for precise distance measurement. While path integration is crucial for animals like desert ants navigating sparse environments with few navigational cues, we argue that flying Hymenopterans in visually complex environments, rich in objects and textures, rely on additional navigational cues rather than precise path integration. As they become more familiar with an environment, they may iteratively refine unreliable distance estimates derived from optic flow. By combining this refined information with directional cues, they could determine a goal vector and improve their ability to navigate efficiently between key locations. In the case of honeybees, this ability also enables them to communicate these refined goal vectors to other bees through the waggle dance.

Follow the flower: approach-flight behaviour of bumblebees landing on a moving target

While landing on flowers, pollinating insects often have to deal with flower movement caused by wind. Here, we determined the landing performance of bumblebees on a moving artificial flower and how bees use their visual-motor system to control their landings. To do this, we built an experimental setup containing a physical model of a flower, moving sideways using sinusoidal kinematics at various oscillation frequencies (up to 0.65 Hz, at constant amplitude of 5 cm). We filmed the landings of Bombus terrestris bumblebees on this moving flower model and extracted the flight kinematics and trajectories using deep neural network-based videography tracking. The bumblebees were capable of compensating for the detrimental effects of flower movement on landing performance for flower movement frequencies up to 0.53 Hz. Only at our maximum frequency of 0.65 Hz did the percentage of successful landings decrease but landing accuracy and duration were not affected. To successfully land on the moving flower, the bumblebees gradually slowed down, aimed towards the middle of the flower and aligned with its movement. Our results indicated that bumblebees use modular visual-motor control feedback to do this: (1) they slow down by maintaining an approximately constant average optic expansion of the approaching flower image; (2) they aim towards the flower by keeping the flower in the middle of their view; (3) they align to the flower movement by minimizing the sideways optic flow of the moving flower image. Our findings increase our understanding of how flying insects land on flowers moved by wind.

Is this scenery worth exploring? Insight into the visual encoding of navigating ants

Solitary foraging insects such as desert ants rely heavily on vision for navigation. Although ants can learn visual scenes, it is unclear what cues they use to decide whether a scene is worth exploring at the first place. To investigate this, we recorded the motor behaviour of Cataglyphis velox ants navigating in a virtual reality setup and measured their lateral oscillations in response to various unfamiliar visual scenes under both closed-loop and open-loop conditions. In naturalistic-looking panorama, ants display regular oscillations as observed outdoors, allowing them to efficiently scan the scenery. Manipulations of the virtual environment revealed distinct functions served by dynamic and static cues. Dynamic cues, mainly rotational optic flow, regulated the amplitude of oscillations but not their regularity. Conversely, static cues had little impact on the amplitude but were essential for producing regular oscillations. Regularity of oscillations decreased in scenes with only horizontal, only vertical or no edges, but was restored in scenes with both edge types together. The actual number of edges, the visual pattern heterogeneity across azimuths, the light intensity or the relative elevation of brighter regions did not affect oscillations. We conclude that ants use a simple but functional heuristic to determine whether the visual world is worth exploring, relying on the presence of at least two different edge orientations in the scene.

Zebra stripes induce aberrant motion analysis in flies through aliasing

The function of zebra stripes has long puzzled biologists: contrasted and conspicuous colours are unusual in mammals. The puzzle appears solved: two lines of evidence indicate that they evolved as a protection against biting flies, the geographical coincidence of stripes and exposure to trypanosomiasis in Africa and field experiments showing flies struggling to navigate near zebras. A logical mechanistic explanation would be that stripes interfere with analysis of the optic flow; however, both spatiotemporal aliasing and the aperture effect seem ruled out following recent experiments showing that randomly checked patterns also interfere with the ability of flies to navigate near zebras. No clear mechanistic hypothesis remains. Here, I model from first principles how flies assess their motion relative to stripes, from image forming to motion analysis. I show that, at short distances, flies would consistently misjudge the motion of a striped object and frequently and saliently misjudge the direction of movement of a randomised check pattern. The range of distances at which the model predicts that stripes should impair flies is consistent with observations. The model shows that image formation is subject to spatial aliasing, preventing any form of motion analysis against a striped pattern at medium distances. The motion computation of flies is subject to a second form of aliasing, which, although independent of the temporal resolution of flies, bears conceptual similarities to spatiotemporal aliasing. The findings highlight the necessity of accounting not only for processing and psychology but also for the optics of image formation when taking a perceptual perspective of animal colours and contrasts.

The visual stimuli attributes instrumental for collective-motion-related decision-making in locusts

Visual interactions play an instrumental role in collective-motion-related decision-making. However, our understanding of the various tentative mechanisms that can serve the visual-based decision-making is limited. We investigated the role that different attributes of the visual stimuli play in the collective-motion-related motor response of locust nymphs. We monitored and analyzed the behavioral responses of individual locusts tethered in a natural-like walking posture over an airflow-suspended trackball to carefully selected stimuli comprising various black rectangular shapes. The experimental findings together with a prediction model relating the level of behavioral response to the visual stimuli attributes indicate a major role of the number of objects in the visual field, and a further important effect of the object's vertical moving edges. While the object's horizontal edges can be utilized in the estimation of conspecifics' heading, the overall area or visual angle subtended by the stimuli do not seem to play any role in inducing the response. Our results offer important novel insights regarding the fundamental visual-based mechanisms underlying animal collective motion and can be useful also in swarm robotics.

A bio-inspired looming detection for stable landing in unmanned aerial vehicles. BIOINSPIRATION & BIOMIMETICS 2024;20:016007. [PMID: 39481235 DOI: 10.1088/1748-3190/ad8d99] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Flying insects, such as flies and bees, have evolved the capability to rely solely on visual cues for smooth and secure landings on various surfaces. In the process of carrying out tasks, micro unmanned aerial vehicles (UAVs) may encounter various emergencies, and it is necessary to land safely in complex and unpredictable ground environments, especially when altitude information is not accurately obtained, which undoubtedly poses a significant challenge. Our study draws on the remarkable response mechanism of the Lobula Giant Movement Detector to looming scenarios to develop a novel UAV landing strategy. The proposed strategy does not require distance estimation, making it particularly suitable for payload-constrained micro aerial vehicles. Through a series of experiments, this strategy has proven to effectively achieve stable and high-performance landings in unknown and complex environments using only a monocular camera. Furthermore, a novel mechanism to trigger the final landing phase has been introduced, further ensuring the safe and stable touchdown of the drone.

trajPredRNN+: A new approach for precipitation nowcasting with weather radar echo images based on deep learning

：Short-term rainfall prediction is a crucial and practical research area, with the accuracy of rainfall prediction, particularly for heavy rainfall, significantly impacting people's lives, property, and even their safety. Existing models, such as ConvLSTM, TrajGRU, and PredRNN, exhibit limitations in capturing fine-grained appearances due to insufficient memory units or addressing positional misalignment issues, thereby compromising the accuracy of model predictions. In this study, we propose trajPredRNN+, an innovative approach that integrates the trajectory segmentation model and the PredRNN deep learning model to address both limitations in nowcasting precipitation using weather radar echo images. By incorporating attention mechanisms, the model demonstrates an enhanced focus on short-term and imminent heavy rainfall events. To ensure improved stability during training, a residual network is introduced. Lastly, a more rational and effective training loss function is proposed, encompassing weight mechanism, SSIM index, and GAN loss. To validate the proposed model, we conducted a comparative experiment and an ablation experiment using the radar echo map dataset obtained from the Shenzhen Meteorological Bureau. The results of these experiments demonstrate that our model has achieved significant improvements across multiple key performance indicators.

Model organisms and systems in neuroethology: one hundred years of history and a look into the future

The Journal of Comparative Physiology lived up to its name in the last 100 years by including more than 1500 different taxa in almost 10,000 publications. Seventeen phyla of the animal kingdom were represented. The honeybee (Apis mellifera) is the taxon with most publications, followed by locust (Locusta migratoria), crayfishes (Cambarus spp.), and fruitfly (Drosophila melanogaster). The representation of species in this journal in the past, thus, differs much from the 13 model systems as named by the National Institutes of Health (USA). We mention major accomplishments of research on species with specific adaptations, specialist animals, for example, the quantitative description of the processes underlying the axon potential in squid (Loligo forbesii) and the isolation of the first receptor channel in the electric eel (Electrophorus electricus) and electric ray (Torpedo spp.). Future neuroethological work should make the recent genetic and technological developments available for specialist animals. There are many research questions left that may be answered with high yield in specialists and some questions that can only be answered in specialists. Moreover, the adaptations of animals that occupy specific ecological niches often lend themselves to biomimetic applications. We go into some depth in explaining our thoughts in the research of motion vision in insects, sound localization in barn owls, and electroreception in weakly electric fish.

Analysis of collision avoidance in honeybee flight

Insects are excellent at flying in dense vegetation and navigating through other complex spatial environments. This study investigates the strategies used by honeybees (Apis mellifera) to avoid collisions with an obstacle encountered frontally during flight. Bees were trained to fly through a tunnel that contained a solitary vertically oriented cylindrical obstacle placed along the midline. Flight trajectories of bees were recorded for six conditions in which the diameter of the obstructing cylinder was systematically varied from 25 mm to 160 mm. Analysis of salient events during the bees' flight, such as the deceleration before the obstacle, and the initiation of the deviation in flight path to avoid collisions, revealed a strategy for obstacle avoidance that is based on the relative retinal expansion velocity generated by the obstacle when the bee is on a collision course. We find that a quantitative model, featuring a controller that extracts specific visual cues from the frontal visual field, provides an accurate characterization of the geometry and the dynamics of the manoeuvres adopted by honeybees to avoid collisions. This study paves the way for the design of unmanned aerial systems, by identifying the visual cues that are used by honeybees for performing robust obstacle avoidance flight.

Finding the gap: neuromorphic motion-vision in dense environments

Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.

The potential underlying mechanisms during learning flights

Hymenopterans, such as bees and wasps, have long fascinated researchers with their sinuous movements at novel locations. These movements, such as loops, arcs, or zigzags, serve to help insects learn their surroundings at important locations. They also allow the insects to explore and orient themselves in their environment. After they gained experience with their environment, the insects fly along optimized paths guided by several guidance strategies, such as path integration, local homing, and route-following, forming a navigational toolkit. Whereas the experienced insects combine these strategies efficiently, the naive insects need to learn about their surroundings and tune the navigational toolkit. We will see that the structure of the movements performed during the learning flights leverages the robustness of certain strategies within a given scale to tune other strategies which are more efficient at a larger scale. Thus, an insect can explore its environment incrementally without risking not finding back essential locations.

Unraveling the neural basis of spatial orientation in arthropods

The neural basis underlying spatial orientation in arthropods, in particular insects, has received considerable interest in recent years. This special issue of the Journal of Comparative Physiology A seeks to take account of these developments by presenting a collection of eight review articles and eight original research articles highlighting hotspots of research on spatial orientation in arthropods ranging from flies to spiders and the underlying neural circuits. The contributions impressively illustrate the wide range of tools available to arthropods extending from specific sensory channels to highly sophisticated neural computations for mastering complex navigational challenges.