Drosophila Spatiotemporally Integrates Visual Signals to Control Saccades

The Hawkmoth Proboscis: An Insect Model for Sensorimotor Control of Reaching and Exploration

Reaching and inspecting objects is an intricate part of human life, which is shared by a diversity of animals across phyla. In addition to appendages like legs and antennae, some insects use their mouthparts to reach and inspect targets. Hawkmoths of the family Sphingidae (Lepidoptera) use their extremely long and straw-like proboscis to drink nectar from flowers. As they approach flowers, hawkmoths uncoil their proboscis and explore the floral surface while hovering to target the proboscis to the nectary hole. Several sensory modalities provide feedback to control and guide these extremely versatile proboscis movements. The control task faced by the hawkmoths' nervous system during such behaviors is not unlike that of an animal guiding limbs or a robotic agent guiding a manipulator to a target. Hawkmoths perform these reaching maneuvers while simultaneously hovering, and hence require rapid and continuous coordination between the proboscis, neck, and flight motor systems, thereby providing a unique invertebrate model for studying appendage guidance and reaching. Here, we review what is known about how hawkmoths use their proboscis for floral inspection and nectar discovery, as well as the role of various sensors in proboscis guidance. We give a brief overview of the morphology and muscular apparatus of the hawkmoth proboscis, and discuss how multimodal sensory feedback might be turned into motor action for appendage guidance.

Moving in an Uncertain World: Robust and Adaptive Control of Locomotion from Organisms to Machine Intelligence

Whether walking, running, slithering, or flying, organisms display a remarkable ability to move through complex and uncertain environments. In particular, animals have evolved to cope with a host of uncertainties-both of internal and external origin-to maintain adequate performance in an ever-changing world. In this review, we present mathematical methods in engineering to highlight emerging principles of robust and adaptive control of organismal locomotion. Specifically, by drawing on the mathematical framework of control theory, we decompose the robust and adaptive hierarchical structure of locomotor control. We show how this decomposition along the robust-adaptive axis provides testable hypotheses to classify behavioral outcomes to perturbations. With a focus on studies in non-human animals, we contextualize recent findings along the robust-adaptive axis by emphasizing two broad classes of behaviors: (1) compensation to appendage loss and (2) image stabilization and fixation. Next, we attempt to map robust and adaptive control of locomotion across some animal groups and existing bio-inspired robots. Finally, we highlight exciting future directions and interdisciplinary collaborations that are needed to unravel principles of robust and adaptive locomotion.

Insect vision: A steady gaze over different landscapes

To gather visual information effectively, animals must stabilize images when they move. But closely related fruit fly species sometimes occupy strikingly diverse visual habitats. A new study shows that they have adopted gaze strategies tailored to their different visual worlds.

Divergent visual ecology of Drosophila species drives object-tracking strategies matched to landscape sparsity

Maintaining stable gaze while tracking moving objects is commonplace across animal taxa, yet how diverse ecological needs impact these processes is poorly understood. During flight, the fruit-eating fly Drosophila melanogaster maintains course by making smooth steering adjustments to fixate the image of the distant visual background on the retina, while executing body saccades to investigate nearby objects such as food sources. Cactophilic Drosophila mojavensis live where there is no canopy; rather, the flora forming visual "background" and "objects" are one and the same. We tested whether D. mojavensis have adapted their flight control strategies for a visually sparse landscape. We used a magnetic tether that allows free movement in the yaw axis. In response to a textured bar moving across a similarly textured stationary background, D. melanogaster fixates the background, thereby stabilizing gaze while integrating bar dynamics to trigger tracking saccades. By contrast, two mojavensis subspecies in the repleta subgroup and one species in the melanogaster subgroup steer to smoothly fixate the bar, seemingly ignoring the stationary surround. Desert flies execute frequent bar-tracking saccades, but theirs are triggered when rotational velocity lags the bar. Thus, D. melanogaster, which lives in visually cluttered cosmopolitan habitats, leverages the optical disparities between nearby objects and distant foliage for a hybrid control strategy: "ground-fixate, object-saccade." Flies in distant phylogenetic subgroups with similar visual ecology use a "fixate-and-saccade" strategy, which would be adaptive in a visually sparse environment where individual landscape features are both approached and used to maintain a straight course.

Bilateral interactions of optic-flow sensitive neurons coordinate course control in flies

Animals rely on compensatory actions to maintain stability and navigate their environment efficiently. These actions depend on global visual motion cues known as optic-flow. While the optomotor response has been the traditional focus for studying optic-flow compensation in insects, its simplicity has been insufficient to determine the role of the intricate optic-flow processing network involved in visual course control. Here, we reveal a series of course control behaviours in Drosophila and link them to specific neural circuits. We show that bilateral electrical coupling of optic-flow-sensitive neurons in the fly's lobula plate are required for a proper course control. This electrical interaction works alongside chemical synapses within the HS-H2 network to control the dynamics and direction of turning behaviours. Our findings reveal how insects use bilateral motion cues for navigation, assigning a new functional significance to the HS-H2 network and suggesting a previously unknown role for gap junctions in non-linear operations.

Flies tune the activity of their multifunctional gyroscope

Members of the order Diptera, the true flies, are among the most maneuverable flying animals. These aerial capabilities are partially attributed to flies' possession of halteres, tiny club-shaped structures that evolved from the hindwings and play a crucial role in flight control. Halteres are renowned for acting as biological gyroscopes that rapidly detect rotational perturbations and help flies maintain a stable flight posture. Additionally, halteres provide rhythmic input to the wing steering system that can be indirectly modulated by the visual system. The multifunctional capacity of the haltere is thought to depend on arrays of embedded mechanosensors called campaniform sensilla that are arranged in distinct groups on the haltere's dorsal and ventral surfaces. Although longstanding hypotheses suggest that each array provides different information relevant to the flight control circuitry, we know little about how the haltere campaniforms are functionally organized. Here, we use in vivo calcium imaging during tethered flight to obtain population-level recordings of the haltere sensory afferents in specific fields of sensilla. We find that haltere feedback from both dorsal fields is continuously active, modulated under closed-loop flight conditions, and recruited during saccades to help flies actively maneuver. We also find that the haltere's multifaceted role may arise from the steering muscles of the haltere itself, regulating haltere stroke amplitude to modulate campaniform activity. Taken together, our results underscore the crucial role of efferent control in regulating sensor activity and provide insight into how the sensory and motor systems of flies coevolved.

A neural circuit architecture for rapid learning in goal-directed navigation

Anchoring goals to spatial representations enables flexible navigation but is challenging in novel environments when both representations must be acquired simultaneously. We propose a framework for how Drosophila uses internal representations of head direction (HD) to build goal representations upon selective thermal reinforcement. We show that flies use stochastically generated fixations and directed saccades to express heading preferences in an operant visual learning paradigm and that HD neurons are required to modify these preferences based on reinforcement. We used a symmetric visual setting to expose how flies' HD and goal representations co-evolve and how the reliability of these interacting representations impacts behavior. Finally, we describe how rapid learning of new goal headings may rest on a behavioral policy whose parameters are flexible but whose form is genetically encoded in circuit architecture. Such evolutionarily structured architectures, which enable rapidly adaptive behavior driven by internal representations, may be relevant across species.

Asynchronous haltere input drives specific wing and head movements in Drosophila

Halteres are multifunctional mechanosensory organs unique to the true flies (Diptera). A set of reduced hindwings, the halteres beat at the same frequency as the lift-generating forewings and sense inertial forces via mechanosensory campaniform sensilla. Though haltere ablation makes stable flight impossible, the specific role of wing-synchronous input has not been established. Using small iron filings attached to the halteres of tethered flies and an alternating electromagnetic field, we experimentally decoupled the wings and halteres of flying Drosophila and observed the resulting changes in wingbeat amplitude and head orientation. We find that asynchronous haltere input results in fast amplitude changes in the wing (hitches), but does not appreciably move the head. In multi-modal experiments, we find that wing and gaze optomotor responses are disrupted differently by asynchronous input. These effects of wing-asynchronous haltere input suggest that specific sensory information is necessary for maintaining wing amplitude stability and adaptive gaze control.

Sensorimotor adaptation to destabilizing dynamics in weakly electric fish

Humans and other animals can readily learn to compensate for changes in the dynamics of movement. Such changes can result from an injury or changes in the weight of carried objects. These changes in dynamics can lead not only to reduced performance but also to dramatic instabilities. We evaluated the impacts of compensatory changes in control policies in relation to stability and robustness in Eigenmannia virescens, a species of weakly electric fish. We discovered that these fish retune their sensorimotor control system in response to experimentally generated destabilizing dynamics. Specifically, we used an augmented reality system to manipulate sensory feedback during an image stabilization task in which a fish maintained its position within a refuge. The augmented reality system measured the fish's movements in real time. These movements were passed through a high-pass filter and multiplied by a gain factor before being fed back to the refuge motion. We adjusted the gain factor to gradually destabilize the fish's sensorimotor loop. The fish retuned their sensorimotor control system to compensate for the experimentally induced destabilizing dynamics. This retuning was partially maintained when the augmented reality feedback was abruptly removed. The compensatory changes in sensorimotor control improved tracking performance as well as control-theoretic measures of robustness, including reduced sensitivity to disturbances and improved phase margins.

Drosophila flying in augmented reality reveals the vision-based control autonomy of the optomotor response

For walking, swimming, and flying animals, the optomotor response is essential to stabilize gaze. How flexible is the optomotor response? Classic work in Drosophila has argued that flies adapt flight control under augmented visual feedback conditions during goal-directed bar fixation. However, whether the lower-level, reflexive optomotor response can similarly adapt to augmented visual feedback (partially autonomous) or not (autonomous) over long timescales is poorly understood. To address this question, we developed an augmented reality paradigm to study the vision-based control autonomy of the yaw optomotor response of flying fruit flies (Drosophila). Flies were placed in a flight simulator, which permitted free body rotation about the yaw axis. By feeding back body movements in real time to a visual display, we augmented and inverted visual feedback. Thus, this experimental paradigm caused a constant visual error between expected and actual visual feedback to study potential adaptive visuomotor control. By combining experiments with control theory, we demonstrate that the optomotor response is autonomous during augmented reality flight bouts of up to 30 min, which exceeds the reported learning epoch during bar fixation. Agreement between predictions from linear systems theory and experimental data supports the notion that the optomotor response is approximately linear and time invariant within our experimental assay. Even under positive visual feedback, which revealed the stability limit of flies in augmented reality, the optomotor response was autonomous. Our results support a hierarchical motor control architecture in flies with fast and autonomous reflexes at the bottom and more flexible behavior at higher levels.

The influence of saccades on yaw gaze stabilization in fly flight

In a way analogous to human vision, the fruit fly D. melanogaster and many other flying insects generate smooth and saccadic movements to stabilize and shift their gaze in flight, respectively. It has been hypothesized that this combination of continuous and discrete movements benefits both flight stability and performance, particularly at high frequencies or speeds. Here we develop a hybrid control system model to explore the effects of saccades on the yaw stabilization reflex of D. melanogaster. Inspired from experimental data, the model includes a first order plant, a Proportional-Integral (PI) continuous controller, and a saccadic reset system that fires based on the integrated error of the continuous controller. We explore the gain, delay and switching threshold parameter space to quantify the optimum regions for yaw stability and performance. We show that the addition of saccades to a continuous controller provides benefits to both stability and performance across a range of frequencies. Our model suggests that Drosophila operates near its optimal switching threshold for its experimental gain set. We also show that based on experimental data, D. melanogaster operates in a region that trades off performance and stability. This trade-off increases flight robustness to compensate for internal perturbations such as wing damage.

Flies adaptively control flight to compensate for added inertia

Animal locomotion is highly adaptive, displaying a large degree of flexibility, yet how this flexibility arises from the integration of mechanics and neural control remains elusive. For instance, animals require flexible strategies to maintain performance as changes in mass or inertia impact stability. Compensatory strategies to mechanical loading are especially critical for animals that rely on flight for survival. To shed light on the capacity and flexibility of flight neuromechanics to mechanical loading, we pushed the performance of fruit flies (Drosophila) near its limit and implemented a control theoretic framework. Flies with added inertia were placed inside a virtual reality arena which permitted free rotation about the vertical (yaw) axis. Adding inertia increased the fly's response time yet had little influence on overall gaze stabilization performance. Flies maintained stability following the addition of inertia by adaptively modulating both visuomotor gain and damping. By contrast, mathematical modelling predicted a significant decrease in gaze stabilization performance. Adding inertia altered saccades, however, flies compensated for the added inertia by increasing saccade torque. Taken together, in response to added inertia flies increase reaction time but maintain flight performance through adaptive neural control. Overall, adding inertia decreases closed-loop flight robustness. Our work highlights the flexibility and capacity of motor control in flight.

Long-timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

A deep learning analysis of Drosophila body kinematics during magnetically tethered flight

Flying Drosophila rely on their vision to detect visual objects and adjust their flight course. Despite their robust fixation on a dark, vertical bar, our understanding of the underlying visuomotor neural circuits remains limited, in part due to difficulties in analyzing detailed body kinematics in a sensitive behavioral assay. In this study, we observed the body kinematics of flying Drosophila using a magnetically tethered flight assay, in which flies are free to rotate around their yaw axis, enabling naturalistic visual and proprioceptive feedback. Additionally, we used deep learning-based video analyses to characterize the kinematics of multiple body parts in flying animals. By applying this pipeline of behavioral experiments and analyses, we characterized the detailed body kinematics during rapid flight turns (or saccades) in two different visual conditions: spontaneous flight saccades under static screen and bar-fixating saccades while tracking a rotating bar. We found that both types of saccades involved movements of multiple body parts and that the overall dynamics were comparable. Our study highlights the importance of sensitive behavioral assays and analysis tools for characterizing complex visual behaviors.

Proprioception gates visual object fixation in flying flies

Visual object tracking in animals as diverse as felines, frogs, and fish supports behaviors including predation, predator avoidance, and landscape navigation. Decades of experimental results show that a rigidly body-fixed tethered fly in a "virtual reality" visual flight simulator steers to follow the motion of a vertical bar, thereby "fixating" it on visual midline. This behavior likely reflects a desire to seek natural features such as plant stalks and has inspired algorithms for visual object tracking predicated on robust responses to bar velocity, particularly near visual midline. Using a modified flight simulator equipped with a magnetic pivot to allow frictionless turns about the yaw axis, we discovered that bar fixation as well as smooth steering responses to bar velocity are attenuated or eliminated in yaw-free conditions. Body-fixed Drosophila melanogaster respond to bar oscillation on a stationary ground with frequency-matched wing kinematics and fixate the bar on midline. Yaw-free flies respond to the same stimulus by ignoring the bar and maintaining their original heading. These differences are driven by proprioceptive signals, rather than visual signals, as artificially "clamping" a bar in the periphery of a yaw-free fly has no effect. When presented with a bar and ground oscillating at different frequencies, a yaw-free fly follows the frequency of the ground only, whereas a body-fixed fly robustly steers at the frequencies of both the bar and ground. Our findings support a model in which proprioceptive feedback promote active damping of high-gain optomotor responses to object motion.

Long timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Dehydrated Drosophila melanogaster track a water plume in tethered flight

Perception of sensory stimuli can be modulated by changes in internal state to drive contextually appropriate behavior. For example, dehydration is a threat to terrestrial animals, especially to Drosophila melanogaster due to their large surface area to volume ratio, particularly under the energy demands of flight. While hydrated D. melanogaster avoid water cues, while walking, dehydration leads to water-seeking behavior. We show that in tethered flight, hydrated flies ignore a water stimulus, whereas dehydrated flies track a water plume. Antennal occlusions eliminate odor and water plume tracking, whereas inactivation of moist sensing neurons in the antennae disrupts water tracking only upon starvation and dehydration. Elimination of the olfactory coreceptor eradicates odor tracking while leaving water-seeking behavior intact in dehydrated flies. Our results suggest that while similar hygrosensory receptors may be used for walking and in-flight hygrotaxis, the temporal dynamics of modulating the perception of water vary with behavioral state.

Flies trade off stability and performance via adaptive compensation to wing damage

Physical injury often impairs mobility, which can have dire consequences for survival in animals. Revealing mechanisms of robust biological intelligence to prevent system failure can provide critical insights into how complex brains generate adaptive movement and inspiration to design fault-tolerant robots. For flying animals, physical injury to a wing can have severe consequences, as flight is inherently unstable. Using a virtual reality flight arena, we studied how flying fruit flies compensate for damage to one wing. By combining experimental and mathematical methods, we show that flies compensate for wing damage by corrective wing movement modulated by closed-loop sensing and robust mechanics. Injured flies actively increase damping and, in doing so, modestly decrease flight performance but fly as stably as uninjured flies. Quantifying responses to injury can uncover the flexibility and robustness of biological systems while informing the development of bio-inspired fault-tolerant strategies.

Active anemosensing hypothesis: how flying insects could estimate ambient wind direction through sensory integration and active movement

Estimating the direction of ambient fluid flow is a crucial step during chemical plume tracking for flying and swimming animals. How animals accomplish this remains an open area of investigation. Recent calcium imaging with tethered flying Drosophila has shown that flies encode the angular direction of multiple sensory modalities in their central complex: orientation, apparent wind (or airspeed) direction and direction of motion. Here, we describe a general framework for how these three sensory modalities can be integrated over time to provide a continuous estimate of ambient wind direction. After validating our framework using a flying drone, we use simulations to show that ambient wind direction can be most accurately estimated with trajectories characterized by frequent, large magnitude turns. Furthermore, sensory measurements and estimates of their derivatives must be integrated over a period of time that incorporates at least one of these turns. Finally, we discuss approaches that insects might use to simplify the required computations, and present a list of testable predictions. Together, our results suggest that ambient flow estimation may be an important driver underlying the zigzagging manoeuvres characteristic of plume tracking animals' trajectories.

Impact of walking speed and motion adaptation on optokinetic nystagmus-like head movements in the blowfly Calliphora

The optokinetic nystagmus is a gaze-stabilizing mechanism reducing motion blur by rapid eye rotations against the direction of visual motion, followed by slower syndirectional eye movements minimizing retinal slip speed. Flies control their gaze through head turns controlled by neck motor neurons receiving input directly, or via descending neurons, from well-characterized directional-selective interneurons sensitive to visual wide-field motion. Locomotion increases the gain and speed sensitivity of these interneurons, while visual motion adaptation in walking animals has the opposite effects. To find out whether flies perform an optokinetic nystagmus, and how it may be affected by locomotion and visual motion adaptation, we recorded head movements of blowflies on a trackball stimulated by progressive and rotational visual motion. Flies flexibly responded to rotational stimuli with optokinetic nystagmus-like head movements, independent of their locomotor state. The temporal frequency tuning of these movements, though matching that of the upstream directional-selective interneurons, was only mildly modulated by walking speed or visual motion adaptation. Our results suggest flies flexibly control their gaze to compensate for rotational wide-field motion by a mechanism similar to an optokinetic nystagmus. Surprisingly, the mechanism is less state-dependent than the response properties of directional-selective interneurons providing input to the neck motor system.

High speed visual insect swarm tracker (Hi-VISTA) used to identify the effects of confinement on individual insect flight

Individual insects flying in crowded assemblies perform complex aerial maneuvers by sensing and feeding back neighbor measurements to small changes in their wing motions. To understand the individual feedback rules that permit these fast, adaptive behaviors in group flight, both experimental preparations inducing crowded flight and high-speed tracking systems capable of tracking both body motions and more subtle wing motion changes for multiple insects in simultaneous flight are needed. This measurement capability extends tracking beyond the previous focus on individual insects to multiple insects. This paper describes an experimental preparation that induces crowded insect flight in more naturalistic conditions (a laboratory-outdoor transition tunnel) and directly compares the resulting flight performance to traditional flight enclosures. Measurements are made possible via the introduction of a multi-agent high speed insect tracker called Hi-VISTA, which provides a capability to track wing and body motions of multiple insects using high speed cameras (9000-12 500 fps). Processing steps consist of automatic background identification, data association, hull reconstruction, segmentation, and feature measurement. To improve the biological relevance of laboratory experiments and develop a platform for interaction studies, this paper applies the Hi-VISTA measurement system toApis melliferaforagers habituated to transit flights through the transparent transition environment. Binary statistical analysis (Welch's t-test, Cohen's d effect size) of 95 flight trajectories is presented, quantifying the differences between flights in an unobstructed environment and in a confined tunnel volume. The results indicate that body pitch angle, heading rate, flapping frequency, and vertical speed (heave) are each affected by confinement, and other flight variables show minor or statistically insignificant changes. These results form a baseline as swarm tracking and analysis begins to isolate the effects of neighbors from environmental enclosures, and improve the connection of high speed insect laboratory experiments to outdoor field experiments.

A Nonlinear Observability Analysis of Ambient Wind Estimation with Uncalibrated Sensors, Inspired by Insect Neural Encoding

Estimating the direction of ambient fluid flow is key for many flying or swimming animals and robots, but can only be accomplished through indirect measurements and active control. Recent work with tethered flying insects indicates that their sensory representation of orientation, apparent wind, direction of movement, and control is represented by a 2-dimensional angular encoding in the central brain. This representation simplifies sensory integration by projecting the direction (but not scale) of measurements with different units onto a universal polar coordinate frame. To align these angular measurements with one another and the motor system does, however, require a calibration of angular gain and offset for each sensor. This calibration could change with time due to changes in the environment or physical structure. The circumstances under which small robots and animals with angular sensors and changing calibrations could self-calibrate and estimate the direction of ambient fluid flow while moving remains an open question. Here, a methodical nonlinear observability analysis is presented to address this. The analysis shows that it is mathematically feasible to continuously estimate flow direction and perform self-calibrations by adopting frequent changes in course (or active prevention thereof) and orientation, and requires fusion and temporal differentiation of three sensory measurements: apparent flow, orientation (or its derivative), and direction of motion (or its derivative). These conclusions are consistent with the zigzagging trajectories exhibited by many plume tracking organisms, suggesting that perhaps flow estimation is a secondary driver of their trajectory structure.

Insect vision: Head saccades to reset the view

Tracking a moving scene requires you to occasionally readjust your gaze as objects slip out of sight. A new study has found how fruit flies use head-turning strategies to reset their gaze and stabilize visual images during flight.

Mechanisms of punctuated vision in fly flight

To guide locomotion, animals control gaze via movements of their eyes, head, and/or body, but how the nervous system controls gaze during complex motor tasks remains elusive. In many animals, shifts in gaze consist of periods of smooth movement punctuated by rapid eye saccades. Notably, eye movements are constrained by anatomical limits, which requires resetting eye position. By studying tethered, flying fruit flies (Drosophila), we show that flies perform stereotyped head saccades to reset gaze, analogous to optokinetic nystagmus in primates. Head-reset saccades interrupted head smooth movement for as little as 50 ms-representing less than 5% of the total flight time-thereby enabling punctuated gaze stabilization. By revealing the passive mechanics of the neck joint, we show that head-reset saccades leverage the neck's natural elastic recoil, enabling mechanically assisted redirection of gaze. The consistent head orientation at saccade initiation, the influence of the head's angular position on saccade rate, the decrease in wing saccade frequency in head-fixed flies, and the decrease in head-reset saccade rate in flies with their head range of motion restricted together implicate proprioception as the primary trigger of head-reset saccades. Wing-reset saccades were influenced by head orientation, establishing a causal link between neck sensory signals and the execution of body saccades. Head-reset saccades were abolished when flies switched to a landing state, demonstrating that head movements are gated by behavioral state. We propose a control architecture for active vision systems with limits in sensor range of motion. VIDEO ABSTRACT.

Facilitation of neural responses to targets moving against optic flow

Target detection in visual clutter is a difficult computational task that insects, with their poor spatial resolution compound eyes and small brains, do successfully and with extremely short behavioral delays. We here show that the responses of target selective descending neurons are attenuated by background motion in the same direction as target motion but facilitated by background motion in the opposite direction. This finding is important for understanding how target pursuit can occur in tandem with gaze stabilization. Indeed, the neural facilitation would come into effect if the hoverfly is subjected to background motion in one direction but the target it is pursuing moves in the opposite direction and could therefore be used to override gaze stabilizing corrective turns.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer’s own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Confidence in predicted position error explains saccadic decisions during pursuit

A fundamental problem in motor control is the coordination of complementary movement types to achieve a common goal. As a common example, humans view moving objects through coordinated pursuit and saccadic eye movements. Pursuit is initiated and continuously controlled by retinal image velocity. During pursuit, eye position may lag behind the target. This can be compensated by the discrete execution of a catch-up saccade. The decision to trigger a saccade is influenced by both position and velocity errors, and the timing of saccades can be highly variable. The observed distributions of saccade frequency and trigger time remain poorly understood, and this decision process remains imprecisely quantified. Here, we propose a predictive, probabilistic model explaining the decision to trigger saccades during pursuit to foveate moving targets. In this model, expected position error and its associated uncertainty are predicted through Bayesian inference across noisy, delayed sensory observations (Kalman filtering). This probabilistic prediction is used to estimate the confidence that a saccade is needed (quantified through log-probability ratio), triggering a saccade upon accumulating to a fixed threshold. The model qualitatively explains behavioral observations on the frequency and trigger time distributions of saccades during pursuit over a range of target motion trajectories. Furthermore, this model makes novel predictions that saccade decisions are highly sensitive to uncertainty for small predicted position errors, but this influence diminishes as the magnitude of predicted position error increases. We suggest that this predictive, confidence-based decision-making strategy represents a fundamental principle for the probabilistic neural control of coordinated movements.NEW & NOTEWORTHY This is the first stochastic dynamical systems model of pursuit-saccade coordination accounting for noise and delays in the sensorimotor system. The model uses Bayesian inference to predictively estimate visual motion, triggering saccades when confidence in predicted position error accumulates to a threshold. This model explains saccade frequency and trigger time distributions across target trajectories and makes novel predictions about the influence of sensory uncertainty in saccade decisions during pursuit.

Hybrid visual control in fly flight: insights into gaze shift via saccades

Flies fly by alternating between periods of fixation and body saccades, analogous to how our own eyes move. Gaze fixation via smooth movement in fly flight has been studied extensively, but comparatively less is known about the mechanism by which flies trigger and control body saccades to shift their gaze. Why do flies implement a hybrid fixate-and-saccade locomotion strategy? Here we review recent developments that provide new insights into this question. We focus on the interplay between smooth movement and saccades, the trigger classes of saccades, and the timeline of saccade execution. We emphasize recent mechanistic advances in Drosophila, where genetic tools have enabled cellular circuit analysis at an unprecedented level in a flying insect. In addition, we review trade-offs in behavioral paradigms used to study saccades. Throughout we highlight exciting avenues for future research in the control of fly flight.

Tuning movement for sensing in an uncertain world

While animals track or search for targets, sensory organs make small unexplained movements on top of the primary task-related motions. While multiple theories for these movements exist-in that they support infotaxis, gain adaptation, spectral whitening, and high-pass filtering-predicted trajectories show poor fit to measured trajectories. We propose a new theory for these movements called energy-constrained proportional betting, where the probability of moving to a location is proportional to an expectation of how informative it will be balanced against the movement's predicted energetic cost. Trajectories generated in this way show good agreement with measured trajectories of fish tracking an object using electrosense, a mammal and an insect localizing an odor source, and a moth tracking a flower using vision. Our theory unifies the metabolic cost of motion with information theory. It predicts sense organ movements in animals and can prescribe sensor motion for robots to enhance performance.

Active vision shapes and coordinates flight motor responses in flies

Animals use active sensing to respond to sensory inputs and guide future motor decisions. In flight, flies generate a pattern of head and body movements to stabilize gaze. How the brain relays visual information to control head and body movements and how active head movements influence downstream motor control remains elusive. Using a control theoretic framework, we studied the optomotor gaze stabilization reflex in tethered flight and quantified how head movements stabilize visual motion and shape wing steering efforts in fruit flies (Drosophila). By shaping visual inputs, head movements increased the gain of wing steering responses and coordination between stimulus and wings, pointing to a tight coupling between head and wing movements. Head movements followed the visual stimulus in as little as 10 ms-a delay similar to the human vestibulo-ocular reflex-whereas wing steering responses lagged by more than 40 ms. This timing difference suggests a temporal order in the flow of visual information such that the head filters visual information eliciting downstream wing steering responses. Head fixation significantly decreased the mechanical power generated by the flight motor by reducing wingbeat frequency and overall thrust. By simulating an elementary motion detector array, we show that head movements shift the effective visual input dynamic range onto the sensitivity optimum of the motion vision pathway. Taken together, our results reveal a transformative influence of active vision on flight motor responses in flies. Our work provides a framework for understanding how to coordinate moving sensors on a moving body.

Non-canonical Receptive Field Properties and Neuromodulation of Feature-Detecting Neurons in Flies

Several fundamental aspects of motion vision circuitry are prevalent across flies and mice. Both taxa segregate ON and OFF signals. For any given spatial pattern, motion detectors in both taxa are tuned to speed, selective for one of four cardinal directions, and modulated by catecholamine neurotransmitters. These similarities represent conserved, canonical properties of the functional circuits and computational algorithms for motion vision. Less is known about feature detectors, including how receptive field properties differ from the motion pathway or whether they are under neuromodulatory control to impart functional plasticity for the detection of salient objects from a moving background. Here, we investigated 19 types of putative feature selective lobula columnar (LC) neurons in the optic lobe of the fruit fly Drosophila melanogaster to characterize divergent properties of feature selection. We identified LC12 and LC15 as feature detectors. LC15 encodes moving bars, whereas LC12 is selective for the motion of discrete objects, mostly independent of size. Neither is selective for contrast polarity, speed, or direction, highlighting key differences in the underlying algorithms for feature detection and motion vision. We show that the onset of background motion suppresses object responses by LC12 and LC15. Surprisingly, the application of octopamine, which is released during flight, reverses the suppressive influence of background motion, rendering both LCs able to track moving objects superimposed against background motion. Our results provide a comparative framework for the function and modulation of feature detectors and new insights into the underlying neuronal mechanisms involved in visual feature detection.

Coriolis and centrifugal forces drive haltere deformations and influence spike timing

The halteres of flies are mechanosensory organs that serve a crucial role in the control of agile flight, providing sensory input for rapid course corrections to perturbations. Derived from hind wings, halteres are actively flapped and are thus subject to a variety of inertial forces as the fly undergoes complex flight trajectories. Previous analyses of halteres modelled them as a point mass, showing that Coriolis forces lead to subtle deflections orthogonal to the plane of flapping. By design, these models could not consider the effects of force gradients associated with a mass distribution, nor could they reveal three-dimensional spatio-temporal patterns of strain that result from those forces. In addition, diversity in the geometry of halteres, such as shape and asymmetries, could not be simply modelled with a point mass on a massless rod. To study the effects of mass distributions and asymmetries, we examine the haltere subject to both flapping and body rotations using three-dimensional finite-element simulations. We focus on a set of simplified geometries, in which we vary the stalk and bulb shape. We find that haltere mass distribution gives rise to two unreported deformation modes: (i) halteres twist with a magnitude that strongly depends on stalk and bulb geometry and (ii) halteres with an asymmetric mass distribution experience out-of-plane bending due to centrifugal forces, independent of body rotation. Since local strains at the base of the haltere drive deformations of mechanosensory neurons, we combined measured neural encoding mechanisms with our structural analyses to predict the spatial and temporal patterns of neural activity. This activity depends on both the flapping and rotation dynamics, and we show how the timing of neural activity is a viable mechanism for rotation-rate encoding. Our results provide new insights in haltere dynamics and show the viability for timing-based encoding of fly body rotations by halteres.

Fly eyes are not still: a motion illusion in Drosophila flight supports parallel visual processing

Most animals shift gaze by a 'fixate and saccade' strategy, where the fixation phase stabilizes background motion. A logical prerequisite for robust detection and tracking of moving foreground objects, therefore, is to suppress the perception of background motion. In a virtual reality magnetic tether system enabling free yaw movement, Drosophila implemented a fixate and saccade strategy in the presence of a static panorama. When the spatial wavelength of a vertical grating was below the Nyquist wavelength of the compound eyes, flies drifted continuously and gaze could not be maintained at a single location. Because the drift occurs from a motionless stimulus - thus any perceived motion stimuli are generated by the fly itself - it is illusory, driven by perceptual aliasing. Notably, the drift speed was significantly faster than under a uniform panorama, suggesting perceptual enhancement as a result of aliasing. Under the same visual conditions in a rigid-tether paradigm, wing steering responses to the unresolvable static panorama were not distinguishable from those to a resolvable static pattern, suggesting visual aliasing is induced by ego motion. We hypothesized that obstructing the control of gaze fixation also disrupts detection and tracking of objects. Using the illusory motion stimulus, we show that magnetically tethered Drosophila track objects robustly in flight even when gaze is not fixated as flies continuously drift. Taken together, our study provides further support for parallel visual motion processing and reveals the critical influence of body motion on visuomotor processing. Motion illusions can reveal important shared principles of information processing across taxa.

Sensory processing by motoneurons: a numerical model for low-level flight control in flies

Rhythmic locomotor behaviour in animals requires exact timing of muscle activation within the locomotor cycle. In rapidly oscillating motor systems, conventional control strategies may be affected by neural delays, making these strategies inappropriate for precise timing control. In flies, wing control thus requires sensory processing within the peripheral nervous system, circumventing the central brain. The underlying mechanism, with which flies integrate graded depolarization of visual interneurons and spiking proprioceptive feedback for precise muscle activation, is under debate. Based on physiological parameters, we developed a numerical model of spike initiation in flight muscles of a blowfly. The simulated Hodgkin–Huxley neuron reproduces multiple experimental findings and explains on the cellular level how vision might control wing kinematics. Sensory processing by single motoneurons appears to be sufficient for control of muscle power during flight in flies and potentially other flying insects, reducing computational load on the central brain during body posture reflexes and manoeuvring flight.

Flies land upside down on a ceiling using rapid visually mediated rotational maneuvers

Flies and other insects routinely land upside down on a ceiling. These inverted landing maneuvers are among the most remarkable aerobatic feats, yet the full range of these behaviors and their underlying sensorimotor processes remain largely unknown. Here, we report that successful inverted landing in flies involves a serial sequence of well-coordinated behavioral modules, consisting of an initial upward acceleration followed by rapid body rotation and leg extension, before terminating with a leg-assisted body swing pivoted around legs firmly attached to the ceiling. Statistical analyses suggest that rotational maneuvers are triggered when flies' relative retinal expansion velocity reaches a threshold. Also, flies exhibit highly variable pitch and roll rates, which are strongly correlated to and likely mediated by multiple sensory cues. When flying with higher forward or lower upward velocities, flies decrease the pitch rate but increase the degree of leg-assisted swing, thereby leveraging the transfer of body linear momentum.

General Visual and Contingent Thermal Cues Interact to Elicit Attraction in Female Aedes aegypti Mosquitoes

Female Aedes aegypti mosquitoes use multiple sensory modalities to hunt human hosts and obtain a blood meal for egg production. Attractive cues include carbon dioxide (CO₂), a major component of exhaled breath [1, 2]; heat elevated above ambient temperature, signifying warm-blooded skin [3, 4]; and dark visual contrast [5, 6], proposed to bridge long-range olfactory and short-range thermal cues [7]. Any of these sensory cues in isolation is an incomplete signal of a human host, and so a mosquito must integrate multimodal sensory information before committing to approaching and biting a person [8]. Here, we study the interaction of visual cues, heat, and CO₂ to investigate the contributions of human-associated stimuli to host-seeking decisions. We show that tethered flying mosquitoes strongly orient toward dark visual contrast, regardless of CO₂ stimulation and internal host-seeking status. This suggests that attraction to visual contrast is general and not contingent on other host cues. In free-flight experiments with CO₂, adding a dark contrasting visual cue to a warmed surface enhanced attraction. Moderate warmth became more attractive to mosquitoes, and mosquitoes aggregated on the cue at all non-noxious temperatures. Gr3 mutants, unable to detect CO₂, were lured to the visual cue at ambient temperatures but fled and did not return when the surface was warmed to host-like temperatures. This suggests that attraction to thermal cues is contingent on the presence of the additional sensory cue CO₂. Our results illustrate that mosquitoes integrate general attractive visual stimuli with context-dependent thermal stimuli to seek promising sites for blood feeding.

Visuomotor strategies for object approach and aversion in Drosophila melanogaster

Animals classify stimuli to generate appropriate motor actions. In flight, Drosophila melanogaster classify equidistant large and small objects with categorically different behaviors: a tall object evokes approach whereas a small object elicits avoidance. We studied visuomotor behavior in rigidly and magnetically tethered D. melanogaster to reveal strategies that generate aversion to a small object. We discovered that small-object aversion in tethered flight is enabled by aversive saccades and smooth movement, which vary with the stimulus type. Aversive saccades in response to a short bar had different dynamics from approach saccades in response to a tall bar and the distribution of pre-saccade error angles was more stochastic for a short bar. Taken together, we show that aversive responses in D. melanogaster are driven in part by processes that elicit signed saccades with distinct dynamics and trigger mechanisms. Our work generates new hypotheses to study brain circuits that underlie classification of objects in D. melanogaster.

Object features and T4/T5 motion detectors modulate the dynamics of bar tracking by Drosophila

Visual objects can be discriminated by static spatial features such as luminance or dynamic features such as relative movement. Flies track a solid dark vertical bar moving on a bright background, a behavioral reaction so strong that for a rigidly tethered fly, the steering trajectory is phase advanced relative to the moving bar, apparently in anticipation of its future position. By contrast, flickering bars that generate no coherent motion or have a surface texture that moves in the direction opposite to the bar generate steering responses that lag behind the stimulus. It remains unclear how the spatial properties of a bar influence behavioral response dynamics. Here,  we show that a dark bar defined by its luminance contrast to the uniform background drives a co-directional steering response that is phase advanced relative to the response to a textured bar defined only by its motion relative to a stationary textured background. The textured bar drives an initial contra-directional turn and phase-locked tracking. The qualitatively distinct response dynamics could indicate parallel visual processing of a luminance versus motion-defined object. Calcium imaging shows that T4/T5 motion-detecting neurons are more responsive to a solid dark bar than a motion-defined bar. Genetically blocking T4/T5 neurons eliminates the phase-advanced co-directional response to the luminance-defined bar, leaving the orientation response largely intact. We conclude that T4/T5 neurons mediate a co-directional optomotor response to a luminance-defined bar, thereby driving phase-advanced wing kinematics, whereas separate unknown visual pathways elicit the contra-directional orientation response.

Multisensory Control of Orientation in Tethered Flying Drosophila

A longstanding goal of systems neuroscience is to quantitatively describe how the brain integrates sensory cues over time. Here, we develop a closed-loop orienting paradigm in Drosophila to study the algorithms by which cues from two modalities are integrated during ongoing behavior. We find that flies exhibit two behaviors when presented simultaneously with an attractive visual stripe and aversive wind cue. First, flies perform a turn sequence where they initially turn away from the wind but later turn back toward the stripe, suggesting dynamic sensory processing. Second, turns toward the stripe are slowed by the presence of competing wind, suggesting summation of turning drives. We develop a model in which signals from each modality are filtered in space and time to generate turn commands and then summed to produce ongoing orienting behavior. This computational framework correctly predicts behavioral dynamics for a range of stimulus intensities and spatial arrangements.

Fruit flies are multistable geniuses

Our sensory systems have evolved to provide us with information about the external world. Such information is useful only insofar as it leads to actions that enhance fitness, and thus, the link between sensation and action has been thoroughly studied in many species. In insects, for example, specific visual stimuli lead to highly stereotyped responses. In contrast, humans can exhibit a wide range of responses to the same stimulus, as occurs most notably in the phenomenon of multistable perception. On this basis, one might think that humans have a fundamentally different way of generating actions from sensory inputs, but Toepfer et al. show that flies show evidence of multistable perception as well. Specifically, when confronted with a sensory stimulus that can yield different motor responses, flies switch from one response to another with temporal dynamics that are similar to those of humans and other animals. This suggests that the mechanisms that give rise to the rich repertoire of sensory experience in humans have correlates in much simpler nervous systems.

Abolishment of Spontaneous Flight Turns in Visually Responsive Drosophila

Animals react rapidly to external stimuli, such as an approaching predator, but in other circumstances, they seem to act spontaneously, without any obvious external trigger. How do the neural processes mediating the execution of reflexive and spontaneous actions differ? We studied this question in tethered, flying Drosophila. We found that silencing a large but genetically defined set of non-motor neurons virtually eliminates spontaneous flight turns while preserving the tethered flies' ability to perform two types of visually evoked turns, demonstrating that, at least in flies, these two modes of action are almost completely dissociable.