Active sensing via movement shapes spatiotemporal patterns of sensory feedback

Intrinsic voltage fluctuations reveal a form of phase-amplitude coupling in electrosensory pyramidal neurons

The stochastic flickering of ion channels is known to cause ongoing membrane potential fluctuations in neurons. This channel noise is often considered negligible when compared with synaptic noise, yet it can shape the integrative properties of neurons. Here, in vitro recordings of electrosensory pyramidal neurons under synaptic blockade are characterized and shown to contain a nontrivial repertoire of dynamical features. Our analyses reveal an intrinsic noise structure that is much richer than what could be expected based on previous studies: we identify rapid, small-amplitude, shot noise-like events and we quantify how their rate and amplitude are modulated by slower, large-amplitude fluctuations. This cross-relation is evidence that, at the single-neuron level, membrane potential dynamics can exhibit a form of phase-amplitude coupling. We also investigate the appearance of fast, intermittent subthreshold oscillations and conclude that they are manifestation of stochastic linear dynamics, possibly with time-varying parameters. Our results, collectively, highlight that neurons can spontaneously display rich intrinsic behavior, which is likely to impact how they process synaptic input.NEW & NOTEWORTHY How do neurons behave in the absence of synaptic input? Can their intrinsic activity convey important information about how they function? Here, we provide evidence that the structure of intrinsic voltage noise in pyramidal neurons contains several nontrivial components, contrary to what is usually assumed. We show, for the first time, that a form of phase-amplitude coupling can exist in the spontaneous electrical activity of single neurons.

Moving in an Uncertain World: Robust and Adaptive Control of Locomotion from Organisms to Machine Intelligence

Whether walking, running, slithering, or flying, organisms display a remarkable ability to move through complex and uncertain environments. In particular, animals have evolved to cope with a host of uncertainties-both of internal and external origin-to maintain adequate performance in an ever-changing world. In this review, we present mathematical methods in engineering to highlight emerging principles of robust and adaptive control of organismal locomotion. Specifically, by drawing on the mathematical framework of control theory, we decompose the robust and adaptive hierarchical structure of locomotor control. We show how this decomposition along the robust-adaptive axis provides testable hypotheses to classify behavioral outcomes to perturbations. With a focus on studies in non-human animals, we contextualize recent findings along the robust-adaptive axis by emphasizing two broad classes of behaviors: (1) compensation to appendage loss and (2) image stabilization and fixation. Next, we attempt to map robust and adaptive control of locomotion across some animal groups and existing bio-inspired robots. Finally, we highlight exciting future directions and interdisciplinary collaborations that are needed to unravel principles of robust and adaptive locomotion.

Sensorimotor adaptation to destabilizing dynamics in weakly electric fish

Humans and other animals can readily learn to compensate for changes in the dynamics of movement. Such changes can result from an injury or changes in the weight of carried objects. These changes in dynamics can lead not only to reduced performance but also to dramatic instabilities. We evaluated the impacts of compensatory changes in control policies in relation to stability and robustness in Eigenmannia virescens, a species of weakly electric fish. We discovered that these fish retune their sensorimotor control system in response to experimentally generated destabilizing dynamics. Specifically, we used an augmented reality system to manipulate sensory feedback during an image stabilization task in which a fish maintained its position within a refuge. The augmented reality system measured the fish's movements in real time. These movements were passed through a high-pass filter and multiplied by a gain factor before being fed back to the refuge motion. We adjusted the gain factor to gradually destabilize the fish's sensorimotor loop. The fish retuned their sensorimotor control system to compensate for the experimentally induced destabilizing dynamics. This retuning was partially maintained when the augmented reality feedback was abruptly removed. The compensatory changes in sensorimotor control improved tracking performance as well as control-theoretic measures of robustness, including reduced sensitivity to disturbances and improved phase margins.

Autogenous cerebral processes: an invitation to look at the brain from inside out

While external stimulation can reliably trigger neuronal activity, cerebral processes can operate independently from the environment. In this study, we conceptualize autogenous cerebral processes (ACPs) as intrinsic operations of the brain that exist on multiple scales and can influence or shape stimulus responses, behavior, homeostasis, and the physiological state of an organism. We further propose that the field should consider exploring to what extent perception, arousal, behavior, or movement, as well as other cognitive functions previously investigated mainly regarding their stimulus-response dynamics, are ACP-driven.

Mormyrid fish as models for investigating sensory-motor integration: A behavioural perspective

Animals possess senses which gather information from their environment. They can tune into important aspects of this information and decide on the most appropriate response, requiring coordination of their sensory and motor systems. This interaction is bidirectional. Animals can actively shape their perception with self-driven motion, altering sensory flow to maximise the environmental information they are able to extract. Mormyrid fish are excellent candidates for studying sensory-motor interactions, because they possess a unique sensory system (the active electric sense) and exhibit notable behaviours that seem to be associated with electrosensing. This review will take a behavioural approach to unpicking this relationship, using active electrolocation as an example where body movements and sensing capabilities are highly related and can be assessed in tandem. Active electrolocation is the process where individuals will generate and detect low-voltage electric fields to locate and recognise nearby objects. We will focus on research in the mormyrid Gnathonemus petersii (G. petersii), given the extensive study of this species, particularly its object recognition abilities. By studying object detection and recognition, we can assess the potential benefits of self-driven movements to enhance selection of biologically relevant information. Finally, these findings are highly relevant to understanding the involvement of movement in shaping the sensory experience of animals that use other sensory modalities. Understanding the overlap between sensory and motor systems will give insight into how different species have become adapted to their environments.

Small-amplitude head oscillations result from a multimodal head stabilization reflex in hawkmoths

In flying insects, head stabilization is an essential reflex that helps to reduce motion blur during fast aerial manoeuvres. This reflex is multimodal and requires the integration of visual and antennal mechanosensory feedback in hawkmoths, each operating as a negative-feedback-control loop. As in any negative-feedback system, the head stabilization system possesses inherent oscillatory dynamics that depend on the rate at which the sensorimotor components of the reflex operate. Consistent with this expectation, we observed small-amplitude oscillations in the head motion (or head wobble) of the oleander hawkmoth, Daphnis nerii, which are accentuated when sensory feedback is aberrant. Here, we show that these oscillations emerge from the inherent dynamics of the multimodal reflex underlying gaze stabilization, and that the amplitude of head wobble is a function of both the visual feedback and antennal mechanosensory feedback from the Johnston's organs. Our data support the hypothesis that head wobble results from a multimodal, dynamically stabilized reflex loop that mediates head positioning.

Integration of visual and antennal mechanosensory feedback during head stabilization in hawkmoths

During flight maneuvers, insects exhibit compensatory head movements which are essential for stabilizing the visual field on their retina, reducing motion blur, and supporting visual self-motion estimation. In Diptera, such head movements are mediated via visual feedback from their compound eyes that detect retinal slip, as well as rapid mechanosensory feedback from their halteres - the modified hindwings that sense the angular rates of body rotations. Because non-Dipteran insects lack halteres, it is not known if mechanosensory feedback about body rotations plays any role in their head stabilization response. Diverse non-Dipteran insects are known to rely on visual and antennal mechanosensory feedback for flight control. In hawkmoths, for instance, reduction of antennal mechanosensory feedback severely compromises their ability to control flight. Similarly, when the head movements of freely flying moths are restricted, their flight ability is also severely impaired. The role of compensatory head movements as well as multimodal feedback in insect flight raises an interesting question: in insects that lack halteres, what sensory cues are required for head stabilization? Here, we show that in the nocturnal hawkmoth Daphnis nerii, compensatory head movements are mediated by combined visual and antennal mechanosensory feedback. We subjected tethered moths to open-loop body roll rotations under different lighting conditions, and measured their ability to maintain head angle in the presence or absence of antennal mechanosensory feedback. Our study suggests that head stabilization in moths is mediated primarily by visual feedback during roll movements at lower frequencies, whereas antennal mechanosensory feedback is required when roll occurs at higher frequency. These findings are consistent with the hypothesis that control of head angle results from a multimodal feedback loop that integrates both visual and antennal mechanosensory feedback, albeit at different latencies. At adequate light levels, visual feedback is sufficient for head stabilization primarily at low frequencies of body roll. However, under dark conditions, antennal mechanosensory feedback is essential for the control of head movements at high frequencies of body roll.

Cockroaches adjust body and appendages to traverse cluttered large obstacles

To traverse complex terrain, animals often transition between locomotor modes. It is well-known that locomotor transitions can be induced by switching in neural control circuits or driven by a need to minimize metabolic energetic cost. Recent work discovered that locomotor transitions in complex 3-D terrain cluttered with large obstacles can emerge from physical interaction with the environment controlled by the nervous system. For example, to traverse cluttered, stiff grass-like beams, the discoid cockroach often transitions from using a strenuous pitch mode pushing across to using a less strenuous roll mode rolling into and through the gaps, and this transition requires overcoming a potential energy barrier. Previous robotic physical modeling demonstrated that kinetic energy fluctuation of body oscillation from self-propulsion can help overcome the barrier and facilitate this transition. However, the animal was observed to transition even when the barrier still exceeded kinetic energy fluctuation. Here, we further studied whether and how the cockroach makes active adjustments to facilitate this transition to traverse cluttered beams. The animal repeatedly flexed its head and abdomen, reduced hind leg sprawl, and depressed one hind leg and elevated the other during the pitch-to-roll transition, which were absent when running on a flat ground. Using a refined potential energy landscape with additional degrees of freedom to model these adjustments, we found that head flexion did not substantially reduce the transition barrier, whereas leg sprawl reduction did so dramatically. We speculate that head flexion is for sensing the terrain to guide the transition via sensory feedback control.

Latency shortening with enhanced sparseness and responsiveness in V1 during active visual sensing

In natural vision, neuronal responses to visual stimuli occur due to self-initiated eye movements. Here, we compare single-unit activity in the primary visual cortex (V1) of non-human primates to flashed natural scenes (passive vision condition) to when they freely explore the images by self-initiated eye movements (active vision condition). Active vision enhances the number of neurons responding, and the response latencies become shorter and less variable across neurons. The increased responsiveness and shortened latency during active vision were not explained by increased visual contrast. While the neuronal activities in all layers of V1 show enhanced responsiveness and shortened latency, a significant increase in lifetime sparseness during active vision is observed only in the supragranular layer. These findings demonstrate that the neuronal responses become more distinct in active vision than passive vision, interpreted as consequences of top-down predictive mechanisms.

Salience of multisensory feedback regulates behavioral variability

Many animal behaviors are robust to dramatic variations in morphophysiological features, both across and within individuals. The control strategies that animals use to achieve such robust behavioral performances are not known. Recent evidence suggests that animals rely on sensory feedback rather than precise tuning of neural controllers for robust control. Here we examine the structure of sensory feedback, including multisensory feedback, for robust control of animal behavior. We re-examined two recent datasets of refuge tracking responses ofEigenmannia virescens, a species of weakly electric fish.Eigenmanniarely on both the visual and electrosensory cues to track the position of a moving refuge. The datasets include experiments that varied the strength of visual and electrosensory signals. Our analyses show that increasing the salience (perceptibility) of visual or electrosensory signals resulted in more robust and precise behavioral responses. Further, we find that robust performance was enhanced by multisensory integration of simultaneous visual and electrosensory cues. These findings suggest that engineers may achieve better system performance by improving the salience of multisensory feedback rather than solely focusing on precisely tuned controllers.

Tuning movement for sensing in an uncertain world

While animals track or search for targets, sensory organs make small unexplained movements on top of the primary task-related motions. While multiple theories for these movements exist-in that they support infotaxis, gain adaptation, spectral whitening, and high-pass filtering-predicted trajectories show poor fit to measured trajectories. We propose a new theory for these movements called energy-constrained proportional betting, where the probability of moving to a location is proportional to an expectation of how informative it will be balanced against the movement's predicted energetic cost. Trajectories generated in this way show good agreement with measured trajectories of fish tracking an object using electrosense, a mammal and an insect localizing an odor source, and a moth tracking a flower using vision. Our theory unifies the metabolic cost of motion with information theory. It predicts sense organ movements in animals and can prescribe sensor motion for robots to enhance performance.

Biologically Inspired Catheter for Endovascular Sensing and Navigation

Minimally invasive treatment of vascular disease demands dynamic navigation through complex blood vessel pathways and accurate placement of an interventional device, which has resulted in increased reliance on fluoroscopic guidance and commensurate radiation exposure to the patient and staff. Here we introduce a guidance system inspired by electric fish that incorporates measurements from a newly designed electrogenic sensory catheter with preoperative imaging to provide continuous feedback to guide vascular procedures without additional contrast injection, radiation, image registration, or external tracking. Electrodes near the catheter tip simultaneously create a weak electric field and measure the impedance, which changes with the internal geometry of the vessel as the catheter advances through the vasculature. The impedance time series is then mapped to a preoperative vessel model to determine the relative position of the catheter within the vessel tree. We present navigation in a synthetic vessel tree based on our mapping technique. Experiments in a porcine model demonstrated the sensor's ability to detect cross-sectional area variation in vivo. These initial results demonstrate the capability and potential of this novel bioimpedance-based navigation technology as a non-fluoroscopic technique to augment existing imaging methods.

Active Sensing: Constancy Requires Change

We constantly generate movements in order to enhance our ability to perceive the external environment. New research on electric fish has used augmented reality to demonstrate that animals dynamically regulate their movements to maintain variability in their sensory input.

Variability in locomotor dynamics reveals the critical role of feedback in task control

Animals vary considerably in size, shape, and physiological features across individuals, but yet achieve remarkably similar behavioral performances. We examined how animals compensate for morphophysiological variation by measuring the system dynamics of individual knifefish (Eigenmannia virescens) in a refuge tracking task. Kinematic measurements of Eigenmannia were used to generate individualized estimates of each fish’s locomotor plant and controller, revealing substantial variability between fish. To test the impact of this variability on behavioral performance, these models were used to perform simulated ‘brain transplants’—computationally swapping controllers and plants between individuals. We found that simulated closed-loop performance was robust to mismatch between plant and controller. This suggests that animals rely on feedback rather than precisely tuned neural controllers to compensate for morphophysiological variability.

People come in different shapes and sizes, but most will perform similarly well if asked to complete a task requiring fine manual dexterity – such as holding a pen or picking up a single grape. How can different individuals, with different sized hands and muscles, produce such similar movements? One explanation is that an individual’s brain and nervous system become precisely tuned to mechanics of the body’s muscles and skeleton. An alternative explanation is that brain and nervous system use a more “robust” control policy that can compensate for differences in the body by relying on feedback from the senses to guide the movements.

To distinguish between these two explanations, Uyanik et al. turned to weakly electric freshwater fish known as glass knifefish. These fish seek refuge within root systems, reed grass and among other objects in the water. They swim backwards and forwards to stay hidden despite constantly changing currents. Each fish shuttles back and forth by moving a long ribbon-like fin on the underside of its body. Uyanik et al. measured the movements of the ribbon fin under controlled conditions in the laboratory, and then used the data to create computer models of the brain and body of each fish. The models of each fish’s brain and body were quite different.

To study how the brain interacts with the body, Uyanik et al. then conducted experiments reminiscent of those described in the story of Frankenstein and transplanted the brain from each computer model into the body of different model fish. These “brain swaps” had almost no effect on the model’s simulated swimming behavior. Instead, these “Frankenfish” used sensory feedback to compensate for any mismatch between their brain and body.

This suggests that, for some behaviors, an animal’s brain does not need to be precisely tuned to the specific characteristics of its body. Instead, robust control of movement relies on many seemingly redundant systems that provide sensory feedback. This has implications for the field of robotics. It further suggests that when designing robots, engineers should prioritize enabling the robots to use sensory feedback to cope with unexpected events, a well-known idea in control engineering.

Novel Functions of Feedback in Electrosensory Processing

Environmental signals act as input and are processed across successive stages in the brain to generate a meaningful behavioral output. However, a ubiquitous observation is that descending feedback projections from more central to more peripheral brain areas vastly outnumber ascending feedforward projections. Such projections generally act to modify how sensory neurons respond to afferent signals. Recent studies in the electrosensory system of weakly electric fish have revealed novel functions for feedback pathways in that their transformation of the afferent input generates neural firing rate responses to sensory signals mediating perception and behavior. In this review, we focus on summarizing these novel and recently uncovered functions and put them into context by describing the more "classical" functions of feedback in the electrosensory system. We further highlight the parallels between the electrosensory system and other systems as well as outline interesting future directions.

The Cone Opsin Repertoire of Osteoglossomorph Fishes: Gene Loss in Mormyrid Electric Fish and a Long Wavelength-Sensitive Cone Opsin That Survived 3R

Vertebrates have four classes of cone opsin genes derived from two rounds of genome duplication. These are short wavelength sensitive 1(SWS1), short wavelength sensitive 2(SWS2), medium wavelength sensitive (RH2), and long wavelength sensitive (LWS). Teleosts had another genome duplication at their origin and it is believed that only one of each cone opsin survived the ancestral teleost duplication event. We tested this by examining the retinal cones of a basal teleost group, the osteoglossomorphs. Surprisingly, this lineage has lost the typical vertebrate green-sensitive RH2 opsin gene and, instead, has a duplicate of the LWS opsin that is green sensitive. This parallels the situation in mammalian evolution in which the RH2 opsin gene was lost in basal mammals and a green-sensitive opsin re-evolved in Old World, and independently in some New World, primates from an LWS opsin gene. Another group of fish, the characins, possess green-sensitive LWS cones. Phylogenetic analysis shows that the evolution of green-sensitive LWS opsins in these two teleost groups derives from a common ancestral LWS opsin that acquired green sensitivity. Additionally, the nocturnally active African weakly electric fish (Mormyroideae), which are osteoglossomorphs, show a loss of the SWS1 opsin gene. In comparison with the independently evolved nocturnally active South American weakly electric fish (Gymnotiformes) with a functionally monochromatic LWS opsin cone retina, the presence of SWS2, LWS, and LWS2 cone opsins in mormyrids suggests the possibility of color vision.

Timing Is of the Essence: Improvement in Perception During Active Sensing

Active sensing refers to the concept of animals perceiving their environment while involving self-initiated motor acts. As a consequence of these motor acts, this activity produces direct and timely changes in the sensory surface. Is the brain able to take advantage of the precise time-locking that occurs during active sensing? Is the intrinsic predictability present during active sensing, impacting the sensory processes? We conjecture that if stimuli presentation is evoked by a self-initiated motor act, sensory discrimination and timing accuracy would improve. We studied this phenomenon when rats had to locate the position of a brief light stimulus, either when it was elicited by a warning light [passive condition (PC)] or when it was generated by a lever press [active condition (AC)]. We found that during the PC, rats had 66% of correct responses, vs. a significantly higher 77% of correct responses in AC. Furthermore, reaction times reduced from 1,181 ms during AC to 816 ms during PC For the latter condition, the probability of detecting the side of the light stimulus was negatively correlated with the time lag between the motor act and the evoked light and with a 38% reduction on performance per second of delay. These experiment shows that the mechanism that underlies sensory improvement during active behaviors have a constrained time dynamic, where the peak performances occur during the motor act, decreasing proportionally to the lag between the motor act and the stimulus presentation. This result is consistent with the evidence already found in humans, of a precise time dynamic of the improvement of sensory acuity after a motor act and reveals an equivalent process in rodents. Our results support the idea that perception and action are precisely coordinated in the brain.

Sensory Cues Modulate Smooth Pursuit and Active Sensing Movements

Animals routinely use autogenous movement to regulate the information encoded by their sensory systems. Weakly electric fish use fore-aft movements to regulate visual and electrosensory feedback as they maintain position within a moving refuge. During refuge tracking, fish produce two categories of movements: smooth pursuit that is approximately linear in its relation to the movement of the refuge and ancillary active sensing movements that are nonlinear. We identified four categories of nonlinear movements which we termed scanning, wiggle, drift, and reset. To examine the relations between sensory cues and production of both linear smooth pursuit and nonlinear active sensing movements, we altered visual and electrosensory cues for refuge tracking and measured the fore-aft movements of the fish. Specifically, we altered the length and structure of the refuge and performed experiments with light and in complete darkness. Linear measures of tracking performance were better for shorter refuges (less than a body length) than longer ones (>1.5 body lengths). The magnitude of nonlinear active sensing movements was strongly modulated by light cues but also increased as a function of both longer refuge length and decreased features. Specifically, fish shifted swimming movements from smooth pursuit to scanning when tracking in dark conditions. Finally, fish appear to use nonlinear movements as an alternate tracking strategy in longer refuges: the fish may use more drifts and resets to avoid exiting the ends of the refuge.

Closed-Loop Control of Active Sensing Movements Regulates Sensory Slip

Active sensing involves the production of motor signals for the purpose of acquiring sensory information [1-3]. The most common form of active sensing, found across animal taxa and behaviors, involves the generation of movements-e.g., whisking [4-6], touching [7, 8], sniffing [9, 10], and eye movements [11]. Active sensing movements profoundly affect the information carried by sensory feedback pathways [12-15] and are modulated by both top-down goals (e.g., measuring weight versus texture [1, 16]) and bottom-up stimuli (e.g., lights on or off [12]), but it remains unclear whether and how these movements are controlled in relation to the ongoing feedback they generate. To investigate the control of movements for active sensing, we created an experimental apparatus for freely swimming weakly electric fish, Eigenmannia virescens, that modulates the gain of reafferent feedback by adjusting the position of a refuge based on real-time videographic measurements of fish position. We discovered that fish robustly regulate sensory slip via closed-loop control of active sensing movements. Specifically, as fish performed the task of maintaining position inside the refuge [17-22], they dramatically up- or downregulated fore-aft active sensing movements in relation to a 4-fold change of experimentally modulated reafferent gain. These changes in swimming movements served to maintain a constant magnitude of sensory slip. The magnitude of sensory slip depended on the presence or absence of visual cues. These results indicate that fish use two controllers: one that controls the acquisition of information by regulating feedback from active sensing movements and another that maintains position in the refuge, a control structure that may be ubiquitous in animals [23, 24].

Application of reduced sensor movement sequences as a precursor for search area partitioning and a selection of discrete EEV contour-ring fragments for active electrolocation

In addition to their visual sense, weakly electric fish use active electrolocation to detect and analyse objects in their nearby environment. Their ability to generate and sense electric fields combined with scanning-like swimming movements are intended to extract further parameters like the size, shape and material properties of objects. Inspired by this biological example, this work introduces an application for active electrolocation based on reduced sensor movement sequences as presented in Wolf-Homeyer et al (2016 Bioinspir. Biomim. 11 055002). Initially, the application is conducted with a simulated receptor-system consisting of an emitter-dipole and an orthogonally arranged pair of sensor-electrodes. Close inspection of a minimal set of scanning movements allows the exclusion of sectors of the general search area early in the proposed localization algorithm (search area partitioning). Furthermore, the proposed algorithm is based on an analytical representation of the electric field and of the so-called EEV (ensemble of electrosensory viewpoints) (Solberg et al 2008 Int. J. Robot. Res. 27 529-48) rather than using computationally expensive FEM simulations, rendering it suitable for embedded computer systems. Two-dimensional discrete EEV contour-ring points (CRPs) of desired accuracy are extracted. In the core of the localization algorithm, fragments of the EEV are selected from valid sectors of the search area, which generates sets of CRPs, one for each sensor-emitter position/orientation. These sets are investigated by means of a nearness metric to find points in different sets which correspond to each other in order to estimate the object position. Two resultant scanning strategies/localization algorithms are introduced.

Motion parallax in electric sensing

A crucial step in forming spatial representations of the environment involves the estimation of relative distance. Active sampling through specific movements is considered essential for optimizing the sensory flow that enables the extraction of distance cues. However, in electric sensing, direct evidence for the generation and exploitation of sensory flow is lacking. Weakly electric fish rely on a self-generated electric field to navigate and capture prey in the dark. This electric sense provides a blurred representation of the environment, making the exquisite sensory abilities of electric fish enigmatic. Stereotyped back-and-forth swimming patterns reminiscent of visual peering movements are suggestive of the active generation of sensory flow, but how motion contributes to the disambiguation of the electrosensory world remains unclear. Here, we show that a dipole-like electric field geometry coupled to motion provides the physical basis for a nonvisual parallax. We then show in a behavioral assay that this cue is used for electrosensory distance perception across phylogenetically distant taxa of weakly electric fish. Notably, these species electrically sample the environment in temporally distinct ways (using discrete pulses or quasisinusoidal waves), suggesting a ubiquitous role for parallax in electric sensing. Our results demonstrate that electrosensory information is extracted from sensory flow and used in a behaviorally relevant context. A better understanding of motion-based electric sensing will provide insight into the sensorimotor coordination required for active sensing in general and may lead to improved electric field-based imaging applications in a variety of contexts.

Dynamic modulation of visual and electrosensory gains for locomotor control

Animal nervous systems resolve sensory conflict for the control of movement. For example, the glass knifefish, Eigenmannia virescens, relies on visual and electrosensory feedback as it swims to maintain position within a moving refuge. To study how signals from these two parallel sensory streams are used in refuge tracking, we constructed a novel augmented reality apparatus that enables the independent manipulation of visual and electrosensory cues to freely swimming fish (n = 5). We evaluated the linearity of multisensory integration, the change to the relative perceptual weights given to vision and electrosense in relation to sensory salience, and the effect of the magnitude of sensory conflict on sensorimotor gain. First, we found that tracking behaviour obeys superposition of the sensory inputs, suggesting linear sensorimotor integration. In addition, fish rely more on vision when electrosensory salience is reduced, suggesting that fish dynamically alter sensorimotor gains in a manner consistent with Bayesian integration. However, the magnitude of sensory conflict did not significantly affect sensorimotor gain. These studies lay the theoretical and experimental groundwork for future work investigating multisensory control of locomotion.

Electrosensory processing in Apteronotus albifrons: implications for general and specific neural coding strategies across wave-type weakly electric fish species

Understanding how the brain processes sensory input to generate behavior remains an important problem in neuroscience. Towards this end, it is useful to compare results obtained across multiple species to gain understanding as to the general principles of neural coding. Here we investigated hindbrain pyramidal cell activity in the weakly electric fish Apteronotus albifrons We found strong heterogeneities when looking at baseline activity. Additionally, ON- and OFF-type cells responded to increases and decreases of sinusoidal and noise stimuli, respectively. While both cell types displayed band-pass tuning, OFF-type cells were more broadly tuned than their ON-type counterparts. The observed heterogeneities in baseline activity as well as the greater broadband tuning of OFF-type cells were both similar to those previously reported in other weakly electric fish species, suggesting that they constitute general features of sensory processing. However, we found that peak tuning occurred at frequencies ∼15 Hz in A. albifrons, which is much lower than values reported in the closely related species Apteronotus leptorhynchus and the more distantly related species Eigenmannia virescens In response to stimuli with time-varying amplitude (i.e., envelope), ON- and OFF-type cells displayed similar high-pass tuning curves characteristic of fractional differentiation and possibly indicate optimized coding. These tuning curves were qualitatively similar to those of pyramidal cells in the closely related species A. leptorhynchus In conclusion, comparison between our and previous results reveals general and species-specific neural coding strategies. We hypothesize that differences in coding strategies, when observed, result from different stimulus distributions in the natural/social environment.

Weakly electric fish learn both visual and electrosensory cues in a multisensory object discrimination task

Weakly electric fish use electrosensory, visual, olfactory and lateral line information to guide foraging and navigation behaviors. In many cases they preferentially rely on electrosensory cues. Do fish also memorize non-electrosensory cues? Here, we trained individuals of gymnotiform weakly electric fish Apteronotus albifrons in an object discrimination task. Objects were combinations of differently conductive materials covered with differently colored cotton hoods. By setting visual and electrosensory cues in conflict we analyzed the sensory hierarchy among the electrosensory and the visual sense in object discrimination. Our experiments show that: (i) black ghost knifefish can be trained to solve discrimination tasks similarly to the mormyrid fish; (ii) fish preferentially rely on electrosensory cues for object discrimination; (iii) despite the dominance of the electrosense they still learn the visual cue and use it when electrosensory information is not available; (iv) fish prefer the trained combination of rewarded cues over combinations that match only in a single feature and also memorize the non-rewarded combination.

Modeling latency code processing in the electric sense: from the biological template to its VLSI implementation

Understanding the coding of sensory information under the temporal constraints of natural behavior is not yet well resolved. There is a growing consensus that spike timing or latency coding can maximally exploit the timing of neural events to make fast computing elements and that such mechanisms are essential to information processing functions in the brain. The electric sense of mormyrid fish provides a convenient biological model where this coding scheme can be studied. The sensory input is a physically ordered spatial pattern of current densities, which is coded in the precise timing of primary afferent spikes. The neural circuits of the processing pathway are well known and the system exhibits the best known illustration of corollary discharge, which provides the reference to decoding the sensory afferent latency pattern. A theoretical model has been constructed from available electrophysiological and neuroanatomical data to integrate the principal traits of the neural processing structure and to study sensory interaction with motor-command-driven corollary discharge signals. This has been used to explore neural coding strategies at successive stages in the network and to examine the simulated network capacity to reproduce output neuron responses. The model shows that the network has the ability to resolve primary afferent spike timing differences in the sub-millisecond range, and that this depends on the coincidence of sensory and corollary discharge-driven gating signals. In the integrative and output stages of the network, corollary discharge sets up a proactive background filter, providing temporally structured excitation and inhibition within the network whose balance is then modulated locally by sensory input. This complements the initial gating mechanism and contributes to amplification of the input pattern of latencies, conferring network hyperacuity. These mechanisms give the system a robust capacity to extract behaviorally meaningful features of the electric image with high sensitivity over a broad working range. Since the network largely depends on spike timing, we finally discuss its suitability for implementation in robotic applications based on neuromorphic hardware.

Adaptation of sensor morphology: an integrative view of perception from biologically inspired robotics perspective

Sensor morphology, the morphology of a sensing mechanism which plays a role of shaping the desired response from physical stimuli from surroundings to generate signals usable as sensory information, is one of the key common aspects of sensing processes. This paper presents a structured review of researches on bioinspired sensor morphology implemented in robotic systems, and discusses the fundamental design principles. Based on literature review, we propose two key arguments: first, owing to its synthetic nature, biologically inspired robotics approach is a unique and powerful methodology to understand the role of sensor morphology and how it can evolve and adapt to its task and environment. Second, a consideration of an integrative view of perception by looking into multidisciplinary and overarching mechanisms of sensor morphology adaptation across biology and engineering enables us to extract relevant design principles that are important to extend our understanding of the unfinished concepts in sensing and perception.

Burst Firing in the Electrosensory System of Gymnotiform Weakly Electric Fish: Mechanisms and Functional Roles

Neurons across sensory systems and organisms often display complex patterns of action potentials in response to sensory input. One example of such a pattern is the tendency of neurons to fire packets of action potentials (i.e., a burst) followed by quiescence. While it is well known that multiple mechanisms can generate bursts of action potentials at both the single-neuron and the network level, the functional role of burst firing in sensory processing is not so well understood to date. Here we provide a comprehensive review of the known mechanisms and functions of burst firing in processing of electrosensory stimuli in gymnotiform weakly electric fish. We also present new evidence from existing data showing that bursts and isolated spikes provide distinct information about stimulus variance. It is likely that these functional roles will be generally applicable to other systems and species.

Active sensing associated with spatial learning reveals memory-based attention in an electric fish

Active sensing behaviors reveal what an animal is attending to and how it changes with learning. Gymnotus sp, a gymnotiform weakly electric fish, generates an electric organ discharge (EOD) as discrete pulses to actively sense its surroundings. We monitored freely behaving gymnotid fish in a large dark "maze" and extracted their trajectories and EOD pulse pattern and rate while they learned to find food with electrically detectable landmarks as cues. After training, they more rapidly found food using shorter, more stereotyped trajectories and spent more time near the food location. We observed three forms of active sensing: sustained high EOD rates per unit distance (sampling density), transient large increases in EOD rate (E-scans) and stereotyped scanning movements (B-scans) were initially strong at landmarks and food, but, after learning, intensified only at the food location. During probe (no food) trials, after learning, the fish's search area and intense active sampling was still centered on the missing food location, but now also increased near landmarks. We hypothesize that active sensing is a behavioral manifestation of attention and essential for spatial learning; the fish use spatial memory of landmarks and path integration to reach the expected food location and confine their attention to this region.

The neural dynamics of sensory focus

Coordinated sensory and motor system activity leads to efficient localization behaviours; but what neural dynamics enable object tracking and what are the underlying coding principles? Here we show that optimized distance estimation from motion-sensitive neurons underlies object tracking performance in weakly electric fish. First, a relationship is presented for determining the distance that maximizes the Fisher information of a neuron's response to object motion. When applied to our data, the theory correctly predicts the distance chosen by an electric fish engaged in a tracking behaviour, which is associated with a bifurcation between tonic and burst modes of spiking. Although object distance, size and velocity alter the neural response, the location of the Fisher information maximum remains invariant, demonstrating that the circuitry must actively adapt to maintain ‘focus' during relative motion.

Animals tracking objects can adapt their movements to optimise sensory coding. Using fish that sense objects as perturbations to an electric field, Clarke et al. reveal that the optimal object distance maintained by the fish is encoded within the firing properties of electrosensory neurons.

Finding and identifying simple objects underwater with active electrosense

Active electrosense is used by some fish for the sensing of nearby objects by means of the perturbations the objects induce in a self-generated electric field. As with echolocation (sensing via perturbations of an emitted acoustic field) active electrosense is particularly useful in environments where darkness, clutter or turbidity makes vision ineffective. Work on engineered variants of active electrosense is motivated by the need for sensors in underwater systems that function well at short range and where vision-based approaches can be problematic, as well as to aid in understanding the computational principles of biological active electrosense. Prior work in robotic active electrosense has focused on tracking and localization of spherical objects. In this study, we present an algorithm for estimating the size, shape, orientation, and location of ellipsoidal objects, along with experimental results. The algorithm is implemented in a robotic active electrosense system whose basic approach is similar to biological active electrosense systems, including the use of movement as part of sensing. At a range up to ≈20 cm, or about half the length of the robot, the algorithm localizes spheroids that are one-tenth the length of the robot with accuracy of better than 1 cm for position and 5° in orientation. The algorithm estimates object size and length-to-width ratio with an accuracy of around 10%.

The Musculature That Drives Active Touch by Vibrissae and Nose in Mice

Coordinated action of facial muscles during whisking, sniffing, and touching objects is an important component of active sensing in rodents. Accumulating evidence suggests that the anatomical schemes that underlie active sensing are similar across the majority of whisking rodents. Intriguingly, however, muscle architecture in the mystacial pad of the mouse was reported to be different, possessing only one extrinsic vibrissa protracting muscle (M. nasalis) in the rostral part of the snout. In this study, the organization of the muscles that move the nose and the mystacial vibrissae in mice was re-examined and compared with that reported previously in other rodents. We found that muscle distribution within the mystacial pad and around the tip of the nose in mice is isomorphic with that found in other whisking rodents. In particular, in the rostral part of the mouse snout, we describe both protractors and retractors of the vibrissae. Nose movements are controlled by the M. dilator nasi and five subunits of the M. nasolabialis profundus, with involvement of the nasal cartilaginous skeleton as a mediator in the muscular effort translation.

Feedback control as a framework for understanding tradeoffs in biology

Control theory arose from a need to control synthetic systems. From regulating steam engines to tuning radios to devices capable of autonomous movement, it provided a formal mathematical basis for understanding the role of feedback in the stability (or change) of dynamical systems. It provides a framework for understanding any system with regulation via feedback, including biological ones such as regulatory gene networks, cellular metabolic systems, sensorimotor dynamics of moving animals, and even ecological or evolutionary dynamics of organisms and populations. Here, we focus on four case studies of the sensorimotor dynamics of animals, each of which involves the application of principles from control theory to probe stability and feedback in an organism's response to perturbations. We use examples from aquatic (two behaviors performed by electric fish), terrestrial (following of walls by cockroaches), and aerial environments (flight control by moths) to highlight how one can use control theory to understand the way feedback mechanisms interact with the physical dynamics of animals to determine their stability and response to sensory inputs and perturbations. Each case study is cast as a control problem with sensory input, neural processing, and motor dynamics, the output of which feeds back to the sensory inputs. Collectively, the interaction of these systems in a closed loop determines the behavior of the entire system.

Motor patterns during active electrosensory acquisition

Motor patterns displayed during active electrosensory acquisition of information seem to be an essential part of a sensory strategy by which weakly electric fish actively generate and shape sensory flow. These active sensing strategies are expected to adaptively optimize ongoing behavior with respect to either motor efficiency or sensory information gained. The tight link between the motor domain and sensory perception in active electrolocation make weakly electric fish like Gnathonemus petersii an ideal system for studying sensory-motor interactions in the form of active sensing strategies. Analyzing the movements and electric signals of solitary fish during unrestrained exploration of objects in the dark, we here present the first formal quantification of motor patterns used by fish during electrolocation. Based on a cluster analysis of the kinematic values we categorized the basic units of motion. These were then analyzed for their associative grouping to identify and extract short coherent chains of behavior. This enabled the description of sensory behavior on different levels of complexity: from single movements, over short behaviors to more complex behavioral sequences during which the kinematics alter between different behaviors. We present detailed data for three classified patterns and provide evidence that these can be considered as motor components of active sensing strategies. In accordance with the idea of active sensing strategies, we found categorical motor patterns to be modified by the sensory context. In addition these motor patterns were linked with changes in the temporal sampling in form of differing electric organ discharge frequencies and differing spatial distributions. The ability to detect such strategies quantitatively will allow future research to investigate the impact of such behaviors on sensing.

Active sensing system with in situ adjustable sensor morphology

Background

Despite the widespread use of sensors in engineering systems like robots and automation systems, the common paradigm is to have fixed sensor morphology tailored to fulfill a specific application. On the other hand, robotic systems are expected to operate in ever more uncertain environments. In order to cope with the challenge, it is worthy of note that biological systems show the importance of suitable sensor morphology and active sensing capability to handle different kinds of sensing tasks with particular requirements.

Methodology

This paper presents a robotics active sensing system which is able to adjust its sensor morphology in situ in order to sense different physical quantities with desirable sensing characteristics. The approach taken is to use thermoplastic adhesive material, i.e. Hot Melt Adhesive (HMA). It will be shown that the thermoplastic and thermoadhesive nature of HMA enables the system to repeatedly fabricate, attach and detach mechanical structures with a variety of shape and size to the robot end effector for sensing purposes. Via active sensing capability, the robotic system utilizes the structure to physically probe an unknown target object with suitable motion and transduce the arising physical stimuli into information usable by a camera as its only built-in sensor.

Conclusions/Significance

The efficacy of the proposed system is verified based on two results. Firstly, it is confirmed that suitable sensor morphology and active sensing capability enables the system to sense different physical quantities, i.e. softness and temperature, with desirable sensing characteristics. Secondly, given tasks of discriminating two visually indistinguishable objects with respect to softness and temperature, it is confirmed that the proposed robotic system is able to autonomously accomplish them. The way the results motivate new research directions which focus on in situ adjustment of sensor morphology will also be discussed.

Statistics of the electrosensory input in the freely swimming weakly electric fish Apteronotus leptorhynchus

The neural computations underlying sensory-guided behaviors can best be understood in view of the sensory stimuli to be processed under natural conditions. This input is often actively shaped by the movements of the animal and its sensory receptors. Little is known about natural sensory scene statistics taking into account the concomitant movement of sensory receptors in freely moving animals. South American weakly electric fish use a self-generated quasi-sinusoidal electric field for electrolocation and electrocommunication. Thousands of cutaneous electroreceptors detect changes in the transdermal potential (TDP) as the fish interact with conspecifics and the environment. Despite substantial knowledge about the circuitry and physiology of the electrosensory system, the statistical properties of the electrosensory input evoked by natural swimming movements have never been measured directly. Using underwater wireless telemetry, we recorded the TDP of Apteronotus leptorhynchus as they swam freely by themselves and during interaction with a conspecific. Swimming movements caused low-frequency TDP amplitude modulations (AMs). Interacting with a conspecific caused additional AMs around the difference frequency of their electric fields, with the amplitude of the AMs (envelope) varying at low frequencies due to mutual movements. Both AMs and envelopes showed a power-law relationship with frequency, indicating spectral scale invariance. Combining a computational model of the electric field with video tracking of movements, we show that specific swimming patterns cause characteristic spatiotemporal sensory input correlations that contain information that may be used by the brain to guide behavior.

Sensory flow shaped by active sensing: sensorimotor strategies in electric fish

Goal-directed behavior in most cases is composed of a sequential order of elementary motor patterns shaped by sensorimotor contingencies. The sensory information acquired thus is structured in both space and time. Here we review the role of motion during the generation of sensory flow focusing on how animals actively shape information by behavioral strategies. We use the well-studied examples of vision in insects and echolocation in bats to describe commonalities of sensory-related behavioral strategies across sensory systems, and evaluate what is currently known about comparable active sensing strategies in electroreception of electric fish. In this sensory system the sensors are dispersed across the animal's body and the carrier source emitting energy used for sensing, the electric organ, is moved while the animal moves. Thus ego-motions strongly influence sensory dynamics. We present, for the first time, data of electric flow during natural probing behavior in Gnathonemus petersii (Mormyridae), which provide evidence for this influence. These data reveal a complex interdependency between the physical input to the receptors and the animal's movements, posture and objects in its environment. Although research on spatiotemporal dynamics in electrolocation is still in its infancy, the emerging field of dynamical sensory systems analysis in electric fish is a promising approach to the study of the link between movement and acquisition of sensory information.