Small or far away? Size and distance perception in the praying mantis

Anatomical organization of the cerebrum of the praying mantis Hierodula membranacea

Many predatory animals, such as the praying mantis, use vision for prey detection and capture. Mantises are known in particular for their capability to estimate distances to prey by stereoscopic vision. While the initial visual processing centers have been extensively documented, we lack knowledge on the architecture of central brain regions, pivotal for sensory motor transformation and higher brain functions. To close this gap, we provide a three-dimensional (3D) reconstruction of the central brain of the Asian mantis, Hierodula membranacea. The atlas facilitates in-depth analysis of neuron ramification regions and aides in elucidating potential neuronal pathways. We integrated seven 3D-reconstructed visual interneurons into the atlas. In total, 42 distinct neuropils of the cerebrum were reconstructed based on synapsin-immunolabeled whole-mount brains. Backfills from the antenna and maxillary palps, as well as immunolabeling of γ-aminobutyric acid (GABA) and tyrosine hydroxylase (TH), further substantiate the identification and boundaries of brain areas. The composition and internal organization of the neuropils were compared to the anatomical organization of the brain of the fruit fly (Drosophila melanogaster) and the two available brain atlases of Polyneoptera-the desert locust (Schistocerca gregaria) and the Madeira cockroach (Rhyparobia maderae). This study paves the way for detailed analyses of neuronal circuitry and promotes cross-species brain comparisons. We discuss differences in brain organization between holometabolous and polyneopteran insects. Identification of ramification sites of the visual neurons integrated into the atlas supports previous claims about homologous structures in the optic lobes of flies and mantises.

Enhancing LGMD-based model for collision prediction via binocular structure

Introduction

Lobular giant motion detector (LGMD) neurons, renowned for their distinctive response to looming stimuli, inspire the development of visual neural network models for collision prediction. However, the existing LGMD-based models could not yet incorporate the invaluable feature of depth distance and still suffer from the following two primary drawbacks. Firstly, they struggle to effectively distinguish the three fundamental motion patterns of approaching, receding, and translating, in contrast to the natural abilities of LGMD neurons. Secondly, due to their reliance on a general determination process employing an activation function and fixed threshold for output, these models exhibit dramatic fluctuations in prediction effectiveness across different scenarios.

Methods

To address these issues, we propose a novel LGMD-based model with a binocular structure (Bi-LGMD). The depth distance of the moving object is extracted by calculating the binocular disparity facilitating a clear differentiation of the motion patterns, after obtaining the moving object's contour through the basic components of the LGMD network. In addition, we introduce a self-adaptive warning depth-distance, enhancing the model's robustness in various motion scenarios.

Results

The effectiveness of the proposed model is verified using computer-simulated and real-world videos.

Discussion

Furthermore, the experimental results demonstrate that the proposed model is robust to contrast and noise.

Stereopsis without correspondence

Stereopsis has traditionally been considered a complex visual ability, restricted to large-brained animals. The discovery in the 1980s that insects, too, have stereopsis, therefore, challenged theories of stereopsis. How can such simple brains see in three dimensions? A likely answer is that insect stereopsis has evolved to produce simple behaviour, such as orienting towards the closer of two objects or triggering a strike when prey comes within range. Scientific thinking about stereopsis has been unduly anthropomorphic, for example assuming that stereopsis must require binocular fusion or a solution of the stereo correspondence problem. In fact, useful behaviour can be produced with very basic stereoscopic algorithms which make no attempt to achieve fusion or correspondence, or to produce even a coarse map of depth across the visual field. This may explain why some aspects of insect stereopsis seem poorly designed from an engineering point of view: for example, paying no attention to whether interocular contrast or velocities match. Such algorithms demonstrably work well enough in practice for their species, and may prove useful in particular autonomous applications. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

A computational model of stereoscopic prey capture in praying mantises

We present a simple model which can account for the stereoscopic sensitivity of praying mantis predatory strikes. The model consists of a single “disparity sensor”: a binocular neuron sensitive to stereoscopic disparity and thus to distance from the animal. The model is based closely on the known behavioural and neurophysiological properties of mantis stereopsis. The monocular inputs to the neuron reflect temporal change and are insensitive to contrast sign, making the sensor insensitive to interocular correlation. The monocular receptive fields have a excitatory centre and inhibitory surround, making them tuned to size. The disparity sensor combines inputs from the two eyes linearly, applies a threshold and then an exponent output nonlinearity. The activity of the sensor represents the model mantis’s instantaneous probability of striking. We integrate this over the stimulus duration to obtain the expected number of strikes in response to moving targets with different stereoscopic disparity, size and vertical disparity. We optimised the parameters of the model so as to bring its predictions into agreement with our empirical data on mean strike rate as a function of stimulus size and disparity. The model proves capable of reproducing the relatively broad tuning to size and narrow tuning to stereoscopic disparity seen in mantis striking behaviour. Although the model has only a single centre-surround receptive field in each eye, it displays qualitatively the same interaction between size and disparity as we observed in real mantids: the preferred size increases as simulated prey distance increases beyond the preferred distance. We show that this occurs because of a stereoscopic “false match” between the leading edge of the stimulus in one eye and its trailing edge in the other; further work will be required to find whether such false matches occur in real mantises. Importantly, the model also displays realistic responses to stimuli with vertical disparity and to pairs of identical stimuli offering a “ghost match”, despite not being fitted to these data. This is the first image-computable model of insect stereopsis, and reproduces key features of both neurophysiology and striking behaviour.

The praying mantis is the only insect so far known to compute depth using stereoscopic (3D) vision. Mantis stereopsis appears to be simpler than human stereopsis and most machine sterovision algorithms. A computational model of mantis stereopsis may therefore be beneficial to the field of robotics, particularly where computational power is limited. Using a combination of behavioural observations and neurophysiological data, we propose a very simple model structure to describe the prey capture response in the praying mantis. We used the limited available data on the mantis’ size and distance preferences for its prey to train our model parameters. Our simple model is able to qualitatively reproduce previously unexplained characteristics of our training data, and predicts key observations in additional empirical data that was not included in the model training. Whilst we believe our model to be only a partial and heavily simplified account of mantis stereopsis, our results are supportive of our model structure as an approximation of the size and disparity sensors used by the mantis when catching its prey.

Evolutionary diversification in the raptorial forelegs of Mantodea: Relations to body size and depth perception

Limb proportions have evolved among animals to meet functional demands among diverse environments. Studies from terrestrial, vertebrate locomotion have demonstrated that variation in limb proportions have adaptively evolved so animals can perform in a given environment. Most of the research on limb proportion evolution is among vertebrates and terrestrial locomotion, with little information on limb segment evolution in invertebrates or for other functional roles. For example, among invertebrates, multisegmented raptorial forelimbs have evolved multiple times independently to capture prey, but there is little information on the adaptive evolution and diversity of these limbs. Furthermore, as feeding performance is influenced by the sensory system, few studies have examined the coevolution of sensory-motor systems. Using mantises (Mantodea) I examined forelimb diversification among 97 species with a combination of methods, including ternary plots for morphospace visualization, phylogenetically informed allometric relationships, and comparison of evolutionary rates of diversification. Furthermore, using head width as a proxy for depth perception, I examined the correlated evolution of foreleg diversity with depth perception. The results show that among the three segments of the foreleg, the tibia is the smallest, most diverse, and has the highest rate of evolution after body size corrections. Furthermore, while all foreleg segments were related to head width, head width explained the most variation in tibial length compared with other foreleg segments. The results suggest a potential adaptive functional role of tibia length related to the displacement or force produced in this mechanical lever. Furthermore, results from this study support distinct ecomorphs of mantises, as several independent evolutions to grass mimicry evolve similar morphologies. RESEARCH HIGHLIGHT: This study demonstrates interspecific variation among segments of an invertebrate raptorial foreleg. Among Mantodea species the tibia is the most diverse and is related to a proxy for depth perception, while the other segments had strong relationships with body size. This suggests an adaptive, functional role of the tibia during prey capture.

Second-order cues to figure motion enable object detection during prey capture by praying mantises

Detecting motion is essential for animals to perform a wide variety of functions. In order to do so, animals could exploit motion cues, including both first-order cues-such as luminance correlation over time-and second-order cues, by correlating higher-order visual statistics. Since first-order motion cues are typically sufficient for motion detection, it is unclear why sensitivity to second-order motion has evolved in animals, including insects. Here, we investigate the role of second-order motion in prey capture by praying mantises. We show that prey detection uses second-order motion cues to detect figure motion. We further present a model of prey detection based on second-order motion sensitivity, resulting from a layer of position detectors feeding into a second layer of elementary-motion detectors. Mantis stereopsis, in contrast, does not require figure motion and is explained by a simpler model that uses only the first layer in both eyes. Second-order motion cues thus enable prey motion to be detected, even when perfectly matching the average background luminance and independent of the elementary motion of any parts of the prey. Subsequent to prey detection, processes such as stereopsis could work to determine the distance to the prey. We thus demonstrate how second-order motion mechanisms enable ecologically relevant behavior such as detecting camouflaged targets for other visual functions including stereopsis and target tracking.

The Science Behind Virtual Reality Displays

Virtual reality (VR) is becoming an increasingly important way to investigate sensory processing. The converse is also true: in order to build good VR technologies, one needs an intimate understanding of how our brain processes sensory information. One of the key advantages of studying perception with VR is that it allows an experimenter to probe perceptual processing in a more naturalistic way than has been possible previously. In VR, one is able to actively explore and interact with the environment, just as one would do in real life. In this article, we review the history of VR displays, including the philosophical origins of VR, before discussing some key challenges involved in generating good VR and how a sense of presence in a virtual environment can be measured. We discuss the importance of multisensory VR and evaluate the experimental tension that exists between artifice and realism when investigating sensory processing.

A neuronal correlate of insect stereopsis

A puzzle for neuroscience—and robotics—is how insects achieve surprisingly complex behaviours with such tiny brains. One example is depth perception via binocular stereopsis in the praying mantis, a predatory insect. Praying mantids use stereopsis, the computation of distances from disparities between the two retinal images, to trigger a raptorial strike of their forelegs when prey is within reach. The neuronal basis of this ability is entirely unknown. Here we show the first evidence that individual neurons in the praying mantis brain are tuned to specific disparities and eccentricities, and thus locations in 3D-space. Like disparity-tuned cortical cells in vertebrates, the responses of these mantis neurons are consistent with linear summation of binocular inputs followed by an output nonlinearity. Our study not only proves the existence of disparity sensitive neurons in an insect brain, it also reveals feedback connections hitherto undiscovered in any animal species.

The praying mantis, a predatory insect, estimates depth via binocular vision. In this way, the animal decides whether prey is within reach. Here, the authors explore the neural correlates of binocular distance estimation and report that individual neurons are tuned to specific locations in 3D space.

Motion-in-depth perception and prey capture in the praying mantis Sphodromantis lineola

Perceiving motion-in-depth is essential to detecting approaching or receding objects, predators and prey. This can be achieved using several cues, including binocular stereoscopic cues such as changing disparity and interocular velocity differences, and monocular cues such as looming. While these have been studied in detail in humans, only looming responses have been well characterized in insects and we know nothing about the role of stereo cues and how they might interact with looming cues. We used our 3D insect cinema in a series of experiments to investigate the role of the stereo cues mentioned above, as well as looming, in the perception of motion-in-depth during predatory strikes by the praying mantis Sphodromantis lineola. Our results show that motion-in-depth does increase the probability of mantis strikes but only for the classic looming stimulus, an expanding luminance edge. Approach indicated by radial motion of a texture or expansion of a motion-defined edge, or by stereoscopic cues, all failed to elicit increased striking. We conclude that mantises use stereopsis to detect depth but not motion-in-depth, which is detected via looming.

The hindgut microbiota of praying mantids is highly variable and includes both prey-associated and host-specific microbes

Praying mantids are predators that consume a wide variety of insects. While the gut microbiome of carnivorous mammals is distinct from that of omnivores and herbivores, the role of the gut microbiome among predatory insects is relatively understudied. Praying mantids are the closest known relatives to termites and cockroaches, which are known for their diverse gut microbiota. However, little is known about the mantid gut microbiota or their importance to host health. In this work, we report the results of a 16S rRNA gene-based study of gut microbiome composition in adults and late-instar larvae of three mantid species. We found that the praying mantis gut microbiome exhibits substantial variation in bacterial diversity and community composition. The hindgut of praying mantids were often dominated by microbes that are present in low abundance or not found in the guts of their insect prey. Future studies will explore the role of these microbes in the digestion of the dietary substrates and/or the degradation of toxins produced by their insect prey.

Stereopsis in animals: evolution, function and mechanisms

Stereopsis is the computation of depth information from views acquired simultaneously from different points in space. For many years, stereopsis was thought to be confined to primates and other mammals with front-facing eyes. However, stereopsis has now been demonstrated in many other animals, including lateral-eyed prey mammals, birds, amphibians and invertebrates. The diversity of animals known to have stereo vision allows us to begin to investigate ideas about its evolution and the underlying selective pressures in different animals. It also further prompts the question of whether all animals have evolved essentially the same algorithms to implement stereopsis. If so, this must be the best way to do stereo vision, and should be implemented by engineers in machine stereopsis. Conversely, if animals have evolved a range of stereo algorithms in response to different pressures, that could inspire novel forms of machine stereopsis appropriate for distinct environments, tasks or constraints. As a first step towards addressing these ideas, we here review our current knowledge of stereo vision in animals, with a view towards outlining common principles about the evolution, function and mechanisms of stereo vision across the animal kingdom. We conclude by outlining avenues for future work, including research into possible new mechanisms of stereo vision, with implications for machine vision and the role of stereopsis in the evolution of camouflage.

Summary: Stereopsis has evolved independently in different animals. We review the various functions it serves and the variety of mechanisms that could underlie stereopsis in different species.

A Novel Form of Stereo Vision in the Praying Mantis

Stereopsis is the ability to estimate distance based on the different views seen in the two eyes [1-5]. It is an important model perceptual system in neuroscience and a major area of machine vision. Mammalian, avian, and almost all machine stereo algorithms look for similarities between the luminance-defined images in the two eyes, using a series of computations to produce a map showing how depth varies across the scene [3, 4, 6-14]. Stereopsis has also evolved in at least one invertebrate, the praying mantis [15-17]. Mantis stereopsis is presumed to be simpler than vertebrates' [15, 18], but little is currently known about the underlying computations. Here, we show that mantis stereopsis uses a fundamentally different computational algorithm from vertebrate stereopsis-rather than comparing luminance in the two eyes' images directly, mantis stereopsis looks for regions of the images where luminance is changing. Thus, while there is no evidence that mantis stereopsis works at all with static images, it successfully reveals the distance to a moving target even in complex visual scenes with targets that are perfectly camouflaged against the background in terms of texture. Strikingly, these insects outperform human observers at judging stereoscopic distance when the pattern of luminance in the two eyes does not match. Insect stereopsis has thus evolved to be computationally efficient while being robust to poor image resolution and to discrepancies in the pattern of luminance between the two eyes. VIDEO ABSTRACT.

Vision in our three-dimensional world

Many aspects of our perceptual experience are dominated by the fact that our two eyes point forward. Whilst the location of our eyes leaves the environment behind our head inaccessible to vision, co-ordinated use of our two eyes gives us direct access to the three-dimensional structure of the scene in front of us, through the mechanism of stereoscopic vision. Scientific understanding of the different brain regions involved in stereoscopic vision and three-dimensional spatial cognition is changing rapidly, with consequent influences on fields as diverse as clinical practice in ophthalmology and the technology of virtual reality devices.This article is part of the themed issue 'Vision in our three-dimensional world'.

Patterns of variation in feeding strike kinematics of juvenile ghost praying mantis (Phyllocrania paradoxa): are components of the strike stereotypic?

Functional systems, such as feeding mechanics, often involve the evolution of several components of the musculoskeletal system that are moved in coordination to capture prey. Because these systems often involve the quick movement of several structures, some feeding systems have been hypothesized to be stereotypic. While the motor activity patterns are often stereotyped, the subsequent kinematics can be variable, many times in response to variation in prey stimulus (e.g., prey position). Patterns of feeding kinematics have been well studied among vertebrates, with less attention on invertebrate systems. The goal of this study was to examine the amount of stereotypy in the feeding strike kinematics of praying mantises. We filmed 8 juvenile ghost praying mantises (Phyllocrania paradox) at 1000 Hz, across several days within instar 7. We digitized several points that represent the movement of the coxa, trochanter-femur and tibia of the raptorial foreleg to obtain a set of kinematics including angles and angular velocities of the joint, as well as body lunge. Using the coefficient of variation, we found less stereotypy in the approach stage of the strike compared to the sweep. Using Bonferroni corrected Pearson's correlations of kinematics with prey position we found few traits related to prey position with the exception of some kinematics of the coxa joint and the amount of lunge used during the strike. Our results suggest that several components of the praying mantis strike are stereotypic, while others exhibit flexibility to ensure successful capture of the prey.