Bumble bees display cross-modal object recognition between visual and tactile senses

Reconfigurable optoelectronic transistors for multimodal recognition

Biological nervous system outperforms in both dynamic and static information perception due to their capability to integrate the sensing, memory and processing functions. Reconfigurable neuromorphic transistors, which can be used to emulate different types of biological analogues in a single device, are important for creating compact and efficient neuromorphic computing networks, but their design remains challenging due to the need for opposing physical mechanisms to achieve different functions. Here we report a neuromorphic electrolyte-gated transistor that can be reconfigured to perform physical reservoir and synaptic functions. The device exhibits dynamics with tunable time-scales under optical and electrical stimuli. The nonlinear volatile property is suitable for reservoir computing, which can be used for multimodal pre-processing. The nonvolatility and programmability of the device through ion insertion/extraction achieved via electrolyte gating, which are required to realize synaptic functions, are verified. The device's superior performance in mimicking human perception of dynamic and static multisensory information based on the reconfigurable neuromorphic functions is also demonstrated. The present study provides an exciting paradigm for the realization of multimodal reconfigurable devices and opens an avenue for mimicking biological multisensory fusion.

Behavioral indicators of heterogeneous subjective experience in animals across the phylogenetic spectrum: Implications for comparative animal phenomenology

This behavioral study was undertaken to provide empirical evidence in favor of or opposed to the notion that animals across a wide breadth of the animal kingdom have subjective (personal) experience that varies with their lifestyles, ecological constraints, or phylogeny. Twelve species representing two invertebrate phyla and six vertebrate classes were observed unobtrusively in 15-min episodes, during which three modes of behavior (volitional, interactive, and egocentric) were quantified according to the frequency, variety, and dynamism of each mode. Volitional behavior was the most prevalent and dynamic mode for nearly all species, largely without regard to phylogenetic position. Interactive behavior likewise varied inconsistently across the entire evolutionary spectrum. Egocentric behavior was concentrated among the avian and mammalian species, but evidence of it were observed in the invertebrate species as well. Diagrams of the matrix constructed from the three qualitative modes and three quantitative attributes for each mode provide a metaphorical representation of the unique experiential profile of each species. To the extent that these behavioral measures correlate with the nature of the animal's subjective experience, they support the growing view that phenomenology is heterogeneous, multimodal, and non-linear in extent across the animal kingdom.

First-sight recognition of touched objects shows that chicks can solve Molyneux's problem

If a congenitally blind person learns to distinguish between a cube and a sphere by touch, would they immediately recognize these objects by sight once their vision is restored? This question, posed by Molyneux in 1688, has puzzled philosophers and scientists since then. To overcome ethical and practical difficulties in the investigation of cross-modal recognition, we studied inexperienced poultry chicks, which can be reared in darkness until the moment of a visual test with no detrimental consequences. After hatching chicks in darkness, we exposed them to either tactile smooth or tactile bumpy stimuli for 24 h. Immediately after the tactile exposure, chicks were tested in a visual recognition task, during their first experience with light. At first sight, chicks that had been exposed in the tactile modality to smooth stimuli approached the visual smooth stimulus significantly more than those exposed to the tactile bumpy stimuli. These results show that visually inexperienced chicks can solve Molyneux's problem, indicating cross-modal recognition does not require previous multimodal experience. At least in this precocial species, supra-modal brain areas appear functional already at birth. This discovery paves the way for the investigation of predisposed cross-modal cognition that does not depend on visual experience.

Efficient visual learning by bumble bees in virtual-reality conditions: Size does not matter

Recent developments allowed establishing virtual-reality (VR) setups to study multiple aspects of visual learning in honey bees under controlled experimental conditions. Here, we adopted a VR environment to investigate the visual learning in the buff-tailed bumble bee Bombus terrestris. Based on responses to appetitive and aversive reinforcements used for conditioning, we show that bumble bees had the proper appetitive motivation to engage in the VR experiments and that they learned efficiently elemental color discriminations. In doing so, they reduced the latency to make a choice, increased the proportion of direct paths toward the virtual stimuli and walked faster toward them. Performance in a short-term retention test showed that bumble bees chose and fixated longer on the correct stimulus in the absence of reinforcement. Body size and weight, although variable across individuals, did not affect cognitive performances and had a mild impact on motor performances. Overall, we show that bumble bees are suitable experimental subjects for experiments on visual learning under VR conditions, which opens important perspectives for invasive studies on the neural and molecular bases of such learning given the robustness of these insects and the accessibility of their brain.

Bumblebees negotiate a trade-off between nectar quality and floral biomechanics

How and why pollinators choose which flowers to visit are fundamental, multifaceted questions in pollination biology, yet most studies of floral traits measure simple relative preferences. Here, we used vertically and horizontally oriented slippery-surfaced artificial flowers to test whether bumblebees could make a trade-off between floral handling difficulty and nectar sucrose concentration. We quantified foraging energetics, thereby resolving the rationale behind the bees' foraging decisions. The bees chose flowers with either a high handling cost or low sucrose concentration, depending on which was the energetically favorable option. Their behavior agreed with the critical currency being the rate of energy return (net energy collected per unit time), not energetic efficiency (net energy collected per unit energy spent). This suggests that bumblebees prioritize immediate carbohydrate flow to the nest rather than energy gain over the working lifespan of each bee. Trade-off paradigms like these are a powerful approach for quantifying pollinator trait preferences.

Using pupae as appetitive reinforcement to study visual and tactile associative learning in the Ponerine ant Diacamma indicum

Associative learning is of great importance to animals, as it enhances their ability to navigate, forage, evade predation and improve fitness. Even though associative learning abilities of Hymenopterans have been explored, many of these studies offered food as appetitive reinforcement. In the current study, we focus on tactile and visual cue learning in an ant Diacamma indicum using a Y-maze setup with pupa as a positive reinforcement. Using pupa as a reward resulted in a significantly higher proportion of ants completing the training in a shorter time as compared to using food as reinforcement. Ants spent significantly more time in the conditioned arm for both visual cues (white dots or black dots) and tactile cues (rough or smooth surfaces) presented on the floor when associated with pupa, thus showing that they were capable of associative learning. On encountering a conflict between visual and tactile cues during the test, ants chose to spend significantly more time on the arm with the tactile cues indicating that they had made a stronger association between pupa and the tactile cue as compared to the visual cue during training. Using pupa as an ecologically relevant reward, we show that these solitary foraging ants living in small colonies are capable of visual and tactile associative learning and are likely to learn tactile cues over visual cues in association with pupa.

Crossmodal association between visual and acoustic cues in a tortoise (Testudo hermanni)

Humans spontaneously match information coming from different senses, in what we call crossmodal associations. For instance, high-pitched sounds are preferentially associated with small objects, and low-pitched sounds with larger ones. Although previous studies reported crossmodal associations in mammalian species, evidence for other taxa is scarce, hindering an evolutionary understanding of this phenomenon. Here, we provide evidence of pitch-size correspondence in a reptile, the tortoise Testudo hermanni. Tortoises showed a spontaneous preference to associate a small disc (i.e. visual information about size) with a high-pitch sound (i.e. auditory information) and a larger disc to a low-pitched sound. These results suggest that crossmodal associations may be an evolutionary ancient phenomenon, potentially an organizing principle of the vertebrate brain.

Mammalian-brain-inspired neuromorphic motion-cognition nerve achieves cross-modal perceptual enhancement

Perceptual enhancement of neural and behavioral response due to combinations of multisensory stimuli are found in many animal species across different sensory modalities. By mimicking the multisensory integration of ocular-vestibular cues for enhanced spatial perception in macaques, a bioinspired motion-cognition nerve based on a flexible multisensory neuromorphic device is demonstrated. A fast, scalable and solution-processed fabrication strategy is developed to prepare a nanoparticle-doped two-dimensional (2D)-nanoflake thin film, exhibiting superior electrostatic gating capability and charge-carrier mobility. The multi-input neuromorphic device fabricated using this thin film shows history-dependent plasticity, stable linear modulation, and spatiotemporal integration capability. These characteristics ensure parallel, efficient processing of bimodal motion signals encoded as spikes and assigned with different perceptual weights. Motion-cognition function is realized by classifying the motion types using mean firing rates of encoded spikes and postsynaptic current of the device. Demonstrations of recognition of human activity types and drone flight modes reveal that the motion-cognition performance match the bio-plausible principles of perceptual enhancement by multisensory integration. Our system can be potentially applied in sensory robotics and smart wearables.

Novel object recognition in Octopus maya

The Novel Object Recognition task (NOR) is widely used to study vertebrates' memory. It has been proposed as an adequate model for studying memory in different taxonomic groups, allowing similar and comparable results. Although in cephalopods, several research reports could indicate that they recognize objects in their environment, it has not been tested as an experimental paradigm that allows studying different memory phases. This study shows that two-month-old and older Octopus maya subjects can differentiate between a new object and a known one, but one-month-old subjects cannot. Furthermore, we observed that octopuses use vision and tactile exploration of new objects to achieve object recognition, while familiar objects only need to be explored visually. To our knowledge, this is the first time showing an invertebrate performing the NOR task similarly to how it is performed in vertebrates. These results establish a guide to studying object recognition memory in octopuses and the ontological development of that memory.

Undercompression errors as evidence for conceptual primitives

The Meaning First Approach offers a model of the relation between thought and language that includes a Generator and a Compressor. The Generator build non-linguistic thought structures and the Compressor is responsible for its articulation through three processes: structure-preserving linearization, lexification, and compression via non-articulation of concepts when licensed. One goal of this paper is to show that a range of phenomena in child language can be explained in a unified way within the Meaning First Approach by the assumption that children differ from adults with respect to compression and, specifically, that they may undercompress in production, an idea that sets a research agenda for the study of language acquisition. We focus on dependencies involving pronouns or gaps in relative clauses and wh-questions, multi-argument verbal concepts, and antonymic concepts involving negation or other opposites. We present extant evidence from the literature that children produce undercompression errors (a type of commission errors) that are predicted by the Meaning First Approach. We also summarize data that children's comprehension ability provides evidence for the Meaning First Approach prediction that decompression should be challenging, when there is no 1-to-1 correspondence.

The best game in town: The reemergence of the language-of-thought hypothesis across the cognitive sciences

Mental representations remain the central posits of psychology after many decades of scrutiny. However, there is no consensus about the representational format(s) of biological cognition. This paper provides a survey of evidence from computational cognitive psychology, perceptual psychology, developmental psychology, comparative psychology, and social psychology, and concludes that one type of format that routinely crops up is the language-of-thought (LoT). We outline six core properties of LoTs: (i) discrete constituents; (ii) role-filler independence; (iii) predicate-argument structure; (iv) logical operators; (v) inferential promiscuity; and (vi) abstract content. These properties cluster together throughout cognitive science. Bayesian computational modeling, compositional features of object perception, complex infant and animal reasoning, and automatic, intuitive cognition in adults all implicate LoT-like structures. Instead of regarding LoT as a relic of the previous century, researchers in cognitive science and philosophy-of-mind must take seriously the explanatory breadth of LoT-based architectures. We grant that the mind may harbor many formats and architectures, including iconic and associative structures as well as deep-neural-network-like architectures. However, as computational/representational approaches to the mind continue to advance, classical compositional symbolic structures - that is, LoTs - only prove more flexible and well-supported over time.

Geometry-based navigation in the dark: layout symmetry facilitates spatial learning in the house cricket, Acheta domesticus, in the absence of visual cues

The capacity to navigate by layout geometry has been widely recognized as a robust strategy of place-finding. It has been reported in various species, although most studies were performed with vision-based paradigms. In the presented study, we aimed to investigate layout symmetry-based navigation in the house cricket, Acheta domesticus, in the absence of visual cues. For this purpose, we used a non-visual paradigm modeled on the Tennessee Williams setup. We ensured that the visual cues were indeed inaccessible to insects. In the main experiment, we tested whether crickets are capable of learning to localize the centrally positioned, inconspicuous cool spot in heated arenas of various shapes (i.e., circular, square, triangular, and asymmetric quadrilateral). We found that the symmetry of the arena significantly facilitates crickets’ learning to find the cool spot, indicated by the increased time spent on the cool spot and the decreased latency in locating it in subsequent trials. To investigate mechanisms utilized by crickets, we analyzed their approach paths to the spot. We found that crickets used both heuristic and directed strategies of approaching the target, with the dominance of a semi-directed strategy (i.e., a thigmotactic phase preceding direct navigation to the target). We propose that the poor performance of crickets in the asymmetrical quadrilateral arena may be explained by the difficulty of encoding its layout with cues from a single modality.

The Scientific Study of Consciousness Cannot and Should Not Be Morally Neutral

A target question for the scientific study of consciousness is how dimensions of consciousness, such as the ability to feel pain and pleasure or reflect on one's own experience, vary in different states and animal species. Considering the tight link between consciousness and moral status, answers to these questions have implications for law and ethics. Here we point out that given this link, the scientific community studying consciousness may face implicit pressure to carry out certain research programs or interpret results in ways that justify current norms rather than challenge them. We show that because consciousness largely determines moral status, the use of nonhuman animals in the scientific study of consciousness introduces a direct conflict between scientific relevance and ethics-the more scientifically valuable an animal model is for studying consciousness, the more difficult it becomes to ethically justify compromises to its well-being for consciousness research. Finally, in light of these considerations, we call for a discussion of the immediate ethical corollaries of the body of knowledge that has accumulated and for a more explicit consideration of the role of ideology and ethics in the scientific study of consciousness.

Insect-inspired AI for autonomous robots

Autonomous robots are expected to perform a wide range of sophisticated tasks in complex, unknown environments. However, available onboard computing capabilities and algorithms represent a considerable obstacle to reaching higher levels of autonomy, especially as robots get smaller and the end of Moore's law approaches. Here, we argue that inspiration from insect intelligence is a promising alternative to classic methods in robotics for the artificial intelligence (AI) needed for the autonomy of small, mobile robots. The advantage of insect intelligence stems from its resource efficiency (or parsimony) especially in terms of power and mass. First, we discuss the main aspects of insect intelligence underlying this parsimony: embodiment, sensory-motor coordination, and swarming. Then, we take stock of where insect-inspired AI stands as an alternative to other approaches to important robotic tasks such as navigation and identify open challenges on the road to its more widespread adoption. Last, we reflect on the types of processors that are suitable for implementing insect-inspired AI, from more traditional ones such as microcontrollers and field-programmable gate arrays to unconventional neuromorphic processors. We argue that even for neuromorphic processors, one should not simply apply existing AI algorithms but exploit insights from natural insect intelligence to get maximally efficient AI for robot autonomy.

Cross-modal perception of identity by sound and taste in bottlenose dolphins

While studies have demonstrated concept formation in animals, only humans are known to label concepts to use them in mental simulations or predictions. To investigate whether other animals use labels comparably, we studied cross-modal, individual recognition in bottlenose dolphins (Tursiops truncatus) that use signature whistles as labels for conspecifics in their own communication. First, we tested whether dolphins could use gustatory stimuli and found that they could distinguish between water and urine samples, as well as between urine from familiar and unfamiliar individuals. Then, we paired playbacks of signature whistles of known animals with urine samples from either the same dolphin or a different, familiar animal. Dolphins investigated the presentation area longer when the acoustic and gustatory sample matched than when they mismatched. This demonstrates that dolphins recognize other individuals by gustation alone and can integrate information from acoustic and taste inputs indicating a modality independent, labeled concept for known conspecifics.

Honey bees respond to multimodal stimuli following the Principle of Inverse Effectiveness

Multisensory integration is assumed to entail benefits for receivers across multiple ecological contexts. However, signal integration effectiveness is constrained by features of the spatiotemporal and intensity domains. How sensory modalities are integrated during tasks facilitated by learning and memory, such as pollination, remains unsolved. Honey bees use olfactory and visual cues during foraging, making them a good model to study the use of multimodal signals. Here, we examined the effect of stimulus intensity on both learning and memory performance of bees trained using unimodal or bimodal stimuli. We measured the performance and the latency response across planned discrete levels of stimulus intensity. We employed the conditioning of the proboscis extension response protocol in honey bees using an electromechanical setup allowing us to control simultaneously and precisely olfactory and visual stimuli at different intensities. Our results show that the bimodal enhancement during learning and memory was higher as the intensity decreased when the separate individual components were least effective. Still, this effect was not detectable for the latency of response. Remarkably, these results support the principle of inverse effectiveness, traditionally studied in vertebrates, predicting that multisensory stimuli are more effectively integrated when the best unisensory response is relatively weak. Thus, we argue that the performance of the bees while using a bimodal stimulus depends on the interaction and intensity of its individual components. We further hold that the inclusion of findings across all levels of analysis enriches the traditional understanding of the mechanics and reliance of complex signals in honey bees.

Summary: Bimodal enhancement during learning and memory tasks in africanized honey bees increases as the stimulus intensity of its unimodal components decreases; this indicates that learning performance depends on the interaction between the intensity of its components and the nature of the sensory modalities involved, supporting the principle of inverse effectiveness.

The best of both worlds: Dual systems of reasoning in animals and AI

Much of human cognition involves two different types of reasoning that operate together. Type 1 reasoning systems are intuitive and fast, whereas Type 2 reasoning systems are reflective and slow. Why has our cognition evolved with these features? Both systems are coherent and in most ecological circumstances either alone is capable of coming up with the right answer most of the time. Neural tissue is costly, and thus far evolutionary models have struggled to identify a benefit of operating two systems of reasoning. To explore this issue we take a broad comparative perspective. We discuss how dual processes of cognition have enabled the emergence of selective attention in insects, transforming the learning capacities of these animals. Modern AIs using dual systems of learning are able to learn how their vast world works and how best to interact with it, allowing them to exceed human levels of performance in strategy games. We propose that the core benefits of dual processes of reasoning are to narrow down a problem space in order to focus cognitive resources most effectively.

The search for invertebrate consciousness

There is no agreement on whether any invertebrates are conscious and no agreement on a methodology that could settle the issue. How can the debate move forward? I distinguish three broad types of approach: theory-heavy, theory-neutral and theory-light. Theory-heavy and theory-neutral approaches face serious problems, motivating a middle path: the theory-light approach. At the core of the theory-light approach is a minimal commitment about the relation between phenomenal consciousness and cognition that is compatible with many specific theories of consciousness: the hypothesis that phenomenally conscious perception of a stimulus facilitates, relative to unconscious perception, a cluster of cognitive abilities in relation to that stimulus. This "facilitation hypothesis" can productively guide inquiry into invertebrate consciousness. What is needed? At this stage, not more theory, and not more undirected data gathering. What is needed is a systematic search for consciousness-linked cognitive abilities, their relationships to each other, and their sensitivity to masking.

Evaluation of Visual and Tactile Perception by Plain-Body Octopus (Callistoctopus aspilosomatis) of Prey-Like Objects

We investigated the characteristic features of perception in octopuses by examining multisensory information from an object simulating prey, which provided different visual and tactile stimuli. In experiments, we presented plain-body octopus with four kinds of models, namely, the Lifelike crab, the Embedded crab, the Translucent crab, and the Black cuboid. These models contain different amounts of visual and tactile information that a crab originally contains: the Lifelike crab resembles a crab both visually and tactilely, the Embedded crab resembles a crab visually but provides different tactile information, the Translucent crab provides tactile information of a crab but contains less visual information, and the Black cuboid lacks both visual and tactile information of a crab. Among these four models, octopuses contacted most with the Lifelike crab, which was similar to their behavior with a crab. Indeed, octopuses were fastest to contact the Lifelike crab and had the longest duration of contacting it among the four models. Octopuses contacted the Embedded crab more than the Translucent crab, both of which had contrasting visuo-tactile information compared to that of a crab. Quickness of octopuses to contact and duration of contact with the Embedded crab were more similar to those with the Lifelike crab than to those with the Translucent crab. Furthermore, octopuses contacted the Black cuboid least among the models. These results suggest that octopuses compositely detect both visual and tactile information in order to perceive an object. Furthermore, octopuses possess the potential priority either for visual or tactile information, by which they process the target object.

Plain-Body Octopus's (Callistoctopus aspilosomatis) Learning about Objects via Both Visual and Tactile Sensory Inputs: A Pilot Study

Although various recognizing abilities have been revealed for octopuses, they predominantly deal with only a few species. Therefore, cognition diversity among other octopus species that have been overlooked needs to be investigated. We investigated whether plain-body octopus can learn a symbolic stimulus, for the reason that this octopus is abundant around Okinawa Island with a complex coral community landscape. Attention was paid to whether an octopus can learn a stimulus based solely on visual information without previous experience of learning it tactilely as well as visually. Furthermore, we examined whether different sensory inputs affect learning in octopuses. First, we tested whether octopuses can be conditioned to three different stimuli (object, picture, and video of a white cross). Octopuses that were presented an object or a picture could learn to touch them. However, octopuses that were presented a video could not learn to touch the stimulus. Second, we showed a video to octopuses that had already learned about an object or a picture to investigate whether the octopuses, having experienced a target using visual and tactile senses, can recognize a video of the target based solely on visual information. Octopuses could learn to touch the video. When a conditioned stimulus and a novel stimulus were simultaneously presented on a computer screen, an octopus that had learned an object more often selected the conditioned stimulus when compared with an octopus that had experienced only a picture. These findings suggest that octopuses use multisensory information to recognize a specific object.

Extraordinary claims, extraordinary evidence? A discussion

Roberts (2020, Learning & Behavior, 48[2], 191-192) discussed research claiming honeybees can do arithmetic. Some readers of this research might regard such claims as unlikely. The present authors used this example as a basis for a debate on the criterion that ought to be used for publication of results or conclusions that could be viewed as unlikely by a significant number of readers, editors, or reviewers.

Insect Consciousness

The question of consciousness in other species, not least species very physically different from humans such as insects, is highly challenging for a number of reasons. One reason is that we do not have any available empirical method to answer the question. Another reason is that current theories of consciousness disagree about the relation between physical structure and consciousness, i.e., whether consciousness requires specific, say, neural structures or whether consciousness can be realized in different ways. This article sets out to analyze if and how there could be an empirical and/or a theoretical approach to the topic on the basis of current consciousness research in humans.

Einstein, von Frisch and the honeybee: a historical letter comes to light

The work of the Nobel Laureate Karl von Frisch, the founder of this journal, was seminal in many ways. He established the honeybee as a key animal model for experimental behavioural studies on sensory perception, learning and memory, and first correctly interpreted its famous dance communication. Here, we report on a previously unknown letter by the Physicist and Nobel Laureate Albert Einstein that was written in October 1949. It briefly addresses the work of von Frisch and also queries how understanding animal perception and navigation may lead to innovations in physics. We discuss records proving that Einstein and von Frisch met in April 1949 when von Frisch visited the USA to present a lecture on bees at Princeton University. In the historical context of Einstein’s theories and thought experiments, we discuss some more recent discoveries of animal sensory capabilities alien to us humans and potentially valuable for bio-inspired design improvements. We also address the orientation of animals like migratory birds mentioned by Einstein 70 years ago, which pushes the boundaries of our understanding nature, both its biology and physics.

Tactile active sensing in an insect plant pollinator

The interaction between insects and the flowers they pollinate has driven the evolutionary diversity of both insects and flowering plants, two groups with the most numerous species on Earth. Insects use vision and olfaction to localize host plants, but we know relatively little about how they find the tiny nectary opening in the flower, which can be well beyond their visual resolution. Especially when vision is limited, touch becomes crucial in successful insect-plant pollination interactions. Here, we studied the remarkable feeding behavior of crepuscular hawkmoths Manduca sexta, which use their long, actively controlled, proboscis to expertly explore flower-like surfaces. Using machine vision and 3D-printed artificial flower-like feeders, we revealed a novel behavior that shows moths actively probe surfaces, sweeping their proboscis from the feeder edge to its center repeatedly until they locate the nectary opening. Moreover, naive moths rapidly learn to exploit these flowers, and they adopt a tactile search strategy to more directly locate the nectary opening in as few as three to five consecutive visits. Our results highlight the proboscis as a unique active sensory structure and emphasize the central role of touch in nectar foraging insect-plant pollinator interactions.

Animal Cognition: The Self-Image of a Bumblebee

Knowing one's body dimensions is a core aspect of individual experience and self-awareness. A recent study illustrates how bees take into account their own body size both in preparation for and while traversing small gaps.

Bioinspired multisensory neural network with crossmodal integration and recognition

The integration and interaction of vision, touch, hearing, smell, and taste in the human multisensory neural network facilitate high-level cognitive functionalities, such as crossmodal integration, recognition, and imagination for accurate evaluation and comprehensive understanding of the multimodal world. Here, we report a bioinspired multisensory neural network that integrates artificial optic, afferent, auditory, and simulated olfactory and gustatory sensory nerves. With distributed multiple sensors and biomimetic hierarchical architectures, our system can not only sense, process, and memorize multimodal information, but also fuse multisensory data at hardware and software level. Using crossmodal learning, the system is capable of crossmodally recognizing and imagining multimodal information, such as visualizing alphabet letters upon handwritten input, recognizing multimodal visual/smell/taste information or imagining a never-seen picture when hearing its description. Our multisensory neural network provides a promising approach towards robotic sensing and perception.

Multisensory integration supports configural learning of a home refuge in the whip spider Phrynus marginemaculatus

Whip spiders (Amblypygi) reside in structurally complex habitats and are nocturnally active yet display notable navigational abilities. From the theory that uncertainty in sensory inputs should promote multisensory representations to guide behavior, we hypothesized that their navigation is supported by a multisensory and perhaps configural representation of navigational inputs, an ability documented in a few insects and never reported in arachnids. We trained Phrynus marginemaculatus to recognize a home shelter characterized by both discriminative olfactory and tactile stimuli. In tests, subjects readily discriminated between shelters based on the paired stimuli. However, subjects failed to recognize the shelter in tests with either of the component stimuli alone. This result is consistent with the hypothesis that the terminal phase of their navigational behavior, shelter recognition, can be supported by the integration of multisensory stimuli as an enduring, configural representation. We hypothesize that multisensory learning occurs in the whip spiders' extraordinarily large mushroom bodies, which may functionally resemble the hippocampus of vertebrates.

Use of Peripheral Sensory Information for Central Nervous Control of Arm Movement by Octopus vulgaris

Octopuses are active predators with highly flexible bodies and rich behavioral repertoires [1-3]. They display advanced cognitive abilities, and the size of their large nervous system rivals that of many mammals. However, only one third of the neurons constitute the CNS, while the rest are located in an elaborate PNS, including eight arms, each containing myriad sensory receptors of various modalities [2-4]. This led early workers to question the extent to which the CNS is privy to non-visual sensory input from the periphery and to suggest that it has limited capacity to finely control arm movement [3-5]. This conclusion seemed reasonable considering the size of the PNS and the results of early behavioral tests [3, 6-8]. We recently demonstrated that octopuses use visual information to control goal-directed complex single arm movements [9]. However, that study did not establish whether animals use information from the arm itself [9-12]. We here report on development of two-choice, single-arm mazes that test the ability of octopuses to perform operant learning tasks that mimic normal tactile exploration behavior and require the non-peripheral neural circuitry to use focal sensory information originating in single arms [1, 10]. We show that the CNS of the octopus uses peripheral information about arm motion as well as tactile input to accomplish learning tasks that entail directed control of movement. We conclude that although octopus arms have a great capacity to act independently, they are also subject to central control, allowing well-organized, purposeful behavior of the organism as a whole.

Honeybees solve a multi-comparison ranking task by probability matching

Honeybees forage on diverse flowers which vary in the amount and type of rewards they offer, and bees are challenged with maximizing the resources they gather for their colony. That bees are effective foragers is clear, but how bees solve this type of complex multi-choice task is unknown. Here, we set bees a five-comparison choice task in which five colours differed in their probability of offering reward and punishment. The colours were ranked such that high ranked colours were more likely to offer reward, and the ranking was unambiguous. Bees' choices in unrewarded tests matched their individual experiences of reward and punishment of each colour, indicating bees solved this test not by comparing or ranking colours but by basing their colour choices on their history of reinforcement for each colour. Computational modelling suggests a structure like the honeybee mushroom body with reinforcement-related plasticity at both input and output can be sufficient for this cognitive strategy. We discuss how probability matching enables effective choices to be made without a need to compare any stimuli directly, and the use and limitations of this simple cognitive strategy for foraging animals.

Validity of Cognitive Tests for Non-human Animals: Pitfalls and Prospects

Comparative psychology assesses cognitive abilities and capacities of non-human animals and humans. Based on performance differences and similarities in various species in cognitive tests, it is inferred how their minds work and reconstructed how cognition might have evolved. Critically, such species comparisons are only valid and meaningful if the tasks truly capture individual and inter-specific variation in cognitive abilities rather than contextual variables that might affect task performance. Unlike in human test psychology, however, cognitive tasks for non-human primates (and most other animals) have been rarely evaluated regarding their measurement validity. We review recent studies that address how non-cognitive factors affect performance in a set of commonly used cognitive tasks, and if cognitive tests truly measure individual variation in cognitive abilities. We find that individual differences in emotional and motivational factors primarily affect performance via attention. Hence, it is crucial to systematically control for attention during cognitive tasks to obtain valid and reliable results. Aspects of test design, however, can also have a substantial effect on cognitive performance. We conclude that non-cognitive factors are a minor source of measurement error if acknowledged and properly controlled for. It is essential, however, to validate and eventually re-design several primate cognition tasks in order to ascertain that they capture the cognitive abilities they were designed to measure. This will provide a more solid base for future cognitive comparisons within primates but also across a wider range of non-human animal species.

Cross-modal sensory transfer: Bumble bees do it

Stored sensory input permits two sensory channels to exchange and compare information