Learning Classification in the Olfactory System of Insects

Decision-making: A new role for insect mushroom bodies

Mushroom bodies are neural structures that are ubiquitous in insect brains, where they integrate sensory inputs to encode stimulus identity and meaning. A new study now adds action selection - decision-making - to those roles.

A linear discriminant analysis model of imbalanced associative learning in the mushroom body compartment

To adapt to their environments, animals learn associations between sensory stimuli and unconditioned stimuli. In invertebrates, olfactory associative learning primarily occurs in the mushroom body, which is segregated into separate compartments. Within each compartment, Kenyon cells (KCs) encoding sparse odor representations project onto mushroom body output neurons (MBONs) whose outputs guide behavior. Associated with each compartment is a dopamine neuron (DAN) that modulates plasticity of the KC-MBON synapses within the compartment. Interestingly, DAN-induced plasticity of the KC-MBON synapse is imbalanced in the sense that it only weakens the synapse and is temporally sparse. We propose a normative mechanistic model of the MBON as a linear discriminant analysis (LDA) classifier that predicts the presence of an unconditioned stimulus (class identity) given a KC odor representation (feature vector). Starting from a principled LDA objective function and under the assumption of temporally sparse DAN activity, we derive an online algorithm which maps onto the mushroom body compartment. Our model accounts for the imbalanced learning at the KC-MBON synapse and makes testable predictions that provide clear contrasts with existing models.

Insect-inspired AI for autonomous robots

Autonomous robots are expected to perform a wide range of sophisticated tasks in complex, unknown environments. However, available onboard computing capabilities and algorithms represent a considerable obstacle to reaching higher levels of autonomy, especially as robots get smaller and the end of Moore's law approaches. Here, we argue that inspiration from insect intelligence is a promising alternative to classic methods in robotics for the artificial intelligence (AI) needed for the autonomy of small, mobile robots. The advantage of insect intelligence stems from its resource efficiency (or parsimony) especially in terms of power and mass. First, we discuss the main aspects of insect intelligence underlying this parsimony: embodiment, sensory-motor coordination, and swarming. Then, we take stock of where insect-inspired AI stands as an alternative to other approaches to important robotic tasks such as navigation and identify open challenges on the road to its more widespread adoption. Last, we reflect on the types of processors that are suitable for implementing insect-inspired AI, from more traditional ones such as microcontrollers and field-programmable gate arrays to unconventional neuromorphic processors. We argue that even for neuromorphic processors, one should not simply apply existing AI algorithms but exploit insights from natural insect intelligence to get maximally efficient AI for robot autonomy.

An incentive circuit for memory dynamics in the mushroom body of Drosophila melanogaster

Insects adapt their response to stimuli, such as odours, according to their pairing with positive or negative reinforcements, such as sugar or shock. Recent electrophysiological and imaging findings in Drosophila melanogaster allow detailed examination of the neural mechanisms supporting the acquisition, forgetting, and assimilation of memories. We propose that this data can be explained by the combination of a dopaminergic plasticity rule that supports a variety of synaptic strength change phenomena, and a circuit structure (derived from neuroanatomy) between dopaminergic and output neurons that creates different roles for specific neurons. Computational modelling shows that this circuit allows for rapid memory acquisition, transfer from short term to long term, and exploration/exploitation trade-off. The model can reproduce the observed changes in the activity of each of the identified neurons in conditioning paradigms and can be used for flexible behavioural control.

Developmental and evolutionary constraints on olfactory circuit selection

SignificanceIn this work, we explore the hypothesis that biological neural networks optimize their architecture, through evolution, for learning. We study early olfactory circuits of mammals and insects, which have relatively similar structure but a huge diversity in size. We approximate these circuits as three-layer networks and estimate, analytically, the scaling of the optimal hidden-layer size with input-layer size. We find that both longevity and information in the genome constrain the hidden-layer size, so a range of allometric scalings is possible. However, the experimentally observed allometric scalings in mammals and insects are consistent with biologically plausible values. This analysis should pave the way for a deeper understanding of both biological and artificial networks.

The anterior paired lateral neuron normalizes odour-evoked activity in the Drosophila mushroom body calyx

To identify and memorize discrete but similar environmental inputs, the brain needs to distinguish between subtle differences of activity patterns in defined neuronal populations. The Kenyon cells (KCs) of the Drosophila adult mushroom body (MB) respond sparsely to complex olfactory input, a property that is thought to support stimuli discrimination in the MB. To understand how this property emerges, we investigated the role of the inhibitory anterior paired lateral (APL) neuron in the input circuit of the MB, the calyx. Within the calyx, presynaptic boutons of projection neurons (PNs) form large synaptic microglomeruli (MGs) with dendrites of postsynaptic KCs. Combining electron microscopy (EM) data analysis and in vivo calcium imaging, we show that APL, via inhibitory and reciprocal synapses targeting both PN boutons and KC dendrites, normalizes odour-evoked representations in MGs of the calyx. APL response scales with the PN input strength and is regionalized around PN input distribution. Our data indicate that the formation of a sparse code by the KCs requires APL-driven normalization of their MG postsynaptic responses. This work provides experimental insights on how inhibition shapes sensory information representation in a higher brain centre, thereby supporting stimuli discrimination and allowing for efficient associative memory formation.

Non-synaptic interactions between olfactory receptor neurons, a possible key feature of odor processing in flies

When flies explore their environment, they encounter odors in complex, highly intermittent plumes. To navigate a plume and, for example, find food, they must solve several challenges, including reliably identifying mixtures of odorants and their intensities, and discriminating odorant mixtures emanating from a single source from odorants emitted from separate sources and just mixing in the air. Lateral inhibition in the antennal lobe is commonly understood to help solving these challenges. With a computational model of the Drosophila olfactory system, we analyze the utility of an alternative mechanism for solving them: Non-synaptic ("ephaptic") interactions (NSIs) between olfactory receptor neurons that are stereotypically co-housed in the same sensilla. We find that NSIs improve mixture ratio detection and plume structure sensing and do so more efficiently than the traditionally considered mechanism of lateral inhibition in the antennal lobe. The best performance is achieved when both mechanisms work in synergy. However, we also found that NSIs decrease the dynamic range of co-housed ORNs, especially when they have similar sensitivity to an odorant. These results shed light, from a functional perspective, on the role of NSIs, which are normally avoided between neurons, for instance by myelination.

Stochastic facilitation in heteroclinic communication channels

Biological neural systems encode and transmit information as patterns of activity tracing complex trajectories in high-dimensional state spaces, inspiring alternative paradigms of information processing. Heteroclinic networks, naturally emerging in artificial neural systems, are networks of saddles in state space that provide a transparent approach to generate complex trajectories via controlled switches among interconnected saddles. External signals induce specific switching sequences, thus dynamically encoding inputs as trajectories. Recent works have focused either on computational aspects of heteroclinic networks, i.e., Heteroclinic Computing, or their stochastic properties under noise. Yet, how well such systems may transmit information remains an open question. Here, we investigate the information transmission properties of heteroclinic networks, studying them as communication channels. Choosing a tractable but representative system exhibiting a heteroclinic network, we investigate the mutual information rate (MIR) between input signals and the resulting sequences of states as the level of noise varies. Intriguingly, MIR does not decrease monotonically with increasing noise. Intermediate noise levels indeed maximize the information transmission capacity by promoting an increased yet controlled exploration of the underlying network of states. Complementing standard stochastic resonance, these results highlight the constructive effect of stochastic facilitation (i.e., noise-enhanced information transfer) on heteroclinic communication channels and possibly on more general dynamical systems exhibiting complex trajectories in state space.

Biological networks across scales

Many biological systems across scales of size and complexity exhibit a time-varying complex network structure that emerges and self-organizes as a result of interactions with the environment. Network interactions optimize some intrinsic cost functions that are unknown and involve for example energy efficiency, robustness, resilience, and frailty. A wide range of networks exist in biology, from gene regulatory networks important for organismal development, protein interaction networks that govern physiology and metabolism, and neural networks that store and convey information to networks of microbes that form microbiomes within hosts, animal contact networks that underlie social systems, and networks of populations on the landscape connected by migration. Increasing availability of extensive (big) data is amplifying our ability to quantify biological networks. Similarly, theoretical methods that describe network structure and dynamics are being developed. Beyond static networks representing snapshots of biological systems, collections of longitudinal data series can help either at defining and characterizing network dynamics over time or analyzing the dynamics constrained to networked architectures. Moreover, due to interactions with the environment and other biological systems, a biological network may not be fully observable. Also, subnetworks may emerge and disappear as a result of the need for the biological system to cope with for example invaders or new information flows. The confluence of these developments renders tractable the question of how the structure of biological networks predicts and controls network dynamics. In particular, there may be structural features that result in homeostatic networks with specific higher-order statistics (e.g., multifractal spectrum), which maintain stability over time through robustness and/or resilience to perturbation. Alternative, plastic networks may respond to perturbation by (adaptive to catastrophic) shifts in structure. Here, we explore the opportunity for discovering universal laws connecting the structure of biological networks with their function, positioning them on the spectrum of time-evolving network structure, i.e. dynamics of networks, from highly stable to exquisitely sensitive to perturbation. If such general laws exist, they could transform our ability to predict the response of biological systems to perturbations-an increasingly urgent priority in the face of anthropogenic changes to the environment that affect life across the gamut of organizational scales.

Predictive olfactory learning in Drosophila

Olfactory learning and conditioning in the fruit fly is typically modelled by correlation-based associative synaptic plasticity. It was shown that the conditioning of an odor-evoked response by a shock depends on the connections from Kenyon cells (KC) to mushroom body output neurons (MBONs). Although on the behavioral level conditioning is recognized to be predictive, it remains unclear how MBONs form predictions of aversive or appetitive values (valences) of odors on the circuit level. We present behavioral experiments that are not well explained by associative plasticity between conditioned and unconditioned stimuli, and we suggest two alternative models for how predictions can be formed. In error-driven predictive plasticity, dopaminergic neurons (DANs) represent the error between the predictive odor value and the shock strength. In target-driven predictive plasticity, the DANs represent the target for the predictive MBON activity. Predictive plasticity in KC-to-MBON synapses can also explain trace-conditioning, the valence-dependent sign switch in plasticity, and the observed novelty-familiarity representation. The model offers a framework to dissect MBON circuits and interpret DAN activity during olfactory learning.

Drosophila reward system - A summary of current knowledge

The fruit fly Drosophila melanogaster brain is the most extensively investigated model of a reward system in insects. Drosophila can discriminate between rewarding and punishing environmental stimuli and consequently undergo associative learning. Functional models, especially those modelling mushroom bodies, are constantly being developed using newly discovered information, adding to the complexity of creating a simple model of the reward system. This review aims to clarify whether its reward system also includes a hedonic component. Neurochemical systems that mediate the 'wanting' component of reward in the Drosophila brain are well documented, however, the systems that mediate the pleasure component of reward in mammals, including those involving the endogenous opioid and endocannabinoid systems, are unlikely to be present in insects. The mushroom body components exhibit differential developmental age and different functional processes. We propose a hypothetical hierarchy of the levels of reinforcement processing in response to particular stimuli, and the parallel processes that take place concurrently. The possible presence of activity-silencing and meta-satiety inducing levels in Drosophila should be further investigated.

The connectome of the adult Drosophila mushroom body provides insights into function

Making inferences about the computations performed by neuronal circuits from synapse-level connectivity maps is an emerging opportunity in neuroscience. The mushroom body (MB) is well positioned for developing and testing such an approach due to its conserved neuronal architecture, recently completed dense connectome, and extensive prior experimental studies of its roles in learning, memory, and activity regulation. Here, we identify new components of the MB circuit in Drosophila, including extensive visual input and MB output neurons (MBONs) with direct connections to descending neurons. We find unexpected structure in sensory inputs, in the transfer of information about different sensory modalities to MBONs, and in the modulation of that transfer by dopaminergic neurons (DANs). We provide insights into the circuitry used to integrate MB outputs, connectivity between the MB and the central complex and inputs to DANs, including feedback from MBONs. Our results provide a foundation for further theoretical and experimental work.

Non-synaptic interactions between olfactory receptor neurons, a possible key feature of odor processing in flies.. [DOI: 10.1101/2020.07.23.217216]

When flies explore their environment, they encounter odors in complex, highly intermittent plumes. To navigate a plume and, for example, find food, they must solve several challenges, including reliably identifying mixtures of odorants and their intensities, and discriminating odorant mixtures emanating from a single source from odorants emitted from separate sources and just mixing in the air. Lateral inhibition in the antennal lobe is commonly understood to help solving these challenges. With a computational model of the Drosophila olfactory system, we analyze the utility of an alternative mechanism for solving them: Non-synaptic (“ephaptic”) interactions (NSIs) between olfactory receptor neurons that are stereotypically co-housed in the same sensilla.

We found that NSIs improve mixture ratio detection and plume structure sensing and they do so more efficiently than the traditionally considered mechanism of lateral inhibition in the antennal lobe. However, we also found that NSIs decrease the dynamic range of co-housed ORNs, especially when they have similar sensitivity to an odorant. These results shed light, from a functional perspective, on the role of NSIs, which are normally avoided between neurons, for instance by myelination.

Author summary

Myelin is important to isolate neurons and avoid disruptive electrical interference between them; it can be found in almost any neural assembly. However, there are a few exceptions to this rule and it remains unclear why. One particularly interesting case is the electrical interaction between olfactory sensory neurons co-housed in the sensilla of insects. Here, we created a computational model of the early stages of the Drosophila olfactory system and observed that the electrical interference between olfactory receptor neurons can be a useful trait that can help flies, and other insects, to navigate the complex plumes of odorants in their natural environment.

With the model we were able to shed new light on the trade-off of adopting this mechanism: We found that the non-synaptic interactions (NSIs) improve both the identification of the concentration ratio in mixtures of odorants and the discrimination of odorant mixtures emanating from a single source from odorants emitted from separate sources – both highly advantageous. However, they also decrease the dynamic range of the olfactory sensory neurons – a clear disadvantage.

Numerical Cognition Based on Precise Counting with a Single Spiking Neuron

Insects are able to solve basic numerical cognition tasks. We show that estimation of numerosity can be realized and learned by a single spiking neuron with an appropriate synaptic plasticity rule. This model can be efficiently trained to detect arbitrary spatiotemporal spike patterns on a noisy and dynamic background with high precision and low variance. When put to test in a task that requires counting of visual concepts in a static image it required considerably less training epochs than a convolutional neural network to achieve equal performance. When mimicking a behavioral task in free-flying bees that requires numerical cognition, the model reaches a similar success rate in making correct decisions. We propose that using action potentials to represent basic numerical concepts with a single spiking neuron is beneficial for organisms with small brains and limited neuronal resources.

Mechanisms of Plastic Rescue in Novel Environments

Adaptive phenotypic plasticity provides a mechanism of developmental rescue in novel and rapidly changing environments. Understanding the underlying mechanism of plasticity is important for predicting both the likelihood that a developmental response is adaptive and associated life-history trade-offs that could influence patterns of subsequent evolutionary rescue. Although evolved developmental switches may move organisms toward a new adaptive peak in a novel environment, such mechanisms often result in maladaptive responses. The induction of generalized physiological mechanisms in new environments is relatively more likely to result in adaptive responses to factors such as novel toxins, heat stress, or pathogens. Developmental selection forms of plasticity, which rely on within-individual selective processes, such as shaping of tissue architecture, trial-and-error learning, or acquired immunity, are particularly likely to result in adaptive plasticity in a novel environment. However, both the induction of plastic responses and the ability to be plastic through developmental selection come with significant costs, resulting in delays in reproduction, increased individual investment, and reduced fecundity. Thus, we might expect complex interactions between plastic responses that allow survival in novel environments and subsequent evolutionary responses at the population level.

Abstract concept learning in a simple neural network inspired by the insect brain

The capacity to learn abstract concepts such as 'sameness' and 'difference' is considered a higher-order cognitive function, typically thought to be dependent on top-down neocortical processing. It is therefore surprising that honey bees apparantly have this capacity. Here we report a model of the structures of the honey bee brain that can learn sameness and difference, as well as a range of complex and simple associative learning tasks. Our model is constrained by the known connections and properties of the mushroom body, including the protocerebral tract, and provides a good fit to the learning rates and performances of real bees in all tasks, including learning sameness and difference. The model proposes a novel mechanism for learning the abstract concepts of 'sameness' and 'difference' that is compatible with the insect brain, and is not dependent on top-down or executive control processing.

A neural network model for familiarity and context learning during honeybee foraging flights

How complex is the memory structure that honeybees use to navigate? Recently, an insect-inspired parsimonious spiking neural network model was proposed that enabled simulated ground-moving agents to follow learned routes. We adapted this model to flying insects and evaluate the route following performance in three different worlds with gradually decreasing object density. In addition, we propose an extension to the model to enable the model to associate sensory input with a behavioral context, such as foraging or homing. The spiking neural network model makes use of a sparse stimulus representation in the mushroom body and reward-based synaptic plasticity at its output synapses. In our experiments, simulated bees were able to navigate correctly even when panoramic cues were missing. The context extension we propose enabled agents to successfully discriminate partly overlapping routes. The structure of the visual environment, however, crucially determines the success rate. We find that the model fails more often in visually rich environments due to the overlap of features represented by the Kenyon cell layer. Reducing the landmark density improves the agents route following performance. In very sparse environments, we find that extended landmarks, such as roads or field edges, may help the agent stay on its route, but often act as strong distractors yielding poor route following performance. We conclude that the presented model is valid for simple route following tasks and may represent one component of insect navigation. Additional components might still be necessary for guidance and action selection while navigating along different memorized routes in complex natural environments.

An insect-like mushroom body in a crustacean brain

Mushroom bodies are the iconic learning and memory centers of insects. No previously described crustacean possesses a mushroom body as defined by strict morphological criteria although crustacean centers called hemiellipsoid bodies, which serve functions in sensory integration, have been viewed as evolutionarily convergent with mushroom bodies. Here, using key identifiers to characterize neural arrangements, we demonstrate insect-like mushroom bodies in stomatopod crustaceans (mantis shrimps). More than any other crustacean taxon, mantis shrimps display sophisticated behaviors relating to predation, spatial memory, and visual recognition comparable to those of insects. However, neuroanatomy-based cladistics suggesting close phylogenetic proximity of insects and stomatopod crustaceans conflicts with genomic evidence showing hexapods closely related to simple crustaceans called remipedes. We discuss whether corresponding anatomical phenotypes described here reflect the cerebral morphology of a common ancestor of Pancrustacea or an extraordinary example of convergent evolution.

With more than four million species, arthropods are the largest and most diverse group of animals on the planet and include, for example, crustaceans, insects and spiders. They are defined by their segmented bodies, hard outer skeletons and jointed limbs. All arthropods share a common ancestor that lived more than 550 million years ago. Exactly how this ancestral arthropod gave rise to the myriad species that exist today is unclear but we know that at some point the arthropod family tree split into branches, one of which went on to become the crustaceans. The crustacean branch then split again, giving rise to a line of descendants that would become the insects.

But although insects evolved from crustaceans, the brains of insects possess structures that those of crustaceans do not. Known as mushroom bodies, these structures help to form and store memories. Their absence in crustaceans has therefore been an enduring mystery. Wolff et al. now add a piece to the puzzle by showing that one group of modern-day crustaceans, the mantis shrimps, does in fact possess mushroom bodies.

By visualizing cells and pathways within the brains of mantis shrimps, and also a number of closely related species, Wolff et al. show that only these shrimps possess true mushroom bodies. However, some of the mantis shrimp’s close relatives possess a few attributes of these structures. This suggests that mushroom bodies are evolutionarily ancient structures that arose in a common ancestor of insects and crustaceans, before being lost or radically modified in most of the crustaceans.

So why did this happen? Mantis shrimps are top predators with excellent vision that hunt over considerable distances, requiring them to evaluate and memorize complex features of their environment. These cognitive demands, which might not be shared by other crustaceans, may have led to the mantis shrimps retaining their mushroom bodies. Further research into the brains and behavior of the mantis shrimp may provide insights into how mushroom bodies construct memories of a complex sensory world.

Olfactory learning without the mushroom bodies: Spiking neural network models of the honeybee lateral antennal lobe tract reveal its capacities in odour memory tasks of varied complexities

The honeybee olfactory system is a well-established model for understanding functional mechanisms of learning and memory. Olfactory stimuli are first processed in the antennal lobe, and then transferred to the mushroom body and lateral horn through dual pathways termed medial and lateral antennal lobe tracts (m-ALT and l-ALT). Recent studies reported that honeybees can perform elemental learning by associating an odour with a reward signal even after lesions in m-ALT or blocking the mushroom bodies. To test the hypothesis that the lateral pathway (l-ALT) is sufficient for elemental learning, we modelled local computation within glomeruli in antennal lobes with axons of projection neurons connecting to a decision neuron (LHN) in the lateral horn. We show that inhibitory spike-timing dependent plasticity (modelling non-associative plasticity by exposure to different stimuli) in the synapses from local neurons to projection neurons decorrelates the projection neurons' outputs. The strength of the decorrelations is regulated by global inhibitory feedback within antennal lobes to the projection neurons. By additionally modelling octopaminergic modification of synaptic plasticity among local neurons in the antennal lobes and projection neurons to LHN connections, the model can discriminate and generalize olfactory stimuli. Although positive patterning can be accounted for by the l-ALT model, negative patterning requires further processing and mushroom body circuits. Thus, our model explains several-but not all-types of associative olfactory learning and generalization by a few neural layers of odour processing in the l-ALT. As an outcome of the combination between non-associative and associative learning, the modelling approach allows us to link changes in structural organization of honeybees' antennal lobes with their behavioural performances over the course of their life.

A Simple Computational Model of the Bee Mushroom Body Can Explain Seemingly Complex Forms of Olfactory Learning and Memory

Honeybees are models for studying how animals with relatively small brains accomplish complex cognition, displaying seemingly advanced (or "non-elemental") learning phenomena involving multiple conditioned stimuli. These include "peak shift" [1-4]-where animals not only respond to entrained stimuli, but respond even more strongly to similar ones that are farther away from non-rewarding stimuli. Bees also display negative and positive patterning discrimination [5], responding in opposite ways to mixtures of two odors than to individual odors. Since Pavlov, it has often been assumed that such phenomena are more complex than simple associate learning. We present a model of connections between olfactory sensory input and bees' mushroom bodies [6], incorporating empirically determined properties of mushroom body circuitry (random connectivity [7], sparse coding [8], and synaptic plasticity [9, 10]). We chose not to optimize the model's parameters to replicate specific behavioral phenomena, because we were interested in the emergent cognitive capacities that would pop out of a network constructed solely based on empirical neuroscientific information and plausible assumptions for unknown parameters. We demonstrate that the circuitry mediating "simple" associative learning can also replicate the various non-elemental forms of learning mentioned above and can effectively multi-task by replicating a range of different learning feats. We found that PN-KC synaptic plasticity is crucial in controlling the generalization-discrimination trade-off-it facilitates peak shift and hinders patterning discrimination-and that PN-to-KC connection number can affect this trade-off. These findings question the notion that forms of learning that have been regarded as "higher order" are computationally more complex than "simple" associative learning.

A Fly-Inspired Mushroom Bodies Model for Sensory-Motor Control Through Sequence and Subsequence Learning

Classification and sequence learning are relevant capabilities used by living beings to extract complex information from the environment for behavioral control. The insect world is full of examples where the presentation time of specific stimuli shapes the behavioral response. On the basis of previously developed neural models, inspired by Drosophila melanogaster, a new architecture for classification and sequence learning is here presented under the perspective of the Neural Reuse theory. Classification of relevant input stimuli is performed through resonant neurons, activated by the complex dynamics generated in a lattice of recurrent spiking neurons modeling the insect Mushroom Bodies neuropile. The network devoted to context formation is able to reconstruct the learned sequence and also to trace the subsequences present in the provided input. A sensitivity analysis to parameter variation and noise is reported. Experiments on a roving robot are reported to show the capabilities of the architecture used as a neural controller.

Neural mechanisms of insect navigation

We know more about the ethology of insect navigation than the neural substrates. Few studies have shown direct effects of brain manipulation on navigational behaviour; or measure brain responses that clearly relate to the animal's current location or spatial target, independently of specific sensory cues. This is partly due to the methodological problems of obtaining neural data in a naturally behaving animal. However, substantial indirect evidence, such as comparative anatomy and knowledge of the neural circuits that provide relevant sensory inputs provide converging arguments for the role of some specific brain areas: the mushroom bodies; and the central complex. Finally, modelling can help bridge the gap by relating the computational requirements of a given navigational task to the type of computation offered by different brain areas.

What insects can tell us about the origins of consciousness

How, why, and when consciousness evolved remain hotly debated topics. Addressing these issues requires considering the distribution of consciousness across the animal phylogenetic tree. Here we propose that at least one invertebrate clade, the insects, has a capacity for the most basic aspect of consciousness: subjective experience. In vertebrates the capacity for subjective experience is supported by integrated structures in the midbrain that create a neural simulation of the state of the mobile animal in space. This integrated and egocentric representation of the world from the animal's perspective is sufficient for subjective experience. Structures in the insect brain perform analogous functions. Therefore, we argue the insect brain also supports a capacity for subjective experience. In both vertebrates and insects this form of behavioral control system evolved as an efficient solution to basic problems of sensory reafference and true navigation. The brain structures that support subjective experience in vertebrates and insects are very different from each other, but in both cases they are basal to each clade. Hence we propose the origins of subjective experience can be traced to the Cambrian.

Using multilayer perceptron computation to discover ideal insect olfactory receptor combinations in the mosquito and fruit fly for an efficient electronic nose

The model organism, Drosophila melanogaster, and the mosquito Anopheles gambiae use 60 and 79 odorant receptors, respectively, to sense their olfactory world. However, a commercial "electronic nose" in the form of an insect olfactory biosensor demands very low numbers of receptors at its front end of detection due to the difficulties of receptor/sensor integration and functionalization. In this letter, we demonstrate how computation via artificial neural networks (ANNs), in the form of multilayer perceptrons (MLPs), can be successfully incorporated as the signal processing back end of the biosensor to drastically reduce the number of receptors to three while still retaining 100% performance of odorant detection to that of a full complement of receptors. In addition, we provide a detailed performance comparison between D. melanogaster and A. gambiae odorant receptors and demonstrate that A. gambiae receptors provide superior olfaction detection performance over D. melanogaster for very low receptor numbers. The results from this study present the possibility of using the computation of MLPs to discover ideal biological olfactory receptors for an olfactory biosensor device to provide maximum classification performance of unknown odorants.

The neuronal architecture of the mushroom body provides a logic for associative learning

We identified the neurons comprising the Drosophila mushroom body (MB), an associative center in invertebrate brains, and provide a comprehensive map describing their potential connections. Each of the 21 MB output neuron (MBON) types elaborates segregated dendritic arbors along the parallel axons of ∼2000 Kenyon cells, forming 15 compartments that collectively tile the MB lobes. MBON axons project to five discrete neuropils outside of the MB and three MBON types form a feedforward network in the lobes. Each of the 20 dopaminergic neuron (DAN) types projects axons to one, or at most two, of the MBON compartments. Convergence of DAN axons on compartmentalized Kenyon cell-MBON synapses creates a highly ordered unit that can support learning to impose valence on sensory representations. The elucidation of the complement of neurons of the MB provides a comprehensive anatomical substrate from which one can infer a functional logic of associative olfactory learning and memory.

Computational models to understand decision making and pattern recognition in the insect brain

Odor stimuli reaching olfactory systems of mammals and insects are characterized by remarkable non-stationary and noisy time series. Their brains have evolved to discriminate subtle changes in odor mixtures and find meaningful variations in complex spatio-temporal patterns. Insects with small brains can effectively solve two computational tasks: identify the presence of an odor type and estimate the concentration levels of the odor. Understanding the learning and decision making processes in the insect brain can not only help us to uncover general principles of information processing in the brain, but it can also provide key insights to artificial chemical sensing. Both olfactory learning and memory are dominantly organized in the Antennal Lobe (AL) and the Mushroom Bodies (MBs). Current computational models yet fail to deliver an integrated picture of the joint computational roles of the AL and MBs. This review intends to provide an integrative overview of the computational literature analyzed in the context of the problem of classification (odor discrimination) and regression (odor concentration estimation), particularly identifying key computational ingredients necessary to solve pattern recognition.

Peak shift in honey bee olfactory learning

If animals are trained with two similar stimuli such that one is rewarding (S+) and one punishing (S-), then following training animals show a greatest preference not for the S+, but for a novel stimulus that is slightly more different from the S- than the S+ is. This peak shift phenomenon has been widely reported for vertebrates and has recently been demonstrated for bumblebees and honey bees. To explore the nature of peak shift in invertebrates further, here we examined the properties of peak shift in honey bees trained in a free-flight olfactory learning assay. Hexanal and heptanol were mixed in different ratios to create a continuum of odour stimuli. Bees were trained to artificial flowers such that one odour mixture was rewarded with 2 molar sucrose (S+), and one punished with distasteful quinine (S-). After training, bees were given a non-rewarded preference test with five different mixtures of hexanal and heptanol. Following training bees' maximal preference was for an odour mixture slightly more distinct from the S- than the trained S+. This effect was not seen if bees were initially trained with two distinct odours, replicating the classic features of peak shift reported for vertebrates. We propose a conceptual model of how peak shift might occur in honey bees. We argue that peak shift does not require any higher level of processing than the known olfactory learning circuitry of the bee brain and suggest that peak shift is a very general feature of discrimination learning.

Olfactory coding in the insect brain: data and conjectures

Much progress has been made recently in understanding how olfactory coding works in insect brains. Here, I propose a wiring diagram for the major steps from the first processing network (the antennal lobe) to behavioral readout. I argue that the sequence of lateral inhibition in the antennal lobe, non-linear synapses, threshold-regulating gated spring network, selective lateral inhibitory networks across glomeruli, and feedforward inhibition to the lateral protocerebrum cover most of the experimental results from different research groups and model species. I propose that the main difference between mushroom bodies and the lateral protocerebrum is not about learned vs. innate behavior. Rather, mushroom bodies perform odor identification, whereas the lateral protocerebrum performs odor evaluation (both learned and innate). I discuss the concepts of labeled line and combinatorial coding and postulate that, under restrictive experimental conditions, these networks lead to an apparent existence of 'labeled line' coding for special odors. Modulatory networks are proposed as switches between different evaluating systems in the lateral protocerebrum. A review of experimental data and theoretical conjectures both contribute to this synthesis, creating new hypotheses for future research.

A spiking network model of decision making employing rewarded STDP

Reward-modulated spike timing dependent plasticity (STDP) combines unsupervised STDP with a reinforcement signal that modulates synaptic changes. It was proposed as a learning rule capable of solving the distal reward problem in reinforcement learning. Nonetheless, performance and limitations of this learning mechanism have yet to be tested for its ability to solve biological problems. In our work, rewarded STDP was implemented to model foraging behavior in a simulated environment. Over the course of training the network of spiking neurons developed the capability of producing highly successful decision-making. The network performance remained stable even after significant perturbations of synaptic structure. Rewarded STDP alone was insufficient to learn effective decision making due to the difficulty maintaining homeostatic equilibrium of synaptic weights and the development of local performance maxima. Our study predicts that successful learning requires stabilizing mechanisms that allow neurons to balance their input and output synapses as well as synaptic noise.

A spiking neural network model of self-organized pattern recognition in the early mammalian olfactory system

Olfactory sensory information passes through several processing stages before an odor percept emerges. The question how the olfactory system learns to create odor representations linking those different levels and how it learns to connect and discriminate between them is largely unresolved. We present a large-scale network model with single and multi-compartmental Hodgkin-Huxley type model neurons representing olfactory receptor neurons (ORNs) in the epithelium, periglomerular cells, mitral/tufted cells and granule cells in the olfactory bulb (OB), and three types of cortical cells in the piriform cortex (PC). Odor patterns are calculated based on affinities between ORNs and odor stimuli derived from physico-chemical descriptors of behaviorally relevant real-world odorants. The properties of ORNs were tuned to show saturated response curves with increasing concentration as seen in experiments. On the level of the OB we explored the possibility of using a fuzzy concentration interval code, which was implemented through dendro-dendritic inhibition leading to winner-take-all like dynamics between mitral/tufted cells belonging to the same glomerulus. The connectivity from mitral/tufted cells to PC neurons was self-organized from a mutual information measure and by using a competitive Hebbian-Bayesian learning algorithm based on the response patterns of mitral/tufted cells to different odors yielding a distributed feed-forward projection to the PC. The PC was implemented as a modular attractor network with a recurrent connectivity that was likewise organized through Hebbian-Bayesian learning. We demonstrate the functionality of the model in a one-sniff-learning and recognition task on a set of 50 odorants. Furthermore, we study its robustness against noise on the receptor level and its ability to perform concentration invariant odor recognition. Moreover, we investigate the pattern completion capabilities of the system and rivalry dynamics for odor mixtures.

A neuromorphic network for generic multivariate data classification

Computational neuroscience has uncovered a number of computational principles used by nervous systems. At the same time, neuromorphic hardware has matured to a state where fast silicon implementations of complex neural networks have become feasible. En route to future technical applications of neuromorphic computing the current challenge lies in the identification and implementation of functional brain algorithms. Taking inspiration from the olfactory system of insects, we constructed a spiking neural network for the classification of multivariate data, a common problem in signal and data analysis. In this model, real-valued multivariate data are converted into spike trains using "virtual receptors" (VRs). Their output is processed by lateral inhibition and drives a winner-take-all circuit that supports supervised learning. VRs are conveniently implemented in software, whereas the lateral inhibition and classification stages run on accelerated neuromorphic hardware. When trained and tested on real-world datasets, we find that the classification performance is on par with a naïve Bayes classifier. An analysis of the network dynamics shows that stable decisions in output neuron populations are reached within less than 100 ms of biological time, matching the time-to-decision reported for the insect nervous system. Through leveraging a population code, the network tolerates the variability of neuronal transfer functions and trial-to-trial variation that is inevitably present on the hardware system. Our work provides a proof of principle for the successful implementation of a functional spiking neural network on a configurable neuromorphic hardware system that can readily be applied to real-world computing problems.

Neural circuits: random design of a higher-order olfactory projection

A recent study in Drosophila has found that the connectivity between the first olfactory processing center, the antennal lobe, and one of its targets, the mushroom body, is apparently random. This supports the idea that the mushroom body is designed for learning arbitrary odor features.

Natural image sequences constrain dynamic receptive fields and imply a sparse code

In their natural environment, animals experience a complex and dynamic visual scenery. Under such natural stimulus conditions, neurons in the visual cortex employ a spatially and temporally sparse code. For the input scenario of natural still images, previous work demonstrated that unsupervised feature learning combined with the constraint of sparse coding can predict physiologically measured receptive fields of simple cells in the primary visual cortex. This convincingly indicated that the mammalian visual system is adapted to the natural spatial input statistics. Here, we extend this approach to the time domain in order to predict dynamic receptive fields that can account for both spatial and temporal sparse activation in biological neurons. We rely on temporal restricted Boltzmann machines and suggest a novel temporal autoencoding training procedure. When tested on a dynamic multi-variate benchmark dataset this method outperformed existing models of this class. Learning features on a large dataset of natural movies allowed us to model spatio-temporal receptive fields for single neurons. They resemble temporally smooth transformations of previously obtained static receptive fields and are thus consistent with existing theories. A neuronal spike response model demonstrates how the dynamic receptive field facilitates temporal and population sparseness. We discuss the potential mechanisms and benefits of a spatially and temporally sparse representation of natural visual input.

Design principles of the sparse coding network and the role of "sister cells" in the olfactory system of Drosophila

Sensory systems face the challenge to represent sensory inputs in a way to allow easy readout of sensory information by higher brain areas. In the olfactory system of the fly drosopohila melanogaster, projection neurons (PNs) of the antennal lobe (AL) convert a dense activation of glomeruli into a sparse, high-dimensional firing pattern of Kenyon cells (KCs) in the mushroom body (MB). Here we investigate the design principles of the olfactory system of drosophila in regard to the capabilities to discriminate odor quality from the MB representation and its robustness to different types of noise. We focus on understanding the role of highly correlated homotypic projection neurons (“sister cells”) found in the glomeruli of flies. These cells are coupled by gap-junctions and receive almost identical sensory inputs, but target randomly different KCs in MB. We show that sister cells might play a crucial role in increasing the robustness of the MB odor representation to noise. Computationally, sister cells thus might help the system to improve the generalization capabilities in face of noise without impairing the discriminability of odor quality at the same time.

Gain control network conditions in early sensory coding

Gain control is essential for the proper function of any sensory system. However, the precise mechanisms for achieving effective gain control in the brain are unknown. Based on our understanding of the existence and strength of connections in the insect olfactory system, we analyze the conditions that lead to controlled gain in a randomly connected network of excitatory and inhibitory neurons. We consider two scenarios for the variation of input into the system. In the first case, the intensity of the sensory input controls the input currents to a fixed proportion of neurons of the excitatory and inhibitory populations. In the second case, increasing intensity of the sensory stimulus will both, recruit an increasing number of neurons that receive input and change the input current that they receive. Using a mean field approximation for the network activity we derive relationships between the parameters of the network that ensure that the overall level of activity of the excitatory population remains unchanged for increasing intensity of the external stimulation. We find that, first, the main parameters that regulate network gain are the probabilities of connections from the inhibitory population to the excitatory population and of the connections within the inhibitory population. Second, we show that strict gain control is not achievable in a random network in the second case, when the input recruits an increasing number of neurons. Finally, we confirm that the gain control conditions derived from the mean field approximation are valid in simulations of firing rate models and Hodgkin-Huxley conductance based models.

Data-driven honeybee antennal lobe model suggests how stimulus-onset asynchrony can aid odour segregation

Insects have a remarkable ability to identify and track odour sources in multi-odour backgrounds. Recent behavioural experiments show that this ability relies on detecting millisecond stimulus asynchronies between odourants that originate from different sources. Honeybees, Apis mellifera, are able to distinguish mixtures where both odourants arrive at the same time (synchronous mixtures) from those where odourant onsets are staggered (asynchronous mixtures) down to an onset delay of only 6ms. In this paper we explore this surprising ability in a model of the insects' primary olfactory brain area, the antennal lobe. We hypothesize that a winner-take-all inhibitory network of local neurons in the antennal lobe has a symmetry-breaking effect, such that the response pattern in projection neurons to an asynchronous mixture is different from the response pattern to the corresponding synchronous mixture for an extended period of time beyond the initial odourant onset where the two mixture conditions actually differ. The prolonged difference between response patterns to synchronous and asynchronous mixtures could facilitate odoursegregation in downstream circuits of the olfactory pathway. We present a detailed data-driven model of the bee antennal lobe that reproduces a large data set of experimentally observed physiological odour responses, successfully implements the hypothesised symmetry-breaking mechanism and so demonstrates that this mechanism is consistent with our current knowledge of the olfactory circuits in the bee brain. This article is part of a Special Issue entitled Neural Coding 2012.

A computational framework for understanding decision making through integration of basic learning rules

Nonassociative and associative learning rules simultaneously modify neural circuits. However, it remains unclear how these forms of plasticity interact to produce conditioned responses. Here we integrate nonassociative and associative conditioning within a uniform model of olfactory learning in the honeybee. Honeybees show a fairly abrupt increase in response after a number of conditioning trials. The occurrence of this abrupt change takes many more trials after exposure to nonassociative trials than just using associative conditioning. We found that the interaction of unsupervised and supervised learning rules is critical for explaining latent inhibition phenomenon. Associative conditioning combined with the mutual inhibition between the output neurons produces an abrupt increase in performance despite smooth changes of the synaptic weights. The results show that an integrated set of learning rules implemented using fan-out connectivities together with neural inhibition can explain the broad range of experimental data on learning behaviors.

Neural sensitivity to odorants in deprived and normal olfactory bulbs

Early olfactory deprivation in rodents is accompanied by an homeostatic regulation of the synaptic connectivity in the olfactory bulb (OB). However, its consequences in the neural sensitivity and discrimination have not been elucidated. We compared the odorant sensitivity and discrimination in early sensory deprived and normal OBs in anesthetized rats. We show that the deprived OB exhibits an increased sensitivity to different odorants when compared to the normal OB. Our results indicate that early olfactory stimulation enhances discriminability of the olfactory stimuli. We found that deprived olfactory bulbs adjusts the overall excitatory and inhibitory mitral cells (MCs) responses to odorants but the receptive fields become wider than in the normal olfactory bulbs. Taken together, these results suggest that an early natural sensory stimulation sharpens the receptor fields resulting in a larger discrimination capability. These results are consistent with previous evidence that a varied experience with odorants modulates the OB's synaptic connections and increases MCs selectivity.

Network architecture underlying maximal separation of neuronal representations

One of the most basic and general tasks faced by all nervous systems is extracting relevant information from the organism's surrounding world. While physical signals available to sensory systems are often continuous, variable, overlapping, and noisy, high-level neuronal representations used for decision-making tend to be discrete, specific, invariant, and highly separable. This study addresses the question of how neuronal specificity is generated. Inspired by experimental findings on network architecture in the olfactory system of the locust, I construct a highly simplified theoretical framework which allows for analytic solution of its key properties. For generalized feed-forward systems, I show that an intermediate range of connectivity values between source- and target-populations leads to a combinatorial explosion of wiring possibilities, resulting in input spaces which are, by their very nature, exquisitely sparsely populated. In particular, connection probability ½, as found in the locust antennal-lobe-mushroom-body circuit, serves to maximize separation of neuronal representations across the target Kenyon cells (KCs), and explains their specific and reliable responses. This analysis yields a function expressing response specificity in terms of lower network parameters; together with appropriate gain control this leads to a simple neuronal algorithm for generating arbitrarily sparse and selective codes and linking network architecture and neural coding. I suggest a straightforward way to construct ecologically meaningful representations from this code.

Ensemble response in mushroom body output neurons of the honey bee outpaces spatiotemporal odor processing two synapses earlier in the antennal lobe

Neural representations of odors are subject to computations that involve sequentially convergent and divergent anatomical connections across different areas of the brains in both mammals and insects. Furthermore, in both mammals and insects higher order brain areas are connected via feedback connections. In order to understand the transformations and interactions that this connectivity make possible, an ideal experiment would compare neural responses across different, sequential processing levels. Here we present results of recordings from a first order olfactory neuropile – the antennal lobe (AL) – and a higher order multimodal integration and learning center – the mushroom body (MB) – in the honey bee brain. We recorded projection neurons (PN) of the AL and extrinsic neurons (EN) of the MB, which provide the outputs from the two neuropils. Recordings at each level were made in different animals in some experiments and simultaneously in the same animal in others. We presented two odors and their mixture to compare odor response dynamics as well as classification speed and accuracy at each neural processing level. Surprisingly, the EN ensemble significantly starts separating odor stimuli rapidly and before the PN ensemble has reached significant separation. Furthermore the EN ensemble at the MB output reaches a maximum separation of odors between 84–120 ms after odor onset, which is 26 to 133 ms faster than the maximum separation at the AL output ensemble two synapses earlier in processing. It is likely that a subset of very fast PNs, which respond before the ENs, may initiate the rapid EN ensemble response. We suggest therefore that the timing of the EN ensemble activity would allow retroactive integration of its signal into the ongoing computation of the AL via centrifugal feedback.

Inhibition in multiclass classification

The role of inhibition is investigated in a multiclass support vector machine formalism inspired by the brain structure of insects. The so-called mushroom bodies have a set of output neurons, or classification functions, that compete with each other to encode a particular input. Strongly active output neurons depress or inhibit the remaining outputs without knowing which is correct or incorrect. Accordingly, we propose to use a classification function that embodies unselective inhibition and train it in the large margin classifier framework. Inhibition leads to more robust classifiers in the sense that they perform better on larger areas of appropriate hyperparameters when assessed with leave-one-out strategies. We also show that the classifier with inhibition is a tight bound to probabilistic exponential models and is Bayes consistent for 3-class problems. These properties make this approach useful for data sets with a limited number of labeled examples. For larger data sets, there is no significant comparative advantage to other multiclass SVM approaches.