A normative theory of forgetting: lessons from the fruit fly

Investigations of forgetting in Caenorhabditis elegans

The traditional view considered forgetting as a passive process where memory traces gradually fade due to the natural weakening of neural connections. However, studies on olfactory memory in Drosophila have revealed that forgetting is an active process controlled by specific neural circuits. Caenorhabditis elegans is a widely used model organism in neurobiological research due to its relatively simple nervous system. Despite its simplicity, C. elegans exhibits complex behaviors influenced by environmental factors and prior experiences. Similar to Drosophila, C. elegans can actively initiate neural circuits based on the type of memory that needs to be forgotten, which supports using C. elegans as a model for studying forgetting. These characteristics facilitate the identification of genes and pathways involved in forgetting in C. elegans. In this review, we discuss recent advances in understanding forgetting mechanisms in C. elegans through three well-characterized olfactory learning paradigms. The insights derived from C. elegans offer a valuable framework for understanding the molecular and cellular mechanisms underlying forgetting, with potentially broader implications for memory regulation in more complex organisms.

Mechanical vibration patterns elicit behavioral transitions and habituation in crawling Drosophila larvae

How animals respond to repeatedly applied stimuli, and how animals respond to mechanical stimuli in particular, are important questions in behavioral neuroscience. We study adaptation to repeated mechanical agitation using the Drosophila larva. Vertical vibration stimuli elicit a discrete set of responses in crawling larvae: continuation, pause, turn, and reversal. Through high-throughput larva tracking, we characterize how the likelihood of each response depends on vibration intensity and on the timing of repeated vibration pulses. By examining transitions between behavioral states at the population and individual levels, we investigate how the animals habituate to the stimulus patterns. We identify time constants associated with desensitization to prolonged vibration, with re-sensitization during removal of a stimulus, and additional layers of habituation that operate in the overall response. Known memory-deficient mutants exhibit distinct behavior profiles and habituation time constants. An analogous simple electrical circuit suggests possible neural and molecular processes behind adaptive behavior.

Interaction time with conspecifics induces food preference or aversion in the wild Algerian mouse

The social transmission of a novel food preference can avoid unnecessary costs arising from tasting nonedible foods. This type of social learning has been demonstrated in laboratory rats and mice. However, among wild animals, there may be several constraints that make it less effective. Using wild Algerian mice (Mus spretus) tested in the laboratory, we demonstrate that a preference for a novel food can be transmitted between Observer and Demonstrator individuals and that it is maintained for at least 30 days. However, only half of the Observers acquired a preference for the same food as the Demonstrators, and only when the duration of oronasal investigation was above a certain threshold (≥122 s); below this threshold (<122 s), Observers acquired a preference for the alternative food offered, which was maintained for a shorter time. Sex, size, and identity of individuals did not influence the transmission of social information. The results show that different interaction times will result in animals copying or avoiding the food choices of others. This suggests that the transmission of social information among wild animals is complex and probably influenced by many factors (e.g., dominance, familiarity, and health condition), ultimately conditioning the type of interaction between individuals and its outcome. Testing wild animals and the ecological and social constraints they face is, therefore, an important step in our understanding of how effectively social information is transmitted in nature.

Roles of Rac1-Dependent Intrinsic Forgetting in Memory-Related Brain Disorders: Demon or Angel

Animals are required to handle daily massive amounts of information in an ever-changing environment, and the resulting memories and experiences determine their survival and development, which is critical for adaptive evolution. However, intrinsic forgetting, which actively deletes irrelevant information, is equally important for memory acquisition and consolidation. Recently, it has been shown that Rac1 activity plays a key role in intrinsic forgetting, maintaining the balance of the brain's memory management system in a controlled manner. In addition, dysfunctions of Rac1-dependent intrinsic forgetting may contribute to memory deficits in neurological and neurodegenerative diseases. Here, these new findings will provide insights into the neurobiology of memory and forgetting, pathological mechanisms and potential therapies for brain disorders that alter intrinsic forgetting mechanisms.

Forgetting Enhances Episodic Control With Structured Memories

Forgetting is a normal process in healthy brains, and evidence suggests that the mammalian brain forgets more than is required based on limitations of mnemonic capacity. Episodic memories, in particular, are liable to be forgotten over time. Researchers have hypothesized that it may be beneficial for decision making to forget episodic memories over time. Reinforcement learning offers a normative framework in which to test such hypotheses. Here, we show that a reinforcement learning agent that uses an episodic memory cache to find rewards in maze environments can forget a large percentage of older memories without any performance impairments, if they utilize mnemonic representations that contain structural information about space. Moreover, we show that some forgetting can actually provide a benefit in performance compared to agents with unbounded memories. Our analyses of the agents show that forgetting reduces the influence of outdated information and states which are not frequently visited on the policies produced by the episodic control system. These results support the hypothesis that some degree of forgetting can be beneficial for decision making, which can help to explain why the brain forgets more than is required by capacity limitations.

Learning as filtering: Implications for spike-based plasticity

Most normative models in computational neuroscience describe the task of learning as the optimisation of a cost function with respect to a set of parameters. However, learning as optimisation fails to account for a time-varying environment during the learning process and the resulting point estimate in parameter space does not account for uncertainty. Here, we frame learning as filtering, i.e., a principled method for including time and parameter uncertainty. We derive the filtering-based learning rule for a spiking neuronal network—the Synaptic Filter—and show its computational and biological relevance. For the computational relevance, we show that filtering improves the weight estimation performance compared to a gradient learning rule with optimal learning rate. The dynamics of the mean of the Synaptic Filter is consistent with spike-timing dependent plasticity (STDP) while the dynamics of the variance makes novel predictions regarding spike-timing dependent changes of EPSP variability. Moreover, the Synaptic Filter explains experimentally observed negative correlations between homo- and heterosynaptic plasticity.

The task of learning is commonly framed as parameter optimisation. Here, we adopt the framework of learning as filtering where the task is to continuously estimate the uncertainty about the parameters to be learned. We apply this framework to synaptic plasticity in a spiking neuronal network. Filtering includes a time-varying environment and parameter uncertainty on the level of the learning task. We show that learning as filtering can qualitatively explain two biological experiments on synaptic plasticity that cannot be explained by learning as optimisation. Moreover, we make a new prediction and improve performance with respect to a gradient learning rule. Thus, learning as filtering is a promising candidate for learning models.

Forgetting as a form of adaptive engram cell plasticity

One leading hypothesis suggests that memories are stored in ensembles of neurons (or 'engram cells') and that successful recall involves reactivation of these ensembles. A logical extension of this idea is that forgetting occurs when engram cells cannot be reactivated. Forms of 'natural forgetting' vary considerably in terms of their underlying mechanisms, time course and reversibility. However, we suggest that all forms of forgetting involve circuit remodelling that switches engram cells from an accessible state (where they can be reactivated by natural recall cues) to an inaccessible state (where they cannot). In many cases, forgetting rates are modulated by environmental conditions and we therefore propose that forgetting is a form of neuroplasticity that alters engram cell accessibility in a manner that is sensitive to mismatches between expectations and the environment. Moreover, we hypothesize that disease states associated with forgetting may hijack natural forgetting mechanisms, resulting in reduced engram cell accessibility and memory loss.

Reward foraging task and model-based analysis reveal how fruit flies learn value of available options

Foraging animals have to evaluate, compare and select food patches in order to increase their fitness. Understanding what drives foraging decisions requires careful manipulation of the value of alternative options while monitoring animals choices. Value-based decision-making tasks in combination with formal learning models have provided both an experimental and theoretical framework to study foraging decisions in lab settings. While these approaches were successfully used in the past to understand what drives choices in mammals, very little work has been done on fruit flies. This is despite the fact that fruit flies have served as model organism for many complex behavioural paradigms. To fill this gap we developed a single-animal, trial-based decision making task, where freely walking flies experienced optogenetic sugar-receptor neuron stimulation. We controlled the value of available options by manipulating the probabilities of optogenetic stimulation. We show that flies integrate reward history of chosen options and forget value of unchosen options. We further discover that flies assign higher values to rewards experienced early in the behavioural session, consistent with formal reinforcement learning models. Finally, we also show that the probabilistic rewards affect walking trajectories of flies, suggesting that accumulated value is controlling the navigation vector of flies in a graded fashion. These findings establish the fruit fly as a model organism to explore the genetic and circuit basis of reward foraging decisions.

Energy efficient synaptic plasticity

Many aspects of the brain’s design can be understood as the result of evolutionary drive toward metabolic efficiency. In addition to the energetic costs of neural computation and transmission, experimental evidence indicates that synaptic plasticity is metabolically demanding as well. As synaptic plasticity is crucial for learning, we examine how these metabolic costs enter in learning. We find that when synaptic plasticity rules are naively implemented, training neural networks requires extremely large amounts of energy when storing many patterns. We propose that this is avoided by precisely balancing labile forms of synaptic plasticity with more stable forms. This algorithm, termed synaptic caching, boosts energy efficiency manifold and can be used with any plasticity rule, including back-propagation. Our results yield a novel interpretation of the multiple forms of neural synaptic plasticity observed experimentally, including synaptic tagging and capture phenomena. Furthermore, our results are relevant for energy efficient neuromorphic designs.

The brain expends a lot of energy. While the organ accounts for only about 2% of a person’s bodyweight, it is responsible for about 20% of our energy use at rest. Neurons use some of this energy to communicate with each other and to process information, but much of the energy is likely used to support learning. A study in fruit flies showed that insects that learned to associate two stimuli and then had their food supply cut off, died 20% earlier than untrained flies. This is thought to be because learning used up the insects’ energy reserves.

If learning a single association requires so much energy, how does the brain manage to store vast amounts of data? Li and van Rossum offer an explanation based on a computer model of neural networks. The advantage of using such a model is that it is possible to control and measure conditions more precisely than in the living brain.

Analysing the model confirmed that learning many new associations requires large amounts of energy. This is particularly true if the memories must be stored with a high degree of accuracy, and if the neural network contains many stored memories already. The reason that learning consumes so much energy is that forming long-term memories requires neurons to produce new proteins. Using the computer model, Li and van Rossum show that neural networks can overcome this limitation by storing memories initially in a transient form that does not require protein synthesis. Doing so reduces energy requirements by as much as 10-fold.

Studies in living brains have shown that transient memories of this type do in fact exist. The current results hence offer a hypothesis as to how the brain can learn in a more energy efficient way. Energy consumption is thought to have placed constraints on brain evolution. It is also often a bottleneck in computers. By revealing how the brain encodes memories energy efficiently, the current findings could thus also inspire new engineering solutions.

Forgetting at biologically realistic levels of neurogenesis in a large-scale hippocampal model

Neurogenesis persists throughout life in the dentate gyrus region of the mammalian hippocampus. Computational models have established that the addition of neurons degrades existing memories (i.e., produces forgetting). These predictions are supported by empirical observations in rodents, where post-training increases in neurogenesis also promote forgetting of hippocampus-dependent memories. However, in these computational models which use 10-1,000 neurons to represent the dentate gyrus, forgetting is only observed at rates of new neuron addition that greatly exceed adult neurogenesis rates observed in vivo. In order to address this, here we generated an artificial neural network which incorporated more realistic features of the hippocampus - including increased network size (with up to 20,000 dentate gyrus neurons), sparse activity, and sparse connectivity - features that were not present in earlier models. In addition, we explored how properties of new neurons - their connectivity, excitability, and plasticity - impact forgetting using a pattern categorization task. Our results revealed that neurogenic networks forget previously learned input-output pattern associations. This forgetting predicted a performance enhancement in subsequent conflictual learning, compared to static networks (with no added neurons). These effects were especially sensitive to changes in increased output connectivity and excitability of new neurons. Crucially, forgetting was observed at much lower rates of neurogenesis in larger networks, with the addition of as little as 0.2% of the total DG population sufficient to induce forgetting.

Analyzing dynamic decision-making models using Chapman-Kolmogorov equations

Decision-making in dynamic environments typically requires adaptive evidence accumulation that weights new evidence more heavily than old observations. Recent experimental studies of dynamic decision tasks require subjects to make decisions for which the correct choice switches stochastically throughout a single trial. In such cases, an ideal observer's belief is described by an evolution equation that is doubly stochastic, reflecting stochasticity in the both observations and environmental changes. In these contexts, we show that the probability density of the belief can be represented using differential Chapman-Kolmogorov equations, allowing efficient computation of ensemble statistics. This allows us to reliably compare normative models to near-normative approximations using, as model performance metrics, decision response accuracy and Kullback-Leibler divergence of the belief distributions. Such belief distributions could be obtained empirically from subjects by asking them to report their decision confidence. We also study how response accuracy is affected by additional internal noise, showing optimality requires longer integration timescales as more noise is added. Lastly, we demonstrate that our method can be applied to tasks in which evidence arrives in a discrete, pulsatile fashion, rather than continuously.

The Biology of Forgetting-A Perspective

Pioneering research studies, beginning with those using Drosophila, have identified several molecular and cellular mechanisms for active forgetting. The currently known mechanisms for active forgetting include neurogenesis-based forgetting, interference-based forgetting, and intrinsic forgetting, the latter term describing the brain's chronic signaling systems that function to slowly degrade molecular and cellular memory traces. The best-characterized pathway for intrinsic forgetting includes "forgetting cells" that release dopamine onto engram cells, mobilizing a signaling pathway that terminates in the activation of Rac1/Cofilin to effect changes in the actin cytoskeleton and neuron/synapse structure. Intrinsic forgetting may be the default state of the brain, constantly promoting memory erasure and competing with processes that promote memory stability like consolidation. A better understanding of active forgetting will provide insights into the brain's memory management system and human brain disorders that alter active forgetting mechanisms.

Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation

It has been suggested that dopamine (DA) represents reward-prediction-error (RPE) defined in reinforcement learning and therefore DA responds to unpredicted but not predicted reward. However, recent studies have found DA response sustained towards predictable reward in tasks involving self-paced behavior, and suggested that this response represents a motivational signal. We have previously shown that RPE can sustain if there is decay/forgetting of learned-values, which can be implemented as decay of synaptic strengths storing learned-values. This account, however, did not explain the suggested link between tonic/sustained DA and motivation. In the present work, we explored the motivational effects of the value-decay in self-paced approach behavior, modeled as a series of ‘Go’ or ‘No-Go’ selections towards a goal. Through simulations, we found that the value-decay can enhance motivation, specifically, facilitate fast goal-reaching, albeit counterintuitively. Mathematical analyses revealed that underlying potential mechanisms are twofold: (1) decay-induced sustained RPE creates a gradient of ‘Go’ values towards a goal, and (2) value-contrasts between ‘Go’ and ‘No-Go’ are generated because while chosen values are continually updated, unchosen values simply decay. Our model provides potential explanations for the key experimental findings that suggest DA's roles in motivation: (i) slowdown of behavior by post-training blockade of DA signaling, (ii) observations that DA blockade severely impairs effortful actions to obtain rewards while largely sparing seeking of easily obtainable rewards, and (iii) relationships between the reward amount, the level of motivation reflected in the speed of behavior, and the average level of DA. These results indicate that reinforcement learning with value-decay, or forgetting, provides a parsimonious mechanistic account for the DA's roles in value-learning and motivation. Our results also suggest that when biological systems for value-learning are active even though learning has apparently converged, the systems might be in a state of dynamic equilibrium, where learning and forgetting are balanced.

Dopamine (DA) has been suggested to have two reward-related roles: (1) representing reward-prediction-error (RPE), and (2) providing motivational drive. Role(1) is based on the physiological results that DA responds to unpredicted but not predicted reward, whereas role(2) is supported by the pharmacological results that blockade of DA signaling causes motivational impairments such as slowdown of self-paced behavior. So far, these two roles are considered to be played by two different temporal patterns of DA signals: role(1) by phasic signals and role(2) by tonic/sustained signals. However, recent studies have found sustained DA signals with features indicative of both roles (1) and (2), complicating this picture. Meanwhile, whereas synaptic/circuit mechanisms for role(1), i.e., how RPE is calculated in the upstream of DA neurons and how RPE-dependent update of learned-values occurs through DA-dependent synaptic plasticity, have now become clarified, mechanisms for role(2) remain unclear. In this work, we modeled self-paced behavior by a series of ‘Go’ or ‘No-Go’ selections in the framework of reinforcement-learning assuming DA's role(1), and demonstrated that incorporation of decay/forgetting of learned-values, which is presumably implemented as decay of synaptic strengths storing learned-values, provides a potential unified mechanistic account for the DA's two roles, together with its various temporal patterns.

Dissecting neural pathways for forgetting in Drosophila olfactory aversive memory

Recent studies have identified molecular pathways driving forgetting and supported the notion that forgetting is a biologically active process. The circuit mechanisms of forgetting, however, remain largely unknown. Here we report two sets of Drosophila neurons that account for the rapid forgetting of early olfactory aversive memory. We show that inactivating these neurons inhibits memory decay without altering learning, whereas activating them promotes forgetting. These neurons, including a cluster of dopaminergic neurons (PAM-β'1) and a pair of glutamatergic neurons (MBON-γ4>γ1γ2), terminate in distinct subdomains in the mushroom body and represent parallel neural pathways for regulating forgetting. Interestingly, although activity of these neurons is required for memory decay over time, they are not required for acute forgetting during reversal learning. Our results thus not only establish the presence of multiple neural pathways for forgetting in Drosophila but also suggest the existence of diverse circuit mechanisms of forgetting in different contexts.

Sleep Facilitates Memory by Blocking Dopamine Neuron-Mediated Forgetting

Early studies from psychology suggest that sleep facilitates memory retention by stopping ongoing retroactive interference caused by mental activity or external sensory stimuli. Neuroscience research with animal models, on the other hand, suggests that sleep facilitates retention by enhancing memory consolidation. Recently, in Drosophila, the ongoing activity of specific dopamine neurons was shown to regulate the forgetting of olfactory memories. Here, we show this ongoing dopaminergic activity is modulated with behavioral state, increasing robustly with locomotor activity and decreasing with rest. Increasing sleep-drive, with either the sleep-promoting agent Gaboxadol or by genetic stimulation of the neural circuit for sleep, decreases ongoing dopaminergic activity, while enhancing memory retention. Conversely, increasing arousal stimulates ongoing dopaminergic activity and accelerates dopaminergic-based forgetting. Therefore, forgetting is regulated by the behavioral state modulation of dopaminergic-based plasticity. Our findings integrate psychological and neuroscience research on sleep and forgetting.

Neurons that Remember How We Got There

In this issue of Neuron, Daie et al. (2015) show that the eye velocity-to-position neural integrator not only encodes the position, but also how it was reached. Representing content and context in the same neuronal population may form a general coding principle.

Information processing in the CNS: a supramolecular chemistry?

How does central nervous system process information? Current theories are based on two tenets: (a) information is transmitted by action potentials, the language by which neurons communicate with each other-and (b) homogeneous neuronal assemblies of cortical circuits operate on these neuronal messages where the operations are characterized by the intrinsic connectivity among neuronal populations. In this view, the size and time course of any spike is stereotypic and the information is restricted to the temporal sequence of the spikes; namely, the "neural code". However, an increasing amount of novel data point towards an alternative hypothesis: (a) the role of neural code in information processing is overemphasized. Instead of simply passing messages, action potentials play a role in dynamic coordination at multiple spatial and temporal scales, establishing network interactions across several levels of a hierarchical modular architecture, modulating and regulating the propagation of neuronal messages. (b) Information is processed at all levels of neuronal infrastructure from macromolecules to population dynamics. For example, intra-neuronal (changes in protein conformation, concentration and synthesis) and extra-neuronal factors (extracellular proteolysis, substrate patterning, myelin plasticity, microbes, metabolic status) can have a profound effect on neuronal computations. This means molecular message passing may have cognitive connotations. This essay introduces the concept of "supramolecular chemistry", involving the storage of information at the molecular level and its retrieval, transfer and processing at the supramolecular level, through transitory non-covalent molecular processes that are self-organized, self-assembled and dynamic. Finally, we note that the cortex comprises extremely heterogeneous cells, with distinct regional variations, macromolecular assembly, receptor repertoire and intrinsic microcircuitry. This suggests that every neuron (or group of neurons) embodies different molecular information that hands an operational effect on neuronal computation.