Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation

Specific Plasticity Loci and Their Synergism Mediate Operant Conditioning

Despite numerous studies examining the mechanisms of operant conditioning (OC), the diversity of OC plasticity loci and their synergism have not been examined sufficiently. In the well-characterized feeding neural circuit of Aplysia, in vivo and in vitro appetitive OC increases neuronal excitability and electrical coupling among several neurons leading to an increase in expression of ingestive behavior. Here, we used the in vitro analog of OC to investigate whether OC reduces the excitability of a neuron, B4, whose inhibitory connections decrease expression of ingestive behavior. We found OC decreased the excitability of B4. This change appeared intrinsic to B4 because it could be replicated with an analog of OC in isolated cultures of B4 neurons. In addition to changes in B4 excitability, OC decreased the strength of B4's inhibitory connection to a key decision-making neuron, B51. The OC-induced changes were specific without affecting the excitability of another neuron critical for feeding behavior, B8, or the B4-to-B8 inhibitory connection. A conductance-based circuit model indicated that reducing the B4-to-B51 synapse, or increasing B51 excitability, mediated the OC phenotype more effectively than did decreasing B4 excitability. We combined these modifications to examine whether they could act synergistically. Combinations including B51 synergistically enhanced feeding. Taken together, these results suggest modifications of diverse loci work synergistically to mediate OC and that some neurons are well suited to work synergistically with plasticity in other loci.SIGNIFICANCE STATEMENT The ways in which synergism of diverse plasticity loci mediate the change in motor patterns in operant conditioning (OC) are poorly understood. Here, we found that OC was in part mediated by decreasing the intrinsic excitability of a critical neuron of Aplysia feeding behavior, and specifically reducing the strength of one of its inhibitory connections that targets a key decision-making neuron. A conductance-based computational model indicated that the known plasticity loci showed a surprising level of synergism to mediate the behavioral changes associated with OC. These results highlight the importance of understanding the diversity, specificity and synergy among different types of plasticity that encode memory. Also, because OC in Aplysia is mediated by dopamine (DA), the present study provides insights into specific and synergistic mechanisms of DA-mediated reinforcement of behaviors.

Dopamine as a Multifunctional Neurotransmitter in Gastropod Molluscs: An Evolutionary Hypothesis

AbstractThe catecholamine 3,4-dihydroxyphenethylamine, or dopamine, acts as a neurotransmitter across a broad phylogenetic spectrum. Functions attributed to dopamine in the mammalian brain include regulation of motor circuits, valuation of sensory stimuli, and mediation of reward or reinforcement signals. Considerable evidence also supports a neurotransmitter role for dopamine in gastropod molluscs, and there is growing appreciation for its potential common functions across phylogeny. This article reviews evidence for dopamine's transmitter role in the nervous systems of gastropods. The functional properties of identified dopaminergic neurons in well-characterized neural circuits suggest a hypothetical incremental sequence by which dopamine accumulated its diverse roles. The successive acquisition of dopamine functions is proposed in the context of gastropod feeding behavior: (1) sensation of potential nutrients, (2) activation of motor circuits, (3) selection of motor patterns from multifunctional circuits, (4) valuation of sensory stimuli with reference to internal state, (5) association of motor programs with their outcomes, and (6) coincidence detection between sensory stimuli and their consequences. At each stage of this sequence, it is proposed that existing functions of dopaminergic neurons favored their recruitment to fulfill additional information processing demands. Common functions of dopamine in other intensively studied groups, ranging from mammals and insects to nematodes, suggest an ancient origin for this progression.

Use of the Aplysia feeding network to study repetition priming of an episodic behavior

Many central pattern generator (CPG)-mediated behaviors are episodic, meaning that they are not continuously ongoing; instead, there are pauses between bouts of activity. This raises an interesting possibility, that the neural networks that mediate these behaviors are not operating under "steady-state" conditions; i.e., there could be dynamic changes in motor activity as it stops and starts. Research in the feeding system of the mollusk Aplysia californica has demonstrated that this can be the case. After a pause, initial food grasping responses are relatively weak. With repetition, however, responses strengthen. In this review we describe experiments that have characterized cellular/molecular mechanisms that produce these changes in motor activity. In particular, we focus on cumulative effects of modulatory neuropeptides. Furthermore, we relate Aplysia research to work in other systems and species, and develop a hypothesis that postulates that changes in response magnitude are a reflection of an efficient feeding strategy.

Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control

Goal-directed decision making in biological systems is broadly based on associations between conditional and unconditional stimuli. This can be further classified as classical conditioning (correlation-based learning) and operant conditioning (reward-based learning). A number of computational and experimental studies have well established the role of the basal ganglia in reward-based learning, where as the cerebellum plays an important role in developing specific conditioned responses. Although viewed as distinct learning systems, recent animal experiments point toward their complementary role in behavioral learning, and also show the existence of substantial two-way communication between these two brain structures. Based on this notion of co-operative learning, in this paper we hypothesize that the basal ganglia and cerebellar learning systems work in parallel and interact with each other. We envision that such an interaction is influenced by reward modulated heterosynaptic plasticity (RMHP) rule at the thalamus, guiding the overall goal directed behavior. Using a recurrent neural network actor-critic model of the basal ganglia and a feed-forward correlation-based learning model of the cerebellum, we demonstrate that the RMHP rule can effectively balance the outcomes of the two learning systems. This is tested using simulated environments of increasing complexity with a four-wheeled robot in a foraging task in both static and dynamic configurations. Although modeled with a simplified level of biological abstraction, we clearly demonstrate that such a RMHP induced combinatorial learning mechanism, leads to stabler and faster learning of goal-directed behaviors, in comparison to the individual systems. Thus, in this paper we provide a computational model for adaptive combination of the basal ganglia and cerebellum learning systems by way of neuromodulated plasticity for goal-directed decision making in biological and bio-mimetic organisms.

Implication of dopaminergic modulation in operant reward learning and the induction of compulsive-like feeding behavior in Aplysia

Feeding in Aplysia provides an amenable model system for analyzing the neuronal substrates of motivated behavior and its adaptability by associative reward learning and neuromodulation. Among such learning processes, appetitive operant conditioning that leads to a compulsive-like expression of feeding actions is known to be associated with changes in the membrane properties and electrical coupling of essential action-initiating B63 neurons in the buccal central pattern generator (CPG). Moreover, the food-reward signal for this learning is conveyed in the esophageal nerve (En), an input nerve rich in dopamine-containing fibers. Here, to investigate whether dopamine (DA) is involved in this learning-induced plasticity, we used an in vitro analog of operant conditioning in which electrical stimulation of En substituted the contingent reinforcement of biting movements in vivo. Our data indicate that contingent En stimulation does, indeed, replicate the operant learning-induced changes in CPG output and the underlying membrane and synaptic properties of B63. Significantly, moreover, this network and cellular plasticity was blocked when the input nerve was stimulated in the presence of the DA receptor antagonist cis-flupenthixol. These results therefore suggest that En-derived dopaminergic modulation of CPG circuitry contributes to the operant reward-dependent emergence of a compulsive-like expression of Aplysia's feeding behavior.

Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia

Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.

Nitric oxide and histamine signal attempts to swallow: A component of learning that food is inedible in Aplysia

Memory that food is inedible in Aplysia arises from training requiring three contingent events. Nitric oxide (NO) and histamine are released by a neuron responding to one of these events, attempts to swallow food. Since NO release during training is necessary for subsequent memory and NO substitutes for attempts to swallow, it was suggested that NO functions during training as a signal of attempts to swallow. However, it has been shown that NO may also be released in other contexts affecting feeding, raising the possibility that its role in learning is unrelated to signaling attempts to swallow. We confirmed that NO during learning signals attempts to swallow, by showing that a variety of behavioral effects on feeding of blocking or adding NO do not affect learning and memory that a food is inedible. In addition, histamine had effects similar to NO on learning that food is inedible, as expected if the transmitters are released together when animals attempt to swallow. Blocking histamine during training blocked long-term memory, and exogenous histamine substituted for attempts to swallow. NO also substituted for histamine during training. Histamine at concentrations relevant to learning activates neuron metacerebral cell (MCC). However, MCC activity is not a good monitor of attempts to swallow during training, since the neuron responds equally well to other stimuli. These findings support and extend the hypothesis that NO and histamine signal efforts to swallow during learning, acting on targets other than the MCC that specifically respond to attempts to swallow.

Neural analog of arousal: persistent conditional activation of a feeding modulator by serotonergic initiators of locomotion

We investigated how a neural analog of a form of arousal induced by a mildly noxious stimulus can promote two antagonistic responses, locomotion and feeding. Two pairs of cerebral serotonergic interneurons in Aplysia, CC9 and CC10, were persistently activated by transient noxious stimuli. Direct stimulation of CC9-10 activated locomotor activity that outlasted the stimulation and enhanced subsequent nerve-evoked locomotor programs. Thus, CC9-10 function both as initiators and as modulators of the locomotor network. CC9-10 also interacted with the feeding circuit but in a fundamentally different manner. CC9-10 did not directly trigger feeding activity or activate feeding command or pattern generating interneurons. CC9-10 did, however, elicit slow EPSPs in serotonergic cells that modulate feeding responses, the metacerebral cells (MCCs). CC9-10 persistently enhanced MCC excitability, but did not activate the MCCs directly. Previous work has demonstrated that the MCCs are activated during food ingestion via a sensory neuron C2. Interestingly, we found that CC9-10 stimulation converted subthreshold C2 mediated excitation of the MCC into suprathreshold excitation. Transient noxious stimuli also enhanced MCC excitability, and this was largely mediated by CC9-10. To summarize, CC9-10 exert actions on the feeding network, but their functional effects appear to be conditional on the presence of food-related inputs to the MCCs. A potential advantage of this arrangement is that it may prevent conflicting responses from being directly evoked by noxious stimuli while also facilitating the ability of food-related stimuli to generate feeding responses in the aftermath of noxious stimulation.

Behavioral and neuronal attributes of short- and long-term habituation in the crab Chasmagnathus

Investigations using invertebrate species have led to a considerable progress in our understanding of the mechanisms underlying learning and memory. In this review we describe the main behavioral and neuronal findings obtained by studying the habituation of the escape response to a visual danger stimulus in the crab Chasmagnathus granulatus. Massed training with brief intertrial intervals lead to a rapid reduction of the escape response that recovers after a short term. Conversely, few trials of spaced training renders a slower escape reduction that endures for many days. As predicted by Wagner's associative theory of habituation, long-term habituation in the crab proved to be determined by an association between the contextual environment of the training and the unconditioned stimulus. By performing intracellular recordings in the brain of the intact animal at the same time it was learning, we identified a group of neurons that remarkably reflects the short- and long-term behavioral changes. Thus, the visual memory abilities of crabs, their relatively simple and accessible nervous system, and the recording stability that can be achieved with their neurons provide an opportunity for uncovering neurophysiological and molecular events that occur in identifiable neurons during learning.

Behavioral and in vitro correlates of compulsive-like food seeking induced by operant conditioning in Aplysia

Motivated behaviors comprise appetitive actions whose occurrence results partly from an internally driven incentive to act. Such impulsive behavior can also be regulated by external rewarding stimuli that, through learning processes, can lead to accelerated and seemingly automatic, compulsive-like recurrences of the rewarded act. Here, we explored such behavioral plasticity in Aplysia by analyzing how appetitive reward stimulation in a form of operant conditioning can modify a goal-directed component of the animal's food-seeking behavior. In naive animals, protraction/retraction cycles of the tongue-like radula are expressed sporadically with highly variable interbite intervals. In contrast, animals that were previously given a food-reward stimulus in association with each spontaneous radula bite now expressed movement cycles with an elevated frequency and a stereotyped rhythmic organization. This rate increase and regularization, which was retained for several hours after training, depended on both the reward quality and its contingency because accelerated, stereotyped biting was not induced in animals that had previously received a less-palatable food stimulus or had been subjected to nonassociative reward stimulation. Neuronal correlates of these learning-induced changes were also expressed in the radula motor pattern-generating circuitry of isolated buccal ganglia. In such in vitro preparations, moreover, manipulation of the burst frequency of the bilateral motor pattern-initiating B63 interneurons indicated that the regularization of radula motor pattern generation in contingently trained animals occurred separately from an increase in cycle rate, thereby suggesting independent processes of network plasticity. These data therefore suggest that operant conditioning can induce compulsive-like actions in Aplysia feeding behavior and provide a substrate for a cellular analysis of the underlying mechanisms.

Feeding behavior of Aplysia: a model system for comparing cellular mechanisms of classical and operant conditioning

Feeding behavior of Aplysia provides an excellent model system for analyzing and comparing mechanisms underlying appetitive classical conditioning and reward operant conditioning. Behavioral protocols have been developed for both forms of associative learning, both of which increase the occurrence of biting following training. Because the neural circuitry that mediates the behavior is well characterized and amenable to detailed cellular analyses, substantial progress has been made toward a comparative analysis of the cellular mechanisms underlying these two forms of associative learning. Both forms of associative learning use the same reinforcement pathway (the esophageal nerve, En) and the same reinforcement transmitter (dopamine, DA). In addition, at least one cellular locus of plasticity (cell B51) is modified by both forms of associative learning. However, the two forms of associative learning have opposite effects on B51. Classical conditioning decreases the excitability of B51, whereas operant conditioning increases the excitability of B51. Thus, the approach of using two forms of associative learning to modify a single behavior, which is mediated by an analytically tractable neural circuit, is revealing similarities and differences in the mechanisms that underlie classical and operant conditioning.

Neural events in the reinforcement contingency

When neural events are analyzed as stimuli and responses, functional relations among them and among overt stimuli and responses can be unveiled. The integration of neuroscience and the experimental analysis of behavior is beginning to provide empirical evidence of involvement of neural events in the three-term contingency relating discriminative stimuli, responses, and consequences. This paper is aimed at highlighting exemplar instances in the development of this issue. It has long been known that the electrical stimulation of certain cerebral areas can have a reinforcing function. Extraordinary technological advances in recent years show that neural activity can be selected by consequences. For example, the activity of in vitro isolated neurons that receive dopamine as a reinforcer functions as a cellular analogue of operant conditioning. The in vivo activity of populations of neurons of rats and monkeys can be recorded on an instant-to-instant basis and can then be used to move mechanical arms or track a target as a function of consequences. Neural stimulation acts as a discriminative stimulus for operant responses that are in turn maintained by neural consequences. Together with investigations on the molecular basis of classical conditioning, those studies are examples of possibilities that are being created for the study of behavior-environment interactions within the organism. More important, they show that, as an element in the three-term contingency, neural activity follows the same laws as other events.

A neuronal network for the logic of Limax learning

We construct a neuronal network to model the logic of associative conditioning as revealed in experimental results using the terrestrial mollusk Limax maximus. We show, in particular, how blocking to a previously conditioned stimulus in the presence of the unconditional stimulus, can emerge as a dynamical property of the network. We also propose experiments to test the new model.

Target-specific regulation of synaptic efficacy in the feeding central pattern generator of Aplysia: potential substrates for behavioral plasticity?

The contributions to this symposium are unified by their focus on the role of synaptic plasticity in sensorimotor learning. Synaptic plasticities are also known to operate within the central pattern generator (CPG) circuits that produce repetitive motor programs, where their relation to adaptive behavior is less well understood. This study examined divergent synaptic plasticity in the signaling of an influential interneuron, B20, located within the CPG that controls consummatory feeding-related behaviors in Aplysia. Previously, B20 was shown to contain markers for catecholamines and GABA (Díaz-Ríos et al., 2002), and its rapid synaptic signaling to two follower motor neurons, B16 and B8, was found to be mediated by dopamine (Díaz-Ríos and Miller, 2005). In this investigation, two incremental forms of increased synaptic efficacy, facilitation and summation, were both greater in the signaling from B20 to B8 than in the signaling from B20 to B16. Manipulation of the membrane potentials of the two postsynaptic motor neurons did not affect facilitation of excitatory postsynaptic potentials (EPSPs) to either follower cell. Striking levels of summation in B8, however, were eliminated at hyperpolarized membrane potentials and could be attributed to distinctive membrane properties of this postsynaptic cell. GABA and the GABAB agonist baclofen increased facilitation and summation of EPSPs from B20 to B8, but not to B16. The enhanced facilitation was not affected when the membrane potential of B8 was pre-set to hyperpolarized levels, but GABAergic effects on summation were eliminated by this manipulation. These observations demonstrate a target-specific amplification of synaptic efficacy that can contribute to channeling the flow of divergent information from an intrinsic interneuron within the buccal CPG. They further suggest that GABA, acting as a cotransmitter in B20, could induce coordinated and target-specific pre- and postsynaptic modulation of these signals. Finally, we speculate that target-specific plasticity and its modulation could be efficient, specific, and flexible substrates for learning-related modifications of CPG function.

Nitric oxide signals that aplysia have attempted to eat, a necessary component of memory formation after learning that food is inedible

Inhibiting nitric oxide (NO) synthesis during learning that food is inedible in Aplysia blocks subsequent memory formation. To gain insight into the function of NO transmission during learning we tested whether blocking NO synthesis affects aspects of feeding that are expressed both in a nonlearning context and during learning. Inhibiting NO synthesis with L-NAME and blocking guanylyl cyclase with methylene blue decreased the efficacy of ad libitum feeding. D-NAME had no effect. L-NAME also decreased rejection responses frequency, but did not affect rejection amplitude. The effect of L-NAME was explained by a decreased signaling that efforts to swallow are not successful, leading to a decreased rejection rate, and a decreased ability to reposition and subsequently consume food in ad libitum feeding. Signaling that animals have made an effort to swallow is a critical component of learning that food is inedible. Stimulation of the lips with food alone did not produce memory, but stimulation combined with the NO donor SNAP did produce memory. Exogenous NO at a concentration causing memory also excited a key neuron responding to NO, the MCC. Block of the cGMP second-messenger cascade during training by methylene blue also blocked memory formation after learning. Our data indicate that memory arises from the contingency of three events during learning that food is inedible. One of the events is efforts to swallow, which are signaled by NO by cGMP.

Operant conditioning of gill withdrawal in Aplysia

A basic question in neuroscience is how different forms of learning are related. To further address that question, we examined whether gill withdrawal in Aplysia, which has already been studied extensively for neuronal mechanisms contributing to habituation, sensitization, and classical conditioning, also undergoes operant conditioning. Animals were run in pairs. During the initial training period, the contingent (experimental) animal received a siphon shock each time its gill relaxed below a criterion level, and the yoked control animal received a shock whenever the experimental animal did, regardless of its own gill position. This was followed by an extinction period when there was no shock, a retraining period when both animals were contingent, and another extinction period. The experimental animals spent more time with their gills contracted above the criterion level than did the control animals during each period, demonstrating operant conditioning. The type of gill behavior modified by learning shifted over time: the experimental animals had a larger increase in the frequency and duration of spontaneous contractions than did the control animals during the first but not the last extinction period and a larger increase in the level of tonic contraction during the last but not the first extinction period. Because many of the neurons controlling spontaneous and tonic gill withdrawal have already been identified, it should now be possible to examine the cellular locus and mechanism of operant conditioning and compare them with those for other forms of learning of the same behavior.

The "prawn-in-the-tube" procedure in the cuttlefish: habituation or passive avoidance learning?

This study examines whether or not habituation contributes to the regulation of the inhibition of predatory behavior observed during the "prawn-in-the-tube" training procedure. When presented with prawns that are visible behind glass but untouchable, cuttlefish promptly learn to inhibit their capture attempts. The first three experiments demonstrated that the acquired response in the course of training cannot be dishabituated. The fourth experiment demonstrated that the repeated application of a brief visual prawn stimulus, one that is terminated before the cuttlefish can strike, decreases attack latencies. Taken together, the results of this study establish that habituation does not play a significant role in this learning task. In fact, the present findings strengthen the results of previous studies suggesting that passive avoidance learning contributes to the regulation of the inhibition of predatory behavior.

Two-color in situ hybridization in the CNS of Aplysia californica

Aplysia californica is an attractive model organism for cellular and systems neuroscience. Currently, there is a growing body of sequence data from Aplysia that includes many interesting genes. To fully exploit this molecular data it must be integrated with the large body of physiological data that are already available for identified neurons in Aplysia networks. In situ hybridization is a powerful technique that enables this to be done. Expression patterns of selected mRNA transcripts can be mapped to individual cells in the central nervous system (CNS). Here, we describe a detailed non-radioactive in situ hybridization protocol optimized for whole-mount preparations of Aplysia ganglia. The indirect alkaline phosphatase-based chromogenic detection method we employ may be used with one or two colors in order to detect one or two different transcripts in the same preparation. The procedure is also compatible with intracellular dye labeling, making it possible to couple localization of transcripts with electrophysiological studies in positively identified neurons. Double labeling was done for transcripts encoding the neuropeptides FMRFamide and sensorin. The sensitive detection of mRNA and great preservation of CNS morphology makes this method a useful tool for analyzing expression patterns of neuron specific genes in Aplysia.

Changes of internal state are expressed in coherent shifts of neuromuscular activity in Aplysia feeding behavior

The multitasking central pattern generator (CPG) that drives consummatory feeding behaviors of Aplysia can produce ingestive, egestive, and intermediate motor programs. External stimuli trigger the programs but, remarkably, do not directly specify which type of program is produced. Rather, recent work has proposed, the type of program is determined by the internal network state of the CPG that has developed in response to the previous history of the stimulation. Here we have tested a key prediction of this network-state hypothesis. If the network state has a real existence and governs real functional behavior, changes in the state should be seen as coherent, coordinated changes along many dimensions of interneuron and motor neuron activity, muscle contraction, and ultimately movement, that underlie functional behavior. In reduced neuromuscular preparations, we elicited repetitive motor programs by continued stimulation of the esophageal nerve while recording the firing of motor neurons B8, B15, B16, B4/5, and B48, and contractions of the accessory radula closer and I7-I10 muscles that respectively close and open the animal's food-grasping organ, the radula. Using sonomicrometric techniques, we similarly recorded the movement of the radula in the complete buccal mass. Successive esophageal nerve programs indeed exhibited clear progressive changes in motor neuron firing, muscle contractions, and the phasing of radula movements within each cycle, from an initially intermediate or even ingestive character to a strongly egestive character. We conclude that the Aplysia feeding CPG really has a coherent internal network state whose dynamics are likely to be reflected in the real behavior of the animal.

Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: blockade by a dopamine antagonist

In a recently developed in vitro analog of appetitive classical conditioning of feeding in Aplysia, the unconditioned stimulus (US) was electrical stimulation of the esophageal nerve (En). This nerve is rich in dopamine (DA)-containing processes, which suggests that DA mediates reinforcement during appetitive conditioning. To test this possibility, methylergonovine was used to antagonize DA receptors. Methylergonovine (1 nM) blocked the pairing-specific increase in fictive feeding that is usually induced by in vitro classical conditioning. The present results and previous observation that methylergonovine also blocks the effects of contingent reinforcement in an in vitro analog of appetitive operant conditioning suggest that DA mediates reinforcement for appetitive associative conditioning of feeding in Aplysia.