Durable secondary reinforcement using brain stimulation as the primary reinforcer

Pavlovian learning and conditioned reinforcement

Conditioned reinforcers are widely used in applied behavior analysis. Basic research evidence reveals that Pavlovian learning plays an important role in the acquisition and efficacy of new conditioned-reinforcer functions. Thus, a better understanding of Pavlovian principles holds the promise of improving the efficacy of conditioned reinforcement in applied research and practice. This paper surveys how (and if) Pavlovian principles are presented in behavior-analytic textbooks; imprecisions and knowledge gaps within contemporary Pavlovian empirical findings are highlighted. Thereafter, six practical principles of Pavlovian conditioning are presented along with empirical support and knowledge gaps that should be filled by applied and translational behavior-analytic researchers. Innovative applications of these principles are outlined for research in language acquisition, token reinforcement, and self-control.

Amphetamine-induced enhancement of responding for conditioned reward in rats: interactions with repeated testing

RATIONALE

The mesolimbic dopamine system underlies the ability of reward-related stimuli to control operant behavior. Previous work has shown that amphetamine potentiates operant responding for conditioned rewards (CRs).

OBJECTIVES

Here, we asked whether the profile of this amphetamine-produced potentiation changes with repeated CR presentation, i.e., as the CR is being extinguished.

METHODS

Amphetamine (0-1.0 mg/kg, i.p.), administered over four daily sessions using a Latin square design, dose-dependently increased lever pressing for a 'lights-off' stimulus previously paired with food in rats.

RESULTS

The amphetamine-produced enhancement of responding for CR was significantly modulated with repeated CR exposure: it was strongest on day 1 and became less pronounced in subsequent sessions whereas the CR effect persisted. In further experiments, rats receiving LiCl devaluation of the primary reward failed to show a significant reduction in the amphetamine-produced enhancement of responding for CR.

CONCLUSIONS

The nature of the dissociable effects of amphetamine on responding for CR versus the CR effect itself remains to be elucidated.

Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats

RATIONALE

Stimulation of ventral tegmental area (VTA) muscarinic acetylcholine receptors (mAChRs) is implicated in feeding.

OBJECTIVE

To investigate the effects of mAChR blockade in the VTA on food-related learning.

METHODS

In experiment 1, rats (N=12) were placed in chambers containing food and received microinjections of 0 or 5 microg/0.5 microl scopolamine prior to the first four feeding sessions and the alternate dose prior to the tenth feeding session. In experiment 2 (N=9), the effects of daily microinjections of scopolamine on lever pressing under a progressive ratio schedule of food reinforcement were tested. In experiment 3 (N=34), the effects of daily microinjections of scopolamine on lever pressing maintained by conditioned reward were investigated.

RESULTS

In experiment 1, all rats demonstrated low consumption during session 1. However, pellet consumption for rats initially pretreated with the 0-microg dose rose to and stayed at maximal levels for the remaining sessions, even when pretreated with the 5-microg dose during the tenth session. Pellet consumption for rats initially pretreated with the 5-microg dose remained low, even for the first two sessions following the cessation of scopolamine pretreatment, and gradually rose to maximal levels by the eighth session. In experiment 2, scopolamine significantly decreased break points. In experiment 3, scopolamine failed to significantly decrease responding specifically on the lever producing the conditioned reward.

CONCLUSIONS

Altogether, these data suggest that VTA mAChR stimulation is involved in feeding and food-related learning but may not be involved in responding maintained by conditioned reward.

Biphasic effects of 7-OH-DPAT on the acquisition of responding for conditioned reward in rats

Previous studies have shown that dopamine (DA) receptor subtype-specific agonists differentially affect responding for conditioned reward D1-like agonists impair, whereas D2-like agonists enhance responding. The present study compared the effects of the D2-like agonists bromocriptine and 7-hydroxy-N,N-di-n-propyl-2-aminotetralin (7-OH-DPAT). Food-deprived rats (N=159) were preexposed to a chamber with two levers, one producing a tone (3 s) and the other turning the house lights off (3 s), for five 40-min sessions. In four subsequent 65-min conditioning sessions with the levers removed, the lights-off stimulus was paired with food (80 presentations per session). During two 40-min test sessions, the lights-off (CR) and tone (NCR) levers were replaced and responses at each lever were recorded. Confirming previous results, bromocriptine (0.50-5.0 mg/kg) dose-dependently enhanced responding on the lever producing conditioned reward. In contrast, 7-OH-DPAT had a biphasic effect on responding for conditioned reward. Low doses (0.10-0.25 mg/kg) reduced CR lever responding, whereas a higher dose of 1.0 mg/kg enhanced such responding. An intermediate dose of 0.50 mg/kg neither impaired nor enhanced CR lever responding. The biphasic profile of 7-OH-DPAT may arise through differential actions at D3 vs. D2 receptors or presynaptic vs. postsynaptic DA receptors at low and high doses, respectively.

The D1 agonist SKF 38393 attenuates amphetamine-produced enhancement of responding for conditioned reward in rats

The present study investigated the hypothesis that the D1 subtype of DA receptors is critically involved in reward-related learning. The effects of SKF 38393, a D1-specific agonist, on amphetamine-produced enhancement of responding for conditioned reward were tested. We exposed 69 male Wistar rats to an experimental design consisting of three phases. The preexposure phase consisted of five sessions during which the rats were exposed to an operant chamber containing two levers. One lever produced a lights-off stimulus (3 s) and the other a tone stimulus (3 s). This was followed by four conditioning sessions during which the levers were removed and the rats were exposed to pairings of the lights-off stimulus with food. This phase was followed by two test sessions during which the levers were present and the number of responses made on each lever was calculated as a ratio of the number of responses made during the preexposure phase. A group receiving saline during the test sessions showed a higher ratio of responding for the lights off stimulus than the tone stimulus, demonstrating that the lights-off stimulus had become a conditioned reward. Amphetamine [2.0 mg/kg, intraperitoneally (i.p.), 5 min before the test] enhanced responding specifically on the lever producing the conditioned reward. Groups receiving SKF 38393 (5.0, 10.0, and 20.0 mg/kg, i.p., 5 min before the test) failed to show significantly greater responding for the lights-off stimulus than the tone, indicating a reduction or elimination of the conditioned reward effect. Moreover, SKF 38393 dose dependently reduced the amphetamine-produced enhancement of responding for conditioned reward.(ABSTRACT TRUNCATED AT 250 WORDS)

Bromocriptine enhancement of responding for conditioned reward depends on intact D1 receptor function

It has been suggested that reward-related learning may require intact functioning at the dopamine D1 receptor. The present experiment tested this hypothesis by challenging the reward-enhancing effects of the D2 agonist, bromocriptine, with a D1 antagonist, SCH 23390. For comparison, the effects of the D2 antagonist, pimozide, were also evaluated. Male rats (n = 240) were pre-exposed to a chamber with two levers, one producing a 3-s lights-off stimulus and the other a 3-s tone stimulus. Four conditioning sessions followed, during which levers were absent and presentations of the lights-off stimulus were paired with food. Testing consisted of comparing presses on each lever after conditioning to before conditioning for each rat. Control groups showed a significantly greater increase in responding for lights-off than tone, indicating that the lights-off stimulus had become a conditioned reward. Results showed that bromocriptine (0.25-10.0 mg/kg, IP, 60 min before test session) enhanced responding at doses of 2.5 and 5.0 mg/kg significantly more on the conditioned reward lever than on the other lever. The lowest dose of SCH 23390 (1.0 microgram/kg, SC, 2 h before testing) eliminated the bromocriptine-produced enhancement at 2.5 mg/kg and a significant enhancement was seen at 10.0 mg/kg. The higher doses of SCH 23390 (5.0 and 10.0 micrograms/kg) eliminated the bromocriptine effect and the conditioned reward effect itself, respectively.(ABSTRACT TRUNCATED AT 250 WORDS)

Dopamine D1 and D2 antagonists attenuate amphetamine-produced enhancement of responding for conditioned reward in rats

It has been suggested that the dopamine D1 receptor may play an important role in reward. The present study was undertaken to investigate the roles of dopamine D1 and D2 receptor subtypes in responding for conditioned reward. This was done by examining the effects of the D1 antagonist SCH 23390 and the D2 antagonists pimozide and metoclopramide on amphetamine-produced enhancement of responding for conditioned reward. The procedure consisted of three distinct phases. During the pre-exposure phase the rats were exposed to an operant chamber containing two levers. One lever produced a lights-off stimulus (3 s) and the other a tone stimulus (3 s). This was followed by four conditioning sessions during which the levers were removed and the rats were exposed to pairings of the lights-off stimulus with food. This phase was followed by two test sessions during which the levers were present and the number of responses made on each was calculated as a ratio of the number of responses made during the pre-exposure phase. A group receiving the vehicle during the test sessions showed a greater ratio of responding for the lights-off stimulus than the tone stimulus, indicating that the lights-off stimulus had become a conditioned reward. Amphetamine (0.1, 1.0, 2.0 and 5.0 mg/kg, IP, 5 min prior to test) specifically enhanced responding on the lever producing conditioned reward. SCH 23390 (5.0 and 10.0 micrograms/kg, SC, 2 h before test) and pimozide (0.1 and 0.2 mg/kg, IP, 4 h before test) dose-dependently shifted the peak in the amphetamine dose-response function to the right, indicating an attenuation of conditioned reward.(ABSTRACT TRUNCATED AT 250 WORDS)

The effects of pimozide on the establishment of conditioned reinforcement as a function of the amount of conditioning

In an attempt to understand some inconsistent findings, the present experiment investigated the effects of pimozide, a dopamine (DA) receptor blocker, on the establishment of conditioned reinforcement as a function of the amount of conditioning. In Experiment 1, rats received three phases of training in a two-lever box. The pre-exposure phase measured the operant rates of pressing the levers; one produced a 3-s tone and the other turned the lights off for 3 s. In the conditioning phase, with the levers absent, the light-off stimulus was paired with food for two or four sessions. The test phase again measured the rate of pressing the levers. Conditioned reinforcement was shown by a relative increase in responding on the light lever during the test. Of the groups receiving four conditioning sessions, pimozide (0.5, 1.0, 2.0 and 4.0 mg/kg) produced a dose-dependent attenuation of conditioned reinforcement, those rats treated with 4.0 mg/kg failing to demonstrate a significant effect. When 2 conditioning days were employed, pimozide treatment also produced a dose-dependent attenuation; however, in these less conditioned animals 2.0 mg/kg blocked the effect. The possibility that pimozide produced a conditioned taste aversion to the food was ruled out in Experiment 2. These data suggest that DA transmission may be necessary for the establishment of conditioned reinforcement and that the effects of receptor blockade may be related to the amount of conditioning.

The acquisition of responding with conditioned reinforcement: effects of cocaine, (+)-amphetamine and pipradrol

1 A procedure for examining the acquisition of a lever-pressing operant with conditioned reinforcement was used to compare the effects of three psychomotor stimulants. 2 Hungry rats were trained to associated and auditory tone (i.e., conditioned reinforcer) with food. Preference for the tone was then measured after treatment with pipradol (5, 10, 15 mg/kg), cocaine (1, 5, 10 mg/kg) or (+)-amphetamine (0.5, 1.5, 5.0 mg/kg). 3 In agreement with previous data, 10 mg/kg of pipradrol enhanced the effect of conditioned reinforcement whereas animals treated with any of the doses of (+)-amphetamine showed no effect. 4 Rats treated with cocaine (1 or 5 mg/kg) showed an effect of conditioned reinforcement but the effect was not significantly greater than in controls. 5 The present data suggest important differences in enhancement of responding for conditioned reinforcement by various drugs in the psychomotor stimulant class. These differences in turn may be related to the pharmacological actions of these compounds on release of catecholamines from different storage pools.

The effects of pimozide during pairing on the transfer of classical conditioning to an operant discrimination

Transfer of classical conditioning to operant learning was demonstrated by showing enhanced acquisition of an operant discrimination in a group of rats (n = 6) previously exposed to pairings of the discriminative stimulus with food as compared to control animals (n = 6). A group (n = 6) that received the classical conditioning sessions while under the influence of the neuroleptic, pimozide (1.0 mg/kg, IP) also showed enhanced acquisition of the discrimination when tested while undrugged but their performance was intermediate between that of the other groups for the first seven sessions. For the remaining sessions, the two groups that had received classical conditioning did not differ from each other and both groups discriminated better than the controls. These data may indicate a role for dopaminergic neurons in the mechanism by which the effects of classical conditioning influence operant responding.

The effects of pipradrol on the acquisitionof responding with conditioned reinforcement: a role for sensory preconditioning

Previous studies have shown that pipradrol enhances the acquisition of responding with conditioned reinforcement. The present experiments replicated this finding and assessed the possible role of nonspecific stimulus change, feeding in the test environment and prior exposure to the conditioned stimulus. The test procedure consisted of three phases: Operant rates of pressing two levers, one of which produced a 3-s tone, were measured in the pre-exposure phase; the tone was paired with food in the four conditioning sessions; and conditioned reinforcement was demonstrated in the test phase by a relative increase in pressing the tone lever. A group (N = 8) receiving food but no tone during the conditioning phase also increased responding on the tone lever in the test phase and this effect was enhanced in a group (N = 8) receiving pipradrol prior to the test. A group (N = 8) receiving neither tones nor pellets during conditioning and pipradrol prior to test failed to show a change in lever bias, ruling out the possibility that pipradrol produced a nonspecific enhancement of responding for stimulus change. It was hypothesized that the conditioned reinforcement effect seen in the food-alone groups resulted from sensory preconditioning. According to this hypothesis, the tone was conditioned to environmental stimuli during the pre-exposure phase; subsequent presentation of pellets resulted in a learned association between environmental stimuli and food, leading to increased responding for the tone in the test phase. The results of two control experiments supported this view. One group (N = 8) received no tone in the pre-exposure phase, pellets alone in the conditioning phase and then pipradrol prior to test. The second group, but under altered environmental stimulus conditions, and then was given pipradrol prior to test. Neither group showed evidence of conditioned reinforcement. It was concluded that pipradrol enhanced acquisition of responding with conditioned reinforcement even if the conditioned reinforceing stimulus was established with sensory preconditioning procedures.

The effect of pimozide on the establishment of conditioned reinforcement

The effect of pimozide on conditioned reinforcement was determined by comparing rate of lever pressing for a tone in groups previously treated with or without the drug when the tone was paired with food. Eight groups of six to eight rats each received three phases of training in a two-lever box. The pre-exposure phase measured the operant rate of pressing the two levers, one of which produced a 3s tone. In the conditioning phase, with the levers absent, the tone was paired with food over four sessions. The test phase again measured the rate of pressing the two levers. In an undrugged experimental group (i.e., Paradigm group), the number of presses on the tone lever significantly increased from the pre-exposure to the test phase, thereby confirming that the procedure could establish conditioned reinforcement. A control group receiving tones and pellets randomly during the conditioning phase also showed conditioned reinforcement but a group receiving negatively correlated tones and pellets did not. Groups receiving the dopamine-receptor blocker pimozide (1.0 mg/kg) prior to each conditioning session failed to show conditioned reinforcement in the test session. Control groups ruled out state dependent learning and drug-induced performance impairments as explanations of this pimozide-related effect. These data may indicate a possible role for dopamine neurons in mediating the control of behavior by certain positive reinforcing stimuli.

Pipradrol enhances reinforcing properties of stimuli paired with brain stimulation

The hypothesis that a psychomotor stimulant drug (pipradrol) enhances the reinforcing effects of stimuli paired with reinforcing brain stimulation was tested using a conditioned reinforcement paradigm. Rats were trained to discriminate between two stimuli (S+ and S-) to obtain ICS in the lateral hypothalamus by pushing a panel in the presence of S+. In a subsequent preference test, ICS was no longer available, but responding on one of two novel levers now produced S+, whereas responding on the other lever produced S-. Four groups of four rats received 0, 5, 10 or 15 mg/kg pipradrol. Doses of 5 and 10 mg/kg significantly enhanced the preference for S+ over S-. These doses increased responding for S+, but had no effect on responding for S-. These results support the hypothesis tested, and suggest that pipradrol potentiates the effects of conditioned reinforcement.

Secondary reinforcement established with intracranial stimulation in rat

Using a single-subject design with the rat secondary reinforcement was demonstrated with intracranial stimulation as the primary reinforcer. The presence of a buzzer (the S') which had previously signalled the availability of intracranial stimulation significantly extended responding during extinction. It was pointed out that previous failures to demonstrate this phenomenon had attempted to apply Bugelski's (1938) classic design without appropriate attention to the discriminative stimulus hypothesis. A careful application of the hypothesis led to the present design which produced strong, replicable results.

Conditioned reinforcement using brain stimulation as a primary reinforcement

The purposes of this study were to condition a secondary reinforcer using rewarding brain stimulation as a primary reinforcer, and to determine whether the absence of stimulation-induced motivational conditions during testing might be an important factor in making this phenomenon so difficult to obtain. A moderate conditioned-reward effect was obtained, but the importance of stimulation-evoked motivational conditions was not definitely established.

Secondary reinforcement based on primary brain stimulation reward

15 rats were trained to run in a straight runway for rewarding brain stimulation (ESB). A consistent color of goal box—ESB reward pairing existed during runway training. When subsequently rested in a T-maze where the goal box associated with ESB was placed on the initially nonpreferred side, Ss learned to run to that side with only the goal-box color as the reward. The results are held to conclusively demonstrate that secondary reward can be based on ESB as a primary reward. Also, results appear to challenge the view that drive is necessary for the demonstration of secondary reward.