Progressive ratio schedules of reinforcement

A reinforcement learning model with choice traces for a progressive ratio schedule

The progressive ratio (PR) lever-press task serves as a benchmark for assessing goal-oriented motivation. However, a well-recognized limitation of the PR task is that only a single data point, known as the breakpoint, is obtained from an entire session as a barometer of motivation. Because the breakpoint is defined as the final ratio of responses achieved in a PR session, variations in choice behavior during the PR task cannot be captured. We addressed this limitation by constructing four reinforcement learning models: a simple Q-learning model, an asymmetric model with two learning rates, a perseverance model with choice traces, and a perseverance model without learning. These models incorporated three behavioral choices: reinforced and non-reinforced lever presses and void magazine nosepokes, because we noticed that male mice performed frequent magazine nosepokes during PR tasks. The best model was the perseverance model, which predicted a gradual reduction in amplitudes of reward prediction errors (RPEs) upon void magazine nosepokes. We confirmed the prediction experimentally with fiber photometry of extracellular dopamine (DA) dynamics in the ventral striatum of male mice using a fluorescent protein (genetically encoded GPCR activation-based DA sensor: GRABDA2m). We verified application of the model by acute intraperitoneal injection of low-dose methamphetamine (METH) before a PR task, which increased the frequency of magazine nosepokes during the PR session without changing the breakpoint. The perseverance model captured behavioral modulation as a result of increased initial action values, which are customarily set to zero and disregarded in reinforcement learning analysis. Our findings suggest that the perseverance model reveals the effects of psychoactive drugs on choice behaviors during PR tasks.

Positive correlation between measures of habitual responding and motivated responding in mice

Habit and motivation are thought to be separate processes, with motivated behavior often considered to be goal directed, whereas habits are defined by the absence of goal-directed control over behavior. However, there has been increasing interrogation of the binary nature of habitual versus goal-directed behavior. Furthermore, although drug and alcohol exposure can promote the formation of habits, drug seeking itself can also be highly flexible, pointing toward the need for complex consideration of the parallel processes that drive behavior. The goal of the current study was to determine whether there was a relation between motivation-as measured by progressive ratio-and habit-as measured by contingency degradation-and whether this relation was affected by ethanol exposure history and sex. The results showed that these measures were positively correlated such that greater contingency insensitivity was associated with achieving higher break points on the progressive-ratio task. However, this relation depended on reinforcement schedule history, ethanol exposure history, and sex. These point to potential relations between measures of habit and motivation and stress the importance of carefully parsing behavioral findings and assays. These findings are also expected to inform future substance use research, as drug history may affect these relations.

Hyperdopaminergia in rats is associated with reverse effort-cost dependent performance

BACKGROUND

Dopamine is implicated in the effort-based control of motivational processes; however, whether tonic dopamine regulates the effort-cost impact on motivation, is still debated.

AIMS

The rats lacking the dopamine transporter (DAT), which have dramatically increased levels of the synaptic dopamine, were used in the present study to elucidate the role of the synaptic dopamine in motivational processes.

METHODS

To study the reward-related processes, the progressive ratio 3 (PR3) operant schedule of food reinforcement (the ratio increases by 3 after each earned reinforcer) was performed in adult male rats (DAT knockouts (DAT-KO), heterozygotes (DAT-HT) and wild-types (DAT-WT)).

RESULTS

During the PR3 session, the response rate of DAT-KO rats was gradually increased following the augmented required number of responses. In contrast, the local response rate of DAT-WT and DAT-HT decreased. d-Amphetamine sulfate salt (3 mg/kg, i.p.) altered the local response rate dynamics in DAT-WT, which became similar to that of DAT-KO. Interestingly, the reduction in response rate at low effort demands was associated with decreased rate of entries into the magazine tray in DAT-WT rats treated with amphetamine (3 mg/kg) but not in DAT-KO rats.

CONCLUSIONS

Our results suggest that the elevated tonic synaptic dopamine can strongly affect motivation/effort-cost relation in rodents.

Theory of reinforcement schedules

The three principles of reinforcement are (1) events such as incentives and reinforcers increase the activity of an organism; (2) that activity is bounded by competition from other responses; and (3) animals approach incentives and their signs, guided by their temporal and physical conditions, together called the "contingencies of reinforcement." Mathematical models of each of these principles comprised mathematical principles of reinforcement (MPR; Killeen, 1994). Over the ensuing decades, MPR was extended to new experimental contexts. This article reviews the basic theory and its extensions to satiation, warm-up, extinction, sign tracking, pausing, and sequential control in progressive-ratio and multiple schedules. In the latter cases, a single equation balancing target and competing responses governs behavioral contrast and behavioral momentum. Momentum is intrinsic in the fundamental equations, as behavior unspools more slowly from highly aroused responses conditioned by higher rates of incitement than it does from responses from leaner contexts. Habits are responses that have accrued substantial behavioral momentum. Operant responses, being predictors of reinforcement, are approached by making them: The sight and feel of a paw on a lever is approached by placing paw on lever, as attempted for any sign of reinforcement. Behavior in concurrent schedules is governed by approach to momentarily richer patches (melioration). Applications of MPR in behavioral pharmacology and delay discounting are noted.

Modulation of cue value and the augmentation of heroin seeking in chronically food-restricted male rats under withdrawal

Food restriction augments drug seeking in abstinent rats. The underlying motivational mechanisms, however, remain unclear. We hypothesized that caloric restriction enhances the incentive value attributed to drug-associated cues and, in turn, augments drug seeking. Male rats were trained to lever-press for heroin, and then moved to the animal colony for a forced-abstinence period. Rats were maintained on free access to food (Sated) or subjected to 14 days of food restriction (FDR). In a series of experiments, we assessed the effect of food-restriction on the incentive value of heroin-associated cues. Tests included performance under a progressive ratio (PR) schedule of reinforcement maintained by heroin-associated cues, acquisition of a novel operant response reinforced by drug-associated cues, effect of food-restriction on operant response reinforced by neutral cues, acquisition of a novel operant response reinforced by drug-associated or neutral cues, and the effect of food-restriction on operant response reinforced by drug-associated or neutral cues, under a discrete choice procedure. Food-restriction did not change breakpoints in PR maintained by heroin-associated cues. FDR rats acquired the novel response at a greater level compared to the Sated group. Food-restriction-induced increase in novel-response rate was observed for both heroin-paired and the neutral cue. Responding for a heroin-associated cue was greater than for the neutral cue in both Sated and FDR groups. Response rate for the neutral cue, however, was greater in the FDR versus Sated group. Our findings suggest that food restriction increases the conditioned motivational properties of environmental stimuli, including, but not exclusive to, heroin-paired cues.

Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control

Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ-1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ-1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ-1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.

Appetitive Pavlovian conditioning of the stimulus effects of nicotine enhances later nicotine self-administration

Nicotine produces robust stimulus effects that can be conditioned to exert stimulus control over behavior through associative learning. Additionally, nicotine has weak reinforcing effects that are inconsistent with its prevalence of use and the tenacity of nicotine dependence. The present study investigated whether conditioned associations to the nicotine drug stimulus may confer additional reinforcing strength to nicotine that thereby increase its use liability, and presents a new methodological approach to investigating the interaction between the stimulus effects and reinforcing effects of drugs. Male and female Sprague-Dawley rats were divided into groups receiving intravenous infusions of either 0.03 mg/kg nicotine or 0.9% saline that were either Paired (30 s delayed) or explicitly Unpaired (4 to 6 min delayed) with sucrose deliveries over 24 daily conditioning sessions. Thereafter, recessed nosepoke response devices were installed in the chambers and infusions of their assigned drug solutions were contingently available according to a progressive ratio schedule. Rats in the Paired Nicotine condition acquired the nosepoke response, expressed active nosepoke discrimination, and self-administered significantly more infusions than rats in any of the other groups. These results demonstrate that the interoceptive stimulus effects of nicotine can form Pavlovian associations with reinforcing events that alter its reinforcement efficacy.

Prenatal metal mixture concentrations and reward motivation in children

Reward motivation is a complex umbrella term encompassing the cognitions, emotions, and behaviors involved in the activation, execution, and persistence of goal-directed behavior. Altered reward motivation in children is characteristic of many neurodevelopmental and psychiatric disorders. Previously difficult to operationalize, the Progressive Ratio (PR) task has been widely used to assess reward motivation in animal and human studies, including children. Because the neural circuitry supporting reward motivation starts developing during pregnancy, and is sensitive to disruption by environmental toxicants, including metals, the goal of this study was to examine the association between prenatal concentrations of a mixture of neurotoxic metals and reward motivation in children. We measured reward motivation by administering a PR test to 373 children ages 6-8 years enrolled in the Programming Research in Obesity, Growth, Environment and Social Stressors (PROGRESS) Study in Mexico City. Children were asked to press a response lever for a token reward; one press on the response lever was required to earn the first token and each subsequent token required an additional 10 lever presses. Maternal blood concentrations of lead, manganese, zinc, arsenic, cadmium, and selenium were measured using inductively-coupled plasma mass spectrometry during the 2nd and 3rd trimesters of pregnancy. We performed generalized Weighted Quantile Sum (gWQS) regression analyses to examine associations between the prenatal metal mixture and reward motivation; adjusting for child sex, birthweight and age; and maternal IQ, education, and socioeconomic status. The prenatal metal mixture was significantly associated with higher motivation as indicated by more lever presses (ß = 0.02, p < 0.001) and a shorter time between receiving the reinforcer and the first press (ß = 0.23, p = 0.01), and between subsequent presses (ß = 0.07, p = 0.005). Contributions of different metals to this association differed by trimester and child sex. These findings suggest that children with increased exposure to metal during the 2nd and 3rd trimesters of gestation demonstrate increased reward motivation, which may reflect a tendency to perseverate or hypersensitivity to positive reinforcement.

Conditioned enhancement of the nicotine reinforcer

This study was designed to assess whether nicotine can acquire additional reinforcing properties through associations with other rewards. To this end, rats self-administered nicotine-alone (0.01 mg/kg) or nicotine paired with access to sucrose during the conditioning phase. In the subsequent challenge phase, we tested the effect of nicotine-sucrose pairings on the reinforcing effects of nicotine using a progressive ratio schedule of reinforcement. Using this approach, we show that (a) rats in both paired and nicotine-alone conditions self-administered similar amounts of nicotine in the initial conditioning phase of the study when intake was limited to 10 infusions per session, (b) nicotine rapidly acquired control over goal-tracking behavior in the paired condition, (c) rats that had a history of nicotine and sucrose pairings worked harder and took more nicotine as measured on a progressive ratio using a distinct response form, and (d) conditioned goal-tracking evoked by nicotine did not show extinction when sucrose was no longer paired with nicotine over the 11 days of nicotine self-administration on a progressive ratio schedule of reinforcement. Overall, our results demonstrate that in addition to the multifaceted nature of nicotine stimulus that includes primary reinforcing effects, conditioned reinforcing effects, and reward enhancing effects, nicotine can also acquire additional reinforcing properties through associations with other rewards. This ability to acquire additional reinforcing properties through associative learning may contribute to the development and perpetuation of tobacco use disorder. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Inactivation of posterior but not anterior dorsomedial caudate-putamen impedes learning with self-administered nicotine stimulus in male rats

The rodent caudate-putamen is a large heterogeneous neural structure with distinct anatomical connections that differ in their control of learning processes. Previous research suggests that the anterior and posterior dorsomedial caudate-putamen (a- and p-dmCPu) differentially regulate associative learning with a non-contingent nicotine stimulus. The current study used bilateral NMDA-induced excitotoxic lesions to the a-dmCPu and p-dmCPu to determine the functional involvement of a-dmCPu and p-dmCPu in appetitive learning with contingent nicotine stimulus. Rats with a-dmCPu, p-dmCPu, or sham lesions were trained to lever-press for intravenous nicotine (0.03 mg/kg/inf) followed by access to sucrose 30 s later. After 1, 3, 9, and 20 nicotine-sucrose training sessions, appetitive learning in the form of a goal-tracking response was assessed using a non-contingent nicotine-alone test. All rats acquired nicotine self-administration and learned to retrieve sucrose from a receptacle at equal rates. However, rats with lesions to p-dmCPu demonstrated blunted learning of the nicotine-sucrose association. Our primary findings show that rats with lesions to p-dmCPu had a blunted goal-tracking response to a non-contingent nicotine administration after 20 consecutive days of nicotine-sucrose pairing. Our findings extend previous reports to a contingent model of nicotine self-administration and show that p-dmCPu is involved in associative learning with nicotine stimulus using a paradigm where rats voluntarily self-administer nicotine infusions that are paired with access to sucrose-a paradigm that closely resembles learning processes observed in humans.

Binge eating behavior and incentive motivation with a cafeteria diet

Binge-like eating behavior (BLE) has been characterized as an eating disorder in which subjects have an enhanced intake of food, mainly fats. However, intake of fats and carbohydrates may have differential effects on motivation. Previously it was shown that BLE produces an increase in operant responding for vegetable shortening, but others were unable to replicate the finding using sucrose as the reinforcer. Our aim was to determine if BLE behavior induced with a cafeteria-like diet (CaLD) with several options with fat content would produce an increment in performance. Male Wistar rats were trained under an exponential progressive ratio schedule of sucrose reinforcement; thereafter, the limited access model was used to induce BLE using CaLD options. Finally, subjects were tested for increments in break points (BPs) in the progressive ratio schedule. Rats with intermittent access to CaLD options showed a clear BLE with an escalation in their intake; however they showed compensatory decrements of chow intake that rendered a similar body weight gain to a continuous access group. Although we were unable to observe an increase in BPs after BLE we were able to observe a protection against the decrements of BP previously observed with sugar. Different mechanisms for processing high fat and high carbohydrate reinforcers are variables worth exploring to gain a better understanding of BLE behavior in rodent models.

Moderate prenatal ethanol exposure leads to attention deficits in both male and female rats

BACKGROUND

Attention deficits caused by prenatal ethanol (EtOH) exposure (PE) are a prevalent condition in fetal alcohol spectrum disorders (FASDs). Importantly, the deficits are observed in individuals with FASD who have normal IQs and show no dysmorphic facial features caused by heavy PE. These observations suggest that even moderate PE could lead to attention deficits. This possibility was investigated in the present study using a rat model.

METHODS

Pregnant Sprague Dawley rats were administered EtOH (3 g/kg/day) or vehicle via intragastric gavage on gestational days 8 to 20. The blood EtOH concentration (BEC) in EtOH-treated rats was 87.7 ± 1.2 mg/dl (1 h after the gavage), similar to the BECs reported in other moderate PE studies in rodents. Moderate PE did not produce teratogenic effects on birthweight or litter size. The adult offspring underwent a 2-choice reaction time task.

RESULTS

Moderate PE led to augmented action impulsivity in both sexes, indicated by more rapid response initiation and more premature responses. Deficits were more marked in males than in females. No greater lapses of attention, assessed by incorrect or relatively slow responses, were observed in rats of either sex with moderate PE. In addition, no deficits in learning or motor function were detected after moderate PE. Interestingly, rats with moderate PE completed more trials than controls.

CONCLUSIONS

Our results confirm that moderate PE leads to attention deficits in both sexes, which is demonstrated by greater action impulsivity, but not more lapses of attention. This effect differs from that of heavy PE, as shown in our previous study, which is manifested as impaired action impulsivity and lapses of attention in both sexes.

Significant association of nicotine reinforcement and cue reactivity: a translational study in humans and rats

Relapse is common amongst smokers attempting to quit and tobacco cue-induced craving is an important relapse mechanism. Preclinical studies commonly use cue-induced reinstatement of nicotine seeking to investigate relapse neurobiology. Previous research suggests dependence severity and nicotine intake history affect smoking resumption and cue-induced reinstatement of nicotine seeking. However, behavioural data may be interpreted in terms of nicotine reinforcement. This translational study investigated if individual differences in objectively assessed nicotine reinforcement strength were associated with cue-reactivity in both rats and human smokers, which to our knowledge has not been investigated before. Rats (n = 16) were trained to self-administer nicotine and were tested on a progressive ratio schedule of nicotine reinforcement, to assess reinforcer strength, and on a test of cue-induced reinstatement of nicotine seeking. Nicotine reinforcement strength was assessed in human smokers (n = 104) using a forced choice task (nicotine containing vs. denicotinised cigarettes) and self-reported cue-induced craving was assessed following exposure to smoking and neutral cues. Responding for nicotine under progressive ratio was strongly positively correlated with cue-induced reinstatement of nicotine seeking in rats. Nicotine choices in human smokers were significantly associated with cue-induced craving controlling for dependence severity, years of smoking, and urge to smoke following neutral cues. Findings suggest nicotine reinforcement strength is associated with both types of cue-induced behaviour, implying some translational commonality between cue-induced craving in human smokers and cue-induced reinstatement of nicotine seeking in rats. Findings are discussed in relation to clinical implications and whether these laboratory tasks assess drug 'wanting'.

Western Diet Consumption During Development: Setting the Stage for Neurocognitive Dysfunction

The dietary pattern in industrialized countries has changed substantially over the past century due to technological advances in agriculture, food processing, storage, marketing, and distribution practices. The availability of highly palatable, calorically dense foods that are shelf-stable has facilitated a food environment where overconsumption of foods that have a high percentage of calories derived from fat (particularly saturated fat) and sugar is extremely common in modern Westernized societies. In addition to being a predictor of obesity and metabolic dysfunction, consumption of a Western diet (WD) is related to poorer cognitive performance across the lifespan. In particular, WD consumption during critical early life stages of development has negative consequences on various cognitive abilities later in adulthood. This review highlights rodent model research identifying dietary, metabolic, and neurobiological mechanisms linking consumption of a WD during early life periods of development (gestation, lactation, juvenile and adolescence) with behavioral impairments in multiple cognitive domains, including anxiety-like behavior, learning and memory function, reward-motivated behavior, and social behavior. The literature supports a model in which early life WD consumption leads to long-lasting neurocognitive impairments that are largely dissociable from WD effects on obesity and metabolic dysfunction.

Within- and across-session increases in work requirement do not produce similar response output

Demand curve fitting is a method of data analysis for interpreting reinforcer consumption. These methods were established and validated by examining increases in unit price (UP) across sessions. An alternative experimental preparation is the progressive-ratio (PR) schedule in which schedule requirements increase within a session. Although PR schedules provide an efficient alternative to traditional evaluations of UP, using demand curves to interpret data obtained via PR schedules has not been systematically evaluated in applied contexts. In this study, the experimenters compared demand curves constructed based on across- and within-session increases in UP and evaluated correspondence between the two methods. Results indicated poor correspondence between demand curves constructed with the two methods. Furthermore, within-session measures of responding suggest that higher rates of problem behavior and longer durations of postreinforcement pauses were more likely under PR schedules than fixed-ratio schedules. Results are discussed in terms of implications for clinical application.

Dissociating the effects of dopamine D2 receptors on effort-based versus value-based decision making using a novel behavioral approach

Cost-benefit decision making is essential for organisms to adapt to their ever-changing environment. Most studies of cost-benefit decision making involve choice conditions in which effort and value are varied simultaneously. This prevents identification of the aspects of cost-benefit decision making that are affected by experimental manipulations. We developed operant assays to isolate the individual impacts of effort and value manipulations on cost-benefit decision making. In the concurrent effort choice (CEC) task, mice choose between exerting two distinct types of effort: the number of responses and the duration of a response, to earn the same reward. By parametrically varying response cost, psychometric functions are obtained that reflect how the two types of effort scale against one another. Direct manipulations of effort shift the functions. Because reward value is held constant in this task, differences in scaling of the two response types must be related to the effort manipulations. In the concurrent value choice (CVC) task, mice make the same type of response to earn rewards of different value (e.g., pellets vs. sucrose solutions). Here the effort required to earn one reward type is parametrically varied to obtain the psychometric function that scales the value of the two rewards into the number of responses subjects will pay to earn one reward over the other. Direct value manipulations shift these functions. We tested the effect of the dopamine D2 receptor antagonist, haloperidol, on performance in the CEC and CVC assays and found that D2R signaling is important for effort-based, but not value-based decision making. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

The term "psychological well-being" is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one's environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

Can Slot-Machine Reward Schedules Induce Gambling Addiction in Rats?

Similar to drugs of abuse, random-ratio reward schedules are highly motivating and, in humans, are thought to foster gambling addiction. Animal gambling models, however, have not yet demonstrated the compulsivity so characteristic of drug addiction. Three criteria have been used to evaluate addiction-like behavior in drug models: (1) response inhibition when reward is not available, (2) persistence under a progressive ratio schedule, in which the response-to-reward ratio is stretched, and (3) persistence in spite of punishment. We tested whether prolonged exposure (6 weeks) to a gambling-like reward schedule would induce addiction-like symptoms in rats. In two studies, separate groups were trained to respond to either random- or fixed-ratio schedules for food reward. We found that rats trained on random-ratio schedules showed higher response rates and dramatically shorter pauses after rewards. Tests of addiction-like behavior, however, were largely negative. Response rates were not different during cued no-reward periods nor when reward was coupled with punishment. We also found no group differences when food was devalued nor in reinstatement of reward-seeking after a 1-week delay. The sole exception to this pattern was that rats in the second experiment showed greater persistence on a progressive ratio test. After experiment two, subjects were also orally administered pramipexole, which caused increased perseveration during progressive ratio testing, especially in the random ratio group. While, it is possible that longer training or more appetitive rewards might have led to addiction-like behavior, our results, on the surface, suggest that random-ratio schedules are motivating but not addictive.

Reinforcer quality matters: A test of the Mathematical Principles of Reinforcement with domestic hens (Gallus gallus domesticus)

This study evaluated the ability of Killeen's (1994) Mathematical Principles of Reinforcement to account for the effects of changes in reinforcer quality on hens' rates of responding on fixed-ratio schedules. Hens were trained to peck a key on a fixed-ratio schedule of reinforcement and then experienced an ascending series of ratio values in two separate conditions. In different conditions, the food reinforcer was either wheat or puffed wheat. Response rates initially increased with increases in ratio requirement before eventually decreasing at larger ratios. Quantitative fits of the model accounted for the data well. The fits revealed that different foods were systematically associated with changes in the specific activation parameter, a, and these were consistent with previous reports of preference for those food items.

Evaluating preference for and reinforcing efficacy of fruits and vegetables compared with salty and sweet foods

Children diagnosed with autism spectrum disorder are often more selective in their food preferences than their typically developing peers. Additionally, many preferred food selections have minimal nutritional value. We evaluated the preference for, and reinforcing efficacy of, fruits and vegetables compared with salty and sweet foods, which are often used as edible reinforcers. Multiple-stimulus preference assessments were conducted to identify preferred foods, and reinforcer assessments were conducted to determine the reinforcing efficacy of more preferred foods. Fruits and vegetables were sometimes preferred over salty or sweet foods and often functioned as reinforcers. Future research should incorporate fruits and vegetables into preference assessments when identifying putative reinforcers.

Conditioned taste aversion versus avoidance: A re-examination of the separate processes hypothesis

Rats not only avoid ingesting a substance associated with LiCl toxicosis, but they display rejection reflexes (e.g., gapes) to its taste; this latter response is thought to reflect disgust or taste aversion. Prior work has shown that rats also avoid consuming foods/fluids associated with other adverse gastrointestinal (GI) effects like lactose indigestion but without the concomitant change in oromotor responses (taste reactivity; TR) indicative of aversion. Because of interpretive limitations of the methods used in those studies, we revisited the taste aversion-avoidance distinction with a design that minimized non-treatment differences among groups. Effects on intake and preference (Experiments 1a, 1b, and 2), as well as consummatory (TR, Experiment 1a and 1b) and appetitive (Progressive Ratio, Experiment 2) behaviors to the taste stimulus were assessed after training. In both experiments, rats were trained to associate 0.2% saccharin (CS) with intraduodenal infusions of LiCl, Lactose, or NaCl control. Rats trained with 18% lactose, 0.3 and 1.5 mEq/kg dose of LiCl subsequently avoided the taste CS in post-training single-bottle intake tests and two-bottle choice tests. However, only those trained with 1.5 mEq/kg LiCl displayed post-conditioning increases in taste CS-elicited aversive TR (Experiment 1a and 1b). This dose of LiCl also led to reductions in breakpoint for saccharin. The fact that conditioned avoidance is not always accompanied by changes in other common appetitive and/or consummatory indices of ingestive motivation further supports a functional dissociation between these processes, and highlights the intricacies of visceral influences on taste-guided ingestive motivation.

Inhibition of Methamphetamine Self-Administration and Reinstatement by Central Blockade of Angiotensin II Receptor in Rats

The molecular mechanism and treatment of methamphetamine (METH) use disorder remain unclear. The current study aimed to investigate the role of central angiotensin II receptor (ATR) in drug taking and seeking behavior associated with METH use disorder. The effect of an ATR type 1 (AT1R) antagonist, candesartan cilexetil, on the reinforcing and motivational effects of METH was first assessed using the animal model of METH self-administration (SA) and reinstatement. The levels of dopamine D2 receptor (D2R) and AT1R were subsequently examined. Furthermore, the present study determined the expression of microRNAs (miRNAs) by comparing METH SA, METH-yoked, and Saline-yoked groups. The target miRNAs were further overexpressed in the nucleus accumbens (NAc) via a lentivirus vector to investigate the effects of target miRNAs on METH SA maintained under a fixed ratio 1, progressive ratio, and cue/drug reinstatement of METH SA. The potential role of the AT1R-PLCβ-CREB signaling pathway was finally investigated. The results suggest that AT1R blockade effectively reduced METH SA and reinstatement, in conjunction with the counter-regulation of D2R and AT1R. A total of 17 miRNAs targeting Ang II in NAc were found to be associated with the voluntary intake of METH. Furthermore, overexpression of specific miR-219a-5p targeting AT1R-regulated METH SA and reinstatement. The AT1R-PLCβ-CREB signaling pathway was found to be associated with the effect of AT1R on the drug-taking and drug-seeking behavior involving METH use disorder.

Evaluating the cyclic ratio schedule as an assay of feeding behaviour in the European starling (Sturnus vulgaris)

The cyclic ratio (CR) schedule is a behavioural assay developed to study feeding in rats, in which the number of operant responses required to obtain food reward (the ratio requirement) increases and then decreases in a repeating cycle. In a recent study, we used the CR schedule with European starlings (Sturnus vulgaris) to investigate the effects of an early-life manipulation on adult feeding behaviour. As this was the first time the CR schedule had been used with any bird species, a more in-depth evaluation is warranted. Here, we performed a fuller CR experiment with the same birds as the prior study, a year later. First, we examine the individual consistency of feeding behaviour between experimental sessions and also between CR schedules comprising different ratio requirement progressions. We found that between-session consistency was poor to moderate, and that a geometric ratio progression provided greater between-session consistency than an arithmetic ratio progression. Second, we tried to replicate some of the canonical findings from rats working on CR schedules. In contrast to findings from rats, we found that defence of feeding rates did not increase when starlings were acutely food deprived. However, as in rats, we found that the post-reinforcement pause increased linearly with the upcoming ratio requirement, suggesting that starlings were able to learn the cyclic nature of the schedule. Third, we compared the results from the present study concerning the impacts of our early-life treatment with those from our earlier study. We found that the majority of our previous findings were replicated in the same individuals one year on, reinforcing our previous conclusion that the early-life manipulation had canalised our birds into two groups with different patterns of feeding rate defence.

Validation and optimisation of a touchscreen progressive ratio test of motivation in male rats

RATIONALE

Across species, effort-related motivation can be assessed by testing behaviour under a progressive ratio (PR) schedule of reinforcement. However, to date, PR tasks for rodents have been available using traditional operant response systems only.

OBJECTIVES

Touchscreen operant response systems allow the assessment of behaviour in laboratory rodents, using tasks that share high face validity with the computerised assessments used in humans. Here, we sought to optimise a rat touchscreen variant of PR and validate it by assessing the effects of a number of manipulations known to affect PR performance in non-touchscreen paradigms.

METHODS

Separate groups of male Sprague-Dawley rats were trained on PR schedules with either linear (PR4) or exponential (PREXP) schedules of reinforcement. PR performance was assessed in response to manipulations in reward outcome. Animals were tested under conditions of increased reward magnitude and following reward devaluation through a prefeeding procedure. Subsequently, the effects of systemic administration of the dopamine D2/D3 receptor antagonist raclopride and the psychostimulant d-amphetamine were examined as traditional pharmacological methods for manipulating motivation.

RESULTS

Rats reinforced under PR4 and PREXP schedules consistently showed differential patterns of response rates within sessions. Furthermore, both PR4 and PREXP schedules were sensitive to suppression by prefeeding or raclopride administration. Performance under both schedules was facilitated by increasing reward magnitude or d-amphetamine administration.

CONCLUSIONS

Taken together, these findings mirror those observed in lever-based PR paradigms in rats. This study therefore demonstrates the successful validation of the rat touchscreen PR task. This will allow for the assessment of motivation in rats, within the same touchscreen apparatus used for the assessment of complex cognitive processes in this species.

A comparison of resetting and nonresetting contingencies in progressive-duration schedules

We compared two progressive schedules of reinforcement in which rats received access to sweetened condensed milk for depressing and holding down a response lever. Duration requirements increased after each reinforcer delivery in a manner similar to progressive-ratio schedules. Under one schedule, any response duration less than that required for reinforcement had no programmed consequences. Under the second schedule, the cumulative duration of all responses could meet the reinforcement criteria. Breaking points were consistently higher when all lever presses, regardless of duration, contributed to meeting the reinforcer requirements. Breaking points under both schedules increased when food deprivation was long enough to result in body-weight reductions, but the sensitivity of the schedules to brief periods of food deprivation was inconsistent. Under both schedules, food deprivation produced an increase in shorter durations, thus reducing the efficiency of responding.

Activation of Ventral Tegmental Area 5-HT2C Receptors Reduces Incentive Motivation

Obesity is primarily due to food intake in excess of the body's energetic requirements, intake that is not only associated with hunger but also the incentive value of food. The 5-hydroxytryptamine 2C receptor (5-HT_2CR) is a target for the treatment of human obesity. Mechanistically, 5-HT_2CRs are positioned to influence both homeostatic feeding circuits within the hypothalamus and reward circuits within the ventral tegmental area (VTA). Here we investigated the role of 5-HT_2CRs in incentive motivation using a mathematical model of progressive ratio (PR) responding in mice. We found that the 5-HT_2CR agonist lorcaserin significantly reduced both ad libitum chow intake and PR responding for chocolate pellets and increased c-fos expression in VTA 5-HT_2CR expressing γ-aminobutyric acid (GABA) neurons, but not 5-HT_2CR expressing dopamine (DA) neurons. We next adopted a chemogenetic approach using a 5-HT_2CR^CRE line to clarify the function of subset of 5-HT_2C receptor expressing VTA neurons in the modulation of appetite and food-motivated behavior. Activation of VTA 5-HT_2C receptor expressing neurons significantly reduced ad libitum chow intake, operant responding for chocolate pellets, and the incentive value of food. In contrast, chemogenetic inhibition of VTA 5-HT_2C receptor expressing neurons had no effect on the feeding behavior. These results indicate that activation of the subpopulation of 5-HT_2CR neurons within the VTA is sufficient to significantly reduce homeostatic feeding and effort-based intake of palatable food, and that this subset has an inhibitory role in motivational processes. These findings are relevant to the treatment of obesity.

Future Research Directions in the Positive Valence Systems: Measurement, Development, and Implications for Youth Unipolar Depression

The Positive Valence Systems (PVS) have been introduced by the National Institute of Mental Health as a domain to help organize multiple constructs focusing on reward-seeking behaviors. However, the initial working model for this domain is strongly influenced by adult constructs and measures. Thus, the present review focuses on extending the PVS into a developmental context. Specifically, the review provides some hypotheses about the structure of the PVS, how PVS components may change throughout development, how family history of depression may influence PVS development, and potential means of intervening on PVS function to reduce onsets of depression. Future research needs in each of these areas are highlighted.

Concurrent progressive-ratio and fixed-ratio schedule performance under geometric and arithmetic progressions by brushtail possums

The aim of this study was to compare the demand for food under concurrent progressive- and fixed-ratio schedules. Twelve brushtail possums participated in 16 conditions where schedule, progression and food type were varied. An incrementing schedule increased the fixed-ratio requirement within and across sessions and was arranged as either a geometric sequence (base 2), or an arithmetic sequence (step 5). Two foods were tested: a flaked barley and coco-pop(®) mix versus rolled oats. Overall, performance was similar for most possums in the within- and across-session incrementing schedules. An analysis of the estimates of essential value and break point produced the same account of demand for foods under the geometric or arithmetic progressions and within- and across-session procedures for 8 of 12 possums. Six possums showed higher demand for rolled oats compared to flaked barley, and two possums showed higher demand for flaked barley compared to rolled oats. Incrementing ratios within, rather than between sessions using an arithmetic progression was demonstrated to be a time efficient procedure for investigating demand for different food types without affecting conclusions about the relative demand for those foods.

Minor Functional Deficits in Basic Response Patterns for Reinforcement after Frontal Traumatic Brain Injury in Rats

Traumatic brain injury (TBI) is a major contributor to numerous psychiatric conditions and chronic behavioral dysfunction. Recent studies in experimental brain injury have begun to adopt operant methodologies to assess these deficits, all of which rely on the process of reinforcement. No studies have directly examined how reinforced behaviors are affected by TBI, however. The current study assessed performance under the four most common schedules of reinforcement (fixed ratio, variable ratio, fixed interval, variable interval) and one higher order schedule assessing motivation (progressive ratio) after bilateral, pre-frontal controlled cortical impact injury. TBI-induced differences on the basic schedules were minor, with the exception of the variable ratio, where increased efficacy (more reinforcers, higher response rates, lower interresponse times) at higher requirements was observed as a result of brain injury. Performance on the progressive ratio schedule showed some gross differences between the groups, in that sham rats became more efficient under this schedule while injured rats perseverated in lever pressing. Further, injured rats were specifically impaired at lower response requirements on the progressive ratio. Taken together, these findings indicate that simple reinforced behaviors are mostly unaffected after TBI, except in the case of variable ratio schedules, but the altered performance on the higher-order progressive ratio schedule suggests changes involving motivation or potentially perseveration. These findings validate operant measures of more complex behaviors for brain injury, all of which rely on reinforcement and can be taken into consideration when adapting and developing novel functional assessments.

Evaluation of a Postoperative Pain-Like State on Motivated Behavior in Rats: Effects of Plantar Incision on Progressive-Ratio Food-Maintained Responding

There has been recent interest in characterizing the effects of pain-like states on motivated behaviors in order to quantify how pain modulates goal-directed behavior and the persistence of that behavior. The current set of experiments assessed the effects of an incisional postoperative pain manipulation on food-maintained responding under a progressive-ratio (PR) operant schedule. Independent variables included injury state (plantar incision or anesthesia control) and reinforcer type (grain pellet or sugar pellet); dependent variables were tactile sensory thresholds and response breakpoint. Once responding stabilized on the PR schedule, separate groups of rats received a single ventral hind paw incision or anesthesia (control condition). Incision significantly reduced breakpoints in rats responding for grain, but not sugar. In rats responding for sugar, tactile hypersensitivity recovered within 24 hr, indicating a faster recovery of incision-induced tactile hypersensitivity compared to rats responding for grain, which demonstrated recovery at PD2. The NSAID analgesic, diclofenac (5.6 mg/kg) completely restored incision-depressed PR operant responding and tactile sensitivity at 3 hr following incision. The PR schedule differentiated between sucrose and grain, suggesting that relative reinforcing efficacy may be an important determinant in detecting pain-induced changes in motivated behavior.

A progressive-duration schedule of reinforcement

We describe a schedule of reinforcement involving systematic, within-session increases in response-duration requirements. Rats received access to appetitive reinforcers for depressing and holding down a response lever. Duration requirements increased after each reinforcer delivery. Sessions ended when reinforcement criteria were unmet for a period of ten minutes. Breaking points, defined as the terminal duration requirement in effect prior to the end of the session, stabilized when environmental conditions were held constant. Breaking points were sensitive to manipulations of both food deprivation and reinforcer quality. Analogous to progressive-ratio schedules, progressive-duration schedules may provide an assay for measuring the amount of behavior an organism will emit for a given reinforcer under current motivational conditions.

Quantitative analysis of performance on a progressive-ratio schedule: effects of reinforcer type, food deprivation and acute treatment with Δ⁹-tetrahydrocannabinol (THC)

Rats' performance on a progressive-ratio schedule maintained by sucrose (0.6M, 50 μl) and corn oil (100%, 25 μl) reinforcers was assessed using a model derived from Killeen's (1994) theory of schedule-controlled behaviour, 'Mathematical Principles of Reinforcement'. When the rats were maintained at 80% of their free-feeding body weights, the parameter expressing incentive value, a, was greater for the corn oil than for the sucrose reinforcer; the response-time parameter, δ, did not differ between the reinforcer types, but a parameter derived from the linear waiting principle (T0), indicated that the minimum post-reinforcement pause was longer for corn oil than for sucrose. When the rats were maintained under free-feeding conditions, a was reduced, indicating a reduction of incentive value, but δ was unaltered. Under the food-deprived condition, the CB1 cannabinoid receptor agonist Δ(9)-tetrahydrocannabinol (THC: 0.3, 1 and 3 mg kg(-1)) increased the value of a for sucrose but not for corn oil, suggesting a selective enhancement of the incentive value of sucrose; none of the other parameters was affected by THC. The results provide new information about the sensitivity of the model's parameters to deprivation and reinforcer quality, and suggest that THC selectively enhances the incentive value of sucrose.

Methods for Dissecting Motivation and Related Psychological Processes in Rodents

Motivational impairments are increasingly recognized as being critical to functional deficits and decreased quality of life in patients diagnosed with psychiatric disease. Accordingly, much preclinical research has focused on identifying psychological and neurobiological processes which underlie motivation . Inferring motivation from changes in overt behavioural responding in animal models, however, is complicated, and care must be taken to ensure that the observed change is accurately characterized as a change in motivation , and not due to some other, task-related process. This chapter discusses current methods for assessing motivation and related psychological processes in rodents. Using an example from work characterizing the motivational impairments in an animal model of the negative symptoms of schizophrenia, we highlight the importance of careful and rigorous experimental dissection of motivation and the related psychological processes when characterizing motivational deficits in rodent models . We suggest that such work is critical to the successful translation of preclinical findings to therapeutic benefits for patients.

Effect of streptozotocin-induced diabetes on performance on a progressive ratio schedule

RATIONALE

It has been suggested that streptozotocin (STZ)-induced diabetes causes a motivational deficit in rodents. However, some of the evidence adduced in support of this suggestion may be interpreted in terms of a motor impairment rather than a motivational deficit.

OBJECTIVE

This experiment examined the effect of STZ-induced diabetes on performance on a progressive ratio schedule. The data were analysed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement model which enables the effects of interventions on motivation or incentive value to be separated from effects on motor function.

METHOD

Animals were trained under a progressive ratio schedule using food-pellet reinforcement. Then they received a single intraperitoneal injection of 50 mg/kg of STZ or the vehicle. Training continued for 30 sessions after treatment. Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between groups.

RESULTS

The parameter expressing incentive value was reduced in the group treated with STZ, whereas the parameters expressing motor capacity and post-reinforcement pausing were not affected by the treatment. Blood glucose concentration was significantly elevated in the STZ-treated group compared to the vehicle-treated group.

CONCLUSIONS

The results are consistent with the suggestion that STZ-induced diabetes is associated with a reduction of the incentive value of food.

Measurement of food reinforcement in preschool children. Associations with food intake, BMI, and reward sensitivity

Progressive ratio (PR) schedules of reinforcement have been used to measure the relative reinforcing value (RRV) of food in humans as young as 8 years old; however, developmentally appropriate measures are needed to measure RRV of food earlier in life. Study objectives were to demonstrate the validity of the RRV of food task adapted for use among for preschool children (3-5 y), and examine individual differences in performance. Thirty-three children completed the RRV of food task in which they worked to access graham crackers. They also completed a snack task where they had free access these foods, liking and hunger assessments, and their heights and weights were measured. Parents reported on their child's reward sensitivity. Overall, children were willing work for palatable snack foods. Boys and older children made more responses in the task, while children with higher BMI z-scores and reward sensitivity responded at a faster rate. Children who worked harder in terms of total responses and response rates consumed more calories in the snack session. This study demonstrates that with slight modifications, the RRV of food task is a valid and developmentally appropriate measure for assessing individual differences in food reinforcement among very young children.

The novel recreational drug 3,4-methylenedioxypyrovalerone (MDPV) is a potent psychomotor stimulant: self-administration and locomotor activity in rats

Recreational use of the cathinone derivative 3,4-methylenedioxypyrovalerone (MDPV; "bath salts") has increased worldwide in past years, accompanied by accounts of health and legal problems in the popular media and efforts to criminalize possession in numerous jurisdictions. Minimal information exists on the effects of MDPV in laboratory models. This study determined the effects of MDPV, alongside those of the better studied stimulant d-methamphetamine (METH), using rodent models of intravenous self-administration (IVSA), thermoregulation and locomotor activity. Male Wistar rats were trained to self-administer MDPV or METH (0.05 mg/kg/infusion, i.v.) or were prepared with radiotelemetry implants for the assessment of body temperature and activity responses to MDPV or METH (0-5.6 mg/kg s.c.). METH and MDPV were consistently self-administered within 10 training sessions (mg/kg/h; METH Mean = 0.4 and Max = 1.15; MDPV Mean = 0.9 and Max = 5.8). Dose-substitution studies demonstrated that behavior was sensitive to dose for both drugs, but MDPV (0.01-0.50 mg/kg/inf) showed greater potency and efficacy than METH (0.1-0.25 mg/kg/inf). In addition, both MDPV and METH increased locomotor activity at lower doses (0.5-1.0 mg/kg, s.c.) and transiently decreased activity at the highest dose (5.6 mg/kg, s.c.). Body temperature increased monotonically with increasing doses of METH but MDPV had a negligible effect on temperature. Stereotypy was associated with relatively high self-administered cumulative doses of MDPV (∼1.5 mg/kg/h) as well as with non-contingent MDPV administration wherein the intensity and duration of stereotypy increased as MDPV dose increased. Thus, MDPV poses a substantial threat for compulsive use that is potentially greater than that for METH.

A quantitative analysis of the effects of qualitatively different reinforcers on fixed ratio responding in inbred strains of mice

Previous studies of inbred mouse strains have shown reinforcer-strain interactions that may potentially mask differences among strains in memory performance. The present research examined the effects of two qualitatively different reinforcers (heterogeneous mix of flavored pellets and sweetened-condensed milk) on responding maintained by fixed-ratio schedules of reinforcement in three inbred strains of mice (BALB/c, C57BL/6, and DBA/2). Responses rates for all strains were a bitonic (inverted U) function of the size of the fixed-ratio schedule and were generally higher when responding was maintained by milk. For the DBA/2 and C57BL/6 and to a lesser extent the BALB/c, milk primarily increased response rates at moderate fixed ratios, but not at the largest fixed ratios tested. A formal model of ratio-schedule performance, Mathematical Principles of Reinforcement (MPR), was applied to the response rate functions of individual mice. According to MPR, the differences in response rates maintained by pellets and milk were mostly due to changes in motoric processes as indicated by changes in the minimum response time (δ) produced by each reinforcer type and not specific activation (a), a model term that represents value and is correlated with reinforcer magnitude and the break point obtained under progressive ratio schedules. MPR also revealed that, although affected by reinforcer type, a parameter interpreted as the rate of saturation of working memory (λ), differed among the strains.

Effects of SKF-83566 and haloperidol on performance on progressive ratio schedules maintained by sucrose and corn oil reinforcement: quantitative analysis using a new model derived from the Mathematical Principles of Reinforcement (MPR)

RATIONALE

Mathematical models can assist the interpretation of the effects of interventions on schedule-controlled behaviour and help to differentiate between processes that may be confounded in traditional performance measures such as response rate and the breakpoint in progressive ratio (PR) schedules.

OBJECTIVE

The effects of a D1-like dopamine receptor antagonist, 8-bromo-2,3,4,5-tetrahydro-3-methyl-5-phenyl-1H-3-benzazepin-7-ol hydrobromide (SKF-83566), and a D2-like receptor antagonist, haloperidol, on rats' performance on PR schedules maintained by sucrose and corn oil reinforcers were assessed using a new model derived from Killeen's (Behav Brain Sci 17:105-172, 1994) Mathematical Principles of Reinforcement.

METHOD

Separate groups of rats were trained under a PR schedule using sucrose or corn oil reinforcers. SKF-83566 (0.015 and 0.03 mg kg(-1)) and haloperidol (0.05 and 0.1 mg kg(-1)) were administered intraperitoneally (five administrations of each treatment). Running and overall response rates in successive ratios were analysed using the new model, and estimates of the model's parameters were compared between treatments.

RESULTS

Haloperidol reduced a (the parameter expressing incentive value) in the case of both reinforcers, but did not affect the parameters related to response time and post-reinforcement pausing. SKF-83566 reduced a and k (the parameter expressing sensitivity of post-reinforcement pausing to the prior inter-reinforcement interval) in the case of sucrose, but did not affect any of the parameters in the case of corn oil.

CONCLUSIONS

The results are consistent with the hypothesis that blockade of both D1-like and D2-like receptors reduces the incentive value of sucrose, whereas the incentive value of corn oil is more sensitive to blockade of D2-like than D1-like receptors.

Effects of amisulpride and aripiprazole on progressive-ratio schedule performance: comparison with clozapine and haloperidol

Clozapine and some other atypical antipsychotics (e.g. quetiapine, olanzapine) have been found to exert a characteristic profile of action on operant behaviour maintained by progressive-ratio schedules, as revealed by Killeen's Mathematical Principles of Reinforcement model of schedule-controlled behaviour. These drugs increase the value of a parameter that expresses the 'incentive value' of the reinforcer (a) and a parameter that is inversely related to the organism's 'motor capacity' (δ). This experiment examined the effects of two further atypical antipsychotics, aripiprazole and amisulpride, on progressive-ratio schedule performance in rats; the effects of clozapine and a conventional antipsychotic, haloperidol, were also examined. In agreement with previous findings, clozapine (4, 8 mg kg⁻¹) increased a and δ, whereas haloperidol (0.05, 0.1 mg kg⁻¹) reduced a and increased δ. Aripiprazole (3,30 mg kg⁻¹) increased δ but did not affect a. Amisulpride (5, 50 mg kg⁻¹) had a delayed and protracted effect: δ was increased 3-6 hours after treatment; a was increased 1.5 hours, and reduced 12-24 hours after treatment. Interpretation based on Killeen's model suggests that aripiprazole does not share clozapine's ability to enhance reinforcer value. Amisulpride produced a short-lived enhancement, followed by a long-lasting reduction, of reinforcer value. Both drugs impaired motor performance.

On distinguishing progressively increasing response requirements for reinforcement

Several different arrangements have been described for increasing the response requirements for reinforcement using the label progressive-ratio schedule. Under the original progressive-ratio schedule, the response requirement is increased after each reinforcer. Subsequently, arrangements have been used in which the number of required responses increases following multiple reinforcers at a single response requirement or between sessions. Following an assessment of the different types of contingencies that result from such progressive response requirements and the labels used to describe them, a set of descriptive labels is suggested for these different types of progressively increasing response requirements.

A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology

RATIONALE

Mathematical principles of reinforcement (MPR) provide the theoretical basis for a family of models of schedule-controlled behaviour. A model of fixed-ratio schedule performance that was applied to behaviour on progressive ratio (PR) schedules showed systematic departures from the data.

OBJECTIVE

This study aims to derive a new model from MPR that will account for overall and running response rates in the component ratios of PR schedules, and their decline toward 0, the breakpoint.

METHOD

The role of pausing is represented in a real-time model containing four parameters: T (0) and k are the intercept and slope of the linear relation between post-reinforcement pause duration and the prior inter-reinforcer interval; a (specific activation) measures the incentive value of the reinforcer; δ (response time) sets biomechanical limits on response rate. Running rate is predicted to decrease with negative acceleration as ratio size increments, overall rate to increase and then decrease. Differences due to type of progression are explained as hysteresis in the control by reinforcement rates. Re-analysis of extant data focuses on the effects of acute treatment with antipsychotic drugs, lesions of the nucleus accumbens core, and destruction of orexinergic neurones of the lateral hypothalamus.

RESULTS

The new model resolves some anomalies evident in earlier analyses, and provides new insights to the results of these interventions.

CONCLUSIONS

Because they can render biologically relevant parameters, mathematical models can provide greater power in interpreting the effects of interventions on the processes underlying schedule-controlled behaviour than is possible for first-order data such as the breakpoint.

A clozapine-like effect of cyproheptadine on progressive ratio schedule performance

The atypical antipsychotic drug clozapine has multiple pharmacological actions, some of which, including 5-hydroxytryptamine (5-HT₂) and histamine (H₁) receptor antagonist effects, are shared by the non-selective 5-HT receptor antagonist cyproheptadine. Atypical antipsychotics have a characteristic profile of action on operant behaviour maintained by progressive ratio schedules, as revealed by Killeen's (1994) mathematical model of schedule controlled behaviour. These drugs increase the values of a parameter that expresses the 'incentive value' of the reinforcer (a) and a parameter that is inversely related to the 'motor capacity' of the organism (δ). This experiment examined the effects of acute treatment with cyproheptadine and clozapine on performance on a progressive ratio schedule of food reinforcement in rats; the effects of a conventional antipsychotic, haloperidol, and two drugs with food intake-enhancing effects, chlordiazepoxide and Δ⁹-tetrahydrocannabinol (THC), were also examined. Cyproheptadine (1, 5 mg kg⁻¹) and clozapine (3.75, 7.5 mg kg⁻¹) increased a and δ. Haloperidol (0.05, 0.1 mg kg⁻¹) reduced a and increased δ. Chlordiazepoxide (3, 10 mg kg⁻¹) increased a but reduced δ. THC (1, 3 mg kg⁻¹) had no effect. Interpretation based on Killeen's (1994) model suggests that cyproheptadine and clozapine enhanced the incentive value of the reinforcer and impaired motor performance. Motor impairment may be due to sedation (possibly reflecting H₁ receptor blockade). Enhancement of incentive value may reflect simultaneous blockade of H₁ and 5-HT₂ receptors, which has been proposed as the mechanism underlying the food intake-enhancing effect of cyproheptadine. In agreement with previous findings, haloperidol impaired motor performance and reduced the incentive value of the reinforcer. Chlordiazepoxide's effect on a is consistent with its food intake-enhancing effect.

Effect of orexin-B-saporin-induced lesions of the lateral hypothalamus on performance on a progressive ratio schedule

It has been suggested that a sub-population of orexinergic neurones whose somata lie in the lateral hypothalamic area (LHA) play an important role in regulating the reinforcing value of both food and drugs. This experiment examined the effect of disruption of orexinergic mechanisms in the LHA on performance on the progressive ratio schedule of reinforcement, in which the response requirement increases progressively for successive reinforcers. The data were analysed using a mathematical model which yields a quantitative index of reinforcer value and dissociates effects of interventions on motor and motivational processes. Rats were trained under a progressive ratio schedule using food-pellet reinforcement. They received bilateral injections of conjugated orexin-B-saporin (OxSap) into the LHA or sham lesions. Training continued for a further 40 sessions after surgery. Equations were fitted to the response rate data from each rat, and the parameters of the model were derived for successive blocks of 10 sessions. The OxSap lesion reduced the number of orexin-containing neurones in the LHA by approximately 50% compared with the sham-lesioned group. The parameter expressing the incentive value of the reinforcer was not significantly altered by the lesion. However, the parameter related to the maximum response rate was significantly affected, suggesting that motor capacity was diminished in the OxSap-lesioned group. The results indicate that OxSap lesions of the LHA disrupted food-reinforced responding on the progressive ratio schedule. It is suggested that this disruption was brought about by a change in non-motivational (motor) processes.

A mixed fixed ratio/progressive ratio procedure reveals an apathy phenotype in the BAC HD and the z_Q175 KI mouse models of Huntington's disease

Apathy, characterized by generally reduced interest in and likelihood to perform goal-directed actions, is a recognized symptom of Huntington's disease (HD), a devastating neurological disorder caused by a CAG repeat expansion of the Htt gene located on chromosome 4. The present experiments used a modified progressive ratio task that incorporated a fixed-ratio schedule of reinforcement component to assess consummatory behavior, and a progressive-ratio schedule component that required increasing numbers of lever-presses for successive reinforcers (0.01 ml of evaporated milk). The studies revealed an apathetic phenotype in two mouse models of HD, with decreased response rates either overall or only at higher ratio requirements in the progressive-ratio component relative to wild-type controls. Based on the procedure used (within-session fixed- and progressive-ratio components), it is proposed that an observed phenotype can be ascribed either specifically to reduced motivation to work for food reinforcement or more generally to deficits in consummatory behavior. This procedure provides a simple means to assess this type of phenotype in rodents, with issues in consummatory vs. incentive motivation reflected in general alterations in fixed- versus progressive alterations on an escalating-ratio schedules respectively, providing translational measures of the amotivation/apathy construct of the human realm to the homologous construct of incentive motivation in preclinical models of human disease.

Assessing motivation in children using a progressive ratio task

The association of age and sex on the performance of a progressive ratio task was studied in 847 children, ages 4-14 years. Variations of this task have been used extensively with animals and to a lesser extent with humans to study factors that affect aspects of motivation. The participants in this study were required to press a response lever for nickel reinforcers during a 10 min period. One response was required to earn the first nickel and each subsequent nickel required an additional 10 more responses. Older children had a significantly higher breakpoint than younger children. This appeared to be mostly the result of older children having significantly shorter inter-response times than younger children. In addition, boys had significantly higher breakpoints than girls, especially at older ages. The results of this study illustrate that both age and sex influence the performance of this task and thus suggest that age and sex influence aspects of motivation in children. Further, characterization of performance of this task by humans facilitates comparisons with animal models and, thus, enhances its translational utility.

The neuropharmacological substrates of nicotine reward: reinforcing versus reinforcement-enhancing effects of nicotine

Compulsive nicotine use is thought to be maintained by the acute reinforcing effects of nicotine and the reinforcement-enhancing effects of nicotine, in addition to the negative consequences of nicotine abstinence. Nicotine self-administration and nicotine-induced enhancement of non-nicotine reinforcers such as intracranial self-stimulation provide measures of these dual rewarding properties of nicotine. First, pharmacological manipulations that modulate the reinforcing and reinforcement-enhancing effects of nicotine are identified and discussed. Second, the interpretation and implications of data that identified shared and specific pharmacological substrates underlying the dual rewarding effects of nicotine are discussed, including implications for the preclinical testing of putative antismoking medications. In conclusion, reinforcement-related behaviors that are mediated by central reinforcement processes are likely to, and generally do, exhibit a number of common pharmacological substrates. Interestingly, however, a few pharmacological classes of compounds seem to exert selective effects on components of the dual nicotine reward mechanisms, indicating differences in the pharmacological substrates of the reinforcing and reinforcement-enhancing effects of nicotine. Further characterization of such compounds may ultimately lead to the identification of novel medications for nicotine dependence in humans.

Effect of reinforcer magnitude on performance maintained by progressive-ratio schedules

This experiment examined the relationship between reinforcer magnitude and quantitative measures of performance on progressive-ratio schedules. Fifteen rats were trained under a progressive-ratio schedule in seven phases of the experiment in which the volume of a 0.6-M sucrose solution reinforcer was varied within the range 6-300 microl. Overall response rates in successive ratios conformed to a bitonic equation derived from Killeen's (1994) Mathematical Principles of Reinforcement. The "specific activation" parameter, a, which is presumed to reflect the incentive value of the reinforcer, was a monotonically increasing function of reinforcer volume; the "response time" parameter, delta, which defines the minimum response time, increased as a function of reinforcer volume; the "currency" parameter, beta, which is presumed to reflect the coupling of responses to the reinforcer, declined as a function of volume. Running response rate (response rate calculated after exclusion of the postreinforcement pause) decayed monotonically as a function of ratio size; the index of curvature of this function increased as a function of reinforcer volume. Postreinforcement pause increased as a function of ratio size. Estimates of a derived from overall response rates and postreinforcement pauses showed a modest positive correlation across conditions and between animals. Implications of the results for the quantification of reinforcer value and for the use of progressive-ratio schedules in behavioral neuroscience are discussed.

Effect of quinolinic acid-induced lesions of the subthalamic nucleus on performance on a progressive-ratio schedule of reinforcement: a quantitative analysis

The subthalamic nucleus (STN), a major relay in the indirect striatofugal pathway, plays an important role in extrapyramidal motor control. Recent evidence indicates that it may also be involved in regulating the incentive value of food reinforcers.

OBJECTIVE

To examine the effect of lesions of the STN on performance on a progressive-ratio schedule using a quantitative model that dissociates effects of interventions on motor and motivational processes [Killeen PR. Mathematical principles of reinforcement. Behav Brain Sci 1994;17:105-72]. Rats with bilateral quinolinic acid-induced lesions of the STN (n=14) or sham lesions (n=14) were trained to press a lever for food-pellet reinforcers under a progressive-ratio schedule. In Phase 1 (90 sessions) the reinforcer was one pellet; in Phase 2 (30 sessions) it was two pellets; in Phase 3 (30 sessions) it was again one pellet.

RESULTS

The performance of both groups conformed to the model of progressive-ratio schedule performance. The motor parameter, delta, was significantly higher in the STN-lesioned than the sham-lesioned group, reflecting lower overall response rates in the lesioned group. The motivational parameter, a, was significantly higher in the STN-lesioned group than in the sham-lesioned group, consistent with enhanced reinforcer value in the STN-lesioned group compared to the sham-lesioned group. In both groups, a was sensitive to changes in reinforcer size, being significantly greater under the two-pellet condition (Phase 2) than under the one-pellet condition (Phases 1 and 3). The results suggest that destruction of the STN impairs response capacity and enhances the incentive value of food reinforcers.