BEHAVIOR STABILITY AND RESPONSE RATE AS FUNCTIONS OF REINFORCEMENT PROBABILITY ON "RANDOM RATIO" SCHEDULES

Observing responses in pigeons: effects of schedule component duration and schedule value

Pigeons were exposed to a procedure under which five pecks on one response key (the observing key) changed the schedule on a second key (the food key) from a mixed schedule to a multiple schedule for 25 sec. In Experiment I, a random-ratio 50 schedule alternated with extinction. The duration of the random-ratio 50 schedule component was varied between 1.25 and 320 sec, and extinction was scheduled for a varying time, ranging from the duration of the random-ratio 50 to four times that value. Each set of values was scheduled for a block of sessions. Before observing-key pecks were allowed at each set of parameter values, the pigeons were exposed to a condition where the mixed and multiple schedule alternated every 10 min, and observing-key pecks were not permitted. Rates of pecking on the observing key were high for all values of random-ratio component durations except 1.25 sec. Experiment II was conducted with the random-ratio component duration equal to 40 sec, and the random-ratio schedule was varied from random-ratio 50 to 100, 200, and 400. Observing-key pecking rates were high for all values of the random-ratio schedule except random-ratio 400. In both experiments, observing response rates were relatively little affected, suggesting that neither schedule component duration nor schedule value is a strong determinant of observing responses.

Two unlike patterns of random-ratio responding associated with different eating habits in rhesus monkeys

Four rhesus monkeys were exposed to an identical series of schedules that specified a uniform probability of reinforcement for every response. As probability was lowered slowly in 10 steps of 20 sessions each from 1.0 through 0.01, two distinct patterns of responding emerged. Two subjects showed high, pause-free response rates that increased with each successive reduction in reinforcement probability. The other two showed consistent post-reinforcement pausing at all probabilities, including 1.0, and substantially lower response rates that peaked at the moderate probability values of 0.04 and 0.03. This low-rate pattern was found to be correlated with a pre-experimental preference in the two subjects for mouthing and chewing food pellets one at a time, while the former high-rate, pause-free pattern was linked to a long-standing habit of "pouch feeding" in the other monkeys. These idiosyncratic collateral behaviors that differentiated the schedule performances appeared neither superstitious in origin, nor useful in the case of the low-rate monkeys.

Toward an Animal Model of Gambling: Delay Discounting and the Allure of Unpredictable Outcomes

Laboratory investigations of gambling are sometimes criticized as lacking ecological validity because the stakes wagered by human subjects are not real or no real monetary losses are experienced. These problems may be partially addressed by studying gambling in laboratory animals. Toward this end, data are summarized which demonstrate that laboratory animals will work substantially harder and prefer to work under gambling-like schedules of reinforcement in which the number of responses per win is unpredictable. These findings are consistent with a delay discounting model of gambling which holds that rewards obtained following unpredictable delays are more valuable than rewards obtained following predictable delays. According to the delay discounting model, individuals that discount delayed rewards at a high rate (like pathological gamblers) perceive unpredictably delayed rewards to be of substantially greater value than predictable rewards. The reviewed findings and empirical model support the utility of studying animal behavior as an ecologically valid first-approximation of human gambling.

Effects of reinforcement rate and delay on the acquisition of lever pressing by rats

The acquisition of lever pressing by naive rats, in the absence of shaping, was studied as a function of different rates and unsignaled delays of reinforcement. Groups of 3 rats were each exposed to tandem schedules that differed in either the first or the second component. First-component schedules were either continuous reinforcement or random-interval 15, 30, 60 or 120 s; second-component schedules were fixed-time 0, 1, 3, 6, 12, or 24 s. Rate of responding was low under continuous immediate reinforcement and higher under random-interval 15 s. Random interval 30-s and 60-s schedules produced lower rates that were similar to each other. Random-interval 120 s controlled the lowest rate in the immediate-reinforcement condition. Adding a constant 12-s delay to each of the first-component schedule parameters controlled lower response rates that did not vary systematically with reinforcement rate. The continuous and random-interval 60-s schedules of immediate reinforcement controlled higher global and first-component response rates than did the same schedules combined with longer delays, and first-component rates showed some graded effects of delay duration. In addition, the same schedules controlled higher second-component response rates in combination with a 1-s delay than in combination with longer delays. These results were related to those from previous studies on acquisition with delayed reinforcement as well as to those from similar reinforcement procedures used during steady-state responding.

Control rate of response or reinforcement and amphetamine's effect on behavior

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.

Contingency spaces and measures in classical and instrumental conditioning

The contingency between conditional and unconditional stimuli in classical conditioning paradigms, and between responses and consequences in instrumental conditioning paradigms, is analyzed. The results are represented in two- and three-dimensional spaces in which points correspond to procedures, or procedures and outcomes. Traditional statistical and psychological measures of association are applied to data in classical conditioning. Root mean square contingency, Ø, is proposed as a measure of contingency characterizing classical conditioning effects at asymptote. In instrumental training procedures, traditional measures of association are inappropriate, since one degree of freedom-response probability-is yielded to the subject. Further analysis of instrumental contingencies yields a surprising result. The well established "Matching Law" in free-operant concurrent schedules subsumes the "Probability Matching" finding of mathematical learning theory, and both are equivalent to zero contingency between responses and consequences.