Quasi-reinforcement: control of responding by a percentage-reinforcement schedule

Topographical aspects of brief-stimulus presentations: A re-examination of the problem of conditioned reinforcement

Three pigeons were exposed to second-order schedules in which responding under a fixed-interval (FI) component schedule was reinforced according to a variable-interval (VI) schedule of food reinforcement. Completion of each component resulted in either (1) brief presentation of a stimulus present during reinforcement (paired brief stimulus), (2) brief presentation of a stimulus not present during reinforcement (nonpaired brief stimulus), or (3) no stimulus presentation (tandem schedule). Under the two nonpaired brief stimulus conditions, either a change in keylight color or onset of houselight illumination was used as the brief stimulus. Similar patterns of keypecking occurred under tandem and nonpaired keylight brief-stimulus presentations, whereas nonpaired houselight brief-stimulus presentations generated positively accelerated within-component keypeck patterning for two pigeons. When the same keylight brief stimulus was paired with food, positively accelerated patterns of keypecking were obtained for all pigeons. Differences in the effects of nonpaired brief-stimulus presentations on second-order schedule performance suggest that component schedule patterning under nonpaired brief-stimulus procedures is a function of the particular type of stimulus used (i.e., houselight versus keylight). These results suggest that (1) brief houselight illumination may function as a sensory reinforcer, and (2) a briefly presented food-paired stimulus can function as an effective conditioned reinforcer.

Task-Dependent Effects of SKF83959 on Operant Behaviors Associated With Distinct Changes of CaMKII Signaling in Striatal Subareas

BACKGROUND

SKF83959, an atypical dopamine (DA) D1 receptor agonist, has been used to test the functions of DA-related receptor complexes in vitro, but little is known about its impact on conditioned behavior. The present study examined the effects of SKF83959 on operant behaviors and assayed the neurochemical mechanisms involved.

METHODS

Male rats were trained and maintained on either a fixed-interval 30-second (FI30) schedule or a differential reinforcement of low-rate response 10-second (DRL10) schedule of reinforcement. After drug treatment tests, western blotting assayed the protein expressions of the calcium-/calmodulin-dependent protein kinase II (CaMKII) and the transcription factor cyclic AMP response element binding protein (CREB) in tissues collected from 4 selected DA-related areas.

RESULTS

SKF83959 disrupted the performance of FI30 and DRL10 behaviors in a dose-dependent manner by reducing the total number of responses in varying magnitudes. Moreover, the distinct profiles of the behavior altered by the drug were manifested by analyzing qualitative and quantitative measures on both tasks. Western-blot results showed that phospho-CaMKII levels decreased in the nucleus accumbens and the dorsal striatum of the drug-treated FI30 and DRL10 subjects, respectively, compared with their vehicle controls. The phospho-CREB levels decreased in the nucleus accumbens and the hippocampus of drug-treated FI30 subjects but increased in the nucleus accumbens of drug-treated DRL10 subjects.

CONCLUSIONS

Our results provide important insight into the neuropsychopharmacology of SKF83959, indicating that the drug-altered operant behavior is task dependent and related to regional-dependent changes of CaMKII-CREB signaling in the mesocorticolimbic DA systems.

Behavioural Contrast in Discrete-Trial Discriminations: Effects of Non-Reinforcement

Six pigeons performed an acquisition and three reversals of a discrete trial simultaneous discrimination in which responses to the positive key were reinforced on a fixed-interval 5-s schedule; trials were terminated by either a response to the negative key or the delivery of a reinforcement. In the initial stage of acquisition and each reversal, where errors were most frequent, response rates rose and latencies fell on positive trials; these effects declined as errors decreased. The birds were also run in two yoked control conditions, and results showed that the critical requirement for the positive trial rate and latency changes was neither the simultaneous presentation of two stimuli nor the formation of a discrimination, but the occurrence of non-reinforced responses. The similarity of these phenomena to conventional behavioural contrast is emphasized, and the results are discussed in terms of frustration theory.

Brief Stimuli Paired with Food: Comparison of Two Schedule Arrangements

Pecks by pigeons produced food according to a fixed-interval schedule. Pecks also produced a brief blackout according to a small fixed-ratio schedule. Each food delivery was immediately preceded by a brief blackout, but not all blackouts were followed by food. The schedule of food and the schedule of blackouts were combined two ways. On one, a second-order schedule, each fixed-ratio completed during the fixed-interval produced the blackout. The first fixed-ratio completed after the fixed-interval elapsed produced the blackout-food compound. The second, a conjoint schedule, was identical except that the first peck after the fixed-interval elapsed produced the blackout-food compound regardless of the number of responses since the last blackout. Although the blackout was paired with food and was produced on a small fixed-ratio schedule under both arrangements, there was evidence of fixed-ratio-like response patterns only on the second-order schedule.

Resistance to change and frequency of response-dependent stimuli uncorrelated with reinforcement

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response-dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable-interval 60-s schedules of food reinforcement in two components of a multiple schedule, and brief response-dependent keylight-color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable-interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.

Second-order schedules: comparison of different procedures for scheduling paired and nonpaired brief stimuli

Pigeons performed on a second-order schedule in which fixed-interval components were maintained under a variable-interval schedule. Completion of each fixed-interval component resulted in a brief-stimulus presentation and/or food. The relation of the brief stimulus and food was varied across conditions. Under some conditions, the brief stimulus was never paired with food. Under other conditions, the brief stimulus was paired with food; three different pairing procedures were used: (a) a response produced the simultaneous onset of the stimulus and food; (b) a response produced the stimulus before food with the stimulus remaining on during food presentation; (c) a response produced the stimulus and the offset of the stimulus was simultaneous with the onset of the food cycle. The various pairing and nonpairing operations all produced similar effects on performance. Under all conditions, response rates were positively accelerated within fixed-interval components. Total response rates and Index of Curvature measures were similar across conditions. In one condition, a blackout was paired with food; with this different stimulus in effect, less curvature resulted. The results suggest that pairing of a stimulus is not a necessary condition for within-component patterning under some second-order schedules.

Schedule-induced drinking as a function of percentage reinforcement

Drinking was recorded in rats while lever pressing was maintained on a series of percentage reinforcement schedules in which the per cent of 1-min fixed intervals terminating with food was 100, 90, 30, 70, 10, 50, and 100%. Intervals in which a pellet was omitted were terminated by brief light flash and click stimuli that were also correlated with food presentations. Drinking failed to develop in five of six subjects following intervals in which the brief stimuli were presented regardless of percentage reinforcement. Postpellet drinking, which followed intervals terminated with pellet delivery, however, increased in both duration and amount ingested per interval as percentage reinforcement was systematically decreased. The increase in postpellet drinking above that produced by 100% reinforcement was interpreted as an analogue of the positive-contrast effect observed with food-reinforced operants.

Signalled and unsignalled percentage reinforcement of performance under a chained schedule

Pigeons were trained to peck a key under a chained fixed-ratio 15 fixed-interval 25-sec schedule of food presentation. In Experiment 1, blocks of sessions in which 100%, 75%, 50%, and 25% of the sequences ended with food presentation were conducted. When food presentation was omitted, a timeout of equal duration replaced it. As the frequency of food presentation decreased so did the frequency of completing the chained schedule. In Experiment 2, 75%, 50%, or 25% of the sequences terminated with food presentation and outcomes were signalled, i.e., completion of the fixed ratio resulted in either a stimulus correlated with the fixed-interval 25-sec schedule or a stimulus correlated with extinction. As the frequency of food presentation decreased, the number of sequences completed per session increased for two pigeons and remained high for a third. In Experiments 3 and 4, assessments of the effects of signalling the outcome of the chained schedule were made with response-independent presentation of events at the end of the sequence. Again, signalling the outcome of the chained schedule led to more chains being completed per session than did not signalling the outcome. Stimuli differentially paired with food presentation have powerful behavioral effects that may be attributed to the potency of these stimuli as conditioned reinforcers.

Delayed reinforcement versus reinforcement after a fixed interval

When interreinforcement intervals were equated, pigeons demonstrated little or no preference between reinforcement after a delay interval and reinforcement presented on a fixed-interval schedule. The small preferences sometimes found for the fixed interval (a) were considerably smaller than when the delay and fixed intervals differed in duration, and (b) were caused by the absence of light during the delay. These results suggest that the effects of delayed reinforcement on prior responding can be reproduced by imposing a temporally equal fixed-interval schedule in place of the delay; and, therefore, that the time between a response and reinforcement controls the probability of that response, whether other responses intervene or not.

Temporal control of periodic schedules: signal properties of reinforcement and blackout

Pigeons were exposed to periodic food-reinforcement schedules in which intervals ended with equal probability in either reinforcement or brief blackout. The effects on the pattern of key pecking of sequential probability of reinforcement, interval duration, and time to reinforcement opportunity were investigated in three experiments. The major results were: (1) at short absolute interval durations, time to reinforcement opportunity determined both postreinforcement and postblackout pause (time to first key peck within an interval); (2) at long intervals, postblackout pause was consistently shorter than postreinforcement pause, even if both events signalled the same time to the next reinforcement opportunity (omission effect); (3) when reinforcement and blackout signalled different times to the next reinforcement opportunity, within the same experiment, there was some evidence for interactions analogous to behavioral contrast.

Contiguity of briefly presented stimuli with food reinforcement

Pigeons performed on second-order schedules of reinforcement consisting of four fixed-interval components. Only the terminal component ended with food. Performance was studied both when a brief stimulus followed the completion of each of the first three fixed intervals (brief-stimulus schedule) and when the stimulus was omitted (tandem schedule). Variations in the temporal contiguity of the last presentation of the stimulus and the presentation of food indicated that the shorter the delay, the greater was the enhancement of rate of responding in comparison with tandem performance. A positively accelerated pattern of responding within fixed-interval components was a function of the contiguity of the brief stimulus and reinforcement; this pattern was absent for all tandem-schedule performance.

Fixed-interval behavior: effects of percentage reinforcement

The percentage of fixed intervals terminating with food presentation was varied parametrically. Intervals that did not end with food were terminated by a stimulus uncorrelated with food presentation (a timeout stimulus). In Experiment I, the pigeons' response rates were an inverted U-shaped function of the percentage of food presentations: decreasing the percentage from 100% to 90%, 70%, or 50% produced an increase in response rates; lower percentages decreased the rates. The patterns of responding in the 100% condition differed from those of the other conditions. In Experiment II, the chamber was darkened after food presentations and timeouts. Response rate was directly related to the percentage of food presentations: decreasing the percentage decreased the response rate. Characteristic fixed-interval patterns of responding were maintained as long as there were occasional food presentations; pausing followed by positively-accelerated responding occurred in percentage conditions ranging from 7% to 100%. The ability to maintain fixed-interval performance with percentage reinforcement suggested that the behavioral sequences occurring in each interval may operate as unitary responses.

Discriminative functions of schedule stimuli and memory: a combination of schedule and choice procedures

Pigeons responded under a combination brief-stimulus schedule and choice procedure. Normally, a fixed-interval schedule was in effect, where completion randomly produced either a brief stimulus or food. Intermittently, this schedule was interrupted by a choice arrangement. Two choice keys were lit, either a short or a long time since a prior event (food or stimulus). One choice response produced food if the time had been short, and the alternate response produced food if the time had been long. Across conditions, the duration of the fixed-interval schedule was varied, the stimuli that comprised the brief-stimulus operation were changed, and the stimuli were presented as paired and nonpaired with food. The focus of the study was the control of both schedule performance and choice responding across conditions. The results showed that choice accuracy was correlated with the degree of fixed-interval curvature, the response pattern of a pause followed by a gradually accelerated rate. As fixed-interval schedule duration was increased, both the degree of fixed-interval curvature and choice accuracy decreased. The particular brief stimulus used affected schedule and choice performance, with a more salient stimulus producing a greater degree of curvature and higher accuracy. Pairing and nonpairing operations produced striking differences in performance with the less salient brief stimulus, but not with the more salient stimulus. The results suggest that brief-stimulus schedule performance may be conceptualized in the context of memory research.

Second-order schedules: brief shock at the completion of each component

Pigeons worked on second-order schedules in which completion of fixed-interval component schedules was reinforced with food according to a variable-interval schedule of reinforcement. The completion of each fixed-interval component resulted in the presentation of a brief electric shock. In one condition (shock-paired), the completion of every fixed-interval component, including those that ended in food, resulted in the shock. In another condition (shock-nonpaired), completion resulted in shock except for those components that ended in food. Shock presentations resulted in a positively accelerated rate within fixed-interval components. This patterning within components was similar whether the shock was intermittently paired with food or not. Response rates tended to decrease as shock intensity increased. The characteristic fixed-interval response pattern within components did not occur when shock presentations were omitted at the end of each component (tandem schedule). When shocks were scheduled but food was no longer presented (extinction) response rates declined to a near-zero level. The performance under shock conditions is similar to that in other studies in which visual and auditory stimuli are presented at the completion of component schedules.

Inverse relation between choice and local response rate with a schedule of response-produced blackouts

Pigeons were exposed to a two-key concurrent chains schedule in which identical frequencies and distributions of food presentations generated different response rates in the terminal links. An inverse relation between local rate of response in the terminal links and relative frequency of response in the initial links was observed. The high response rate was produced in one terminal link by a second-order schedule in which responding produced brief blackouts of the response key. Responding under the same schedule in the other terminal link did not produce blackouts. Under initial training and after spatial reversal of the terminal-link schedules, two of three pigeons had lower relative frequencies of response in the initial member of the chain with the higher terminal link response rate. The third pigeon showed no change in preference at reversal.

Discriminative properties of briefly presented stimuli

In Experiment I, pigeons' responses produced food according to a fixed-interval schedule while responses on the key also produced brief stimuli according to a variable-interval schedule. Each brief stimulus reset the fixed interval. Thus, a brief stimulus occurred irregularly but a fixed minimum time separated the occurrence of food from a brief stimulus. Pauses followed brief stimuli and were followed by an accelerated response rate until another brief stimulus or food occurred. In Experiment II, four control procedures were examined. (1) Brief-stimulus presentations were omitted, producing a loss of response patterning. (2) A second-order schedule was studied with fixed-interval components. This schedule produced patterning following brief stimuli similar in kind and degree to that found in Experiment I. (3) A conjoint schedule was arranged in which food was no longer separated from the stimulus by a fixed time; pauses following the stimulus no longer resulted. (4) A brief food reinforcer replaced the brief visual stimulus, resulting in a constant response rate with no pausing following the brief food stimulus. The results suggest that the brief-stimulus effects were due to discriminative functions produced by the fixed temporal relation separating the stimulus from food.

Choice, changeover, and travel

Since foraging in nature can be viewed as instrumental behavior, choice between sources of food, known as "patches," can be viewed as choice between instrumental response alternatives. Whereas the travel required to change alternatives deters changeover in nature, the changeover delay (COD) usually deters changeover in the laboratory. In this experiment, pigeons were exposed to laboratory choice situations, concurrent variable-interval schedules, that were standard except for the introduction of a travel requirement for changeover. As the travel requirement increased, rate of changeover decreased and preference for a favored alternative strengthened. When the travel requirement was small, the relations between choice and relative reinforcement revealed the usual tendencies toward matching and undermatching. When the travel requirement was large, strong overmatching occurred. These results, together with those from experiments in which changeover was deterred by punishment or a fixed-ratio requirement, deviate from the matching law, even when a correction is made for cost of changeover. If one accepted an argument that the COD is analogous to travel, the results suggest that the norm in choice relations would be overmatching. This overmatching, however, might only be the sign of an underlying strategy approximating optimization.

Effect of reinforcement duration on fixed-interval responding

Five different reinforcement durations occurred randomly within each session on fixed interval 60-sec. Postreinforcement pause was directly related (and "running" rate inversely related) to the duration of reinforcement initiating each fixed interval.

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.

Schedules of response-independent conditioned reinforcement

Rates and patterns of responding of pigeons under response-independent and response-dependent schedules of brief-stimulus presentation were compared by superimposing 3-min brief-stimulus schedules on a 15-min fixed-interval schedule of food presentation. The brief-stimulus schedules were fixed time, fixed interval, variable time, and variable interval. When the brief stimulus was paired with food presentation, its effects depended upon the schedule and ongoing rates. Fixed- and variable-interval brief-stimulus schedules enhanced the low rates normally occurring early in the 15-min interval, whereas fixed- and variable-time schedules suppressed these rates. Although the overall rates later in the interval were not affected to any great extent, the fixed brief-stimulus schedules generated patterns of positively accelerated responding between stimulus presentations. These patterns appeared less frequently under the variable brief-stimulus schedules. Initially, when not paired with food delivery, presentations of the brief stimulus produced relatively little effect on either response rate or patterning. However, once the stimulus had accompanied food presentation, the original performance under the nonpaired condition was not recovered. The effects were more like those occurring when the stimulus was paired with food.

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.

Effects of component length and of the transitions among components in multiple schedules

Pigeons received equal variable-interval reinforcement during presentations of two line-orientation stimuli while five other orientations appeared in extinction. Component duration was 30 seconds for all orientations and the sequence was arranged so that each orientation preceded itself and each other orientation equally often. The duration of one component (0 degrees ) was shortened to 10 seconds and the other (90 degrees ) was lengthened to 50 seconds. All animals showed large increases in response rate in the shortened component and this increase was recoverable after an interpolated condition in which all components were again 30 seconds in duration. This effect was replicated in a second experiment in which component duration was changed from 150 seconds to 50 seconds and 250 seconds. An examination of local contrast effects during the first experiment showed that the shortened component produced local contrast during subsequent presentations of the lengthened component, just as would a component associated with more frequent reinforcement. When the presentation sequence was changed so that the lengthened component was always followed by the shortened component, response rates generally increased during the lengthened component. When the sequence was arranged so that the shortened component always preceded the longer component, response rate decreased in the former. These effects, as well as the increases in response rate following change in component length, seem not to be the product of local contrast effects among components.

Mathematical principles of reinforcement

Effective conditioning requires a correlation between the experimenter's definition of a response and an organism's, but an animal's perception of its behavior differs from ours. These experiments explore various definitions of the response, using the slopes of learning curves to infer which comes closest to the organism's definition. The resulting exponentially weighted moving average provides a model of memory that is used to ground a quantitative theory of reinforcement. The theory assumes that: incentives excite behavior and focus the excitement on responses that are contemporaneous in memory. The correlation between the organism's memory and the behavior measured by the experimenter is given by coupling coefficients, which are derived for various schedules of reinforcement. The coupling coefficients for simple schedules may be concatenated to predict the effects of complex schedules. The coefficients are inserted into a generic model of arousal and temporal constraint to predict response rates under any scheduling arrangement. The theory posits a response-indexed decay of memory, not a time-indexed one. It requires that incentives displace memory for the responses that occur before them, and may truncate the representation of the response that brings them about. As a contiguity-weighted correlation model, it bridges opposing views of the reinforcement process. By placing the short-term memory of behavior in so central a role, it provides a behavioral account of a key cognitive process.

Conditional reinforcers and informative stimuli in a constant environment

Five pigeons responded on steady-state concurrent variable-interval variable-interval schedules of food presentation in which half of the foods were removed and replaced with nonfood stimuli. Across conditions, the stimuli were either paired or unpaired with food, and the correlation between the ratio of food deliveries on the two alternatives and the ratio of nonfood stimuli was either -1, 0, or +1. Neither the pairing of stimuli with food, nor the correlation between stimuli and food, affected generalized-matching performance, but paired stimuli had a demonstrable effect at a local level of analysis. This effect was independent of the food-stimulus correlation. These results differ from results previously obtained in a frequently changing environment. We attribute this difference in results to differences in the information value of response-contingent stimuli in frequently changing versus relatively constant environments, as well as to differences between forward pairing and simultaneous pairing of the stimuli with food.

A comparison of responses and stimuli as time markers

A rat's behavior, as well as a stimulus, may be a time marker. But do they lead to similar performance? Eight rats were trained on a 20-s DRL procedure in which head-entry responses were time markers, i.e., each head-entry response indicated that food would not be delivered for 20s. Concurrently, eight rats were trained on a control procedure in which light stimuli, yoked to the responses of a rat in the DRL procedure, were time markers, i.e., each light stimulus indicated that food would not be delivered for 20s. A comparison of performance between the two groups showed a lower response rate in the DRL procedure than in the yoked control procedure. However, similar response patterns between the two groups were observed, suggesting that rats anticipated the food similarly with a stimulus or a response as the time marker.

Do conditional reinforcers count?

Six pigeons were trained on a procedure in which seven components arranged different food-delivery ratios on concurrent variable-interval schedules each session. The components were unsignaled, lasted for 10 food deliveries, and occurred in random order with a 60-s blackout between components. The schedules were arranged using a switching-key procedure in which two responses on a center key changed the schedules and associated stimuli on two side keys. In Experiment 1, over five conditions, an increasing proportion of food deliveries accompanied by a magazine light was replaced with the presentation of the magazine light only. Local analyses of preference showed preference pulses toward the alternative that had just produced either a food-plus-magazine-light or magazine-light-only presentation, but pulses after food deliveries were always greater than those after magazine lights. Increasing proportions of magazine lights did not change the size of preference pulses after food or magazine-light presentations. Experiment 2 investigated the effects of correlations between food ratios and magazine-light ratios: In Condition 6, magazine-light ratios in components were inversely correlated (-1.0) with food ratios, and in Condition 7, magazine-light ratios were uncorrelated with food ratios. In Conditions 8 and 9, pecks also produced occasional 2.5-s flashes of a green keylight. In Condition 8, food and magazine-light ratios were correlated 1.0 whereas food and green-key ratios were correlated -1.0. In Condition 9, food and green-key ratios were correlated 1.0 whereas food and magazine-light ratios were correlated -1.0. Preference pulses toward alternatives after magazine lights and green keys depended on the correlation between these event ratios and the food ratios: If the ratios were correlated +1.0, positive preference pulses resulted; if the correlation was -1.0, preference pulses were negative. These results suggest that the Law of Effect has more to do with events signaling consequences than with strengthening responses.

Within-subject testing of the signaled-reinforcement effect on operant responding as measured by response rate and resistance to change

Response rates under random-interval schedules are lower when a brief (500 ms) signal accompanies reinforcement than when there is no signal. The present study examined this signaled-reinforcement effect and its relation to resistance to change. In Experiment 1, rats responded on a multiple random-interval 60-s random-interval 60-s schedule, with signaled reinforcement in only one component. Response resistance to alternative reinforcement, prefeeding, and extinction was compared between these components. Lower response rates, and greater resistance to change, occurred in the component with the reinforcement signal. In Experiment 2, response rates and resistance to change were compared after training on a multiple random-interval 60-s random-interval 60-s schedule in which reinforcer delivery was unsignaled in one component and a response-produced uncorrelated stimulus was presented in the other component. Higher response rates and greater resistance to change occurred with the uncorrelated stimulus. These results highlight the significance of considering the effects of an uncorrelated signal when used as a control condition, and challenge accounts of resistance to change that depend solely on reinforcer rate.

Brief-stimulus presentations on multiform tandem schedules

Three experiments examined the influence of a brief stimulus (a light) on the behavior of food-deprived rats whose lever pressing on tandem schedules comprising components of different schedule types resulted in food presentation. In Experiment 1, either a tandem variable-ratio variable-interval or a tandem variable-interval variable-ratio schedule was used. The variable-interval requirement in the tandem variable-ratio variable-interval schedule was yoked to the time taken to complete the variable-ratio component in the tandem variable-interval variable-ratio schedule, and the length of the variable-interval component in the latter schedule was yoked to the variable-ratio component in the former schedule. If a brief stimulus occurred following completion of the first component, then behavior was differentiated in the two components; subjects responded more quickly in the variable-ratio than in the variable-interval component. If the stimulus was removed, then response rate was determined by the nature of the final component. Similar results were obtained in Experiments 2 and 3 with the use of a three-component tandem variable-ratio variable-interval variable-ratio schedule or tandem variable-interval variable-ratio variable-interval schedule. Thus, a brief stimulus that was not explicitly paired with reinforcement engendered behavior typical of the component schedule preceding its presentation.