Temporal control on interval schedules: what determines the postreinforcement pause?

Dynamics of temporal control in rats: the effects of a brief transition in interval duration

Five rats lever-pressed for liquid reinforcers delivered according to a fixed-interval (FI) reinforcement schedule, where the interval requirement changed at an unpredictable point within a session. In a short square wave (SSW) condition, eight 30-s intervals were intercalated in a series of 120-s intervals so that the intervals changed from 120 to 30 s then back to 120 s. In a long square wave (LSW) condition the intervals changed from 120 to 480 s then back to 120 s. We observed rapid temporal control of post-reinforcement wait time duration by the IFI duration in the SSW condition only: Wait times decreased significantly during a transition to shorter (30 s) intervals; whereas wait times did not reliably increase during a transition to longer (480 s) intervals. Furthermore, in the SSW condition, wait time in post-transition intervals was shorter than that observed during pre-transition intervals. The results show that rats' wait times are sensitive to moment-to-moment changes in interval duration and that the dynamics depend on the direction in which the intervals change.

Effects of a signaled delay to reinforcement in the previous and upcoming ratios on between-ratio pausing in fixed-ratio schedules

Domestic hens responded under multiple fixed-ratio fixed-ratio schedules with equal fixed ratios. One component provided immediate reinforcement and the other provided reinforcement after a delay, signaled by the offset of the key light. The components were presented quasirandomly so that all four possible transitions occurred in each session. The delay was varied over 0, 4, 8, 16, and 32 s with fixed-ratio 5 schedules, and over 0, 8 and 32 s with fixed-ratio 1, 15 and 40 schedules. Main effects of fixed-ratio value and delay duration were detected on between-ratio pauses. Pauses were longer when the multiple-schedule stimulus correlated with a delayed-reinforcer component was presented, with the longest pauses occurring at the transition from a component with an immediate reinforcer to one with a delayed reinforcer. Pause durations were shortest during immediate components. Overall, both the presence or absence of a delay in the upcoming component, and the presence or absence of a delay in the preceding component affected pause length, but the upcoming delay had the larger effect. Thus changes in delay had similar effects to past reports of the effects of changes in response force, response requirement, and reinforcer magnitude in multiple fixed-ratio fixed-ratio schedules.

Dynamics of waiting in pigeons

Two experiments used response-initiated delay schedules to test the idea that when food reinforcement is available at regular intervals, the time an animal waits before its first operant response (waiting time) is proportional to the immediately preceding interfood interval (linear waiting; Wynne & Staddon, 1988). In Experiment 1 the interfood intervals varied from cycle to cycle according to one of four sinusoidal sequences with different amounts of added noise. Waiting times tracked the input cycle in a way which showed that they were affected by interfood intervals earlier than the immediately preceding one. In Experiment 2 different patterns of long and short interfood intervals were presented, and the results implied that waiting times are disproportionately influenced by the shortest of recent interfood intervals. A model based on this idea is shown to account for a wide range of results on the dynamics of timing behavior.

Cognitive impulsivity in animal models: role of response time and reinforcing rate in delay intolerance with two-choice operant tasks

Impulsivity, a key symptom of ADHD (attention-deficit hyperactivity disorder), is also common in obsessive-compulsive and addictive disorders. There is rising interest in animal models of inhibitory-control impairment. Adolescent rats were tested daily in the intolerance-to-delay (ID) task (session 25 min, timeout 20 s), involving choice between either immediate small amount of food (SS), or larger amount of food after a delay (LL). The mixed 5-HT(1A/7) agonist (8-OH-DPAT, 0 or 0.060 mg/kg i.p.) was administered acutely just before the last three sessions at highest delays. In addition to the classical choice parameter (percent LL preference), the spontaneous waiting (termed response time, RT) occurring between end of a timeout (TO) and next nose-poke was calculated. The pace between consecutive reinforcer deliveries is given by the mean inter-trial interval (mITI, i.e. TO + RT). Hence, the impact of any given delay may be proportional to this pace and be expressed as delay-equivalent odds, i.e. the extent by which delays are multiples of the mITI. Data revealed that RT/mITI increased sharply from around 15 s/35 s to around 30 s/50 s when imposed delay changed from 30 s to 45 s (i.e. odds from 0.91 to 1.06). This suggests that rats adopted a strategy allowing them to keep in pace with perceived reinforcing rate. The increasing delay constraint directly influenced the length of rats' spontaneous waiting (RT) before next decision. For higher delays, with odds >1, rats shifted to a clear-cut SS preference, which is devoid of any exogenous temporal constraint. A challenge with 8-OH-DPAT (0 or 0.060 mg/kg i.p.) decreased impulsive choice but also increased RT. Thus, tapping onto 5-HT(1A/7) receptors slightly enhanced RT/mITI values, possibly reflecting ability of rats to cope with slower reinforcing rates and/or with delay-cancelled reward paces. In summary, delay-induced states of aversion may arise from the innate tendency to rely on a regular rate of reinforcement. Conversely, a drug-enhanced capacity to cope with delay may involve an internal ability to adjust expectancy about such a reinforcing rate.

Operant conditioning

Operant behavior is behavior "controlled" by its consequences. In practice, operant conditioning is the study of reversible behavior maintained by reinforcement schedules. We review empirical studies and theoretical approaches to two large classes of operant behavior: interval timing and choice. We discuss cognitive versus behavioral approaches to timing, the "gap" experiment and its implications, proportional timing and Weber's law, temporal dynamics and linear waiting, and the problem of simple chain-interval schedules. We review the long history of research on operant choice: the matching law, its extensions and problems, concurrent chain schedules, and self-control. We point out how linear waiting may be involved in timing, choice, and reinforcement schedules generally. There are prospects for a unified approach to all these areas.

Nonlinear time perception

Sensitivity to time was investigated to test the linear-timing hypothesis. A long duration was adjusted until accuracy was 75% correct for a short duration in a two-choice procedure. Short durations (2, 4, 6, 8, 10, 12, 14, 16 and 18 s) were selected from previous research that suggests that sensitivity to time is nonlinear in this range. Rats were tested with a single short interval (Experiment 1, n=13) or a random order (Experiment 2, n=7). A local maximum in sensitivity (d' from signal detection theory) was observed at approximately 8-12 s. Sensitivity to time was reliably correlated (r's=0.759-0.941) with previous data. Weber fractions exhibited a U-shape and were negatively correlated with sensitivity to time (r=-0.800). These results provide additional evidence that sensitivity to time is nonlinearly related to physical time.

Models of ratio schedule performance

Predictions of P. R. Killeen's (1994) mathematical principles of reinforcement were tested for responding on ratio reinforcement schedules. The type of response key, the number of sessions per condition, and first vs. second half of a session had negligible effects on responding. Longer reinforcer durations and larger grain types engendered more responding, affecting primarily the parameter alpha (specific activation). Key pecking was faster than treadle pressing, affecting primarily the parameter delta (response time). Longer intertrial intervals led to higher overall response rates and shorter postreinforcement pauses and higher running rates, and ruled out some competing explanations. The treadle data required a distinction between the energetic requirements and rate-limiting properties of extended responses. The theory was extended to predict pause durations and run rates on ratio schedules.

Models of ratio schedule performance

Predictions of P. R. Killeen's (1994) mathematical principles of reinforcement were tested for responding on ratio reinforcement schedules. The type of response key, the number of sessions per condition, and first vs. second half of a session had negligible effects on responding. Longer reinforcer durations and larger grain types engendered more responding, affecting primarily the parameter alpha (specific activation). Key pecking was faster than treadle pressing, affecting primarily the parameter delta (response time). Longer intertrial intervals led to higher overall response rates and shorter postreinforcement pauses and higher running rates, and ruled out some competing explanations. The treadle data required a distinction between the energetic requirements and rate-limiting properties of extended responses. The theory was extended to predict pause durations and run rates on ratio schedules.