Mechanisms of within-session sequential behavior in pigeons

Correctly and efficiently selecting among options is critical to the organization of behavior across different time scales (minutes, days, seasons). As a result, understanding the mechanisms underlying the sequential behavior of animals has been a long-standing aim. In three experiments, four pigeons were tested in a four-choice simultaneous color discrimination. Across a session, they had to sequentially select a colored stimulus, and the correct color changed over four 24-trial phases (A→B→C→D). After learning this ABCD within-session sequence, tests identified that both timing and outcome feedback mechanisms contributed to the organization of pigeons' behavior. Different representational mechanisms are considered as accounts for the pigeons' observed sequential behavior.

Kea (Nestor notabilis) show flexibility and individuality in within-session reversal learning tasks

The midsession reversal paradigm confronts an animal with a two-choice discrimination task where the reward contingencies are reversed at the midpoint of the session. Species react to the reversal with either win-stay/lose-shift, using local information of reinforcement, or reversal estimation, using global information, e.g. time, to estimate the point of reversal. Besides pigeons, only mammalian species were tested in this paradigm so far and analyses were conducted on pooled data, not considering possible individually different responses. We tested twelve kea parrots with a 40-trial midsession reversal test and additional shifted reversal tests with a variable point of reversal. Birds were tested in two groups on a touchscreen, with the discrimination task having either only visual or additional spatial information. We used Generalized Linear Mixed Models to control for individual differences when analysing the data. Our results demonstrate that kea can use win-stay/lose-shift independently of local information. The predictors group, session, and trial number as well as their interactions had a significant influence on the response. Furthermore, we discovered notable individual differences not only between birds but also between sessions of individual birds, including the ability to quite accurately estimate the reversal position in alternation to win-stay/lose-shift. Our findings of the kea’s quick and flexible responses contribute to the knowledge of diversity in avian cognitive abilities and emphasize the need to consider individuality as well as the limitation of pooling the data when analysing midsession reversal data.

Pigeons' midsession reversal: Greater magnitude of reinforcement on the first half of the session leads to improved accuracy

In the midsession reversal task, pigeons are trained on a simultaneous two-alternative discrimination in which S1 is correct for the first half of the session and S2 is correct for the second half of the session. Optimally, pigeons should choose S1 until it stops being correct and choose S2 afterward. Instead, pigeons anticipate S2 too early and continue choosing S1 even after the reversal. Research suggests that they attempt to time the reversal rather than use the feedback from the preceding response(s). Recently, there is evidence that performance is almost optimized by generating an asymmetry between S1 and S2. For example, pigeons' accuracy improves if correct S1 responses are reinforced 100% of the time, but correct S2 responses are reinforced only 20% of the time. Similarly, accuracy improves if S1 requires one peck but S2 requires 10 pecks. Accuracy does not improve, however, if the value of S1 is less than the value of S2. In the current experiment, we manipulated the magnitude of reinforcement. For the experimental group, correct responses to S1 were reinforced with five pellets of food and correct responses to S2 were reinforced with one pellet. For the control group, all correct responses were reinforced with three pellets. Consistent with the earlier findings, results indicated that there was a significant reduction in anticipatory errors in the experimental group compared with the control, and there was no significant increase in perseverative errors.

The midsession reversal task: A theoretical analysis

In the midsession reversal task, choice of one stimulus (S1) is correct for the first half of each session and choice of the other stimulus (S2) is correct for the last half of each session. Although humans and rats develop very close to what has been called a win-stay/lose-shift response strategy, pigeons do not. Pigeons start choosing S2 before the reversal, making anticipatory errors, and they keep choosing S1 after the reversal, making perseverative errors. Research suggests that the pigeons are timing the reversal from the start of the session. However, making the reversal unpredictable does not discourage the pigeons from timing. Curiously, pigeons' accuracy improves if one decreases the value of the S2 stimulus relative to the S1 stimulus. Another form of asymmetry between S1 and S2 can be found by varying, over trials, the number of S1 or S2 stimuli. Counterintuitively, if the number of S2 stimuli varies, it results in a large increase in anticipatory errors but little increase in perseverative errors. However, if the number of S1 stimuli varies over trials, it results in a large increase in perseverative errors but no increase in anticipatory errors. These last two effects suggest that in the original midsession reversal task, the pigeons are learning to reject S2 during the first half of each session and learning to reject S1 during the last half of each session. These results suggest that reject learning may also play an important role in the learning of simple simultaneous discriminations.

Past outcomes and time flexibly exert joint control over midsession reversal performance in the rat

In a midsession reversal task, subjects choose between two stimuli on every trial; only responses to one stimulus are reinforced. Halfway throughout the session, contingencies are reversed: previously reinforced responses are now extinguished and vice versa. Both, the outcome of the previous trial and the time elapsed since the beginning of the session, may predict the availability of reinforcement and determine choice. Thus, this task has typically been used to study cognitive flexibility and the temporal organization of behavior. This study assessed how past outcomes and time interact for behavioral control when each cue predicts the availability of reinforcement to a different extent. Eight rats were trained in four variations of the midsession reversal task differing in the reliability of outcomes and time as predictors of the reinforced response. We manipulated the reliability of the outcomes by providing either continuous or partial reinforcement, and the reliability of time by fixing the moment of reversal (middle of the session) or making the reversal unpredictable (semi-random trial). Results suggest that behavioral control alternates between outcomes and time according to the relative reliability of each cue. Model simulations show that outcomes and time may jointly determine behavior, and that momentary reinforcement rate may determine their relative influence.

Midsession reversal learning by pigeons: Effect on accuracy of increasing the number of stimuli associated with one of the alternatives

The midsession reversal task involves a simultaneous discrimination in which choice of one stimulus (S1) is correct for the first 40 trials and choice of the other stimulus (S2) is correct for the last 40 trials of each 80-trial session. When pigeons are trained on the midsession reversal task, they appear to use the passage of time from the start of the session as a cue to reverse. As the reversal approaches, they begin to make anticipatory errors, choosing S2 early, and following the reversal they make perseverative errors, continuing to choose S1. Recent research suggests that anticipatory errors can be reduced (while not increasing perseverative errors) by reducing the probability of reinforcement for correct S2 choices from 100% to 20%. A similar effect can be found by increasing the response requirement for choice of S2 from one peck to ten pecks. In the present experiments, we asked if a similar effect could be attained by increasing the number of stimuli that, over trials, could serve as S2. Instead, in both experiments, we found that increasing the number of S2 stimuli actually increased the number of anticipatory errors. Several interpretations of this result are provided, including the possibility that attention to the variable S2 stimuli may have interfered with attention to the S1 stimulus.

Anticipation of a midsession reversal in humans

In a two-stimulus visual discrimination choice task with a reversal in reward contingencies midway through each session, pigeons produce a surprising number of anticipatory errors (i.e., responding to the second-correct stimulus before the reversal) based on failure to inhibit timing-based intrusion errors; limited prior research has suggested humans' performance is qualitatively different. Here we illustrate a partial replication of previous findings in humans, but suggest based on our results that humans process these tasks in a manner similar to pigeons. Humans made relatively few but consistent errors across both simultaneous- and successive-choice experiments. Anticipation errors were limited when the identity of the first-correct stimulus alternated between sessions, consistent with the behaviour of pigeons. Subsequent experiments found evidence for anticipation on a purely temporal simultaneous choice task, and fewer errors with symmetrical reinforcement and punishment of responses on a sequential choice task. Interval timing causes conflicts with decision-making processes on the midsession reversal task that are consistent, but differ in magnitude, across species.

Within-session reversal learning in rhesus macaques (Macaca mulatta)

In a midsession reversal (MSR) task, animals are typically presented with a simple, simultaneous discrimination (S1+, S2-) where contingencies are reversed (S1-, S2+) half-way through each session. This paradigm creates multiple, relevant cues that can aid in maximizing overall reinforcement. Recent research has shown that pigeons show systematic anticipatory and perseverative errors across the session, which increase as a function of proximity to the reversal trial. This behavior has been theorized to indicate primary control by temporal cues across the session, instead of the cues provided by recent reinforcement history that appear to control behavior shown by humans. Rats, however, appear to be guided by recent reinforcement history when tested in an operant context, thereby demonstrating behavior that parallels that seen in humans, but they appear to be guided by temporal cues when tested in an open-field apparatus, showing behavior more akin to that seen in pigeons. We tested rhesus macaques (Macaca mulatta) on the MSR with a computerized simultaneous visual discrimination to assess whether they would show errors indicative of control by time or by recent reinforcement history. When a single reversal point occurred midsession, rhesus macaques showed no anticipation of the reversal and a similar level of perseveration to rats tested in an operant setting. Nearly identical results also were observed when the monkeys were trained with a single, variable reversal point or with multiple, variable reversal points within a session. These results indicate that temporal cues are not guiding response flexibility in rhesus macaque visual discrimination.

Rats' midsession reversal performance: the nature of the response

The midsession reversal task involves a simple simultaneous discrimination that predictably reverses midway through a session. Under various conditions, pigeons generally both anticipate the reversal and perseverate once it has occurred, whereas rats tend to make very few of either kind of error. In the present research, we investigated the hypothesis that the difference in performance between rats and pigeons is related to the nature of the responses made. We hypothesized that rats could have been better at bridging the intertrial interval by keeping the relevant paw close to the lever while eating, whereas the pigeons had to remove their beak from the response key and insert it into the feeder, thus making it difficult to mediate the response last made. In the present experiment, in successive phases, rats were trained to leverpress or nose-poke on a 40-trial midsession reversal, an 80-trial midsession reversal, and a variable-location reversal. The results showed that the leverpress group acquired the task faster than the nose-poke group, but that both groups reached comparable levels of performance. Thus, the difference in the natures of the responses cannot fully account for the differences in accuracy between rats and pigeons. Additionally, differences in the types of errors made by the two groups suggest that the nature of the response plays different roles in the performance of this task.