What makes the ephemeral reward task so difficult?

The ephemeral reward task involves providing subjects with a choice between two distinctive stimuli, A and B, each containing an identical reward. If A is chosen, the reward associated with A is obtained and the trial is over. If B is chosen, the reward associated with B is obtained but A remains, and the reward associated with A can be obtained as well. Thus, the reward-maximizing solution is to choose B first. Although cleaner fish (wrasse) and parrots easily acquire the optimal response by choosing B, paradoxically, several nonhuman primate species, as well as rats and pigeons, do not. It appears that some species do not associate their choice and reward with the second reward. Surprisingly, research in an operant context with pigeons and rats suggests that inserting a delay between the choice and reward facilitates optimal choice. It is suggested that impulsivity may be, in part, responsible for the difficulty of the task. In an attempt to better understand this task, we trained human subjects on an operant version of this task, with and without a brief delay between choice and reward and found that many subjects failed to learn to choose optimally, independent of the delay. Furthermore, performance on this task was not correlated with a task thought to measure impulsivity, the Balloon Analog Risk Task or with the Abbreviated Impulsivity Survey. We concluded that, for humans, the task is confusing because there is no incorrect response, only good and better, and better is not easily discriminated. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Does precrastination explain why some observers are suboptimal in a visual search task?

How do we decide where to search for a target? Optimal search relies on first considering the relative informational value of different locations and then executing eye movements to the best options. However, many participants consistently move their eyes to locations that can be easily ascertained to neither contain the target nor provide new information about the target's location. Here, we asked whether this suboptimal search behaviour represents a specific example of a general tendency towards precrastination: starting sub-goals of a task before they are needed, and in so doing, spending longer time on doing the task than is necessary. To test this hypothesis, we asked 200 participants to do two tasks: retrieve two heavy buckets (one close and one far) and search for a line segment. Precrastination is defined as consistently picking up the closer bucket first, versus the more efficient strategy of picking up the farther bucket first. Search efficiency is the proportion of fixations directed to more cluttered regions of the search array. Based on the pilot data, we predicted an association of precrastination with inefficient search strategies. Personality inventories were also administered to identify stable characteristics associated with these strategies. In the final dataset, there was no clear association between search strategy and precrastination, nor did these correlate strongly with any of the personality measures collected. This article received in-principle acceptance (IPA) at Royal Society Open Science on 29 January 2020. The accepted Stage 1 version of the manuscript, not including results and discussion, may be found at https://osf.io/p2sjx. This preregistration was performed prior to data collection and analysis.

Serial pattern learning: The anticipation of worsening conditions by pigeons

In general, animals are known to be sensitive to the immediacy of reinforcers. That is, they are generally impulsive and outcomes that occur in the future are generally heavily discounted. Furthermore, they should prefer alternatives that provide reinforcers that require less rather than greater effort to obtain. In the present research, pigeons were given a choice between (1) obtaining reinforcers on a progressively more difficult schedule of reinforcement; starting with four pecks, then eight pecks, then 16 pecks, then 32 pecks, and finally 64 pecks on each trial, and (2) a color signaling a number of pecks for a single reinforcer: red = six, green = 11, blue = 23, or yellow = 45. If pigeons choose optimally, most of the time they should choose the progressive schedule to obtain five reinforcers rather than switch to a color to receive only one. However, if they are sensitive primarily to the number of pecks to the next reinforcer, they should choose the progressive schedule once before switching to red, twice before switching to green, three times before switching to blue, and four times before switching to yellow. Instead, they systematically switched too early. Rather than choose based on the rate of reinforcement or even based on the time or effort to the next reinforcer, they appear to anticipate that the progressive schedule is going to get more difficult, and they base their choice suboptimally on the serial pattern of the worsening progressive schedule.

How language and agriculture promote culture- and peace-promoting norms

Humans are predisposed to form in-groups and out-groups that are remarkably flexible in their definition due largely to the complex language that has evolved in them. Language has allowed for the creation of shared "background stories" that can unite people who do not know each other. Second, the discovery of agriculture has resulted in the critical need to negotiate boundaries, a process that can lead to peace (but also war).

Conditional discrimination learning by pigeons: Stimulus-response chains or occasion setters?

In conditional discrimination, the conditional stimulus or sample indicates which of two choice or comparison stimuli is associated with a reinforcer. Two hypotheses have been proposed concerning the role of the sample stimulus. According to Hull (1952), the sample and the response to the correct comparison form a stimulus-response chain. According to Skinner (1938), however, the sample serves as an occasion setter, setting the occasion for the choice of the correct comparison stimulus. In a conditional discrimination, if the sample stimulus forms part of a stimulus-response chain, then presenting the sample in the absence of the comparison stimuli should weaken the association. If the sample serves as an occasion setter, however, presenting the sample alone should not weaken its occasion-setting ability. In two experiments we tested these predictions. In Experiment 1, following conditional discrimination training with vertical and horizontal line samples and red and green comparison stimuli, we found that the presentation of the samples without the comparison stimuli (followed sometimes by a reinforcer) had little effect on conditional discrimination accuracy. In Experiment 2, two different houselights served as samples. When we presented the samples without comparison stimuli and without the reinforcers we found similar results. The results support the hypothesis that in conditional discrimination, the samples serve as occasion setters. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Memory for where and when: pigeons use single-code/default strategy

Memory for what, where, and when an event took place has been interpreted as playing a critical role in episodic memory. Moreover, such memory is likely to be important to an animal's ability to efficiently forage for food. In Experiment 1 of the present study, pigeons were trained on a task in which on each trial, one lit stimulus color and location was presented and then another. A cue presented after the last stimulus location signaled that the pigeon was to choose either the first location presented, or the last location presented, to receive a reinforcer. After learning this task, in Experiment 2, the color cue was removed, requiring the pigeons to choose based on location and order alone. In Experiment 3, when a delay was inserted between presentation of the two locations, it had little effect on task accuracy. Results suggested that the pigeons had acquired the task using a single-code/default rule. When presented with the cue indicating that the last location was correct, pigeons selected the location just presented. When presented with the cue indicating that the first location was correct, pigeons chose the other location, by default. In support of this hypothesis, in Experiment 4, when a delay was inserted, prior to receiving the instructional cue, it had a disruptive effect on task accuracy proportional to the delay. Although the present results do not provide evidence for episodic memory, they do suggest that the pigeons have developed a single-code/default strategy that appears to be an efficient means of performing this task.

Interference of same/different learning by a spatial discrimination

Same/different learning by pigeons has long been of interest to experimental psychologists. In one of these procedures, matching-to-sample, responses to a sample stimulus result in the presentation of two comparison stimuli, one of which matches the sample, the other of which does not, and choice of the matching stimulus is reinforced. Evidence of a matching concept has been found when transfer has been found to new stimuli. Given the transfer results, it is surprising that acquisition of two matching tasks (or two mismatching tasks), has not been found to be any faster than one matching and one mismatching task (i.e., two compatible tasks do not appear to facilitate each other). In the present experiment, we asked if matching acquisition involving three colors would be retarded if the correct response to a fourth color was not matching but was spatial (e.g., if the sample is red choose the red comparison, if the sample is green choose the green comparison, if the sample is yellow choose the yellow comparison, but if the sample is blue choose the left comparison). We found that acquisition of this task was slower than acquisition of a four color matching task (i.e., when the sample was blue, the blue comparison was correct). The results suggest that there is an interaction among matching associations, such that common rules facilitate learning compared with having to learn an inconsistent (spatial) rule. This result provides further evidence of the development of a matching concept by pigeons.

1-Back reinforcement symbolic-matching by humans: How do they learn it?

For humans, a distinction has been made between implicit and explicit learning. Implicit learning is thought to involve automatic processes of the kind involved in much Pavlovian conditioning, while explicit learning is thought to involve conscious hypothesis testing and rule formation, in which the subject's statement of the rule has been taken as evidence of explicit learning. Various methods have been used to determine if nonverbal animals are able to learn a task explicitly - among these is the 1-back reinforcement task in which feedback from performance on the current conditional discrimination trial is provided only after completion of the following trial. We propose that it is not whether an organism can learn the task, but whether they learn it rapidly, all-or-none, that provides a better distinction between the two kinds of learning. We had humans learn a symbolic matching, 1-back reinforcement task. Almost half of the subjects failed to learn the task, and of those who did, none described the 1-back rule. Thus, it is possible to learn this task without learning the 1-back rule. Furthermore, the backward learning functions for humans differ from those of pigeons. Human subjects who learned the task did so all-or-none, suggesting explicit learning. In earlier research with pigeons, they too showed significant learning of this task; however, backward learning functions suggested that they did so gradually over the course of several sessions of training and to a lower level of asymptotic accuracy than the humans, a result suggesting implicit learning was involved.

What enables "distraction" to reduce delay discounting for pigeons (Columba livia)

In a successive delay-discounting task, a small reward can be obtained immediately but a larger reward can be obtained if one waits. There is evidence that the larger reward can be obtained more easily if one is "distracted" from obtaining the small reward. It is proposed here that a distractor stimulus may function as a Pavlovian conditioned stimulus (sign tracking) because orienting to it may be directly associated with the larger reinforcer. In the present study with pigeons, we examined two successive procedures: (a) a peck to a red light resulted in one pellet of food, and waiting for the red light to turn off resulted in five pellets (Red-Only). (b) If the pigeon pecked a red light, it received one pellet of food, and if it waited for the red light to turn to green, a peck to the green light resulted in five pellets of food (Red-Green). For both groups, on some trials, a concurrent (distractor) stimulus appeared with the red light but responses to it had no programed consequence. Results indicated that the pigeons in both groups waited for the larger reward more often when the distractor was present than when it was absent and that pigeons in the Red-Only group waited longer than those in the Red-Green group. The results are consistent with the hypothesis that the concurrent stimulus served as a conditioned stimulus for the Red-Only group and as a higher order conditioned stimulus for the Red-Green group. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Implicit learning of the one-back reinforcement matching-mismatching task by pigeons

Humans can learn tasks explicitly, as they can often describe the rules they have used to learn the task.^1,2,3 Animals, however, are thought to learn tasks implicitly (i.e., purely associatively).^2,3 That is, they gradually learn the correlation or association between the stimulus (or response) and the outcome. Both humans and pigeons can learn matching, where a sample stimulus indicates which one of two stimuli matches the sample. The 1-back reinforcement task is a difficult version of matching in which a correct response on trial N is rewarded only following a response on trial N + 1 (independent of the response on trial N + 1),⁴ and the correct response on trial N + 1 indicates whether a reward will occur on trial N + 2, and so forth. Humans do not appear to be able to learn the 1-back rule.⁵ Pigeons, however, do show 1-back reinforcement learning,^6,7 and they appear to do so implicitly by gradually learning the correlation between their response on one trial and the outcome on the next trial (because all other relations are uncorrelated with the outcome). They learn the task slowly and to a level below what would be expected had they learned it explicitly. The present results, together with research with humans,⁷ suggest that there are times when human explicit learning may interfere with the ability of humans to learn. Pigeons, however, are not "distracted" by attempts at explicit learning, and thus they are able to learn this and other similar tasks.^6,7,8.

Comparative Cognition Research Demonstrates the Similarity between Humans and Other Animals

The field of comparative cognition represents the interface between the cognitive behavior of humans and other animals. In some cases, research demonstrates that other animals are capable of showing similar cognitive processes. In other cases, when animals show behavior thought to be culturally determined in humans, it suggests that simpler processes may be involved. This review examines research primarily with pigeons (out of convenience because of their visual ability). I start with the concept of sameness and follow with the concept of stimulus equivalence, the building blocks of human language. This is followed by research on directed forgetting, the cognitive ability to maintain or forget information. A hallmark of cognition is transitive inference performance (if A < B, and B < C, the understanding that A < C), but the variety of species that show this ability suggests that there may be simpler accounts of this behavior. Similarly, experiments that demonstrate a form of cognitive dissonance in animals suggest that dissonance may not be necessary to explain this biased behavior. Furthermore, examples of sunk cost in pigeons suggests that the human need to continue working on a failing project may also have a biological basis. Finally, pigeons show a preference for a suboptimal choice that is similar to unskilled human gambling, a finding that may clarify why humans are so prone to engage in this typically losing activity.

"Distractor" effects in delay discounting of probability by pigeons

Impulsive behavior can be measured by performance on a successive delay-discounting task, in which a response to a stimulus provides a small reinforcer sooner (SS), but in the absence of a response, a larger reinforcer later (LL). Previous research suggests that the presence of a concurrent "distractor" stimulus, to which responding has no programed consequence, can result in increased LL reinforcers. In the present experiments, we used differences in the probability of reinforcement between SS and LL (rather than magnitude of reinforcement) and tested the hypothesis that the concurrent stimulus may become a Pavlovian conditioned stimulus. For the Red-Only group, a response to the SS stimulus resulted in a reinforcer with a low probability (SS), whereas the absence of a response resulted in a reinforcer with a high probability (LL). For the Red-Green group, (analogous to the more typical simultaneous choice between an SS and LL stimulus) the absence of a response to the SS stimulus replaced the SS stimulus with the LL stimulus and a response to the LL stimulus resulted in the reinforcer. Thus, for the Red-Green group, the concurrent stimulus should have been less effective because responding to the concurrent stimulus was not immediately followed by the reinforcer. In Experiment 1, the concurrent stimulus was a yellow key-light; in Experiment 2, it was a houselight. In both experiments, the concurrent stimulus was effective in increasing the number of LL reinforcers and the effect was larger for the Red-Only group than for the Red-Green group.

Matching is acquired faster than mismatching by pigeons when salient stimuli are presented manually

Same/different learning by pigeons has been studied using several different procedures. One of these procedures is matching-to-sample or mismatching-from-sample in which responses to a sample stimulus result in the presentation of two comparison stimuli, one of which matches the sample, the other of which does not. In the matching task, choice of the matching stimulus is reinforced. In the mismatching task, choice of the stimulus that does not match the sample is reinforced. Most research that has compared acquisition of the two tasks has not reported a difference between them. Research with transfer of training, in which either the matching stimulus or the mismatching stimulus is replaced with a new stimulus, suggests that the matching stimulus is selected in the matching task, but the matching stimulus is rejected in the mismatching task. In the present experiment, pigeons were trained on either matching or mismatching with salient stimuli presented manually and the reinforcer was presented under a colored slide that covered it. In Phase 1 with a noncorrection procedure and a reinforcer for pecking the sample, pigeons did not acquire either task, however, in Phase 2 they learned both tasks readily without reinforcement for pecking the sample and with a correction procedure. Furthermore, the pigeons learned matching significantly faster than mismatching, suggesting that sameness may be a more natural stimulus relation than mismatching.

Pigeon's choice depends primarily on the value of the signal for the outcome rather than its frequency or contrast

Pigeons typically prefer a 20% probability of signaled reinforcement over a 50% probability of unsignaled reinforcement. There is even evidence that they prefer 50% signaled reinforcement over 100% reinforcement. It has been suggested that this effect results from contrast between the expected probability of reinforcement (e.g., 50%) at the time of choice and the value of the positive signal for reinforcement (100%). Alternatively, it is primarily the value of the positive signal for reinforcement itself that determines suboptimal choice. To attempt to distinguish between these two hypotheses, in Experiment 1, we gave pigeons a choice between (a) a 50% reinforcement alternative that was followed by one of two signals for 100% reinforcement, each 25% of the time, or a signal for the absence of reinforcement 50% of the time (50% contrast) and (b) a 25% reinforcement alternative that was followed by a signal for 100% reinforcement 25% of the time, or a signal for the absence of reinforcement 75% of the time (75% contrast). In spite of the difference in contrast, the pigeons were indifferent between the two alternatives. In Experiment 2, when contrast was held constant at 50% and the value of the positive signals for reinforcement were different, we found support for choice based on the value of the positive signal for reinforcement. Thus, it appears that pigeons' choice depends primarily on the value of the outcome rather than its frequency or contrast. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Basic Behavioral Processes Involved in Procrastination

Procrastination involves an irrational putting off of engaging in a course of action, in spite of expecting to be worse off for the delay. I suggest that to understand the processes underlying procrastination one should examine its relation to several behavioral procedures that have been studied in humans and other animals. For example, in delay discounting, smaller rewards that come sooner are often preferred over larger rewards that come later. In the context of delay discounting, procrastination can be viewed as the preference for an immediate competing activity over the delay to work on a required task. Another process similar to procrastination can be seen in free operant, temporal avoidance (or Sidman avoidance) in which an animal will receive a shock (a deadline not met) if an interval passes without a specified response (task completion). Once animals learn about the interval, they often procrastinate by waiting until the interval has almost passed before responding. Finally, research with animals suggests that the persistence of procrastination may involve a form of negative reinforcement associated with the sudden decline in anxiety or fear (relief) when the task is completed prior to the deadline. Research with animals suggests that the mechanisms responsible for human procrastination may involve systems that derive from several procedures known to produce similar behavior animals.

1-Back reinforcement matching and mismatching by pigeons: Implicit or explicit learning?

In human learning a distinction has been made between implicit and explicit learning. Implicit learning is thought involve automatic processes of the kind involved in Pavlovian conditioning, while explicit learning is thought to involve conscious hypothesis testing and rule formation, in which the ability to report the rule used to learn the task is taken as evidence. Because non-verbal animals cannot provide such evidence, several indirect methods have been proposed. One of these methods is faster learning by humans of certain explicitly learned tasks than implicitly learned tasks, but pigeons do not show a similar difference. Another method involves the 1-back-reinforcement conditional discrimination (if A choose X, if B choose Y) in which feedback following the conditional response is delayed until the next trial. It has been argued that implicit learning cannot occur over the delay between the conditional response and the reinforcer on the next trial, yet, it has been found that monkeys can learn this 1-back reinforcement task. We have argued that such learning can occur implicitly. We have found that pigeons, a species not thought to learn explicitly, can show significant learning of both 1-back reinforcement matching and 1-back reinforcement mismatching, two versions of the 1-back-reinforcement conditional discrimination. We propose that the evidence for explicit learning by non-verbal animals suffers from alternative simpler accounts because the rationale for explicit learning is based on assumptions that likely are not correct.

Flexible conditional discrimination learning: Pigeons can learn to select the correct comparison stimulus, reject the incorrect comparison, or both

In a simultaneous discrimination, pigeons are presumed to learn to about the correct stimulus, but they may also learn to avoid the incorrect stimulus. Similarly, in a conditional discrimination, they are presumed to learn about the relation between the sample stimulus and the correct comparison stimulus but not about the incorrect comparison stimulus. In the present research, we encouraged pigeons to learn about the incorrect comparison stimulus by increasing, over trials, the number of correct comparison stimuli with one sample, to compare with increasing the number of incorrect comparison stimuli over trials with the other sample. In Experiment 1, using colors and shapes, we found no difference in acquisition between the 2 sample types. However, when we replaced either the correct or incorrect comparison from training with a novel stimulus, the pigeons showed that they had learned to avoid the incorrect comparison when there were multiple correct comparisons and to select the single correct comparison when there were multiple incorrect comparisons. In Experiment 2, using national flags as stimuli, when tested with a novel flag stimulus, once again, the pigeons learned about the single correct comparison but not about the multiple incorrect comparisons. However, with the other sample, they appeared to learn about both the multiple correct comparisons and about the single incorrect comparison. This research indicates that pigeons can show considerable flexibility in what they learn in a conditional discrimination. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Effect of Environmental Enrichment on the Brain and on Learning and Cognition by Animals

Simple Summary

Most people consider the environment in which animals are kept to be an ethical matter, separate from the research that we conduct with them. Those of us who do research on the cognitive behavior of animals try to consider their welfare, but what we often fail to recognize is that the welfare of the animals we study can affect the results of experiments that we investigate. We have but scratched the surface of the question, how do enriched environments affect the cognitive behavior of animals, in our case pigeons. We have found that pigeons with experience in an enriched environment are less impulsive. The reduction in impulsivity results in a reduced tendency to make the suboptimal choice. It also has been claimed to make animals more optimistic, as assessed by their tendency to make choices of more favorable alternatives, under ambiguous conditions.

Abstract

The humane treatment of animals suggests that they should be housed in an environment that is rich in stimulation and allows for varied activities. However, even if one’s main concern is an accurate assessment of their learning and cognitive abilities, housing them in an enriched environment can have an important effect on the assessment of those abilities. Research has found that the development of the brain of animals is significantly affected by the environment in which they live. Not surprisingly, their ability to learn both simple and complex tasks is affected by even modest time spent in an enriched environment. In particular, animals that are housed in an enriched environment are less impulsive and make more optimal choices than animals housed in isolation. Even the way that they judge the passage of time is affected by their housing conditions. Some researchers have even suggested that exposing animals to an enriched environment can make them more “optimistic” in how they treat ambiguous stimuli. Whether that behavioral effect reflects the subtlety of differences in optimism/pessimism or something simpler, like differences in motivation, incentive, discriminability, or neophobia, it is clear that the conditions of housing can have an important effect on the learning and cognition of animals.

Putting the Self in Self-Correction: Findings From the Loss-of-Confidence Project

Science is often perceived to be a self-correcting enterprise. In principle, the assessment of scientific claims is supposed to proceed in a cumulative fashion, with the reigning theories of the day progressively approximating truth more accurately over time. In practice, however, cumulative self-correction tends to proceed less efficiently than one might naively suppose. Far from evaluating new evidence dispassionately and infallibly, individual scientists often cling stubbornly to prior findings. Here we explore the dynamics of scientific self-correction at an individual rather than collective level. In 13 written statements, researchers from diverse branches of psychology share why and how they have lost confidence in one of their own published findings. We qualitatively characterize these disclosures and explore their implications. A cross-disciplinary survey suggests that such loss-of-confidence sentiments are surprisingly common among members of the broader scientific population yet rarely become part of the public record. We argue that removing barriers to self-correction at the individual level is imperative if the scientific community as a whole is to achieve the ideal of efficient self-correction.

Revisited: Pigeons Have Much Cognitive Behavior in Common With Humans

The hypothesis proposed by Macphail (1987) is that differences in intelligent behavior thought to distinguish different species were likely attributed to differences in the context of the tasks being used. Once one corrects for differences in sensory input, motor output, and incentive, it is likely that all vertebrate animals have comparable intellectual abilities. In the present article I suggest a number of tests of this hypothesis with pigeons. In each case, the evidence suggests that either there is evidence for the cognitive behavior, or the pigeons suffer from biases similar to those of humans. Thus, Macphail’s hypothesis offers a challenge to researchers to find the appropriate conditions to bring out in the animal the cognitive ability being tested.

Should I stay or should I go? Pigeons' (Columba livia) performance of a foraging task has implications for optimal foraging theory and serial pattern learning

Optimal foraging theory suggests that animals have evolved to maximize their net rate of energy intake; all things being equal, they should leave a current depleting patch when an alternative patch would provide either more or sooner food. In nature, however, typically all things are not equal. For example, uncertainty about the value of alternative patches, time to travel to those patches, and potential dangers incurred in changing patches may delay leaving the depleting patch, when it would otherwise be optimal to do so. We tested the hypothesis that leaving the current patch may be delayed, by providing pigeons (Columba livia) with a continuous choice between a progressive schedule, in which each access to food could be obtained with an increasing number of pecks, and a multiple schedule, in which a colored light signaled the number of pecks required for food. The pigeons could switch from the progressive schedule to the multiple schedule at any time. We asked if pigeons would tend to switch when the signaled multiple schedule required fewer pecks than the next reinforcer provided by the progressive schedule. We found that pigeons tended to switch to the multiple schedule sooner than would have been optimal-one might say they precrastinated. We propose that, on the progressive schedule, the signal to switch was not just the number of pecks required for the next reinforcer but also the more general cue that reinforcement was becoming more difficult to obtain-a form of serial pattern learning. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Pigeons are attracted to a perceived gain without an actual gain

Reference dependence refers to the reduced value of a reward that is less than expected, or the added value of a reward that is greater than expected. There is evidence that when pigeons are offered an alternative that has 1 pellet versus an alternative that has 2 pellets, but one of the two pellets offered will be removed, the pigeons prefer the originally presented 1 pellet (loss aversion). In the present research, we tested for the opposite effect (gain attraction). In Experiments 1 and 2, pigeons could choose between 2 pellets, each one on a distinctive background. If they chose the optimal alternative, they received a second pellet. In Experiment 2, the second pellet obtained was the one not initially chosen (a task sometimes referred to as the ephemeral reward task). Pigeons learned to choose optimally in both experiments. In Experiment 3, we tested the pigeons for reference dependence. Pigeons were given an alternative that offered them one pellet or two pellets, if they chose the one-pellet alternative, they received an additional pellet, and if they chose the two-pellet alternative, they received the two pellets. In keeping with the reference dependence hypothesis, the pigeons preferred the 1-pellet alternative that gave them an extra pellet. These effects are related to similar findings with humans, including the endowment effect.

Within-trial contrast or Wagner's SOP model: Can they both account for two presumed complex cognitive phenomena?

When humans make biased or suboptimal choices, they are often attributed to complex cognitive processes that are viewed as being uniquely human. Alternatively, several phenomena, such as suboptimal gambling behavior and cognitive dissonance (justification of effort) may be explained more simply as examples of the contrast between what is expected and what occurs as well as Wagner's Standard Operating Procedure model based on reward prediction error. For example, when pigeons are attracted to choices involving a suboptimal, low probability of a high payoff, as in unskilled gambling behavior, it may be attributed to reward prediction error or the contrast between the low probability of reward expected and the sometimes high probability of reward obtained (when one wins). Similarly, justification of effort, the tendency to attribute greater value to rewards that are difficult to obtain, is typically explained in terms of the tendency to inflate the value of a reward to justify the effort required to obtain it. When pigeons prefer outcomes that require more effort to obtain, however, it is more likely to be explained in terms of contrast between the effort and the reward that follows. We readily attribute the behavior of animals to contrast-like effects or reward prediction error, however, when similar behavior occurs in humans, we also should be prepared to explain it in terms of simpler learning mechanisms. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Midsession Reversal Task with Pigeons Does a Brief Delay Between Choice and Reinforcement Facilitate Reversal Learning?

In a midsession reversal task, the session begins with a simple simultaneous discrimination in which one stimulus (S1) is correct and the other stimulus (S2) is incorrect (S1+/S2-). At the midpoint of the session, the discrimination reverses and S2 becomes the correct choice (S2+/S1-). When choosing optimally, a pigeon should choose S1 until the first trial in which its choice is not reinforced and then it should shift to S2 (win-stay/lose-shift). With this task, pigeons have been shown to respond suboptimally by anticipating the reversal (making anticipatory errors) and continuing to choose S1 after the reversal (making perseverative errors). This suboptimal behavior may result from a pigeon's relative impulsivity due to the immediacy of reinforcement following choice. In other choice tasks, there is evidence that the introduction of a short delay between choice and reinforcement may decrease pigeons' impulsivity. In the present experiment, a delay was introduced between stimulus selection and reinforcement to assess whether it results in a decrease in anticipatory and perseverative errors. Pigeons that had a delay between choice and reinforcement were a bit slower in acquiring the midsession reversal task compared to those without a delay, but showed no decrease in either anticipatory or perseverative errors. It is likely that the pigeons' natural tendency to use time from the start of the session to the reversal as a cue to reverse prevented the delay from increasing accuracy on this task.

Midsession reversal learning: Pigeons learn what stimulus to avoid

The midsession reversal task involves a simple simultaneous discrimination in which, each session, choice of 1 stimulus (S1) is correct for the first 40 trials of each session, and choice of the other stimulus (S2) is correct for the remaining 40 trials. After considerable training with this task, pigeons typically continue to choose S2 too early (making anticipatory errors) and continue choosing S1 for following the reversal (making perseverative errors). Errors can be reduced, however, by decreasing the probability of reinforcement for correct S2 choices or by increasing the response requirement for S2 choices. Increasing the number of S2 stimuli (over trials, 1 S2 stimulus on each trial), however, does not reduce errors. Instead, it results in an increase in anticipatory errors but no change in perseverative errors. In the present experiment, we increased the number of S1 stimuli (over trials, 1 S1 stimulus on each trial) and found an increase in the number of perseverative errors but no change in anticipatory errors. The results suggest that the pigeons acquire this task by learning which stimuli to avoid, rather than which stimuli to choose, although it is also possible that these effects result from attention drawn to the variable stimuli when they are incorrect. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Pigeons can learn a difficult discrimination if reinforcement is delayed following choice

Delaying reinforcement typically has been thought to retard the rate of acquisition of an association, but there is evidence that it may facilitate acquisition of some difficult simultaneous discriminations. After describing several cases in which delaying reinforcement can facilitate acquisition, we suggest that under conditions in which the magnitude of reinforcement is difficult to discriminate, the introduction of a delay between choice and reinforcement can facilitate the discrimination. In the present experiment, we tested the hypothesis that the discrimination between one pellet of food for choice of one alternative and two pellets of food for choice of another may be a difficult discrimination when choice consists of a single peck. If a 10-s delay occurs between choice and reinforcement, however, the discrimination is significantly easier. It is suggested that when discrimination between the outcomes of a choice is difficult and impulsive choice leads to immediate reinforcement, acquisition may be retarded. Under these conditions, the introduction of a brief delay may facilitate acquisition.

Less information results in better midsession reversal accuracy by pigeons

The midsession reversal task involves a simultaneous discrimination between Stimulus 1 (S1) and Stimulus 2 (S2) in which, for the first half of each session, choice of S1 is reinforced and S2 is not, and for the last half of each session, choice of S2 is reinforced and S1 is not. With this task, even after considerable training, pigeons tend to make anticipatory errors as they approach the reversal and they continue to make perseverative errors following the reversal. In the present research, we tested the hypothesis that reversal accuracy would improve by devaluing choice of S2 relative to S1. In Experiment 1, correct choice of S1 was reinforced 100% of the time, whereas correct choice of S2 was reinforced only 20% of the time. This manipulation reduced anticipatory errors but did not increase perseverative errors. In Experiment 2, choice of S1 required a single peck, whereas choice of S2 was devalued by requiring 10 pecks. A similar result was found. In Experiment 3 we devalued S1 by requiring 10 pecks and found decreased accuracy in the form of increased anticipatory errors. Paradoxically, in Experiments 1 and 2, by encouraging the pigeons to avoid using the feedback from choice of S2, and rely solely on feedback from choice of S1, discrimination reversal errors were reduced. The results have implications for attentional theories of learning and theories of behavior change. They also have implications for the conditions responsible for pigeons' tendency to time the occurrence of the change in reinforcement contingencies. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Transitive inference in pigeons may result from differential tendencies to reject the test stimuli acquired during training

In the five-term, transitive inference task used with animals, pigeons are trained on four simultaneous discrimination premise pairs: A + B -, B + C -, C + D -, D + E -. Typically, when tested with the BD pair, most pigeons show a transitive inference effect, choosing B over D. Two non-inferential hypotheses have been proposed to account for this effect but neither has been reliably supported by research. Here we test a third non-inferential hypothesis that the preference for B arises because the animals have not had as much experience with B - in the A + B - discrimination as they have had with the D - in the C + D - discrimination. To test this hypothesis we trained the Experimental Group with the A + B - discrimination in which, over trials, there were four possible A + stimuli that could appear. This was done to encourage the pigeons to learn to reject the B - stimulus. For the Control Group there was only one A + stimulus over trials, as is typically the case. We also varied the nature of the stimuli between groups, such that colors served as the stimuli for half of the pigeons, whereas flags of different counties served as stimuli for the remaining pigeons. In both stimulus conditions, for the Experiment Group, we found little preference for stimulus B over stimulus D, whereas for the Control Group we found the typical preference for stimulus B. Thus, we propose that it is not necessary to attribute the transitive inference effect to an inferential process.

Sooner Rather Than Later: Precrastination Rather Than Procrastination

Putting things off as long as possible (procrastination) is a well-known tendency. Less well known is the tendency to attempt to get things done as soon as possible, even if that involves extra effort ( precrastination). Since its discovery in 2014, precrastination has been demonstrated in humans and animals and has recently been revealed in an analogous tendency called the mere-urgency effect. Trying to get things done as soon as one can may reflect optimal foraging, but another less obvious factor may also contribute—reducing cognitive demands associated with having to remember what to do when. Individual differences may also play a role. Understanding precrastination will have important implications for explaining why hurrying happens as often as it does and may help reduce the chance that haste makes waste.

Object permanence in the pigeon (Columba livia): Insertion of a delay prior to choice facilitates visible- and invisible-displacement accuracy

Object permanence, often viewed as a measure of human cognitive development, has also been used to assess animals' cognitive abilities. Tests of object permanence have distinguished between visible displacement, in which an object may be placed into one of two (or more) containers to be retrieved, and invisible displacement, in which after the object is placed into the container, the container is moved before retrieval is attempted. We tested pigeons' accuracy on both visible and invisible displacement using a rotational beam with a container at either end. In Experiment 1, the pigeons showed some evidence of object permanence on an initial visible displacement test, but they did not maintain accurate choice. With training, their accuracy improved but only to about 70% correct. When tested on a 90° invisible displacement (rotation), accuracy transferred but once again dropped with further training. In Experiment 2, a 5-s delay was inserted between container baiting and choice. Once again, the pigeons showed some evidence of object permanence on an initial visible displacement test, although on the first test session, choice accuracy was not much better than in Experiment 1. With training, choice accuracy improved greatly. Furthermore, pigeons showed good transfer when they were tested on the 90° invisible displacement. Finally, and importantly, they also transferred well to a 180° invisible displacement, a displacement on which dogs failed. The results of these experiments suggest that under the right conditions, pigeons can show a moderate degree of object permanence. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Suboptimal choice in pigeons: Does the predictive value of the conditioned reinforcer alone determine choice?

Prior research has found that pigeons are indifferent between an option that always provides a signal for reinforcement and an alternative that provides a signal for reinforcement only 50% of the time (and a signal for the absence of reinforcement 50% of the time). This suboptimal choice suggests that the frequency of the signal for reinforcement plays virtually no role and choice depends only on the predictive value of the signal for reinforcement associated with each alternative. In the present research we tested the hypothesis that if there are two or three signals for reinforcement associated with the suboptimal alternative but each occurs only 25% or 17% of the time, respectively, pigeons would show a greater preference for the suboptimal alternative. Although we found that increasing the number of signals for reinforcement associated with the suboptimal alternative did not increase the preference for the suboptimal alternative (relative to a single signal for reinforcement) extended training on this task resulted in a significant preference for the suboptimal alternative by both groups. This result suggests that contrast between the expected outcome at the time of choice (50% reinforcement) and the value of the signal for reinforcement (100% reinforcement) is also responsible for choice of the suboptimal alternative.

Sameness May Be a Natural Concept That Does Not Require Learning

It has been assumed that when pigeons learn how to match to sample, they learn simple stimulus-response chains but not the concept of sameness. However, transfer to novel stimuli has been influenced by pigeons' tendency to be neophobic. We trained pigeons on matching ( n = 7) and mismatching ( n = 8) with colors as samples and, with each sample, one color as the nonmatching comparison. We then replaced either the matching or the nonmatching stimulus with a familiar stimulus never presented with that sample. Results suggest that for both matching and mismatching, pigeons locate the stimulus that matches the sample: If the task involves matching, they chose it; if it involves mismatching, they avoid it. Thus, the concept of sameness is the basis for correct choice with both tasks. This finding suggests that sameness is a basic concept that does not have to be learned and may have evolved in many species, including humans.

The role of 'jackpot' stimuli in maladaptive decision-making: dissociable effects of D1/D2 receptor agonists and antagonists

RATIONALE

Laboratory experiments often model risk through a choice between a large, uncertain (LU) reward against a small, certain (SC) reward as an index of an individual's risk tolerance. An important factor generally lacking from these procedures are reward-associated cues that may modulate risk preferences.

OBJECTIVE

We tested whether the addition of cues signaling 'jackpot' wins to LU choices would modulate risk preferences and if these cue effects were mediated by dopaminergic signaling.

METHODS

Three groups of rats chose between LU and SC rewards for which the LU probability of reward decreased across blocks. The unsignaled group received a non-informative stimulus of trial outcome. The signaled group received a jackpot signal prior to reward delivery and blackout on losses. The signaled-light group received a similar jackpot for wins, but a salient loss signal distinct from the win signal.

RESULTS

Presenting win signals decreased the discounting of LU value for both signaled groups regardless of loss signal, while the unsignaled group showed discounting similar to previous research without cues. Pharmacological challenges with D1/D2 agonists and antagonists revealed that D1 antagonism increased and decreased sensitives to the relative probability of reward for unsignaled and signaled groups, respectively, while D2 agonists decreased sensitivities to the relative magnitude of reward.

CONCLUSION

The results highlight how signals predictive of wins can promote maladaptive risk taking in individuals, while loss signals have reduced effect. Additionally, the presence of reward-predictive cues may change the underlying neurobehavioral mechanisms mediating decision-making under risk.

The Value of Research in Comparative Cognition

Most research of comparative cognition has focused on the degree to which cognitive phenomena that have been reported in humans, especially children, can also be demonstrated in other animals. The value of such comparative research has not only been the finding that other animals show behavior that is qualitatively similar to that of humans but because the comparative approach calls for the careful control of variables often confounded with the mechanisms being tested, the comparative approach has identified procedures that could also improve the design of research with humans. The comparative approach has also been used to study the degree to which other animals demonstrate human biases and suboptimal behavior (e.g., commercial gambling). When applied to this field of research, the comparative approach has generally taken the position that human biases generally thought to be established by complex social and societal mechanisms (e.g., social reinforcement and entertainment) may be more parsimoniously accounted for by simpler mechanisms (i.e., conditioned reinforcement and positive contrast). When explained in terms of these mechanisms, the results have implications for explaining in simpler and more general terms the results of similar research with humans. Thus, comparative psychology tells us not only about the similarities and possible differences in behavior among species but it also may have implications for our understanding of similar behavior in humans.

Prior commitment: Its effect on suboptimal choice in a gambling-like task

Animals choose suboptimally when provided with cues that signal whether reinforcement is coming or not. For example, pigeons do not prefer an alternative that always provides them with a signal for reinforcement over an alternative that provides them with a signal for reinforcement only half of the time and a signal for the absence of reinforcement the rest of the time. In the present research, we tested the hypothesis that if the results of the choice are delayed, pigeons will choose less suboptimally. We tested this hypothesis by forcing pigeons to wait following their choice, requiring them to complete a fixed-interval 20-s schedule prior to receiving the signals for reinforcement. In Experiment 1, we gave the pigeons a choice between (a) a 50% chance of receiving a signal for reinforcement or a 50% chance of receiving a signal for the absence of reinforcement and (b) a 100% chance of receiving a signal for reinforcement. When the signal for reinforcement was delayed, most of the pigeons chose optimally. When it was not delayed, most of the pigeons chose suboptimally. In Experiment 2, we gave the pigeons a choice between (a) a 25% chance of receiving a signal for reinforcement or a 75% chance of receiving a signal for nonreinforcement and (b) a 100% chance of receiving an unreliable signal for reinforcement (predicting reinforcement 75% of the time). When the signal was not delayed, the pigeons showed a strong tendency to choose suboptimally but they chose suboptimally much less when the signal was delayed.