Biological Constraints on Instrumental and Classical Conditioning: Implications for General Process Theory

Applied behavior analysis and the zoo: Forthman and Ogden (1992) thirty years later

The field of applied behavior analysis has been directly involved in both research and applications of behavioral principles to improve the lives of captive zoo animals. Thirty years ago, Forthman and Ogden (1992) wrote one of the first papers documenting some of these efforts. Since that time, considerable work has been done using behavioral principles and procedures to guide zoo welfare efforts. The current paper reexamines and updates Forthman and Ogden's original points, with attention to the 5 categories they detailed: (a) promotion of species-typical behavior, (b) reintroduction and repatriation of endangered species, (c) animal handling, (d) pest control, and (e) animal performances. In addition, we outline 3 current and future directions for behavior analytic endeavors: (a) experimental analyses of behavior and the zoo, (b) applied behavior analysis and the zoo, and (c) single-case designs and the zoo. The goal is to provide a framework that can guide future behavioral research in zoos, as well as create applications based on these empirical evaluations.

A-learning: A new formulation of associative learning theory

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

Meta-analytic techniques reveal that corvid causal reasoning in the Aesop's Fable paradigm is driven by trial-and-error learning

The classic Aesop's fable, Crow and the Pitcher, has inspired a major line of research in comparative cognition. Over the past several years, five articles (over 32 experiments) have examined the ability of corvids (e.g., rooks, crows, and jays) to complete lab-based analogs of this fable, by requiring them to drop stones and other objects into tubes of water to retrieve a floating worm (Bird and Emery in Curr Biol 19:1-5, 2009b; Cheke et al. in Anim Cogn 14:441-455, 2011; Jelbert et al. in PLoS One 3:e92895, 2014; Logan et al. in PLoS One 7:e103049, 2014; Taylor et al. in Gray R D 12:e26887, 2011). These researchers have stressed the unique potential of this paradigm for understanding causal reasoning in corvids. Ghirlanda and Lind (Anim Behav 123:239-247, 2017) re-evaluated trial-level data from these studies and concluded that initial preferences for functional objects, combined with trial-and-error learning, may account for subjects' performance on key variants of the paradigm. In the present paper, we use meta-analytic techniques to provide more precise information about the rate and mode of learning that occurs within and across tasks. Within tasks, subjects learned from successful (but not unsuccessful) actions, indicating that higher-order reasoning about phenomena such as mass, volume, and displacement is unlikely to be involved. Furthermore, subjects did not transfer information learned in one task to subsequent tasks, suggesting that corvids do not engage with these tasks as variants of the same problem (i.e., how to generate water displacement to retrieve a floating worm). Our methodological analysis and empirical findings raise the question: Can Aesop's fable studies distinguish between trial-and-error learning and/or higher-order causal reasoning? We conclude they cannot.

A common heritage of behaviour systems

Behaviour systems theory had its beginnings with Nikolaas Tinbergen’s “hierarchical systems”, an aspect of his thinking and writing that he conspicuously left out of his very memorable 1963 manifesto. This starting point has since been developed within psychology, where it has provided numerous advances. Tinbergen’s aspiration for behaviour systems had been principled integration of ethology with physiology, but the bridge among sciences it ultimately provided led to psychology. To an ethology audience, this paper attempts to reintroduce behaviour systems as a part of Tinbergen’s legacy to make accessible the theoretical developments of behaviour systems theory that have occurred outside of ethology over the last several decades. To a psychology audience, the paper serves as a reminder of the ethological origins of behaviour systems. Both sciences and their integration stand to benefit from recognising this point of common heritage.

Elicited versus emitted behavior: Time to abandon the distinction

The concept of emitted behavior was formulated as a part of the original argument for the validity of a new kind of learning called operant conditioning. The rationale for operant conditioning contrasted it with Pavlovian or classical conditioning, which was (and remains) fundamentally based on responses to conditioned and unconditioned stimuli. Classical conditioned responses were said to be elicited. In contrast, operant behavior was viewed as emitted and controlled primarily by response consequences rather than antecedents. I argue that the distinction between emitted and elicited behavior is no longer warranted for three major reasons. First, the distinction was based on a view of Pavlovian conditioning that is no longer viable. Second, the distinction is incompatible with both empirical data and contemporary conceptualizations of operant behavior. Third, the only way to overcome these problems is to define emitted and elicited in terms of the type of conditioning (operant and classical) that produces these behaviors, but that approach makes the definitions circular and does not avoid implications of the terms that are misleading and counterproductive in light of contemporary research and thinking.

Experimental evolution of prepared learning

Animals learn some things more easily than others. To explain this so-called prepared learning, investigators commonly appeal to the evolutionary history of stimulus-consequence relationships experienced by a population or species. We offer a simple model that formalizes this long-standing hypothesis. The key variable in our model is the statistical reliability of the association between stimulus, action, and consequence. We use experimental evolution to test this hypothesis in populations of Drosophila. We systematically manipulated the reliability of two types of experience (the pairing of the aversive chemical quinine with color or with odor). Following 40 generations of evolution, data from learning assays support our basic prediction: Changes in learning abilities track the reliability of associations during a population's selective history. In populations where, for example, quinine-color pairings were unreliable but quinine-odor pairings were reliable, we find increased sensitivity to learning the quinine-odor experience and reduced sensitivity to learning quinine-color. To the best of our knowledge this is the first experimental demonstration of the evolution of prepared learning.

Stepping outside the box in considering the C/T ratio

Rate expectancy theory (RET) predicts that in Pavlovian procedures conditioned responding will be directly related to the ratio of time spent in the experimental context (C) relative to the trial time (T) or duration of the conditioned stimulus (CS). This prediction was discussed in the context of three experiments. The first and third experiments involved sexual conditioning in quail [Learn Motiv 31 (2000) 211; J Exp Psychol Anim Behav Processes 27 (2001) 269]; the second experiment involved conditioning rats with food as the unconditioned stimulus (US) [Learn Motiv 28(1997) 465]. In each experiment, one type of conditioned response was directly related to the C/T ratio (as predicted by RET) but another conditioned response was inversely related. In addition, the conditioned behavior that occurred with low C/T ratios was controlled by contextual cues rather than the putative CS. The implications of these findings for possible boundary conditions of RET were discussed. The discussion revealed serious shortcomings in the characterization of context conditioning by RET.

The evolution of intelligence: adaptive specializations versus general process

Darwin argued that between-species differences in intelligence were differences of degree, not of kind. The contemporary ecological approach to animal cognition argues that animals have evolved species-specific and problem-specific processes to solve problems associated with their particular ecological niches: thus different species use different processes, and within a species, different processes are used to tackle problems involving different inputs. This approach contrasts both with Darwin's view and with the general process view, according to which the same central processes of learning and memory are used across an extensive range of problems involving very different inputs. We review evidence relevant to the claim that the learning and memory performance of non-human animals varies according to the nature of the stimuli involved. We first discuss the resource distribution hypothesis, olfactory learning-set formation, and the 'biological constraints' literature, but find no convincing support from these topics for the ecological account of cognition. We then discuss the claim that the performance of birds in spatial tasks of learning and memory is superior in species that depend heavily upon stored food compared to species that either show less dependence upon stored food or do not store food. If it could be shown that storing species enjoy a superiority specifically in spatial (and not non-spatial) tasks, this would argue that spatial tasks are indeed solved using different processes from those used in non-spatial tasks. Our review of this literature does not find a consistent superiority of storing over non-storing birds in spatial tasks, and, in particular, no evidence of enhanced superiority of storing species when the task demands are increased, by, for example, increasing the number of items to be recalled or the duration of the retention period. We discuss also the observation that the hippocampus of storing birds is larger than that of non-storing birds, and find evidence contrary to the view that hippocampal enlargement is associated with enhanced spatial memory; we are, however, unable to suggest a convincing alternative explanation for hippocampal enlargement. The failure to find solid support for the ecological view supports the view that there are no qualitative differences in cognition between animal species in the processes of learning and memory. We also argue that our review supports our contention that speculation about the phylogenetic development and function of behavioural processes does not provide a solid basis for gaining insight into the nature of those processes. We end by confessing to a belief in one major qualitative difference in cognition in animals: we believe that humans alone are capable of acquiring language, and that it is this capacity that divides our intelligence so sharply from non-human intelligence.

Fears, phobias, and preparedness: toward an evolved module of fear and fear learning

An evolved module for fear elicitation and fear learning with 4 characteristics is proposed. (a) The fear module is preferentially activated in aversive contexts by stimuli that are fear relevant in an evolutionary perspective. (b) Its activation to such stimuli is automatic. (c) It is relatively impenetrable to cognitive control. (d) It originates in a dedicated neural circuitry, centered on the amygdala. Evidence supporting these propositions is reviewed from conditioning studies, both in humans and in monkeys; illusory correlation studies; studies using unreportable stimuli; and studies from animal neuroscience. The fear module is assumed to mediate an emotional level of fear learning that is relatively independent and dissociable from cognitive learning of stimulus relationships.

Vertebrate intelligence: the null hypothesis

Human cognitive capacities have so evolved that man is able to solve an extensive range of problems having very different properties. Comparative psychologists have endeavoured to throw light on the evolution and nature of this general intellectual capacity by exploring performance of non-hum an vertebrates in a variety of learning tasks, in the expectation of demonstrating superior intelligence in species more closely related to man. It has, however, proved difficult to establish that any observed difference in performance is due to a difference in intellectual capacity rather than to a difference in such contextual variables as, for example, perception or motivation. Three hypotheses that might account for the lack of experimentally demonstrable differences in intelligence amongst non-humans are discussed. The first proposes that the data currently available may have been misinterpreted: that, for example, the potential role of contextual variables has been exaggerated. According to the second hypothesis, the questions posed by comparative psychologists have been inappropriate: learning mechanisms are adaptations evolved for life in a specific ecological niche, so that mechanisms available to species from different niches are not properly comparable. It is argued that neither of these two hypotheses receives convincing empirical support. A third hypothesis proposes that there are, in fact, neither quantitative nor qualitative differences among the intellects of non-hum an vertebrates. It is argued that this null hypothesis is currently to be preferred, and that m an’s intellectual superiority may be due solely to our possession of a species-specific language-acquisition device.

Acute alcohol intoxication paired with appetitive reinforcement: effects upon ethanol intake in infant rats

A recent study suggested that infant rats process alcohol odor and/or taste during acute ethanol intoxication probably due to ethanol elimination via respiration and salivation. The present set of experiments was meant to analyze the possibility that this orosensory processing may act as a conditioned stimulus when an appetitive reinforcer is paired with the state of intoxication. In the first experiment it was observed that intragastric administration of a mildly intoxicating ethanol dose (1.5 g/kg), paired during postabsorptive time intervals with oral infusion of sucrose, was sufficient to promote a significant preference to ethanol. In Experiment 2 different doses of ethanol were either paired or explicitly unpaired with sucrose administration. The result reported in Experiment 1 was replicated and it was observed that a higher dose (3.0 g/kg) unpaired with the reinforcer resulted in alcohol aversions in terms of alcohol consumption patterns. However, when the reinforcer was paired with this dose, the aversion was inhibited. Finally, in the third experiment results indicated that preexposure to alcohol odor eliminates sucrose-conditioned alcohol preferences. These results indicate that, in physiologically immature rats, alcohol preference can be regulated by prior associative experiences involving the state of intoxication and consequences internal and/or inherent to this state.