A computational theory for the learning of equivalence relations

Computational models of stimulus equivalence: An intersection for the study of symbolic behavior

Stimulus equivalence is a central paradigm in the analysis of symbolic behavior, language, and cognition. It describes emergent relations between stimuli that were not explicitly trained and cannot be explained by primary stimulus generalization. In recent years, researchers have developed computational models to simulate the learning of equivalence relations. These models have been used to address primary theoretical and methodological issues in this field, such as exploring the underlying mechanisms that explain emergent equivalence relations and analyzing the effects of training and testing protocols on equivalence outcomes. Nonetheless, although these models build upon general learning principles, their operation is usually obscure for nonmodelers, and in the field of stimulus equivalence computational models have been developed with a variety of approaches, architectures, and algorithms that make it difficult to understand the scope and contributions of these tools. In this paper, we present the state of the art in computational modeling of stimulus equivalence. We seek to provide concise and accessible descriptions of the models' functioning and operation, highlight their main theoretical and methodological contributions, identify the existing software available for researchers to run experiments, and suggest future directions in the emergent field of computational modeling of stimulus equivalence.

Enhanced Equivalence Projective Simulation: A Framework for Modeling Formation of Stimulus Equivalence Classes

Formation of stimulus equivalence classes has been recently modeled through equivalence projective simulation (EPS), a modified version of a projective simulation (PS) learning agent. PS is endowed with an episodic memory that resembles the internal representation in the brain and the concept of cognitive maps. PS flexibility and interpretability enable the EPS model and, consequently the model we explore in this letter, to simulate a broad range of behaviors in matching-to-sample experiments. The episodic memory, the basis for agent decision making, is formed during the training phase. Derived relations in the EPS model that are not trained directly but can be established via the network's connections are computed on demand during the test phase trials by likelihood reasoning. In this letter, we investigate the formation of derived relations in the EPS model using network enhancement (NE), an iterative diffusion process, that yields an offline approach to the agent decision making at the testing phase. The NE process is applied after the training phase to denoise the memory network so that derived relations are formed in the memory network and retrieved during the testing phase. During the NE phase, indirect relations are enhanced, and the structure of episodic memory changes. This approach can also be interpreted as the agent's replay after the training phase, which is in line with recent findings in behavioral and neuroscience studies. In comparison with EPS, our model is able to model the formation of derived relations and other features such as the nodal effect in a more intrinsic manner. Decision making in the test phase is not an ad hoc computational method, but rather a retrieval and update process of the cached relations from the memory network based on the test trial. In order to study the role of parameters on agent performance, the proposed model is simulated and the results discussed through various experimental settings.

Equivalence Projective Simulation as a Framework for Modeling Formation of Stimulus Equivalence Classes

Stimulus equivalence (SE) and projective simulation (PS) study complex behavior, the former in human subjects and the latter in artificial agents. We apply the PS learning framework for modeling the formation of equivalence classes. For this purpose, we first modify the PS model to accommodate imitating the emergence of equivalence relations. Later, we formulate the SE formation through the matching-to-sample (MTS) procedure. The proposed version of PS model, called the equivalence projective simulation (EPS) model, is able to act within a varying action set and derive new relations without receiving feedback from the environment. To the best of our knowledge, it is the first time that the field of equivalence theory in behavior analysis has been linked to an artificial agent in a machine learning context. This model has many advantages over existing neural network models. Briefly, our EPS model is not a black box model, but rather a model with the capability of easy interpretation and flexibility for further modifications. To validate the model, some experimental results performed by prominent behavior analysts are simulated. The results confirm that the EPS model is able to reliably simulate and replicate the same behavior as real experiments in various settings, including formation of equivalence relations in typical participants, nonformation of equivalence relations in language-disabled children, and nodal effect in a linear series with nodal distance five. Moreover, through a hypothetical experiment, we discuss the possibility of applying EPS in further equivalence theory research.

A Neurocomputational Approach to Trained and Transitive Relations in Equivalence Classes

A stimulus class can be composed of perceptually different but functionally equivalent stimuli. The relations between the stimuli that are grouped in a class can be learned or derived from other stimulus relations. If stimulus A is equivalent to B, and B is equivalent to C, then the equivalence between A and C can be derived without explicit training. In this work we propose, with a neurocomputational model, a basic learning mechanism for the formation of equivalence. We also describe how the relatedness between the members of an equivalence class is developed for both trained and derived stimulus relations. Three classic studies on stimulus equivalence are simulated covering typical and atypical populations as well as nodal distance effects. This model shows a mechanism by which certain stimulus associations are selectively strengthened even when they are not co-presented in the environment. This model links the field of equivalence classes to accounts of Hebbian learning and categorization, and points to the pertinence of modeling stimulus equivalence to explore the effect of variations in training protocols.

Dopamine bioavailability in the mPFC modulates operant learning performance in rats: an experimental study with a computational interpretation

Dopamine encodes reward and its prediction in reinforcement learning. Catechol-O-methyltransferase (COMT) activity in the medial prefrontal cortex (mPFC) has been shown to influence cognitive abilities by modifying dopamine clearance. Nevertheless, it is unknown how COMT in the mPFC influences operant learning. Systemic entacapone (50mg/kg), as well as local entacapone (3 pg) and recombinant COMT (17 μg) in the mPFC were administered to male Long Evans rats prior to training in an operant conditioning task. We found that systemic and local administration of the COMT inhibitor entacapone significantly improves learning performance. Conversely, recombinant COMT administration totally impaired learning. These data have been interpreted through a computational model where the phasic firing of dopaminergic neurons was computed by means of a temporal difference algorithm and dopamine bioavailability in the mPFC was simulated with a gating window. The duration of this window was selected to simulate the effects of inhibited or enhanced COMT activity (by entacapone or recombinant COMT respectively). The model accounts for an improved performance reproducing the entacapone effects, and a detrimental impact on learning when the clearance is increased reproducing the recombinant COMT effects. The experimental and computational results show that learning performance can be deeply influenced by COMT manipulations in the mPFC.