On the convergence of projective-simulation-based reinforcement learning in Markov decision processes

Learning how to find targets in the micro-world: the case of intermittent active Brownian particles

Finding the best strategy to minimize the time needed to find a given target is a crucial task both in nature and in reaching decisive technological advances. By considering learning agents able to switch their dynamics between standard and active Brownian motion, here we focus on developing effective target-search behavioral policies for microswimmers navigating a homogeneous environment and searching for targets of unknown position. We exploit projective simulation, a reinforcement learning algorithm, to acquire an efficient stochastic policy represented by the probability of switching the phase, i.e. the navigation mode, in response to the type and the duration of the current phase. Our findings reveal that the target-search efficiency increases with the particle's self-propulsion during the active phase and that, while the optimal duration of the passive case decreases monotonically with the activity, the optimal duration of the active phase displays a non-monotonic behavior.

Computational models of stimulus equivalence: An intersection for the study of symbolic behavior

Stimulus equivalence is a central paradigm in the analysis of symbolic behavior, language, and cognition. It describes emergent relations between stimuli that were not explicitly trained and cannot be explained by primary stimulus generalization. In recent years, researchers have developed computational models to simulate the learning of equivalence relations. These models have been used to address primary theoretical and methodological issues in this field, such as exploring the underlying mechanisms that explain emergent equivalence relations and analyzing the effects of training and testing protocols on equivalence outcomes. Nonetheless, although these models build upon general learning principles, their operation is usually obscure for nonmodelers, and in the field of stimulus equivalence computational models have been developed with a variety of approaches, architectures, and algorithms that make it difficult to understand the scope and contributions of these tools. In this paper, we present the state of the art in computational modeling of stimulus equivalence. We seek to provide concise and accessible descriptions of the models' functioning and operation, highlight their main theoretical and methodological contributions, identify the existing software available for researchers to run experiments, and suggest future directions in the emergent field of computational modeling of stimulus equivalence.

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

We consider the problem of autonomous acquisition of manipulation skills where problem-solving strategies are initially available only for a narrow range of situations. We propose to extend the range of solvable situations by autonomous play with the object. By applying previously-trained skills and behaviors, the robot learns how to prepare situations for which a successful strategy is already known. The information gathered during autonomous play is additionally used to train an environment model. This model is exploited for active learning and the generation of novel preparatory behaviors compositions. We apply our approach to a wide range of different manipulation tasks, e.g., book grasping, grasping of objects of different sizes by selecting different grasping strategies, placement on shelves, and tower disassembly. We show that the composite behavior generation mechanism enables the robot to solve previously-unsolvable tasks, e.g., tower disassembly. We use success statistics gained during real-world experiments to simulate the convergence behavior of our system. Simulation experiments show that the learning speed can be improved by around 30% by using active learning.