The discovery of novel actions is affected by very brief reinforcement delays and reinforcement modality

Exploration biases forelimb reaching strategies

The brain can generate actions, such as reaching to a target, using different movement strategies. We investigate how such strategies are learned in a task where perched head-fixed mice learn to reach to an invisible target area from a set start position using a joystick. This can be achieved by learning to move in a specific direction or to a specific endpoint location. As mice learn to reach the target, they refine their variable joystick trajectories into controlled reaches, which depend on the sensorimotor cortex. We show that individual mice learned strategies biased to either direction- or endpoint-based movements. This endpoint/direction bias correlates with spatial directional variability with which the workspace was explored during training. Model-free reinforcement learning agents can generate both strategies with similar correlation between variability during training and learning bias. These results provide evidence that reinforcement of individual exploratory behavior during training biases the reaching strategies that mice learn.

Contiguity and overshadowing interactions in the rapid-streaming procedure

When multiple cues are associated with the same outcome, organisms tend to select between the cues, with one revealing greater behavioral control at the expense of the others (i.e., cue competition). However, non-human and human studies have not always observed this competition, creating a puzzling scenario in which the interaction between cues can result in competition, no interaction, or facilitation as a function of several learning parameters. In five experiments, we assessed whether temporal contiguity and overshadowing effects are reliably observed in the streamed-trial procedure, and whether there was an interaction between them. We anticipated that weakening temporal contiguity (ranging from 500 to 1,000 ms) should attenuate competition. Using within-subject designs, participants experienced independent series of rapid streams in which they had to learn the relationship between visual cues (presented either alone or with another cue) and an outcome, with the cue-outcome pairings being presented with either a delay or trace relationship. Across experiments, we observed overshadowing (Experiments 1, 2, 4, and 5) and temporal contiguity effects (Experiments 2, 3, and 4). Despite the frequent occurrence of both effects, we did not find that trace conditioning abolished competition between cues. Overall, these results suggest that the extent to which contiguity determines cue interactions depends on multiple variables, some of which we address in the General discussion.

Exploration biases how forelimb reaches to a spatial target are learned

The brain can learn to generate actions, such as reaching to a target, using different movement strategies. Understanding how different variables bias which strategies are learned to produce such a reach is important for our understanding of the neural bases of movement. Here we introduce a novel spatial forelimb target task in which perched head-fixed mice learn to reach to a circular target area from a set start position using a joystick. These reaches can be achieved by learning to move into a specific direction or to a specific endpoint location. We find that mice gradually learn to successfully reach the covert target. With time, they refine their initially exploratory complex joystick trajectories into controlled targeted reaches. The execution of these controlled reaches depends on the sensorimotor cortex. Using a probe test with shifting start positions, we show that individual mice learned to use strategies biased to either direction or endpoint-based movements. The degree of endpoint learning bias was correlated with the spatial directional variability with which the workspace was explored early in training. Furthermore, we demonstrate that reinforcement learning model agents exhibit a similar correlation between directional variability during training and learned strategy. These results provide evidence that individual exploratory behavior during training biases the control strategies that mice use to perform forelimb covert target reaches.

A Mobile App Delivering a Gamified Battery of Cognitive Tests Designed for Repeated Play (OU Brainwave): App Design and Cohort Study

BACKGROUND

Mobile phone and tablet apps are an increasingly common platform for collecting data. A key challenge for researchers has been participant "buy-in" and attrition for designs requiring repeated testing.

OBJECTIVE

The objective of this study was to develop and assess the utility of 1-2 minute versions of both classic and novel cognitive tasks using a user-focused and user-driven mobile phone and tablet app designed to encourage repeated play.

METHODS

A large sample of app users (N=13,979 at first data collection) participated in multiple, self-paced sessions of classic working memory (N-back), spatial cognition (mental rotation), sustained attentional focus (persistent vigilance task), and split attention (multiple object tracking) tasks, along with the implementation of a comparatively novel action-learning task. The "OU Brainwave" app was designed to measure time-of-day variation in cognitive performance and did not offer any training program or promise any cognitive enhancement. To record participants' chronotype, a full Morningness-Eveningness questionnaire was also included, which measures whether a person's circadian rhythm produces peak alertness in the morning, in the evening, or in between. Data were collected during an 18-month period. While the app prompted re-engagement at set intervals, participants were free to complete each task as many times as they wished.

RESULTS

We found a significant relationship between morningness and age (r=.298, n=12,755, P<.001), with no effect of gender (t_13,539=-1.036, P=.30). We report good task adherence, with ~4000 participants repeatedly playing each game >4 times each-our minimum engagement level for analysis. Repeated plays of these games allowed us to replicate commonly reported gender effects in gamified spatial cognition (F_1,4216=154.861, P<.001, η²_ρ=.035), split attention (F_1,4185=11.047, P=.001, η²_ρ=.003), and sustained attentional focus (F_1,4238=15.993, P<.001, η²_ρ=.004) tasks. We also report evidence of a small gender effect in an action-learning task (F_1,3988=90.59, P<.001, η²_ρ=.022). Finally, we found a strong negative effect of self-reported age on performance, when controlling for number of plays, in sustained attentional focus (n=1596, F_6,1595=30.23, P<.001, η²=.102), working memory (n=1627, F_6,1626=19.78, P<.001, η²=.068), spatial cognition (n=1640, F_6,1639=23.74, P<.001, η²=.080), and split attention tasks (n=1616, F_6,1615=2.48, P=.02, η²=.009).

CONCLUSIONS

Using extremely short testing periods and permitting participants to decide their level of engagement-both in terms of which gamified task they played and how many sessions they completed-we were able to collect a substantial and valid dataset. We suggest that the success of OU Brainwave should inform future research oriented apps-particularly in issues of balancing participant engagement with data fidelity.

Infants with complex congenital heart diseases show poor short-term memory in the mobile paradigm at 3 months of age

The purpose of this study was to examine learning, short-term memory and general development including cognitive, motor, and language domains in infants with Complex Congenital Heart Defects (CCDH). Ten infants with CCHD (4 males, 6 females) and 14 infants with typical development (TD) were examined at 3 months of age. The mobile paradigm, where an infant's leg is tethered to an overhead mobile, was used to evaluate learning and short-term memory. The Bayley Scales of Infant Development 3rd edition (Bayley-III) was used to evaluate general development in cognitive, motor, and language domains. Infants with CCHD and infants with TD both showed learning with significant increase in kicking rate (p<0.001) across periods of the mobile paradigm, but only infants with TD demonstrated short-term memory (p=0.017) in the mobile paradigm. There were no differences on cognitive, motor, and language development between infants with CCHD and infants with TD on the Bayley-III. Early assessment is necessary to guide targeted treatment in infants with CCHD. One-time assessment may fail to detect potential cognitive impairments during early infancy in infants with CCHD. Supportive intervention programs for infants with CCHD that focuses on enhancing short-term memory are recommended.

No learning where to go without first knowing where you're coming from: action discovery is trajectory, not endpoint based

Intrinsic motivations drive an agent to explore, providing essential data for linking behaviors with novel outcomes and so laying the foundation for future flexible action. We present experiments using a new behavioral task which allows us to interrogate the connection between exploration and action learning. Human participants used a joystick to search repeatedly for a target location, only receiving feedback on successful discovery. Feedback delay was manipulated, as was the starting position. Experiment 1 employed stable starting positions, so the task could be learnt with respect to a target location or a target trajectory. Participants were able to learn the correct movement under all delay conditions. Experiment 2 used a variable starting location, so the correct movement could only be learnt in terms of target location. Participants displayed little to no learning in this experiment. These results suggest that movements on this scale are stored as trajectories rather than in terms of target location. Overall the experiments demonstrate the potential of this task for uncovering the native representational substrates of action learning.