Reward Learning over Weeks Versus Minutes Increases the Neural Representation of Value in the Human Brain

Reward timescale controls the rate of behavioural and dopaminergic learning

Learning the causes of rewards is necessary for survival. Thus, it is critical to understand the mechanisms of such a vital biological process. Cue-reward learning is controlled by mesolimbic dopamine and improves with spacing of cue-reward pairings. However, whether a mathematical rule governs such improvements in learning rate, and if so, whether a unifying mechanism captures this rule and dopamine dynamics during learning remain unknown. Here, we investigate the behavioral, algorithmic, and dopaminergic mechanisms governing cue-reward learning rate. Across a range of conditions in mice, we show a strong, mathematically proportional relationship between both behavioral and dopaminergic learning rates and the duration between rewards. Due to this relationship, removing up to 19 out of 20 cue-reward pairings over a fixed duration has no influence on overall learning. These findings are explained by a dopamine-based model of retrospective learning, thereby providing a unified account of the biological mechanisms of learning.

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

A novel free-operant framework enables experimental habit induction in humans

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

Causal learning with delays up to 21 hours

Considerable delays between causes and effects are commonly found in real life. However, previous studies have only investigated how well people can learn probabilistic relations with delays on the order of seconds. In the current study we tested whether people can learn a cause-effect relation with delays of 0, 3, 9, or 21 hours, and the study lasted 16 days. We found that learning was slowed with longer delays, but by the end of 16 days participants had learned the cause-effect relation in all four conditions, and they had learned the relation about equally well in all four conditions. This suggests that in real-world situations people may still be fairly accurate at inferring cause-effect relations with delays if they have enough experience. We also discuss ways that delays may interact with other real-world factors that could complicate learning.

Naturalistic reinforcement learning

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

Cortical and subcortical substrates of minutes and days-long object value memory in humans

Obtaining valuable objects motivates many of our daily decisions. However, the neural underpinnings of object processing based on human value memory are not yet fully understood. Here, we used whole-brain functional magnetic resonance imaging (fMRI) to examine activations due to value memory as participants passively viewed objects before, minutes after, and 1-70 days following value training. Significant value memory for objects was evident in the behavioral performance, which nevertheless faded over the days following training. Minutes after training, the occipital, ventral temporal, interparietal, and frontal areas showed strong value discrimination. Days after training, activation in the frontal, temporal, and occipital regions decreased, whereas the parietal areas showed sustained activation. In addition, days-long value responses emerged in certain subcortical regions, including the caudate, ventral striatum, and thalamus. Resting-state analysis revealed that these subcortical areas were functionally connected. Furthermore, the activation in the striatal cluster was positively correlated with participants' performance in days-long value memory. These findings shed light on the neural basis of value memory in humans with implications for object habit formation and cross-species comparisons.

Promoting Empathy and Affiliation in Relationships (PEAR) study: protocol for a longitudinal study investigating the development of early childhood callous-unemotional traits

INTRODUCTION

Children with callous-unemotional (CU) traits are at high lifetime risk of antisocial behaviour. Low affiliation (ie, social bonding difficulties) and fearlessness (ie, low threat sensitivity) are proposed risk factors for CU traits. Parenting practices (eg, harshness and low warmth) also predict risk for CU traits. However, few studies in early childhood have identified attentional or physiological markers of low affiliation and fearlessness. Moreover, no studies have tested whether parenting practices are underpinned by low affiliation or fearlessness shared by parents, which could further shape parent-child interactions and exacerbate risk for CU traits. Addressing these questions will inform knowledge of how CU traits develop and isolate novel parent and child targets for future specialised treatments for CU traits.

METHODS AND ANALYSIS

The Promoting Empathy and Affiliation in Relationships (PEAR) study aims to establish risk factors for CU traits in children aged 3-6 years. The PEAR study will recruit 500 parent-child dyads from two metropolitan areas of the USA. Parents and children will complete questionnaires, computer tasks and observational assessments, alongside collection of eye-tracking and physiological data, when children are aged 3-4 (time 1) and 5-6 (time 2) years. The moderating roles of child sex, race and ethnicity, family and neighbourhood disadvantage, and parental psychopathology will also be assessed. Study aims will be addressed using structural equation modelling, which will allow for flexible characterisation of low affiliation, fearlessness and parenting practices as risk factors for CU traits across multiple domains.

ETHICS AND DISSEMINATION

Ethical approval was granted by Boston University (#6158E) and the University of Pennsylvania (#850638). Results will be disseminated through conferences and open-access publications. All study and task materials will be made freely available on lab websites and through the Open Science Framework (OSF).

Functional Organization of Auditory and Reward Systems in Aging

The intrinsic organization of functional brain networks is known to change with age, and is affected by perceptual input and task conditions. Here, we compare functional activity and connectivity during music listening and rest between younger (n = 24) and older (n = 24) adults, using whole-brain regression, seed-based connectivity, and ROI-ROI connectivity analyses. As expected, activity and connectivity of auditory and reward networks scaled with liking during music listening in both groups. Younger adults show higher within-network connectivity of auditory and reward regions as compared with older adults, both at rest and during music listening, but this age-related difference at rest was reduced during music listening, especially in individuals who self-report high musical reward. Furthermore, younger adults showed higher functional connectivity between auditory network and medial prefrontal cortex that was specific to music listening, whereas older adults showed a more globally diffuse pattern of connectivity, including higher connectivity between auditory regions and bilateral lingual and inferior frontal gyri. Finally, connectivity between auditory and reward regions was higher when listening to music selected by the participant. These results highlight the roles of aging and reward sensitivity on auditory and reward networks. Results may inform the design of music-based interventions for older adults and improve our understanding of functional network dynamics of the brain at rest and during a cognitively engaging task.

Distinct replay signatures for prospective decision-making and memory preservation

Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.

Anesthesia Resistant Memories in Drosophila, a Working Perspective

Memories are lasting representations over time of associations between stimuli or events. In general, the relatively slow consolidation of memories requires protein synthesis with a known exception being the so-called Anesthesia Resistant Memory (ARM) in Drosophila. This protein synthesis-independent memory type survives amnestic shocks after a short, sensitive window post training, and can also emerge after repeated cycles of training in a negatively reinforced olfactory conditioning task, without rest between cycles (massed conditioning—MC). We discussed operational and molecular mechanisms that mediate ARM and differentiate it from protein synthesis-dependent long-term memory (LTM) in Drosophila. Based on the notion that ARM is unlikely to specifically characterize Drosophila, we examined protein synthesis and MC-elicited memories in other species and based on intraspecies shared molecular components and proposed potential relationships of ARM with established memory types in Drosophila and vertebrates.

Spaced training enhances equine learning performance

This field experiment examined whether the well-documented benefit of spaced over massed training for humans and other animals generalizes to horses. Twenty-nine randomly selected horses (Equus ferus caballus) repeatedly encountered a novel obstacle-crossing task while under saddle. Horses were randomly assigned to the spaced-training condition (2 min work, 2 min rest, 2 min work, 2 min rest) or the massed-training condition (4 min work, 4 min rest). Total training time per session and total rest per session were held constant. Days between sessions (M = 3) were held as consistent as possible given the constraints of conducting research on a working ranch and safety-threatening weather conditions. During each training session, the same hypothesis-naïve rider shaped horses to cross a novel obstacle. Fifteen of 16 horses in the spaced-training condition reached performance criterion (94% success) while only 5 of 13 horses in the massed-training condition reached performance criterion (39% success). Horses in the spaced-training condition also initiated their first obstacle-crossing faster than horses in the massed-training condition and were faster at completing eight crossings than horses in the massed-training condition. Overall, task acquisition was higher for horses undergoing spaced training despite both groups experiencing the same total work and rest time per session. These findings generalize the learning-performance benefit observed in human spaced practice to horses and offer applied benefit to equine training.

Mixing memory and desire: How memory reactivation supports deliberative decision-making

Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.

Minimal impact of consolidation on learned switch-readiness

Adaptive behavior is characterized by our ability to create, maintain, and update (or switch) rules by which we categorize and respond to stimuli across changing contexts (cognitive flexibility). Recent research suggests that people can link the control process of task-switching to contextual cues through associative learning, whereby the behavioral cost of switching is reduced for contexts that require frequent switching. One example is the listwide proportion switch (LWPS) effect, denoting smaller switch costs in blocks of trials where switching is more frequent. However, the conditions that govern such learned cognitive flexibility are poorly understood. One major unanswered question is whether this type of learning benefits from memory consolidation effects. To address this question, we manipulated whether task-sets and/or specific task stimuli were more frequently linked with task-switching (vs. repeating), and ran participants over two experimental sessions, separated by a 24-hr delay. We expected that consolidation would facilitate learned cognitive flexibility, resulting in a greater reduction of switch costs with increasing task-switch likelihood on Session 2 compared with Session 1. Across two experiments, we observed robust LWPS effects in both sessions. However, we found little evidence for effects of consolidation on learned cognitive flexibility: The magnitude of the LWPS effect did not change from Session 1 to 2. Altogether our results suggest that people reliably and quickly acquire task-set and stimulus-based switch associations, but this form of control learning-unlike many instances of reward-based learning-does not benefit from long-term memory consolidation. Possible reasons for these findings are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Reward learning and working memory: Effects of massed versus spaced training and post-learning delay period

Neuroscience research has illuminated the mechanisms supporting learning from reward feedback, demonstrating a critical role for the striatum and midbrain dopamine system. However, in humans, short-term working memory that is dependent on frontal and parietal cortices can also play an important role, particularly in commonly used paradigms in which learning is relatively condensed in time. Given the growing use of reward-based learning tasks in translational studies in computational psychiatry, it is important to understand the extent of the influence of working memory and also how core gradual learning mechanisms can be better isolated. In our experiments, we manipulated the spacing between repetitions along with a post-learning delay preceding a test phase. We found that learning was slower for stimuli repeated after a long delay (spaced-trained) compared to those repeated immediately (massed-trained), likely reflecting the remaining contribution of feedback learning mechanisms when working memory is not available. For massed learning, brief interruptions led to drops in subsequent performance, and individual differences in working memory capacity positively correlated with overall performance. Interestingly, when tested after a delay period but not immediately, relative preferences decayed in the massed condition and increased in the spaced condition. Our results provide additional support for a large role of working memory in reward-based learning in temporally condensed designs. We suggest that spacing training within or between sessions is a promising approach to better isolate and understand mechanisms supporting gradual reward-based learning, with particular importance for understanding potential learning dysfunctions in addiction and psychiatric disorders.

Memory consolidation as an adaptive process

We rely on our long-term memories to guide future behaviors, making it adaptive to prioritize the retention of goal-relevant, salient information in memory. In this review, we discuss findings from rodent and human research to demonstrate that active processes during post-encoding consolidation support the selective stabilization of recent experience into adaptive, long-term memories. Building upon literatures focused on dynamics at the cellular level, we highlight that consolidation also transforms memories at the systems level to support future goal-relevant behavior, resulting in more generalized memory traces in the brain and behavior. We synthesize previous literatures spanning animal research, human cognitive neuroscience, and cognitive psychology to propose an integrative framework for adaptive consolidation by which goal-relevant memoranda are "tagged" for subsequent consolidation, resulting in selective transformations to the structure of memories that support flexible, goal-relevant behaviors.

Value network engagement and effects of memory-related processing during encoding and retrieval of value

Decision makers rely on episodic memory to calculate choice values in everyday life, yet it is unclear how neural mechanisms of valuation differ when value-related information is encoded versus retrieved from episodic memory. The current fMRI study compared neural correlates of value while information was encoded versus retrieved from memory. Scanned tasks were followed by a behavioral episodic memory test for item-attribute associations. Our analyses sought to (i) identify neural correlates of value that were distinct and common across encoding and retrieval, and (ii) determine whether neural mechanisms of valuation and episodic memory interact. The study yielded three primary findings. First, value-related activation in the fronto-striatal reward circuit and posterior parietal cortex was comparable across valuation phases. Second, value-related activation in select fronto-parietal and salience regions was significantly greater at value retrieval than encoding. Third, there was no interaction between neural correlates of valuation and episodic memory. Taken with prior research, the present study indicates that fronto-parietal and salience regions play a key role in retrieval-dependent valuation and context-specific effects likely determine whether neural correlates of value interact with episodic memory.

Retention of Value Representations Across Time in People With Schizophrenia and Healthy Control Subjects

BACKGROUND

The current study aimed to further etiological understanding of the psychological mechanisms underlying negative symptoms in people with schizophrenia. Specifically, we tested whether negative symptom severity is associated with reduced retention of reward-related information over time and thus a degraded ability to utilize such information to guide future action selection.

METHODS

Forty-four patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy control volunteers performed a probabilistic reinforcement-learning task involving stimulus pairs in which choices resulted in reward or in loss avoidance. Following training, participants indicated their valuation of learned stimuli in a test/transfer phase. The test/transfer phase was administered immediately following training and 1 week later. Percent retention was defined as accuracy at week-long delay divided by accuracy at immediate delay.

RESULTS

Healthy control subjects and people with schizophrenia showed similarly robust retention of reinforcement learning over a 1-week delay interval. However, in the schizophrenia group, negative symptom severity was associated with reduced retention of information regarding the value of actions across a week-long interval. This pattern was particularly notable for stimuli associated with reward compared with loss avoidance.

CONCLUSIONS

Our results show that although individuals with schizophrenia may initially learn about rewarding aspects of their environment, such learning decays at a more rapid rate in patients with severe negative symptoms. Thus, previously learned reward-related information may be more difficult to access to guide future decision making and to motivate action selection.

Overcoming Pavlovian bias in semantic space

Action is invigorated in the presence of reward-predicting stimuli and inhibited in the presence of punishment-predicting stimuli. Although valuable as a heuristic, this Pavlovian bias can also lead to maladaptive behaviour and is implicated in addiction. Here we explore whether Pavlovian bias can be overcome through training. Across five experiments, we find that Pavlovian bias is resistant to unlearning under most task configurations. However, we demonstrate that when subjects engage in instrumental learning in a verbal semantic space, as opposed to a motoric space, not only do they exhibit the typical Pavlovian bias, but this Pavlovian bias diminishes with training. Our results suggest that learning within the semantic space is necessary, but not sufficient, for subjects to unlearn their Pavlovian bias, and that other task features, such as gamification and spaced stimulus presentation may also be necessary. In summary, we show that Pavlovian bias, whilst robust, is susceptible to change with experience, but only under specific environmental conditions.

Neural Systems for Memory-based Value Judgment and Decision-making

Real-life choices often require that we draw inferences about the value of options based on structured, schematic knowledge about their utility for our current goals. Other times, value information may be retrieved directly from a specific prior experience with an option. In an fMRI experiment, we investigated the neural systems involved in retrieving and assessing information from different memory sources to support value-based choice. Participants completed a task in which items could be conferred positive or negative value based on schematic associations (i.e., schema value) or learned directly from experience via deterministic feedback (i.e., experienced value). We found that ventromedial pFC (vmPFC) activity correlated with the influence of both experience- and schema-based values on participants' decisions. Connectivity between the vmPFC and middle temporal cortex also tracked the inferred value of items based on schematic associations on the first presentation of ingredients, before any feedback. In contrast, the striatum responded to participants' willingness to bet on ingredients as a function of the unsigned strength of their memory for those options' values. These results argue that the striatum and vmPFC play distinct roles in memory-based value judgment and decision-making. Specifically, the vmPFC assesses the value of options based on information inferred from schematic knowledge and retrieved from prior direct experience, whereas the striatum controls a decision to act on options based on memory strength.

A Theoretical Framework for How We Learn Aesthetic Values

How do we come to like the things that we do? Each one of us starts from a relatively similar state at birth, yet we end up with vastly different sets of aesthetic preferences. These preferences go on to define us both as individuals and as members of our cultures. Therefore, it is important to understand how aesthetic preferences form over our lifetimes. This poses a challenging problem: to understand this process, one must account for the many factors at play in the formation of aesthetic values and how these factors influence each other over time. A general framework based on basic neuroscientific principles that can also account for this process is needed. Here, we present such a framework and illustrate it through a model that accounts for the trajectories of aesthetic values over time. Our framework is inspired by meta-analytic data of neuroimaging studies of aesthetic appraisal. This framework incorporates effects of sensory inputs, rewards, and motivational states. Crucially, each one of these effects is probabilistic. We model their interactions under a reinforcement-learning circuitry. Simulations of this model and mathematical analysis of the framework lead to three main findings. First, different people may develop distinct weighing of aesthetic variables because of individual variability in motivation. Second, individuals from different cultures and environments may develop different aesthetic values because of unique sensory inputs and social rewards. Third, because learning is stochastic, stemming from probabilistic sensory inputs, motivations, and rewards, aesthetic values vary in time. These three theoretical findings account for different lines of empirical research. Through our study, we hope to provide a general and unifying framework for understanding the various aspects involved in the formation of aesthetic values over time.

Effective brain connectivity at rest is associated with choice-induced preference formation

Preferences can change as a consequence of making hard decisions whereby the value of chosen options increases and the value of rejected options decreases. Such choice-induced preference changes have been associated with brain areas detecting choice conflict (anterior cingulate cortex, ACC), updating stimulus value (dorsolateral prefrontal cortex, dlPFC) and supporting memory of stimulus value (hippocampus and ventromedial prefrontal cortex, vmPFC). Here we investigated whether resting-state neuronal activity within these regions is associated with the magnitude of individuals' preference updates. We fitted a dynamic causal model (DCM) to resting-state neuronal activity in the spectral domain (spDCM) and estimated the causal connectivity among core regions involved in preference formation following hard choices. The extent of individuals' choice-induced preference changes were found to be associated with a diminished resting-state excitation between the left dlPFC and the vmPFC, whereas preference consistency was related to a higher resting-state excitation from the ACC to the left hippocampus and vmPFC. Our results point to a model of preference formation during which the dynamic network configurations between left dlPFC, ACC, vmPFC and left hippocampus at rest are linked to preference change or stability.

Previously Reward-Associated Stimuli Capture Spatial Attention in the Absence of Changes in the Corresponding Sensory Representations as Measured with MEG

Studies of selective attention typically consider the role of task goals or physical salience, but attention can also be captured by previously reward-associated stimuli, even if they are currently task irrelevant. One theory underlying this value-driven attentional capture (VDAC) is that reward-associated stimulus representations undergo plasticity in sensory cortex, thereby automatically capturing attention during early processing. To test this, we used magnetoencephalography to probe whether stimulus location and identity representations in sensory cortex are modulated by reward learning. We furthermore investigated the time course of these neural effects, and their relationship to behavioral VDAC. Male and female human participants first learned stimulus-reward associations. Next, we measured VDAC in a separate task by presenting these stimuli in the absence of reward contingency and probing their effects on the processing of separate target stimuli presented at different time lags. Using time-resolved multivariate pattern analysis, we found that learned value modulated the spatial selection of previously rewarded stimuli in posterior visual and parietal cortex from ∼260 ms after stimulus onset. This value modulation was related to the strength of participants' behavioral VDAC effect and persisted into subsequent target processing. Importantly, learned value did not influence cortical signatures of early processing (i.e., earlier than ∼200 ms); nor did it influence the decodability of stimulus identity. Our results suggest that VDAC is underpinned by learned value signals that modulate spatial selection throughout posterior visual and parietal cortex. We further suggest that VDAC can occur in the absence of changes in early visual processing in cortex.SIGNIFICANCE STATEMENT Attention is our ability to focus on relevant information at the expense of irrelevant information. It can be affected by previously learned but currently irrelevant stimulus-reward associations, a phenomenon termed "value-driven attentional capture" (VDAC). The neural mechanisms underlying VDAC remain unclear. It has been speculated that reward learning induces visual cortical plasticity, which modulates early visual processing to capture attention. Although we find that learned value modulates spatial signals in visual cortical areas, an effect that correlates with VDAC, we find no relevant signatures of changes in early visual processing in cortex.

Role of the hippocampus in the spacing effect during memory retrieval

It is well known that distributed learning (DL) leads to improved memory performance compared with massed learning (ML) (i.e., spacing effect). However, the extent to which the hippocampus is involved in the spacing effect at shorter and longer retention intervals remains unclear. To address this issue, two groups of participants were asked to encode face-scene pairs at 20-min, 1-day, and 1-month intervals before they were scanned using fMRI during an associative recognition task. The pairs were repeated six times in either a massed (i.e., six times in 1 day) or a distributed (i.e., six times over 3 days, twice per day) manner. The results showed that compared with that in the ML group, the activation of the left hippocampus was stronger in the DL group when the participants retrieved old pairs correctly and rejected new pairs correctly at different retention intervals. In addition, the posterior hippocampus was more strongly activated when the new associations were rejected correctly after DL than ML, especially at the 1-month interval. Hence, our results provide evidence that the hippocampus is involved in better memory performance after DL compared to ML at both shorter and longer retention intervals.

Ten simple rules for the computational modeling of behavioral data

Computational modeling of behavior has revolutionized psychology and neuroscience. By fitting models to experimental data we can probe the algorithms underlying behavior, find neural correlates of computational variables and better understand the effects of drugs, illness and interventions. But with great power comes great responsibility. Here, we offer ten simple rules to ensure that computational modeling is used with care and yields meaningful insights. In particular, we present a beginner-friendly, pragmatic and details-oriented introduction on how to relate models to data. What, exactly, can a model tell us about the mind? To answer this, we apply our rules to the simplest modeling techniques most accessible to beginning modelers and illustrate them with examples and code available online. However, most rules apply to more advanced techniques. Our hope is that by following our guidelines, researchers will avoid many pitfalls and unleash the power of computational modeling on their own data.

Learning of distant state predictions by the orbitofrontal cortex in humans

Representations of our future environment are essential for planning and decision making. Previous research in humans has demonstrated that the hippocampus is a critical region for forming and retrieving associations, while the medial orbitofrontal cortex (OFC) is an important region for representing information about recent states. However, it is not clear how the brain acquires predictive representations during goal-directed learning. Here, we show using fMRI that while participants learned to find rewards in multiple different Y-maze environments, hippocampal activity was highest during initial exposure and then decayed across the remaining repetitions of each maze, consistent with a role in rapid encoding. Importantly, multivariate patterns in the OFC-VPFC came to represent predictive information about upcoming states approximately 30 s in the future. Our findings provide a mechanism by which the brain can build models of the world that span long-timescales to make predictions.