(Reinforcement?) Learning to forage optimally

Research on re-searching: interrupted foraging is not disrupted foraging

In classic visual search, observers typically search for the presence of a target in a scene or display. In foraging tasks, there may be multiple targets in the same display (or "patch"). Observers typically search for and collect these target items in one patch until they decide to leave that patch and move to the next one. This is a highly rule-governed behavior. The current study investigated whether these rules are disrupted when the foraging is interrupted in various manners. In Experiment 1, the foraging was briefly interrupted and then resumed in the same patch. In Experiments 2 and 3, the foraging in each patch either ended voluntarily or compulsorily after a fixed amount of time. In these cases, foraging resumed in a patch only after all patches were visited. Overall, the rules of foraging remained largely intact, though Experiment 2 shows that foraging rules can be overridden by the demand characteristics of the task. The results show that participants tended to perform approximately consistently despite interruptions. The results suggest that foraging behavior in a relatively simple foraging environment is resilient and not easily disrupted by interruption.

Hierarchical control over foraging behavior by anterior cingulate cortex

Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).

Understanding patch foraging strategies across development

Patch foraging is a near-ubiquitous behaviour across the animal kingdom and characterises many decision-making domains encountered by humans. We review how a disposition to explore in adolescence may reflect the evolutionary conditions under which hunter-gatherers foraged for resources. We propose that neurocomputational mechanisms responsible for reward processing, learning, and cognitive control facilitate the transition from exploratory strategies in adolescence to exploitative strategies in adulthood - where individuals capitalise on known resources. This developmental transition may be disrupted by psychopathology, as there is emerging evidence of biases in explore/exploit choices in mental health problems. Explore/exploit choices may be an informative marker for mental health across development and future research should consider this feature of decision-making as a target for clinical intervention.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

Sex differences in patch-leaving foraging decisions in rats

The ubiquity, importance, and sophistication of foraging behavior makes it an ideal platform for studying naturalistic decision making in animals. We developed a spatial patch-foraging task for rats, in which subjects chose how long to remain in one foraging patch as the rate of food earnings steadily decreased. The cost of seeking out a new location was varied across sessions. The behavioral task was designed to mimic the structure of natural foraging problems, where distinct spatial locations are associated with different reward statistics, and decisions require navigation and movement through space. Male and female Long-Evans rats generally followed the predictions of theoretical models of foraging, albeit with a consistent tendency to persist with patches for too long compared to behavioral strategies that maximize food intake rate. The tendency to choose overly-long patch residence times was stronger in male rats. We also observed sex differences in locomotion as rats performed the task, but these differences in movement only partially accounted for the differences in patch residence durations observed between male and female rats. Together, these results suggest a nuanced relationship between movement, sex, and foraging decisions.

Learning and organization of within-session sequences by pigeons (Columba livia)

Most animals engage in complex activities that are the combination of simpler actions expressed over a period of time. The mechanisms organizing such sequential behavior have been of long-standing biological and psychological interest. Previously, we observed pigeons' anticipatory behavior with a within-session sequence involving four choice alternatives suggestive of a potential understanding of the overall order and sequence of the items within a session. In that task, each colored alternative was correct for 24 consecutive trials as presented in a predictable sequence (i.e., A first, then B, then C, then D). To test whether these four already-trained pigeons possessed a sequential and linked representation of the ABCD items, we added a second four-item sequence involving new and distinct colored choice alternatives (i.e., E first for 24 trials, then F, then G, then H) and then alternated these ABCD and EFGH sequences over successive sessions. Over three manipulations, we tested and trained trials composed of combinations of elements from both sequences. We determined that pigeons did not learn any within-sequence associations among the elements. Despite the availability and explicit utility of such sequence cues, the data suggest instead that pigeons learned the discrimination tasks as a series of temporal associations among independent elements. This absence of any sequential linkage is consistent with the hypothesis that such representations are difficult to form in pigeons. This pattern of data suggests that for repeated sequential activities in birds, and potentially other animals including humans, there are highly effective, but underappreciated, clock-like mechanisms that control the ordering of behaviors.

Never Gonna Give You Up Even When It Is Suboptimal

Previous research showed that animals adopt different foraging strategies in different environment settings. However, research on whether humans adapt their foraging strategies to the foraging environment has shown little evidence of a change in strategies. This study aims to investigate whether humans will adapt their foraging strategies when performance differences between strategies are large and why participants may fixate on a single strategy. We conducted two foraging experiments and identified the strategies used by the participants. Most participants used the Give-Up Time (GUT) strategy regardless of the environment they encountered. GUT was used even in environments where other strategies such as the Fixed-Time strategy or the Fixed-Number strategy performed better. Using computer simulations, we further examined the conditions under which the GUT strategy will perform well compared to the other strategies. We found that even though the GUT strategy is not always the best strategy, it performs consistently on a satisfactory level and had an advantage when variance in the quality of patches was introduced. The consistently good performance of the GUT strategy could thus explain participants' lack of strategy switching.

Overharvesting in human patch foraging reflects rational structure learning and adaptive planning

Patch foraging presents a sequential decision-making problem widely studied across organisms-stay with a current option or leave it in search of a better alternative? Behavioral ecology has identified an optimal strategy for these decisions, but, across species, foragers systematically deviate from it, staying too long with an option or "overharvesting" relative to this optimum. Despite the ubiquity of this behavior, the mechanism underlying it remains unclear and an object of extensive investigation. Here, we address this gap by approaching foraging as both a decision-making and learning problem. Specifically, we propose a model in which foragers 1) rationally infer the structure of their environment and 2) use their uncertainty over the inferred structure representation to adaptively discount future rewards. We find that overharvesting can emerge from this rational statistical inference and uncertainty adaptation process. In a patch-leaving task, we show that human participants adapt their foraging to the richness and dynamics of the environment in ways consistent with our model. These findings suggest that definitions of optimal foraging could be extended by considering how foragers reduce and adapt to uncertainty over representations of their environment.

Resource limitations: A taxonomy

Decision making within the context of resource limitations requires balancing the short-term benefits of obtaining a resource and the long-term consequences of depleting those resources. The present manuscript focuses on four types of tasks that share this tradeoff to develop a taxonomy that will encourage a deeper understanding of the psychological processes at play. The four types considered are foraging, common pool traps, deterioration traps, and a novel designation referred to as resource cliffs. All four will be shown to include two opposite processes - depletion of the resource and its replenishment over time. By considering the unique and shared features of these tasks, a taxonomy of features emerges that can be combined to not only create novel tasks but also to shift the research focus to task features rather than specific tasks. The paper closes with a consideration of current theoretical frameworks previously applied to one or more of these resource-limitation tasks as well as the promise of reinforcement learning as a unifying theory.

A robotic model of hippocampal reverse replay for reinforcement learning

Hippocampal reverse replay, a phenomenon in which recently active hippocampal cells reactivate in the reverse order, is thought to contribute to learning, particularly reinforcement learning (RL), in animals. Here, we present a novel computational model which exploits reverse replay to improve stability and performance on a homing task. The model takes inspiration from the hippocampal-striatal network, and learning occurs via a three-factor RL rule. To augment this model with hippocampal reverse replay, we derived a policy gradient learning rule that associates place-cell activity with responses in cells representing actions and a supervised learning rule of the same form, interpreting the replay activity as a 'target' frequency. We evaluated the model using a simulated robot spatial navigation task inspired by the Morris water maze. Results suggest that reverse replay can improve performance stability over multiple trials. Our model exploits reverse reply as an additional source for propagating information about desirable synaptic changes, reducing the requirements for long-time scales in eligibility traces combined with low learning rates. We conclude that reverse replay can positively contribute to RL, although less stable learning is possible in its absence. Analogously, we postulate that reverse replay may enhance RL in the mammalian hippocampal-striatal system rather than provide its core mechanism.

Using optimal foraging theory to infer how groups make collective decisions

Studying animal behavior as collective phenomena is a powerful tool for understanding social processes, including group coordination and decision-making. However, linking individual behavior during group decision-making to the preferences underlying those actions poses a considerable challenge. Optimal foraging theory, and specifically the marginal value theorem (MVT), can provide predictions about individual preferences, against which the behavior of groups can be compared under different models of influence. A major strength of formally linking optimal foraging theory to collective behavior is that it generates predictions that can easily be tested under field conditions. This opens the door to studying group decision-making in a range of species; a necessary step for revealing the ecological drivers and evolutionary consequences of collective decision-making.

The effect of apathy and compulsivity on planning and stopping in sequential decision-making

Real-life decision-making often comprises sequences of successive decisions about whether to take opportunities as they are encountered or keep searching for better ones instead. We investigated individual differences related to such sequential decision-making and link them especially to apathy and compulsivity in a large online sample (discovery sample: n = 449 and confirmation sample: n = 756). Our cognitive model revealed distinct changes in the way participants evaluated their environments and planned their own future behaviour. Apathy was linked to decision inertia, i.e., automatically persisting with a sequence of searches for longer than appropriate given the value of searching. Thus, despite being less motivated, they did not avoid the effort associated with longer searches. In contrast, compulsivity was linked to self-reported insensitivity to the cost of continuing with a sequence of searches. The objective measures of behavioural cost insensitivity were clearly linked to compulsivity only in the discovery sample. While the confirmation sample showed a similar effect, it did not reach significance. Nevertheless, in both samples, participants reported awareness of such bias (experienced as "overchasing"). In addition, this awareness made them report preemptively avoiding situations related to the bias. However, we found no evidence of them actually preempting more in the task, which might mean a misalignment of their metacognitive beliefs or that our behavioural measures were incomplete. In summary, individual variation in distinct, fundamental aspects of sequential decision-making can be linked to variation in 2 measures of behavioural traits associated with psychological illness in the normal population.

Abrupt, Asynchronous Changes in Action Representations by Anterior Cingulate Cortex Neurons during Trial and Error Learning

The ability to act on knowledge about the value of stimuli or actions factors into simple foraging behaviors as well as complex forms of decision-making. In striatal regions, action representations are thought to acquire value through a gradual (reinforcement-learning based) process. It is unclear whether this is also true for anterior cingulate cortex (ACC) where neuronal representations tend to change abruptly. We recorded from ensembles of ACC neurons as rats deduced which of 3 levers was rewarded each day. The rat's lever preferences changed gradually throughout the sessions as they eventually came to focus on the rewarded lever. Most individual neurons changed their responses to both rewarded and nonrewarded lever presses abruptly (<2 trials). These transitions occurred asynchronously across the population but peaked near the point where the rats began to focus on the rewarded lever. Because the individual transitions were asynchronous, the overall change at the population level appeared gradual. Abrupt transitions in action representations of ACC neurons may be part of a mechanism that alters choice strategies as new information is acquired.

Foraging optimally in social neuroscience: computations and methodological considerations

Research in social neuroscience has increasingly begun to use the tools of computational neuroscience to better understand behaviour. Such approaches have proven fruitful for probing underlying neural mechanisms. However, little attention has been paid to how the structure of experimental tasks relates to real-world decisions, and the problems that brains have evolved to solve. To go significantly beyond current understanding, we must begin to use paradigms and mathematical models from behavioural ecology, which offer insights into the decisions animals must make successfully in order to survive. One highly influential theory-marginal value theorem (MVT)-precisely characterises and provides an optimal solution to a vital foraging decision that most species must make: the patch-leaving problem. Animals must decide when to leave collecting rewards in a current patch (location) and travel somewhere else. We propose that many questions posed in social neuroscience can be approached as patch-leaving problems. A richer understanding of the neural mechanisms underlying social behaviour will be obtained by using MVT. In this 'tools of the trade' article, we outline the patch-leaving problem, the computations of MVT and discuss the application to social neuroscience. Furthermore, we consider the practical challenges and offer solutions for designing paradigms probing patch leaving, both behaviourally and when using neuroimaging techniques.

Foraging for foundations in decision neuroscience: insights from ethology

Modern decision neuroscience offers a powerful and broad account of human behaviour using computational techniques that link psychological and neuroscientific approaches to the ways that individuals can generate near-optimal choices in complex controlled environments. However, until recently, relatively little attention has been paid to the extent to which the structure of experimental environments relates to natural scenarios, and the survival problems that individuals have evolved to solve. This situation not only risks leaving decision-theoretic accounts ungrounded but also makes various aspects of the solutions, such as hard-wired or Pavlovian policies, difficult to interpret in the natural world. Here, we suggest importing concepts, paradigms and approaches from the fields of ethology and behavioural ecology, which concentrate on the contextual and functional correlates of decisions made about foraging and escape and address these lacunae.

Foraging as an evidence accumulation process

The patch-leaving problem is a canonical foraging task, in which a forager must decide to leave a current resource in search for another. Theoretical work has derived optimal strategies for when to leave a patch, and experiments have tested for conditions where animals do or do not follow an optimal strategy. Nevertheless, models of patch-leaving decisions do not consider the imperfect and noisy sampling process through which an animal gathers information, and how this process is constrained by neurobiological mechanisms. In this theoretical study, we formulate an evidence accumulation model of patch-leaving decisions where the animal averages over noisy measurements to estimate the state of the current patch and the overall environment. We solve the model for conditions where foraging decisions are optimal and equivalent to the marginal value theorem, and perform simulations to analyze deviations from optimal when these conditions are not met. By adjusting the drift rate and decision threshold, the model can represent different “strategies”, for example an incremental, decremental, or counting strategy. These strategies yield identical decisions in the limiting case but differ in how patch residence times adapt when the foraging environment is uncertain. To describe sub-optimal decisions, we introduce an energy-dependent marginal utility function that predicts longer than optimal patch residence times when food is plentiful. Our model provides a quantitative connection between ecological models of foraging behavior and evidence accumulation models of decision making. Moreover, it provides a theoretical framework for potential experiments which seek to identify neural circuits underlying patch-leaving decisions.

Foraging is a ubiquitous animal behavior, performed by organisms as different as worms, birds, rats, and humans. Although the behavior has been extensively studied, it is not known how the brain processes information obtained during foraging activity to make subsequent foraging decisions. We form an evidence accumulation model of foraging decisions that describes the process through which an animal gathers information and uses it to make foraging decisions. By building on studies of the neural decision mechanisms within systems neuroscience, this model connects the foraging decision process with ecological models of patch-leaving decisions, such as the marginal value theorem. The model suggests the existence of different foraging strategies, which optimize for different environmental conditions and their potential implementation by neural decision making circuits. The model also shows how state-dependence, such as satiation level, can affect evidence accumulation to lead to sub-optimal foraging decisions. Our model provides a framework for future experimental studies which seek to elucidate how neural decision making mechanisms have been shaped by evolutionary forces in an animal’s surrounding environment.

Foraging decisions as multi-armed bandit problems: Applying reinforcement learning algorithms to foraging data

Finding resources is crucial for animals to survive and reproduce, but the understanding of the decision-making underlying foraging decisions to explore new resources and exploit old resources remains lacking. Theory predicts an 'exploration-exploitation trade-off' where animals must balance their effort into either stay and exploit a seemingly good resource or move and explore the environment. To date, however, it has been challenging to generate flexible yet tractable statistical models that can capture this trade-off, and our understanding of foraging decisions is limited. Here, I suggest that foraging decisions can be seen as multi-armed bandit problems, and apply deterministic (i.e., the Upper-Confidence-Bound or 'UCB') and Bayesian algorithms (i.e., Thompson Sampling or 'TS') to demonstrate how these algorithms generate testable a priori predictions from simulated data. Next, I use UCB and TS to analyse empirical foraging data from the tephritid fruit fly larvae Bactrocera tryoni to provide a qualitative and quantitative framework to quantify animal foraging behaviour. Qualitative analysis revealed that TS display shorter exploration period than UCB, although both converged to similar qualitative results. Quantitative analysis demonstrated that, overall, UCB is more accurate in predicting the observed foraging patterns compared with TS, even though both algorithms failed to quantitatively estimate the empirical foraging patterns in high-density groups (i.e., groups with 50 larvae and, more strikingly, groups with 100 larvae), likely due to the influence of intraspecific competition on animal behaviour. The framework proposed here demonstrates how reinforcement learning algorithms can be used to model animal foraging decisions.

Human Foragers: Searchers by Nature and Experience

Diverse studies of human foraging have revealed behavioral strategies that may have evolved as adaptations for foraging. Here, we used an outdoor experimental search task to explore the effect of three sources of information on participants' performance: (i) information obtained directly from performing a search, (ii) information obtained prior to testing in the form of a distilled snippet of knowledge intended to experimentally simulate information acquired culturally about the environment, and (iii) information obtained from experience of foraging for natural resources for economic gain. We found that (i) immediate searching experience improved performance from the beginning to the end of the short, 2-min task, (ii) information priming improved performance notably from the very beginning of the task, and (iii) natural resource foraging experience improved performance to a lesser extent. Our results highlight the role of culturally transmitted information as well as the presence of mechanisms to rapidly integrate and implement new information into searching choices, which ultimately influence performance in a foraging task.

Revisiting foraging approaches in neuroscience

Many complex real-world decisions, such as deciding which house to buy or whether to switch jobs, involve trying to maximize reward across a sequence of choices. Optimal Foraging Theory is well suited to study these kinds of choices because it provides formal models for reward-maximization in sequential situations. In this article, we review recent insights from foraging neuroscience, behavioral ecology, and computational modelling. We find that a commonly used approach in foraging neuroscience, in which choice items are encountered at random, does not reflect the way animals direct their foraging efforts in certain real-world settings, nor does it reflect efficient reward-maximizing behavior. Based on this, we propose that task designs allowing subjects to encounter choice items strategically will further improve the ecological validity of foraging approaches used in neuroscience, as well as give rise to new behavioral and neural predictions that deepen our understanding of sequential, value-based choice.

Local field potentials in dorsal anterior cingulate sulcus reflect rewards but not travel time costs during foraging

To maximise long-term reward rates, foragers deciding when to leave a patch must compute a decision variable that reflects both the immediately available reward and the time costs associated with travelling to the next patch. Identifying the mechanisms that mediate this computation is central to understanding how brains implement foraging decisions. We previously showed that firing rates of dorsal anterior cingulate sulcus neurons incorporate both variables. This result does not provide information about whether integration of information reflected in dorsal anterior cingulate sulcus spiking activity arises locally or whether it is inherited from upstream structures. Here, we examined local field potentials gathered simultaneously with our earlier recordings. In the majority of recording sites, local field potential spectral bands - specifically theta, beta, and gamma frequency ranges - encoded immediately available rewards but not time costs. The disjunction between information contained in spiking and local field potentials can constrain models of foraging-related processing. In particular, given the proposed link between local field potentials and inputs to a brain area, it raises the possibility that local processing within dorsal anterior cingulate sulcus serves to more fully bind immediate reward and time costs into a single decision variable.

Prospection, Perseverance, and Insight in Sequential Behavior

Real-world decisions have benefits occurring only later and dependent on additional decisions taken in the interim. We investigated this in a novel decision-making task in humans (n = 76) while measuring brain activity with fMRI (n = 24). Modeling revealed that participants computed the prospective value of decisions: they planned their future behavior taking into account how their decisions might affect which states they would encounter and how they themselves might respond in these states. They considered their own likely future behavioral biases (e.g., failure to adapt to changes in prospective value) and avoided situations in which they might be prone to such biases. Three neural networks in adjacent medial frontal regions were linked to distinct components of prospective decision making: activity in dorsal anterior cingulate cortex, area 8 m/9, and perigenual anterior cingulate cortex reflected prospective value, anticipated changes in prospective value, and the degree to which prospective value influenced decisions.

State-change decisions and dorsomedial prefrontal cortex: the importance of time

Different kinds of decision making can be categorized by their differential effect on the agent’s current and future states as well as the computational challenges they pose. Here, we draw a distinction between within-state and state-change decision-making, and propose that a dedicated decision mechanism exists in dorsomedial prefrontal cortex (dmPFC) that is specialized for state-change decisions. We set out a formal framework in which state change decisions may be made on the basis of the integrated momentary reward rate, over the intended time to be spent in a state. A key feature of this framework is that reward rate is expressed as a function of continuous time. We argue that dmPFC is suited for this type of decision making partly due to its ability to track the passage of time. This proposed function of dmPFC is placed in contrast to other evaluative systems such as the orbitofrontal cortex, which is important for careful deliberation within a specific model-space or option-space and within a decision strategy.

Mice learn to avoid regret

Regret can be defined as the subjective experience of recognizing that one has made a mistake and that a better alternative could have been selected. The experience of regret is thought to carry negative utility. This typically takes two distinct forms: augmenting immediate postregret valuations to make up for losses, and augmenting long-term changes in decision-making strategies to avoid future instances of regret altogether. While the short-term changes in valuation have been studied in human psychology, economics, neuroscience, and even recently in nonhuman-primate and rodent neurophysiology, the latter long-term process has received far less attention, with no reports of regret avoidance in nonhuman decision-making paradigms. We trained 31 mice in a novel variant of the Restaurant Row economic decision-making task, in which mice make decisions of whether to spend time from a limited budget to achieve food rewards of varying costs (delays). Importantly, we tested mice longitudinally for 70 consecutive days, during which the task provided their only source of food. Thus, decision strategies were interdependent across both trials and days. We separated principal commitment decisions from secondary reevaluation decisions across space and time and found evidence for regret-like behaviors following change-of-mind decisions that corrected prior economically disadvantageous choices. Immediately following change-of-mind events, subsequent decisions appeared to make up for lost effort by altering willingness to wait, decision speed, and pellet consumption speed, consistent with past reports of regret in rodents. As mice were exposed to an increasingly reward-scarce environment, we found they adapted and refined distinct economic decision-making strategies over the course of weeks to maximize reinforcement rate. However, we also found that even without changes in reinforcement rate, mice transitioned from an early strategy rooted in foraging to a strategy rooted in deliberation and planning that prevented future regret-inducing change-of-mind episodes from occurring. These data suggest that mice are learning to avoid future regret, independent of and separate from reinforcement rate maximization.

Regret describes a unique postdecision phenomenon in which losses are realized as a fault of one’s own actions. Regret is often hypothesized to have an inherent negative utility, and humans will often incur costs so as to avoid the risk of future regret. However, current models of nonhuman decision-making are based on reward maximization hypotheses. We recently found that rats express regret behaviorally and neurophysiologically on neuroeconomic foraging tasks; however, it remains unknown whether nonhuman animals will change strategies so as to avoid regret, even in the absence of changes in the achieved rate of reinforcement. Here, we provide the first evidence that mice change strategies to avoid future regret, independent of and separate from reinforcement rate maximization. Our data suggest mice accomplish this by shifting from a foraging decision-making strategy that produces change-of-mind decisions after investment mistakes to one rooted in deliberation that learns to plan ahead.

Altering gain of the infralimbic-to-accumbens shell circuit alters economically dissociable decision-making algorithms

The nucleus accumbens shell (NAcSh) is involved in reward valuation. Excitatory projections from infralimbic cortex (IL) to NAcSh undergo synaptic remodeling in rodent models of addiction and enable the extinction of disadvantageous behaviors. However, how the strength of synaptic transmission of the IL-NAcSh circuit affects decision-making information processing and reward valuation remains unknown, particularly because these processes can conflict within a given trial and particularly given recent data suggesting that decisions arise from separable information-processing algorithms. The approach of many neuromodulation studies is to disrupt information flow during on-going behaviors; however, this limits the interpretation of endogenous encoding of computational processes. Furthermore, many studies are limited by the use of simple behavioral tests of value which are unable to dissociate neurally distinct decision-making algorithms. We optogenetically altered the strength of synaptic transmission between glutamatergic IL-NAcSh projections in mice trained on a neuroeconomic task capable of separating multiple valuation processes. We found that induction of long-term depression in these synapses produced lasting changes in foraging processes without disrupting deliberative processes. Mice displayed inflated reevaluations to stay when deciding whether to abandon continued reward-seeking investments but displayed no changes during initial commitment decisions. We also developed an ensemble-level measure of circuit-specific plasticity that revealed individual differences in foraging valuation tendencies. Our results demonstrate that alterations in projection-specific synaptic strength between the IL and the NAcSh are capable of augmenting self-control economic valuations within a particular decision-making modality and suggest that the valuation mechanisms for these multiple decision-making modalities arise from different circuits.