Independent and interacting value systems for reward and information in the human brain

A causal role of the right dorsolateral prefrontal cortex in random exploration

Decision to explore new options with uncertain outcomes or exploit familiar options with known outcomes is a fundamental challenge that the brain faces in almost all real-life decisions. Previous studies have shown that humans use two main explorative strategies to negotiate this explore-exploit tradeoff. Exploring for the sake of information is called directed exploration, and exploration driven by behavioral variability is known as random exploration. While previous neuroimaging studies have shown different neural correlates for these explorative strategies, including right frontopolar cortex (FPC), right dorsolateral prefrontal cortex (DLPFC), and dorsal anterior cingulate cortex (dACC), there is still a lack of causal evidence for most of these brain regions. Here, we focused on the right DLPFC, which was previously supported to be involved in exploration. Using the continuous theta burst stimulation (cTBS) and Horizon task on twenty-five healthy right-handed adult participants, we showed that inhibiting rDLPFC did not change directed exploration but selectively reduced random exploration, by increasing reward sensitivity over the average reward of each option. This suggests a causal role for rDLPFC in random exploration, and further supports dissociable neural implementations for these two explorative strategies.

Neural mechanisms of information seeking

We ubiquitously seek information to make better decisions. Particularly in the modern age, when more information is available at our fingertips than ever, the information we choose to collect determines the quality of our decisions. Decision neuroscience has long adopted empirical approaches where the information available to decision-makers is fully controlled by the researchers, leaving neural mechanisms of information seeking less understood. Although information seeking has long been studied in the context of the exploration-exploitation trade-off, recent studies have widened the scope to investigate more overt information seeking in a way distinct from other decision processes. Insights gained from these studies, accumulated over the last few years, raise the possibility that information seeking is driven by the reward system signaling the subjective value of information. In this piece, we review findings from the recent studies, highlighting the conceptual and empirical relationships between distinct literatures, and discuss future research directions necessary to establish a more comprehensive understanding of how individuals seek information as a part of value-based decision-making.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Curiosity-driven exploration: foundations in neuroscience and computational modeling

Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity.

People adaptively use information to improve their internal states and external outcomes

Information can strongly impact people's affect, their level of uncertainty and their decisions. It is assumed that people seek information with the goal of improving all three. But are they successful at achieving this goal? Answering this question is important for assessing the impact of self-driven information consumption on people's well-being. Here, over five experiments (total N = 727) we show that participants accurately predict the impact of information on their internal states (e.g., affect and cognition) and external outcomes (e.g., material rewards), and use these predictions to guide information-seeking choices. A model incorporating participants' subjective expectations regarding the impact of information on their affective, cognitive, and material outcomes accounted for information-seeking choices better than a model that included only objective proxies of those measures. This model also accounted for individual differences in information-seeking choices. By balancing considerations of the impact of information on affective, cognitive and material outcomes when seeking knowledge, participants became happier, more certain and made better decisions when they sought information relative to when they did not, suggesting that the actual consequences of receiving information aligned with their subjective expectations.