What do I do now? An electroencephalographic investigation of the explore/exploit dilemma

Adaptations of the balloon analog risk task for neuroimaging settings: a systematic review

Introduction

The Balloon Analog Risk Task (BART), a computerized behavioral paradigm, is one of the most common tools used to assess the risk-taking propensity of an individual. Since its initial behavioral version, the BART has been adapted to neuroimaging technique to explore brain networks of risk-taking behavior. However, while there are a variety of paradigms adapted to neuroimaging to date, no consensus has been reached on the best paradigm with the appropriate parameters to study the brain during risk-taking assessed by the BART. In this review of the literature, we aimed to identify the most appropriate BART parameters to adapt the initial paradigm to neuroimaging and increase the reliability of this tool.

Methods

A systematic review focused on the BART versions adapted to neuroimaging was performed in accordance with PRISMA guidelines.

Results

A total of 105 articles with 6,879 subjects identified from the PubMed database met the inclusion criteria. The BART was adapted in four neuroimaging techniques, mostly in functional magnetic resonance imaging or electroencephalography settings.

Discussion

First, to adapt the BART to neuroimaging, a delay was included between each trial, the total number of inflations was reduced between 12 and 30 pumps, and the number of trials was increased between 80 and 100 balloons, enabling us to respect the recording constraints of neuroimaging. Second, explicit feedback about the balloon burst limited the decisions under ambiguity associated with the first trials. Third, employing an outcome index that provides more informative measures than the standard average pump score, along with a model incorporating an exponential monotonic increase in explosion probability and a maximum explosion probability between 50 and 75%, can yield a reliable estimation of risk profile. Additionally, enhancing participant motivation can be achieved by increasing the reward in line with the risk level and implementing payment based on their performance in the BART. Although there is no universal adaptation of the BART to neuroimaging, and depending on the objectives of a study, an adjustment of parameters optimizes its evaluation and clinical utility in assessing risk-taking.

Distinct neural dynamics underlying risk and ambiguity during valued-based decision making

Uncertainty can be fractioned into risk and ambiguity psychologically and neurobiologically. However, whether and how risk and ambiguity are dissociated in terms of neural dynamics during value-based decision making remain elusive. The present event-related potential (ERP) study addressed these issues by asking participants to perform a wheel-of-fortune task either during a risky context (Experiment 1; N = 30) where outcome probability was known or during an ambiguous context (Experiment 2; N = 30) where outcome probability was unknown. Results revealed that the cue-P3 was more enhanced for risk versus ambiguity during the anticipatory phase, whereas the RewP was more increased for ambiguity than risk during the consummatory phase. Moreover, the SPN and the fb-P3 components were further modulated by the levels of risk and ambiguity, respectively. These findings demonstrate a neural dissociation between risk and ambiguity, which unfolds from the anticipatory phase to the consummatory phase.

Investigating the consistency of ERPs across threatening situations among children and adolescents

Threat sensitivity is thought to be a precursor for anxiety. Yet it remains unknown whether individuals have consistently high neural activation to different threatening situations. The current study (N = 161, M_age = 11.26, SD = 1.79) used three ERPs from different threat-related events: 1) the P3 to receiving negative feedback; 2) the ERN to making mistakes; 3) the N170 to viewing angry faces. Participants also completed self-report measures of threat sensitivity, impulsivity, and demographics. In a follow-up analysis, we also investigated whether the results replicate when using the difference score for each ERP. Youth with higher self-reported sensitivity to threats and lower self-reported impulsivity had consistently higher neural activation to threatening situations. Males also had consistently higher neural activation to threats compared with females. When using the difference score, we found that youth with higher self-reported threat sensitivity had consistently higher neural activation to threats than nonthreats. Although it is common for youth to have high neural activation during at least one threatening situation (e.g., making mistakes), only ~12% of youth have consistently high neural activation across a variety of different threats. Thus, detecting youth who are sensitive to a variety of different threats may be an important avenue to investigate to identify youth most at risk for the development of anxiety.

To learn or to gain: neural signatures of exploration in human decision-making

Individuals not only take actions to obtain immediate rewards but also to gain more information to guide future choices. An ideal exploration-exploitation balance is crucial for maximizing reward over the long run. However, the neural signatures of exploration in humans remain unclear. Using quantitative meta-analyses of functional magnetic resonance imaging experiments on exploratory behaviors, we sought to identify the concordant activity pertaining to exploration over a range of experiments. The results revealed that exploration activates concordant brain activity associated with risk (e.g., dorsal medial prefrontal cortex and anterior insula), cognitive control (e.g., dorsolateral prefrontal cortex and inferior frontal gyrus), and motor processing (e.g., premotor cortex). These stereotaxic maps of exploration may indicate that exploration is highly linked to risk processing, but is also specifically associated with regions involved in executive control processes. Although this explanation should be treated as exploratory, these findings support theories positing an important role for the prefrontal-insular-motor cortical network in exploration.

An ERP investigation of children and adolescents' sensitivity to wins and losses during a peer observation manipulation

The purpose of this ERP P3 study was to test a peer observation manipulation (being observed by a peer versus being alone) on neural markers of attention to reward (win-feedback) and punishment (loss-feedback) during the Balloon Analogue Risk Task. Participants (126 children, 53 % male, 8–10 years; 196 early adolescents, 50 % male, 11–13 years; and 121 mid-adolescents, 52 % male, 14–16 years) were assessed by age group and pubertal status. Individual differences in how participants felt about being observed by a peer, and self-report personality factors, also were examined. Findings indicated that early and mid-adolescents (and individuals in mid-puberty and late-puberty) were sensitive to peer observation as both groups showed larger neural responses to loss-feedback in the peer condition than in the alone condition. Conversely, children (and individuals in pre- and early-puberty) were unaffected by peer observation. In addition, there clearly were individual differences in how rewarding versus anxiety-provoking participants found the peer experience. Early adolescents and mid-adolescents (and individuals in mid- and late-puberty) who reported feeling more anxious about the peer observation elicited larger neural responses to loss-feedback, and individuals in mid- and late-puberty in particular reported higher worry and lower sensation-seeking scores than those who reported a positive experience.

What happens when right means wrong? The impact of conflict arising from competing feedback responses

Humans often rely on feedback to learn. Indeed, in learning the difference between feedback and an expected outcome is computed to inform future actions. Further, recent work has found that reward and feedback have a unique role in modulating conflict processing and cognitive control. However, it is still not clear how conflict, especially concerning the processing and evaluation of feedback, impacts learning. To address this, we examined the effects of feedback competition on feedback evaluation in a reinforcement learning task. Specifically, we had participants play a simple two-choice gambling game while electroencephalographic (EEG) data were recorded. On half of the experiment blocks, we reversed the meaning of performance feedback for each trial from its prepotent meaning to induce response conflict akin to the Stroop effect (e.g., '✓' meant incorrect). Behaviourally, we found that participants' accuracy was reduced as a result of incongruent feedback. Paralleling this, an analysis of our EEG revealed that incongruent feedback resulted in a reduction in amplitude of the reward positivity and the P300, components of the human event-related brain potential implicated in reward processing. Our results demonstrate the negative impact of conflict on feedback evaluation and the impact of this on subsequent performance.

Feedback processing is enhanced following exploration in continuous environments

Decision-making is typically studied by presenting participants with a small set of options. However, real-world behaviour, like foraging, often occurs in continuous environments. The degree to which human decision-making in discrete tasks generalizes to continuous tasks is questionable. For example, successful foraging comprises both exploration (learning about the environment) and exploitation (taking advantage of what is known). Although progress has been made in understanding the neural processes related to this trade-off in discrete tasks, it is currently unclear how, or whether, the same processes are involved in continuous tasks. To address this, we recorded electroencephalographic data while participants "dug for gold" by selecting locations on a map. Participants were cued beforehand that the map contained either a single patch of gold, or many patches of gold. We then used a computational model to classify participant responses as either exploitations, which were driven by previous reward locations and amounts, or explorations. Our participants were able to adjust their strategy based on reward distribution, exploring more in multi-patch environments and less in single-patch environments. We observed an enhancement of the feedback-locked P300, a neural signal previously linked to exploration in discrete tasks, which suggests the presence of a general neural system for managing the explore-exploit trade-off. Furthermore, the P300 was accompanied by an exploration-related enhancement of the late positive potential that was greatest in the multi-patch environment, suggesting a role for motivational processes during exploration.

Decision- and feedback-related brain potentials reveal risk processing mechanisms in patients with alcohol use disorder

Individuals with alcohol use disorder (AUD) are aware of the risks of alcohol abuse yet continue risky drinking. Research indicates that dysfunctional decision processes and trait variables such as impulsivity contribute to this awareness-behavior discrepancy. The present study focused on decision-related versus feedback-related processes as potential contributors to decision making in AUD by examining the relationship between decision choices and decision- and feedback-related ERP phenomena in the balloon analogue risk task (BART). N = 39 AUD and n = 35 healthy comparison participants (HC) performed the BART modified for EEG assessment. In each of 100 runs, participants made a series of choices about whether to pump up a virtual balloon, which popped pseudorandomly, ending the run. Alternatively, participants ended the run by pressing a "cash-out" button. Each pump not producing a pop provided .05 €; popping resulted in loss of the run's accumulated gain. Groups made similar choices, though AUD responded more slowly. The decision P3 200-400 ms after decision prompt (balloon) was larger in AUD than in HC, and decision P3 enhancement on high-risk trials predicted choices to pump. Feedback-related negativity (FRN) after loss (relative to cash out) feedback was smaller in AUD than in HC, suggesting indifference to negative feedback. In AUD, high impulsivity was associated with risk-modulated decision P3 but not FRN. Results indicate atypical decision- and feedback-related processes that could contribute to difficulties in engaging with daily challenges effectively.

Ready, set, explore! Event-related potentials reveal the time-course of exploratory decisions

The decision trade-off between exploiting the known and exploring the unknown has been studied using a variety of approaches and techniques. Surprisingly, electroencephalography (EEG) has been underused in this area of study, even though its high temporal resolution has the potential to reveal the time-course of exploratory decisions. We addressed this issue by recording EEG data while participants tried to win as many points as possible in a two-choice gambling task called a two-armed bandit. After using a computational model to classify responses as either exploitations or explorations, we examined event-related potentials locked to two events preceding decisions to exploit/explore: the arrival of feedback, and the subsequent appearance of the next trial's choice stimuli. In particular, we examined the feedback-locked P300 component, thought to index a phasic release of norepinephrine (a neural interrupt signal), and the reward positivity, thought to index a phasic release of dopamine (a neural prediction error signal). We observed an exploration-dependent enhancement of the P300 only, suggesting a critical role of norepinephrine (but not dopamine) in triggering decisions to explore. Similarly, we examined the N200/P300 components evoked by the appearance of the choice stimuli. In this case, exploration was characterized by an enhancement of the N200, but not P300, a result we attribute to increased response conflict. These results demonstrate the usefulness of combining computational and EEG methodologies, and suggest that exploratory decisions are preceded by two characterizing events: a feedback-locked neural interrupt signal (enhanced P300), and a choice-locked increase in response conflict (enhanced N200).

Predicting risk decisions in a modified Balloon Analogue Risk Task: Conventional and single-trial ERP analyses

Event-related potential (ERP) has the potential to reveal the temporal neurophysiological dynamics of risk decision-making, but this potential has not been fully explored in previous studies. When predicting risk decision with ERPs, most studies focus on between-trial analysis that reflects feedback learning, while within-trial analysis that could directly link option assessment with behavioral output has been largely ignored. Suitable task design is crucial for applying within-trial prediction. In this study, we used a modified version of the classic Balloon Analogue Risk Task (BART). In each trial of the task, participants made multiple rounds of decisions between a risky option (pump up the balloon) and a safe option (cash out). Behavioral results show that as the level of economic risk increased, participants were less willing to make a risky decision and also needed a longer response time to do so. In general, the ERP results showed distinct characteristics compared with previous findings based on between-trial prediction, particularly about the role of the P1 component. Specifically, both the P1 (amplitude and latency) and P3 (amplitude) components evoked by current outcomes predicted subsequent decisions. We suggest that these findings indicate the importance of selective attention (indexed by the P1) and motivational functions (indexed by the P3), which may help clarify the cognitive mechanism of risk decision-making. The theoretical significance of these findings is discussed.

A Neuroeconomic Framework for Creative Cognition

Neuroeconomics is the study of the neurobiological bases of subjective preferences and choices. We present a novel framework that synthesizes findings from the literatures on neuroeconomics and creativity to provide a neurobiological description of creative cognition. We propose that value-based decision-making processes and activity in the locus ceruleus-norepinephrine (LC-NE) neuromodulatory system underlie creative cognition, as well as the large-scale brain network dynamics shown to be associated with creativity. This reconceptualization leads to several falsifiable hypotheses that can further understanding of creativity, decision making, and brain network dynamics.

Event-Related Potentials in Relation to Risk-Taking: A Systematic Review

Event-related potentials (ERPs) have been used to investigate neural mechanisms underlying risk-related decisions over the last 16 years. We aimed to systematically evaluate associations between risk-taking and ERP components elicited during decisions and following feedback. A total of 79 articles identified from PsychINFO and PubMed databases met the inclusion criteria. Selected articles assessed early ERP components (feedback-related negativity/FRN, error-related negativity/ERN, and medial frontal negativity/MFN) and the mid-latency P3 component, all using gambling paradigms that involved selecting between choices of varying risk (e.g., Iowa Gambling Task, Balloon Analogue Risk Task, and two-choice gambling tasks). The P3 component was consistently enhanced to the selection of risky options and when positive feedback (as compared to negative feedback) was provided. Also consistently, the early negative components were found to be larger following feedback indicating monetary losses as compared to gains. In the majority of studies reviewed here, risk was conceptualized in the context of simple economical decisions in gambling tasks. As such, this narrow concept of risk might not capture the diversity of risky decisions made in other areas of everyday experience, for example, social, health, and recreational risk-related decisions. It therefore remains to be seen whether the risk-sensitivity of the ERP components reviewed here generalizes to other domains of life.

Sour Promotes Risk-Taking: An Investigation into the Effect of Taste on Risk-Taking Behaviour in Humans

Taking risks is part of everyday life. Some people actively pursue risky activities (e.g., jumping out of a plane), while others avoid any risk (e.g., people with anxiety disorders). Paradoxically, risk-taking is a primitive behaviour that may lead to a happier life by offering a sense of excitement through self-actualization. Here, we demonstrate for the first time that sour - amongst the five basic tastes (sweet, bitter, sour, salty, and umami) - promotes risk-taking. Based on a series of three experiments, we show that sour has the potential to modulate risk-taking behaviour across two countries (UK and Vietnam), across individual differences in risk-taking personality and styles of thinking (analytic versus intuitive). Modulating risk-taking can improve everyday life for a wide range of people.

Age-related characteristics of risky decision-making and progressive expectation formation

During daily encounters, it is inevitable that people take risks. Investigating the sequential processing of risk hazards involve expectation formation about outcome contingencies. The present study aimed to explore risk behavior and its neural correlates in sequences of decision making, particularly in old age, which represents a critical period regarding risk-taking propensity. The Balloon Analogue Risk Task was used in an electrophysiological setting with young and elderly age groups. During the task each additional pump on a virtual balloon increased the likelihood of a balloon burst but also increased the chance to collect more reward. Event-related potentials associated with rewarding feedback were analyzed based on the forthcoming decisions (whether to continue or to stop) in order to differentiate between states of expectation towards gain or loss. In the young, the reward positivity ERP component increased as a function of reward contingencies with the largest amplitude for rewarding feedback followed by the decision to stop. In the elderly, however, reward positivity did not reflect the effect of reward structure. Behavioral indices of risk-taking propensity suggest that the performance of the young and the elderly were dissociable only with respect to response times: The elderly was characterized by hesitation and more deliberative decision making throughout the experiment. These findings signify that sequential tracking of outcome contingencies has a key role in cost-efficient action planning and progressive expectation formation.

Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration-Exploitation Trade-Offs

Optimal behavior requires striking a balance between exploiting tried-and-true options or exploring new possibilities. Neuroimaging studies have identified different brain regions in humans where neural activity is correlated with exploratory or exploitative behavior, but it is unclear whether this activity directly implements these choices or simply reflects a byproduct of the behavior. Moreover, it remains unknown whether arbitrating between exploration and exploitation can be influenced with exogenous methods, such as brain stimulation. In our study, we addressed these questions by selectively upregulating and downregulating neuronal excitability with anodal or cathodal transcranial direct current stimulation over right frontopolar cortex during a reward-learning task. This caused participants to make slower, more exploratory or faster, more exploitative decisions, respectively. Bayesian computational modeling revealed that stimulation affected how much participants took both expected and obtained rewards into account when choosing to exploit or explore: Cathodal stimulation resulted in an increased focus on the option expected to yield the highest payout, whereas anodal stimulation led to choices that were less influenced by anticipated payoff magnitudes and were more driven by recent negative reward prediction errors. These findings suggest that exploration is triggered by a neural mechanism that is sensitive to prior less-than-expected choice outcomes and thus pushes people to seek out alternative courses of action. Together, our findings establish a parsimonious neurobiological mechanism that causes exploration and exploitation, and they provide new insights into the choice features used by this mechanism to direct decision-making.

We are more selfish than we think: The endowment effect and reward processing within the human medial-frontal cortex

Perceived ownership has been shown to impact a variety of cognitive processes: attention, memory, and--more recently--reward processing. In the present experiment we examined whether or not perceived ownership would interact with the construct of value-the relative worth of an object. Participants completed a simple gambling game in which they gambled either for themselves or for another while electroencephalographic data were recorded. In a key manipulation, gambles for oneself or for another were for either small or large rewards. We tested the hypothesis that value affects the neural response to self-gamble outcomes, but not other-gamble outcomes. Our experimental data revealed that while participants learned the correct response option for both self and other gambles, the reward positivity evoked by wins was impacted by value only when gambling for oneself. Importantly, our findings provide additional evidence for a self-ownership bias in cognitive processing and further demonstrate the insensitivity of the medial-frontal reward system to gambles for another.

Using brain potentials to understand prism adaptation: the error-related negativity and the P300

Prism adaptation (PA) is both a perceptual-motor learning task as well as a promising rehabilitation tool for visuo-spatial neglect (VSN)—a spatial attention disorder often experienced after stroke resulting in slowed and/or inaccurate motor responses to contralesional targets. During PA, individuals are exposed to prism-induced shifts of the visual-field while performing a visuo-guided reaching task. After adaptation, with goggles removed, visuomotor responding is shifted to the opposite direction of that initially induced by the prisms. This visuomotor aftereffect has been used to study visuomotor learning and adaptation and has been applied clinically to reduce VSN severity by improving motor responding to stimuli in contralesional (usually left-sided) space. In order to optimize PA's use for VSN patients, it is important to elucidate the neural and cognitive processes that alter visuomotor function during PA. In the present study, healthy young adults underwent PA while event-related potentials (ERPs) were recorded at the termination of each reach (screen-touch), then binned according to accuracy (hit vs. miss) and phase of exposure block (early, middle, late). Results show that two ERP components were evoked by screen-touch: an error-related negativity (ERN), and a P300. The ERN was consistently evoked on miss trials during adaptation, while the P300 amplitude was largest during the early phase of adaptation for both hit and miss trials. This study provides evidence of two neural signals sensitive to visual feedback during PA that may sub-serve changes in visuomotor responding. Prior ERP research suggests that the ERN reflects an error processing system in medial-frontal cortex, while the P300 is suggested to reflect a system for context updating and learning. Future research is needed to elucidate the role of these ERP components in improving visuomotor responses among individuals with VSN.