Maximizing Electrochemical Information: A Perspective on Background-Inclusive Fast Voltammetry

This perspective encompasses a focused review of the literature leading to a tipping point in electroanalytical chemistry. We tie together the threads of a "revolution" quietly in the making for years through the work of many authors. Long-held misconceptions about the use of background subtraction in fast voltammetry are addressed. We lay out future advantages that accompany background-inclusive voltammetry, particularly when paired with modern machine-learning algorithms for data analysis.

Dopamine and serotonin in human substantia nigra track social context and value signals during economic exchange

Dopamine and serotonin are hypothesized to guide social behaviours. In humans, however, we have not yet been able to study neuromodulator dynamics as social interaction unfolds. Here, we obtained subsecond estimates of dopamine and serotonin from human substantia nigra pars reticulata during the ultimatum game. Participants, who were patients with Parkinson's disease undergoing awake brain surgery, had to accept or reject monetary offers of varying fairness from human and computer players. They rejected more offers in the human than the computer condition, an effect of social context associated with higher overall levels of dopamine but not serotonin. Regardless of the social context, relative changes in dopamine tracked trial-by-trial changes in offer value-akin to reward prediction errors-whereas serotonin tracked the current offer value. These results show that dopamine and serotonin fluctuations in one of the basal ganglia's main output structures reflect distinct social context and value signals.

Subjective feelings associated with expectations and rewards during risky decision-making in impulse control disorder

Impulse Control Disorder (ICD) in Parkinson's disease is a behavioral addiction induced by dopaminergic therapies, but otherwise unclear etiology. The current study investigates the interaction of reward processing variables, dopaminergic therapy, and risky decision-making and subjective feelings in patients with versus without ICD. Patients with (n = 18) and without (n = 12) ICD performed a risky decision-making task both 'on' and 'off' standard-of-care dopaminergic therapies (the task was performed on 2 different days with the order of on and off visits randomized for each patient). During each trial of the task, participants choose between two options, a gamble or a certain reward, and reported how they felt about decision outcomes. Subjective feelings of 'pleasure' are differentially driven by expectations of possible outcomes in patients with, versus without ICD. While off medication, the influence of expectations about risky-decisions on subjective feelings is reduced in patients with ICD versus without ICD. While on medication, the influence of expected outcomes in patients with ICD versus without ICD becomes similar. Computational modeling of behavior supports the idea that latent decision-making factors drive subjective feelings in patients with Parkinson's disease and that ICD status is associated with a change in the relationship between factors associated with risky behavior and subjective feelings about the experienced outcomes. Our results also suggest that dopaminergic medications modulate the impact expectations have on the participants' subjective reports. Altogether our results suggest that expectations about risky decisions may be decoupled from subjective feelings in patients with ICD, and that dopaminergic medications may reengage these circuits and increase emotional reactivity in patients with ICD.

Sub-second and multi-second dopamine dynamics underlie variability in human time perception

Timing behaviour and the perception of time are fundamental to cognitive and emotional processes in humans. In non-human model organisms, the neuromodulator dopamine has been associated with variations in timing behaviour, but the connection between variations in dopamine levels and the human experience of time has not been directly assessed. Here, we report how dopamine levels in human striatum, measured with sub-second temporal resolution during awake deep brain stimulation surgery, relate to participants' perceptual judgements of time intervals. Fast, phasic, dopaminergic signals were associated with underestimation of temporal intervals, whereas slower, tonic, decreases in dopamine were associated with poorer temporal precision. Our findings suggest a delicate and complex role for the dynamics and tone of dopaminergic signals in the conscious experience of time in humans.

Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

In the mammalian brain, midbrain dopamine neuron activity is hypothesized to encode reward prediction errors that promote learning and guide behavior by causing rapid changes in dopamine levels in target brain regions. This hypothesis (and alternatives regarding dopamine's role in punishment-learning) has limited direct evidence in humans. We report intracranial, subsecond measurements of dopamine release in human striatum measured, while volunteers (i.e., patients undergoing deep brain stimulation surgery) performed a probabilistic reward and punishment learning choice task designed to test whether dopamine release encodes only reward prediction errors or whether dopamine release may also encode adaptive punishment learning signals. Results demonstrate that extracellular dopamine levels can encode both reward and punishment prediction errors within distinct time intervals via independent valence-specific pathways in the human brain.

Noradrenaline tracks emotional modulation of attention in human amygdala

The noradrenaline (NA) system is one of the brain's major neuromodulatory systems; it originates in a small midbrain nucleus, the locus coeruleus (LC), and projects widely throughout the brain.1,2 The LC-NA system is believed to regulate arousal and attention3,4 and is a pharmacological target in multiple clinical conditions.5,6,7 Yet our understanding of its role in health and disease has been impeded by a lack of direct recordings in humans. Here, we address this problem by showing that electrochemical estimates of sub-second NA dynamics can be obtained using clinical depth electrodes implanted for epilepsy monitoring. We made these recordings in the amygdala, an evolutionarily ancient structure that supports emotional processing8,9 and receives dense LC-NA projections,10 while patients (n = 3) performed a visual affective oddball task. The task was designed to induce different cognitive states, with the oddball stimuli involving emotionally evocative images,11 which varied in terms of arousal (low versus high) and valence (negative versus positive). Consistent with theory, the NA estimates tracked the emotional modulation of attention, with a stronger oddball response in a high-arousal state. Parallel estimates of pupil dilation, a common behavioral proxy for LC-NA activity,12 supported a hypothesis that pupil-NA coupling changes with cognitive state,13,14 with the pupil and NA estimates being positively correlated for oddball stimuli in a high-arousal but not a low-arousal state. Our study provides proof of concept that neuromodulator monitoring is now possible using depth electrodes in standard clinical use.

Sub-second Dopamine Signals during Risky Decision-Making in Patients with Impulse Control Disorder

Background

Impulse Control Disorder (ICD) in Parkinson's disease is a behavioral addiction arising secondary to dopaminergic therapies, most often dopamine receptor agonists. Prior research implicates changes in striatal function and heightened dopaminergic activity in the dorsal striatum of patients with ICD. However, this prior work does not possess the temporal resolution required to investigate dopaminergic signaling during real-time progression through various stages of decision-making involving anticipation and feedback.

Methods

We recorded high-frequency (10Hz) measurements of extracellular dopamine in the striatum of patients with (N=3) and without (N=3) a history of ICD secondary to dopamine receptor agonist therapy for Parkinson's disease symptoms. These measurements were made using carbon fiber microelectrodes during awake DBS neurosurgery and while participants performed a sequential decision-making task involving risky investment decisions and real monetary gains and losses. Per clinical standard-of-care, participants withheld all dopaminergic medications prior to the procedure.

Results

Patients with ICD invested significantly more money than patients without ICD. On each trial, patients with ICD made smaller adjustments to their investment levels compared to patients without ICD. In patients with ICD, dopamine levels rose or fell on sub-second timescales in anticipation of investment outcomes consistent with increased or decreased confidence in a positive outcome, respectively; dopamine levels in patients without ICD were significantly more stable during this phase. After outcome revelation, dopamine levels in patients with ICD rose significantly more than in inpatients without ICD for better-than-expected gains. For worse-than-expected losses, dopamine levels in patients with ICD remained level whereas dopamine levels in patients without ICD fell.

Conclusion

We report significantly increased risky behavior and exacerbated phasic dopamine signaling, on sub-second timescales, anticipating and following the revelation of the outcomes of risky decisions in patients with ICD. Notably, these results were obtained when patients who had demonstrated ICD in the past but were, at the time of surgery, in an off-medication state. Thus, it is unclear whether observed signals reflect an inherent predisposition for ICD that was revealed when dopamine receptor agonists were introduced or whether these observations were caused by the introduction of dopamine receptor agonists and the patients having experienced ICD symptoms in the past. Regardless, future work investigating dopamine's role in human cognition, behavior, and disease should consider the signals this system generates on sub-second timescales.

Time perception reflects individual differences in motor and non-motor symptoms of Parkinson's disease

Decreasing dopaminergic function is at the core of Parkinson's disease (PD) motor symptoms and changes in dopaminergic action are associated with many comorbid non-motor symptoms in PD. Notably, dopaminergic signaling in the striatum has been shown to play a critical role in the perception of time. We hypothesize that patients with PD perceive time differently and in accordance with their specific comorbid non-motor symptoms and clinical state. This means that individual differences in clinical symptoms may be reflected in individual differences in timing behavior. To test this hypothesis, we recruited patients with PD and compared individual differences in patients' clinical state with their ability to judge intervals of time ranging from 500 ms to 1100 ms while on and off their prescribed dopaminergic medications. We show that medication state (on vs. off medications) did not affect timing behavior, but individual differences in timing behavior are able to predict individual differences in comorbid non-motor symptoms, duration of PD diagnosis, and prescribed dopaminergic medications. We show that comorbid impulse control disorder is associated with temporal overestimation; depression is associated with decreased temporal accuracy; and increased PD duration and prescribed levodopa monotherapy are associated with reduced temporal precision and accuracy. Observed differences in time perception are consistent with hypothesized dopaminergic mechanisms thought to underlie the respective motor and non-motor symptoms in PD. In future work, time perception tasks may augment clinical diagnosis strategies, or help disentangle the neural and cognitive mechanisms underlying PD motor and non-motor symptom etiology.

Valence-partitioned learning signals drive choice behavior and phenomenal subjective experience in humans

How the human brain generates conscious phenomenal experience is a fundamental problem. In particular, it is unknown how variable and dynamic changes in subjective affect are driven by interactions with objective phenomena. We hypothesize a neurocomputational mechanism that generates valence-specific learning signals associated with 'what it is like' to be rewarded or punished. Our hypothesized model maintains a partition between appetitive and aversive information while generating independent and parallel reward and punishment learning signals. This valence-partitioned reinforcement learning (VPRL) model and its associated learning signals are shown to predict dynamic changes in 1) human choice behavior, 2) phenomenal subjective experience, and 3) BOLD-imaging responses that implicate a network of regions that process appetitive and aversive information that converge on the ventral striatum and ventromedial prefrontal cortex during moments of introspection. Our results demonstrate the utility of valence-partitioned reinforcement learning as a neurocomputational basis for investigating mechanisms that may drive conscious experience.

Time perception reflects individual differences in motor and non-motor symptoms of Parkinson's disease

Dopaminergic signaling in the striatum has been shown to play a critical role in the perception of time. Decreasing striatal dopamine efficacy is at the core of Parkinson's disease (PD) motor symptoms and changes in dopaminergic action have been associated with many comorbid non-motor symptoms in PD. We hypothesize that patients with PD perceive time differently and in accordance with their specific comorbid non-motor symptoms and clinical state. We recruited patients with PD and compared individual differences in patients' clinical features with their ability to judge millisecond to second intervals of time (500ms-1100ms) while on or off their prescribed dopaminergic medications. We show that individual differences in comorbid non-motor symptoms, PD duration, and prescribed dopaminergic pharmacotherapeutics account for individual differences in time perception performance. We report that comorbid impulse control disorder is associated with temporal overestimation; depression is associated with decreased temporal accuracy; and PD disease duration and prescribed levodopa monotherapy are associated with reduced temporal precision and accuracy. Observed differences in time perception are consistent with hypothesized dopaminergic mechanisms thought to underlie the respective motor and non-motor symptoms in PD, but also raise questions about specific dopaminergic mechanisms. In future work, time perception tasks like the one used here, may provide translational or reverse translational utility in investigations aimed at disentangling neural and cognitive systems underlying PD symptom etiology.

One Sentence Summary

Quantitative characterization of time perception behavior reflects individual differences in Parkinson's disease motor and non-motor symptom clinical presentation that are consistent with hypothesized neural and cognitive mechanisms.

Intracranial subsecond dopamine measurements during a "sure bet or gamble" decision-making task in patients with alcohol use disorder suggest diminished dopaminergic signals about relief

OBJECTIVE

To the authors' knowledge, no data have been reported on dopamine fluctuations on subsecond timescales in humans with alcohol use disorder (AUD). In this study, dopamine release was monitored in 2 patients with and 2 without a history of AUD during a "sure bet or gamble" (SBORG) decision-making task to begin to characterize how subsecond dopamine responses to counterfactual information, related to psychological notions of regret and relief, in AUD may be altered.

METHODS

Measurements of extracellular dopamine levels were made once every 100 msec using human voltammetric methods. Measurements were made in the caudate during deep brain stimulation electrode implantation surgeries (for treatment of movement disorders) in patients who did (AUD, n = 2) or did not (non-AUD, n = 2) have a history of AUD. Participants performed an SBORG decision-making task in which they made choices between sure bets and 50%-chance monetary gamble outcomes.

RESULTS

Fast changes were found in dopamine levels that appear to be modulated by "what could have been" and by patients' AUD status. Positive counterfactual prediction errors (related to relief) differentiated patients with versus without a history of AUD.

CONCLUSIONS

Dopaminergic encoding of counterfactual information appears to differ between patients with and without AUD. The current study has a major limitation of a limited sample size, but these data provide a rare insight into dopaminergic physiology during real-time decision-making in humans with an addiction disorder. The authors hope future work will expand the sample size and determine the generalizability of the current results.

Hierarchical resampling for bagging in multistudy prediction with applications to human neurochemical sensing

We propose the "study strap ensemble", which combines advantages of two common approaches to fitting prediction models when multiple training datasets ("studies") are available: pooling studies and fitting one model versus averaging predictions from multiple models each fit to individual studies. The study strap ensemble fits models to bootstrapped datasets, or "pseudo-studies." These are generated by resampling from multiple studies with a hierarchical resampling scheme that generalizes the randomized cluster bootstrap. The study strap is controlled by a tuning parameter that determines the proportion of observations to draw from each study. When the parameter is set to its lowest value, each pseudo-study is resampled from only a single study. When it is high, the study strap ignores the multi-study structure and generates pseudo-studies by merging the datasets and drawing observations like a standard bootstrap. We empirically show the optimal tuning value often lies in between, and prove that special cases of the study strap draw the merged dataset and the set of original studies as pseudo-studies. We extend the study strap approach with an ensemble weighting scheme that utilizes information in the distribution of the covariates of the test dataset. Our work is motivated by neuroscience experiments using real-time neurochemical sensing during awake behavior in humans. Current techniques to perform this kind of research require measurements from an electrode placed in the brain during awake neurosurgery and rely on prediction models to estimate neurotransmitter concentrations from the electrical measurements recorded by the electrode. These models are trained by combining multiple datasets that are collected in vitro under heterogeneous conditions in order to promote accuracy of the models when applied to data collected in the brain. A prevailing challenge is deciding how to combine studies or ensemble models trained on different studies to enhance model generalizability. Our methods produce marked improvements in simulations and in this application. All methods are available in the studyStrap CRAN package.

Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

Sub-second Dopamine and Serotonin Signaling in Human Striatum during Perceptual Decision-Making

Recent animal research indicates that dopamine and serotonin, neuromodulators traditionally linked to appetitive and aversive processes, are also involved in sensory inference and decisions based on such inference. We tested this hypothesis in humans by monitoring sub-second striatal dopamine and serotonin signaling during a visual motion discrimination task that separates sensory uncertainty from decision difficulty in a factorial design. Caudate nucleus recordings (n = 4) revealed multi-scale encoding: in three participants, serotonin tracked sensory uncertainty, and, in one participant, both dopamine and serotonin tracked deviations from expected trial transitions within our factorial design. Putamen recordings (n = 1) supported a cognition-action separation between caudate nucleus and putamen—a striatal sub-division unique to primates—with both dopamine and serotonin tracking decision times. These first-of-their-kind observations in the human brain reveal a role for sub-second dopamine and serotonin signaling in non-reward-based aspects of cognition and action.

•

Dopamine and serotonin are measured in human striatum during awake decision-making

•

Serotonin tracks sensory uncertainty in caudate nucleus

•

Dopamine and serotonin track sensory statistics in caudate nucleus

•

Dopamine and serotonin track decision times in putamen

Diminished single-stimulus response in vmPFC to favorite people in children diagnosed with Autism Spectrum Disorder

From an early age, individuals with autism spectrum disorder (ASD) spend less time engaged in social interaction compared to typically developing peers (TD). One reason behind this behavior may be that the brains of children diagnosed with ASD do not attribute enough value to potential social exchanges as compared to the brains of typically developing children; thus, potential social exchanges are avoided because other environmental stimuli are more highly valued by default. Neurobiological investigations into the mechanisms underlying value-based decision-making has shown that the ventral medial prefrontal cortex (vmPFC) is critical for encoding the expected outcome value of different actions corresponding to distinct environmental cues. Here, we tested the hypothesis that the responsiveness of the vmPFC in children diagnosed with ASD (compared to TD controls) is diminished for visual cues that represent highly valued social interaction. Using a passive picture viewing task and functional magnetic resonance imaging (fMRI) we measured the response of an a priori defined region of interest in the vmPFC in children diagnosed with ASD and an age-matched TD cohort. We show that the average response of the vmPFC is significantly diminished in the ASD group. Further, we demonstrate that a single-stimulus and less than 30 s of fMRI data are sufficient to differentiate the ASD and TD cohorts. These findings are consistent with the hypothesis that the brains of children with ASD do not encode the value of social exchange in the same manner as TD children. The latter finding suggests the possibility of utilizing single-stimulus fMRI as a potential biologically based diagnostic tool to augment traditional clinical approaches.

Computational Underpinnings of Neuromodulation in Humans

We summarize a new approach to neuromodulator detection that provides colocalized detection of dopamine, serotonin, and norepinephrine at subsecond timescales and promises to provide submillisecond estimates of the same. The methodology, elastic net electrochemistry, is used to estimate dopamine and serotonin in the striatum of conscious human subjects during active decision-making. We show a proof-of-principle example of the same method working on commercially available depth electrodes in common use for epilepsy monitoring and neurosurgical planning in humans, which further promises to make such electrodes sources of fast neuromodulator information never before available in human subjects. We discuss the implications of this methodology for making direct tests in humans of the computations carried by these three important neuromodulatory systems. The methods also promise great utility in model organisms, but this chapter focuses on the possibilities for human use.

The Protective Action Encoding of Serotonin Transients in the Human Brain

The role of serotonin in human brain function remains elusive due, at least in part, to our inability to measure rapidly the local concentration of this neurotransmitter. We used fast-scan cyclic voltammetry to infer serotonergic signaling from the striatum of 14 brains of human patients with Parkinson's disease. Here we report these novel measurements and show that they correlate with outcomes and decisions in a sequential investment game. We find that serotonergic concentrations transiently increase as a whole following negative reward prediction errors, while reversing when counterfactual losses predominate. This provides initial evidence that the serotonergic system acts as an opponent to dopamine signaling, as anticipated by theoretical models. Serotonin transients on one trial were also associated with actions on the next trial in a manner that correlated with decreased exposure to poor outcomes. Thus, the fluctuations observed for serotonin appear to correlate with the inhibition of over-reactions and promote persistence of ongoing strategies in the face of short-term environmental changes. Together these findings elucidate a role for serotonin in the striatum, suggesting it encodes a protective action strategy that mitigates risk and modulates choice selection particularly following negative environmental events.

BOLD and its connection to dopamine release in human striatum: a cross-cohort comparison

Activity in midbrain dopamine neurons modulates the release of dopamine in terminal structures including the striatum, and controls reward-dependent valuation and choice. This fluctuating release of dopamine is thought to encode reward prediction error (RPE) signals and other value-related information crucial to decision-making, and such models have been used to track prediction error signals in the striatum as encoded by BOLD signals. However, until recently there have been no comparisons of BOLD responses and dopamine responses except for one clear correlation of these two signals in rodents. No such comparisons have been made in humans. Here, we report on the connection between the RPE-related BOLD signal recorded in one group of subjects carrying out an investment task, and the corresponding dopamine signal recorded directly using fast-scan cyclic voltammetry in a separate group of Parkinson's disease patients undergoing DBS surgery while performing the same task. The data display some correspondence between the signal types; however, there is not a one-to-one relationship. Further work is necessary to quantify the relationship between dopamine release, the BOLD signal and the computational models that have guided our understanding of both at the level of the striatum.

This article is part of the themed issue ‘Interpreting BOLD: a dialogue between cognitive and cellular neuroscience’.

An efficiency framework for valence processing systems inspired by soft cross-wiring

•

Serotonin has been proposed as an opponent to dopamine.

•

This review explores positive and negative value pathways for structuring this opponency.

•

The positive and negative pathways co-mingle through transmitter cross-loading.

•

Cross-loading is proposed as a way to tile ‘valence space.’

Recent experiments suggest that subsecond dopamine delivery to human striatum encodes a combination of reward prediction errors and counterfactual errors thus composing the actual with the possible into one neurochemical signal. Here, we present a model where the counterfactual part of these striatal dopamine fluctuations originates in another valuation system that shadows the dopamine system by acting as its near-antipode in terms of spike-rate encoding yet co-releases dopamine alongside its own native neurotransmitter. We show that such a hypothesis engenders important representational consequences where valence processing appears subject to the efficient encoding considerations common to the visual and auditory systems. This new perspective opens up important computational consequences for understanding how value-predicting information should integrate with sensory processing streams.

Economic probes of mental function and the extraction of computational phenotypes

Economic games are now routinely used to characterize human cognition across multiple dimensions. These games allow for effective computational modeling of mental function because they typically come equipped with notions of optimal play, which provide quantitatively prescribed target functions that can be tracked throughout an experiment. The combination of these games, computational models, and neuroimaging tools open up the possibility for new ways to characterize normal cognition and associated brain function. We propose that these tools may also be used to characterize mental dysfunction, such as that found in a range of psychiatric illnesses. We describe early efforts using a multi-round trust game to probe brain responses associated with healthy social exchange and review how this game has provided a novel and useful characterization of autism spectrum disorder. Lastly, we use the multi-round trust game as an example to discuss how these kinds of games could produce novel bases for representing healthy behavior and brain function and thus provide objectively identifiable subtypes within a broad spectrum of mental function.

A computational approach to "free will" constrained by the games we play

Human choice is not free—we are bounded by a multitude of biological constraints. Yet, within the various landscapes we face, we do express choice, preference, and varying degrees of so-called willful behavior. Moreover, it appears that the capacity for choice in humans is variable. Empirical studies aimed at investigating the experience of “free will” will benefit from theoretical disciplines that constrain the language used to frame the relevant issues. The combination of game theory and computational reinforcement learning theory with empirical methods is already beginning to provide valuable insight into the biological variables underlying capacity for choice in humans and how things may go awry in individuals with brain disorders. These disciplines operate within abstract quantitative landscapes, but have successfully been applied to investigate strategic and adaptive human choice guided by formal notions of optimal behavior. Psychiatric illness is an extreme, but interesting arena for studying human capacity for choice. The experiences and behaviors of patients suggest these individuals fundamentally suffer from a diminished capacity of willful choice. Herein, I will briefly discuss recent applications of computationally guided approaches to human choice behavior and the underlying neurobiology. These approaches can be integrated into empirical investigation at multiple temporal scales of analysis including the growing body of experiments in human functional magnetic resonance imaging (fMRI), and newly emerging sub-second electrochemical and electrophysiological measurements in the human brain. These cross-disciplinary approaches hold promise for revealing the underlying neurobiological mechanisms for the variety of choice capacity in humans.

Imaging models of valuation during social interaction in humans

The role of dopamine neurons in value-guided behavior has been described in computationally explicit terms. These developments have motivated new model-based probes of reward processing in healthy humans, and in recent years these same models have also been used to design and understand neural responses during simple social exchange. These latter applications have opened up the possibility of identifying new endophenotypes characteristic of biological substrates underlying psychiatric disease. In this report, we review model-based approaches to functional magnetic resonance imaging in healthy individuals and the application of these paradigms to psychiatric disorders. We show early results from the application of model-based human interaction at three disparate levels: 1) interaction with a single human, 2) interaction within small groups, and 3) interaction with signals generated by large groups. In each case, we show how reward-prediction circuitry is engaged by abstract elements of each paradigm with blood oxygen level-dependent imaging as a read-out; and, in the last case (i.e., signals generated by large groups) we report on direct electrochemical dopamine measurements during decision making in humans. Lastly, we discuss how computational approaches can be used to objectively assess and quantify elements of complex and hidden social decision-making processes.

New approaches to investigating social gestures in autism spectrum disorder

The combination of economic games and human neuroimaging presents the possibility of using economic probes to identify biomarkers for quantitative features of healthy and diseased cognition. These probes span a range of important cognitive functions, but one new use is in the domain of reciprocating social exchange with other humans - a capacity perturbed in a number of psychopathologies. We summarize the use of a reciprocating exchange game to elicit neural and behavioral signatures for subjects diagnosed with autism spectrum disorder (ASD). Furthermore, we outline early efforts to capture features of social exchange in computational models and use these to identify quantitative behavioral differences between subjects with ASD and matched controls. Lastly, we summarize a number of subsequent studies inspired by the modeling results, which suggest new neural and behavioral signatures that could be used to characterize subtle deficits in information processing during interactions with other humans.

Implicit signals in small group settings and their impact on the expression of cognitive capacity and associated brain responses

Measures of intelligence, when broadcast, serve as salient signals of social status, which may be used to unjustly reinforce low-status stereotypes about out-groups' cultural norms. Herein, we investigate neurobehavioural signals manifest in small (n = 5) groups using functional magnetic resonance imaging and a ‘ranked group IQ task’ where implicit signals of social status are broadcast and differentiate individuals based on their expression of cognitive capacity. We report an initial overall decrease in the expression of cognitive capacity in the small group setting. However, the environment of the ‘ranked group IQ task’ eventually stratifies the population into two groups (‘high performers’, HP and ‘low performers’, LP) identifiable based on changes in estimated intelligence quotient and brain responses in the amygdala and dorsolateral prefrontal cortex. In addition, we demonstrate signals in the nucleus accumbens consistent with prediction errors in expected changes in status regardless of group membership. Our results suggest that individuals express diminished cognitive capacity in small groups, an effect that is exacerbated by perceived lower status within the group and correlated with specific neurobehavioural responses. The impact these reactions have on intergroup divisions and conflict resolution requires further investigation, but suggests that low-status groups may develop diminished capacity to mitigate conflict using non-violent means.

Sub-second dopamine detection in human striatum

Fast-scan cyclic voltammetry at carbon fiber microelectrodes allows rapid (sub-second) measurements of dopamine release in behaving animals. Herein, we report the modification of existing technology and demonstrate the feasibility of making sub-second measurements of dopamine release in the caudate nucleus of a human subject during brain surgery. First, we describe the modification of our electrodes that allow for measurements to be made in a human brain. Next, we demonstrate in vitro and in vivo, that our modified electrodes can measure stimulated dopamine release in a rat brain equivalently to previously determined rodent electrodes. Finally, we demonstrate acute measurements of dopamine release in the caudate of a human patient during DBS electrode implantation surgery. The data generated are highly amenable for future work investigating the relationship between dopamine levels and important decision variables in human decision-making tasks.

Neuroeconomic approaches to mental disorders

The pervasiveness of decision-making in every area of human endeavor highlights the importance of understanding choice mechanisms and their detailed relationship to underlying neurobiological function. This review surveys the recent and productive application of game-theoretic probes (economic games) to mental disorders. Such games typically possess concrete concepts of optimal play, thus providing quantitative ways to track when subjects' choices match or deviate from optimal. This feature equips economic games with natural classes of control signals that should guide learning and choice in the agents that play them. These signals and their underlying physical correlates in the brain are now being used to generate objective biomarkers that may prove useful for exposing and understanding the neurogenetic basis of normal and pathological human cognition. Thus, game-theoretic probes represent some of the first steps toward producing computationally principled, objective measures of cognitive function and dysfunction useful for the diagnosis, treatment, and understanding of mental disorders.

Sources and targets of reactive oxygen species in synaptic plasticity and memory

Increasing evidence suggests that reactive oxygen species (ROS), such as superoxide and hydrogen peroxide, act as necessary signaling molecules in processes underlying cognition. Moreover, ROS have been shown to be necessary in molecular process underlying signal transduction, synaptic plasticity, and memory formation. Research from several laboratories suggests that NADPH oxidase is an important source of superoxide in the brain. Evidence is presented here to show that ROS are in fact important signaling molecules involved in synaptic plasticity and memory formation. Moreover, evidence that the NADPH oxidase complex is a key regulator of ROS generation in synaptic plasticity and memory formation is discussed. Understanding redox signaling in the brain, including the sources and molecular targets of ROS, are important for a full understanding of the signaling pathways that underlie synaptic plasticity and memory. Knowledge of ROS function in the brain also is critical for understanding aging and neurodegenerative diseases of the brain given that several of these disorders, including Alzheimer's disease and Parkinson disease, may be exacerbated by the unregulated generation of ROS.

Synaptic plasticity deficits and mild memory impairments in mouse models of chronic granulomatous disease

Reactive oxygen species (ROS) are required in a number of critical cellular signaling events, including those underlying hippocampal synaptic plasticity and hippocampus-dependent memory; however, the source of ROS is unknown. We previously have shown that NADPH oxidase is required for N-methyl-D-aspartate (NMDA) receptor-dependent signal transduction in the hippocampus, suggesting that NADPH oxidase may be required for NMDA receptor-dependent long-term potentiation (LTP) and hippocampus-dependent memory. Herein we present the first evidence that NADPH oxidase is involved in hippocampal synaptic plasticity and memory. We have found that pharmacological inhibitors of NADPH oxidase block LTP. Moreover, mice that lack the NADPH oxidase proteins gp91(phox) and p47(phox), both of which are mouse models of human chronic granulomatous disease (CGD), also lack LTP. We also found that the gp91(phox) and p47(phox) mutant mice have mild impairments in hippocampus-dependent memory. The gp91(phox) mutant mice exhibited a spatial memory deficit in the Morris water maze, and the p47(phox) mutant mice exhibited impaired context-dependent fear memory. Taken together, our results are consistent with NADPH oxidase being required for hippocampal synaptic plasticity and memory and are consistent with reports of cognitive dysfunction in patients with CGD.

NADPH oxidase is required for NMDA receptor-dependent activation of ERK in hippocampal area CA1

Previous studies have shown that N-methyl-D-aspartate (NMDA) receptor activation results in production of reactive oxygen species (ROS) and activation of extracellular signal-regulated kinase (ERK) in hippocampal area CA1. In addition, application of ROS to hippocampal slices has been shown to result in activation of ERK in area CA1. To determine whether these events were linked causally, we investigated whether ROS are required for NMDA receptor-dependent activation of ERK. In agreement with previous studies, we found that treatment of hippocampal slices with NMDA resulted in activation of ERK in area CA1. The NMDA receptor-dependent activation of ERK was either blocked or attenuated by a number of antioxidants, including the general antioxidant N-acetyl-L-cysteine (L-NAC), the superoxide-scavenging enzyme superoxide dismutase (SOD), the membrane-permeable SOD mimetic Mn(III) tetrakis (4-benzoic acid) porphyrin (MnTBAP), the hydrogen peroxide-scavenging enzyme catalase, and the catalase mimetic ebselen. The NMDA receptor-dependent activation of ERK also was blocked by the NADPH oxidase inhibitor diphenylene iodonium (DPI) and was absent in mice that lacked p47(phox), one of the required protein components of NADPH oxidase. Taken together, our results suggest that ROS production, especially superoxide production via NADPH oxidase, is required for NMDA receptor-dependent activation of ERK in hippocampal area CA1.