Pupil-linked phasic arousal predicts a reduction of choice bias across species and decision domains

The Medial Olivocochlear Efferent Pathway Potentiates Cochlear Amplification in Response to Hearing Loss

The mammalian cochlea receives efferent feedback from the brain. Many functions for this feedback have been hypothesized, including on short timescales, such as mediating attentional states, and long timescales, such as buffering acoustic trauma. Testing these hypotheses has been impeded by an inability to make direct measurements of efferent effects in awake animals. Here, we assessed the role of the medial olivocochlear (MOC) efferent nerve fibers on cochlear amplification by measuring organ of Corti vibratory responses to sound in both sexes of awake and anesthetized mice. We studied long-term effects by genetically ablating the efferents and/or afferents. Cochlear amplification increased with deafferentation using VGLUT3-/- mice, but only when the efferents were intact, associated with increased activity within OHCs and supporting cells. Removing both the afferents and the efferents using VGLUT3-/- Alpha9-/- mice did not cause this effect. To test for short-term effects, we recorded sound-evoked vibrations while using pupillometry to measure neuromodulatory brain state. We found no state dependence of cochlear amplification or of the auditory brainstem response. However, state dependence was apparent in the downstream inferior colliculus. Thus, MOC efferents upregulate cochlear amplification chronically with hearing loss, but not acutely with brain state fluctuations. This pathway may partially compensate for hearing loss while mediating associated symptoms, such as tinnitus and hyperacusis.

Neurophysiology of Perceptual Decision-Making and Its Alterations in Attention-Deficit Hyperactivity Disorder

Despite the prevalence of attention-deficit hyperactivity disorder (ADHD), efforts to develop a detailed understanding of the neuropsychology of this neurodevelopmental condition are complicated by the diversity of interindividual presentations and the inability of current clinical tests to distinguish between its sensory, attentional, arousal, or motoric contributions. Identifying objective methods that can explain the diverse performance profiles across individuals diagnosed with ADHD has been a long-held goal. Achieving this could significantly advance our understanding of etiological processes and potentially inform the development of personalized treatment approaches. Here, we examine key neuropsychological components of ADHD within an electrophysiological (EEG) perceptual decision-making paradigm that is capable of isolating distinct neural signals of several key information processing stages necessary for sensory-guided actions from attentional selection to motor responses. Using a perceptual decision-making task (random dot motion), we evaluated the performance of 79 children (aged 8-17 years) and found slower and less accurate responses, along with a reduced rate of evidence accumulation (drift rate parameter of drift diffusion model), in children with ADHD (n = 37; 13 female) compared with typically developing peers (n = 42; 18 female). This was driven by the atypical dynamics of discrete electrophysiological signatures of attentional selection, the accumulation of sensory evidence, and strategic adjustments reflecting urgency of response. These findings offer an integrated account of decision-making in ADHD and establish discrete neural signals that might be used to understand the wide range of neuropsychological performance variations in individuals with ADHD.

Aperiodic spectral slope tracks the effects of brain state on saliency responses in the human auditory cortex

Alteration of responses to salient stimuli occurs in a wide range of brain disorders and may be rooted in pathophysiological brain state dynamics. Specifically, tonic and phasic modes of activity in the reticular activating system (RAS) influence, and are influenced by, salient stimuli, respectively. The RAS influences the spectral characteristics of activity in the neocortex, shifting the balance between low- and high-frequency fluctuations. Aperiodic '1/f slope' has emerged as a promising composite measure of these brain state dynamics. However, the relationship of 1/f slope to state-dependent processes, such as saliency, is less explored, particularly intracranially in humans. Here, we record pupil diameter as a measure of brain state and intracranial local field potentials in auditory cortical regions of human patients during an auditory oddball stimulus paradigm. We find that phasic high-gamma band responses in auditory cortical regions exhibit an inverted-u shaped relationship to tonic state, as reflected in the 1/f slope. Furthermore, salient stimuli trigger state changes, as indicated by shifts in the 1/f slope. Taken together, these findings suggest that 1/f slope tracks tonic and phasic arousal state dynamics in the human brain, increasing the interpretability of this metric and supporting it as a potential biomarker in brain disorders.

Sensory representations and pupil-indexed listening effort provide complementary contributions to multi-talker speech intelligibility

Multi-talker speech intelligibility requires successful separation of the target speech from background speech. Successful speech segregation relies on bottom-up neural coding fidelity of sensory information and top-down effortful listening. Here, we studied the interaction between temporal processing measured using Envelope Following Responses (EFRs) to amplitude modulated tones, and pupil-indexed listening effort, as it related to performance on the Quick Speech-in-Noise (QuickSIN) test in normal-hearing adults. Listening effort increased at the more difficult signal-to-noise ratios, but speech intelligibility only decreased at the hardest signal-to-noise ratio. Pupil-indexed listening effort and EFRs did not independently relate to QuickSIN performance. However, the combined effects of both EFRs and listening effort explained significant variance in QuickSIN performance. Our results suggest a synergistic interaction between sensory coding and listening effort as it relates to multi-talker speech intelligibility. These findings can inform the development of next-generation multi-dimensional approaches for testing speech intelligibility deficits in listeners with normal-hearing.

Affective integration in experience, judgment, and decision-making

The role of affect in value-based judgment and decision-making has attracted increasing interest in recent decades. Most previous approaches neglect the temporal dependence of mental states leading to mapping a relatively well-defined, but largely static, feeling state to a behavioral tendency. In contrast, we posit that expected and experienced consequences of actions are integrated over time into a unified overall affective experience reflecting current resources under current demands. This affective integration is shaped by context and continually modulates judgments and decisions. Changes in affective states modulate evaluation of new information (affect-as-information), signal changes in the environment (affect-as-a-spotlight) and influence behavioral tendencies in relation to goals (affect-as-motivation). We advocate for an approach that integrates affective dynamics into decision-making paradigms. This dynamical account identifies the key variables explaining how changes in affect influence information processing may provide us with new insights into the role of affect in value-based judgment and decision-making.

Task-irrelevant stimuli reliably boost phasic pupil-linked arousal but do not affect decision formation

The arousal systems of the brainstem, specifically the locus coeruleus-noradrenaline system, respond "phasically" during decisions. These central arousal transients are accompanied by dilations of the pupil. Mechanistic attempts to understand the impact of phasic arousal on cognition would benefit from temporally precise experimental manipulations. Here, we evaluated a non-invasive candidate approach to manipulate arousal in humans: presenting task-irrelevant auditory stimuli at different latencies during the execution of a challenging task. Task-irrelevant auditory stimuli drive responses of brainstem nuclei involved in the control of pupil size, but it is unknown whether such sound-evoked responses mimic the central arousal transients evoked during cognitive computations. A large body of evidence has implicated central arousal transients in reducing bias during challenging perceptual decisions. We thus used challenging visual decisions as a testbed, combining them with task-irrelevant sounds of varying onset latency or duration. Across three experiments, the sounds consistently elicited well-controlled pupil responses that superimposed onto task-evoked responses. While we replicated a negative correlation between task-evoked pupil responses and bias, the task-irrelevant sounds had no behavioral effect. This dissociation suggests that cognitive task engagement and task-irrelevant sounds may recruit distinct neural systems contributing to the control of pupil size.

Mapping computational cognitive profiles of aging to dissociable brain and sociodemographic factors

Aging is associated with declines in cognition and brain structural integrity. However, there is equivocality over (1) the specificity of affected domains in different people, (2) the location of associated patterns of brain structural deterioration, and (3) the sociodemographic factors contributing to 'unhealthy' cognition. We aimed to identify cognitive profiles displayed by older adults and determine brain and sociodemographic features potentially shaping these profiles. A sample of Southeast-Asian older adults (N = 386) participated in a multi-session study comprising cognitive testing, neuroimaging, and a structured interview. We used computational models to extract latent mechanisms underlying cognitive flexibility and response inhibition. Data-driven methods were used to construct cognitive profiles based on standard performance measures and model parameters. We also investigated grey matter volume and machine-learning derived 'brain-ages'. A profile associated with poor set-shifting and rigid focusing was associated with widespread grey matter reduction in cognitive control regions. A slow responding profile was associated with advanced brain-age. Both profiles were correlated with poor socioeconomic standing and cognitive reserve. We found that the impact of sociodemographic factors on cognitive profiles was partially mediated by total grey and white matter, and dorsolateral prefrontal and cerebellar volumes. This study furthers understanding of how distinct aging profiles of cognitive impairment uniquely correspond to specific vs. global brain deterioration and the significance of socioeconomic factors in informing cognitive performance in older age.

Neurobehavioral meaning of pupil size

Pupil size is a widely used metric of brain state. It is one of the few signals originating from the brain that can be readily monitored with low-cost devices in basic science, clinical, and home settings. It is, therefore, important to investigate and generate well-defined theories related to specific interpretations of this metric. What exactly does it tell us about the brain? Pupils constrict in response to light and dilate during darkness, but the brain also controls pupil size irrespective of luminosity. Pupil size fluctuations resulting from ongoing "brain states" are used as a metric of arousal, but what is pupil-linked arousal and how should it be interpreted in neural, cognitive, and computational terms? Here, we discuss some recent findings related to these issues. We identify open questions and propose how to answer them through a combination of well-defined tasks, neurocomputational models, and neurophysiological probing of the interconnected loops of causes and consequences of pupil size.

Increased listening effort and cochlear neural degeneration underlie behavioral deficits in speech perception in noise in normal hearing middle-aged adults

Middle-age is a critical period of rapid changes in brain function that presents an opportunity for early diagnostics and intervention for neurodegenerative conditions later in life. Hearing loss is one such early indicator linked to many comorbidities later in life. However, current clinical tests fail to capture hearing difficulties for ∼10% of middle-aged adults seeking help at hearing clinics. Cochlear neural degeneration (CND) could play a role in these hearing deficits, but our current understanding is limited by the lack of objective diagnostics and uncertainty regarding its perceptual consequences. Here, using a cross-species approach, we measured envelope following responses (EFRs) - neural ensemble responses to sound originating from the peripheral auditory pathway - in young and middle-aged adults with normal audiometric thresholds, and compared these responses to young and middle-aged Mongolian gerbils, where CND was histologically confirmed. We observed near identical changes in EFRs across species that were associated with CND. Perceptual effects measured as behavioral readouts showed deficits in the most challenging listening conditions and were associated with CND. Additionally, pupil-indexed listening effort increased even at moderate task difficulties where behavioral outcomes were matched. Our results reveal perceptual deficits in middle-aged adults driven by CND and increases in listening effort, which may result in increased listening fatigue and conversational disengagement.

Pupil-Linked Arousal Modulates Precision of Stimulus Representation in Cortex

Neural responses are naturally variable from one moment to the next, even when the stimulus is held constant. What factors might underlie this variability in neural population activity? We hypothesized that spontaneous fluctuations in cortical stimulus representations are created by changes in arousal state. We tested the hypothesis using a combination of fMRI, probabilistic decoding methods, and pupillometry. Human participants (20 female, 12 male) were presented with gratings of random orientation. Shortly after viewing the grating, participants reported its orientation and gave their level of confidence in this judgment. Using a probabilistic fMRI decoding technique, we quantified the precision of the stimulus representation in the visual cortex on a trial-by-trial basis. Pupil size was recorded and analyzed to index the observer's arousal state. We found that the precision of the cortical stimulus representation, reported confidence, and variability in the behavioral orientation judgments varied from trial to trial. Interestingly, these trial-by-trial changes in cortical and behavioral precision and confidence were linked to pupil size and its temporal rate of change. Specifically, when the cortical stimulus representation was more precise, the pupil dilated more strongly prior to stimulus onset and remained larger during stimulus presentation. Similarly, stronger pupil dilation during stimulus presentation was associated with higher levels of subjective confidence, a secondary measure of sensory precision, as well as improved behavioral performance. Taken together, our findings support the hypothesis that spontaneous fluctuations in arousal state modulate the fidelity of the stimulus representation in the human visual cortex, with clear consequences for behavior.

Pupillary responses to directional uncertainty while intercepting a moving target

Pupillary responses serve as sensitive indicators of cognitive processes, attentional shifts and decision-making dynamics. Our study investigates how directional uncertainty and target speed (V T) influence pupillary responses in a foveal tracking task involving the interception of a moving dot. Directional uncertainty, reflecting the unpredictability of the target's direction changes, was manipulated by altering the angular range (AR) from which random directions for the moving dot were extracted. Higher AR values were associated with reduced pupillary diameters, indicating that heightened uncertainty led to smaller pupil sizes. Additionally, an inverse U-shaped relationship between V T and pupillary responses suggested maximal diameters at intermediate speeds. Analysis of saccade-triggered responses showed a negative correlation between pupil diameter and directional uncertainty. Dynamic linear modelling revealed the influence of past successful collisions and other behavioural parameters on pupillary responses, emphasizing the intricate interaction between task variables and cognitive processing. Our results highlight the dynamic interplay between the directional uncertainty of a single moving target, V T and pupillary responses, with implications for understanding attentional mechanisms, decision-making processes and potential applications in emerging technologies.

Regularizing hyperparameters of interacting neural signals in the mouse cortex reflect states of arousal

In natural behaviors, multiple neural signals simultaneously drive activation across overlapping brain networks. Due to limitations in the amount of data that can be acquired in common experimental designs, the determination of these interactions is commonly inferred via modeling approaches, which reduce overfitting by finding appropriate regularizing hyperparameters. However, it is unclear whether these hyperparameters can also be related to any aspect of the underlying biological phenomena and help interpret them. We applied a state-of-the-art regularization procedure-automatic locality determination-to interacting neural activations in the mouse posterior cortex associated with movements of the body and eyes. As expected, regularization significantly improved the determination and interpretability of the response interactions. However, regularizing hyperparameters also changed considerably, and seemingly unpredictably, from animal to animal. We found that these variations were not random; rather, they correlated with the variability in visually evoked responses and with the variability in the state of arousal of the animals measured by pupillometry-both pieces of information that were not included in the modeling framework. These observations could be generalized to another commonly used-but potentially less informative-regularization method, ridge regression. Our findings demonstrate that optimal model hyperparameters can be discovery tools that are informative of factors not a priori included in the model's design.

Toward a computational role for locus coeruleus/norepinephrine arousal systems

Brain and behavior undergo measurable changes in their underlying state and neuromodulators are thought to contribute to these fluctuations. Why do we undergo such changes, and what function could the underlying neuromodulatory systems perform? Here we examine theoretical answers to these questions with respect to the locus coeruleus/norepinephrine system focusing on peripheral markers for arousal, such as pupil diameter, that are thought to provide a window into brain wide noradrenergic signaling. We explore a computational role for arousal systems in facilitating internal state transitions that facilitate credit assignment and promote accurate perceptions in non-stationary environments. We summarize recent work that supports this idea and highlight open questions as well as alternative views of how arousal affects cognition.

Strategic stabilization of arousal boosts sustained attention

Arousal and motivation interact to profoundly influence behavior. For example, experience tells us that we have some capacity to control our arousal when appropriately motivated, such as staying awake while driving a motor vehicle. However, little is known about how arousal and motivation jointly influence decision computations, including if and how animals, such as rodents, adapt their arousal state to their needs. Here, we developed and show results from an auditory, feature-based, sustained-attention task with intermittently shifting task utility. We use pupil size to estimate arousal across a wide range of states and apply tailored signal-detection theoretic, hazard function, and accumulation-to-bound modeling approaches in a large cohort of mice. We find that pupil-linked arousal and task utility both have major impacts on multiple aspects of task performance. Although substantial arousal fluctuations persist across utility conditions, mice partially stabilize their arousal near an intermediate and optimal level when task utility is high. Behavioral analyses show that multiple elements of behavior improve during high task utility and that arousal influences some, but not all, of them. Specifically, arousal influences the likelihood and timescale of sensory evidence accumulation but not the quantity of evidence accumulated per time step while attending. In sum, the results establish specific decision-computational signatures of arousal, motivation, and their interaction in attention. So doing, we provide an experimental and analysis framework for studying arousal self-regulation in neurotypical brains and in diseases such as attention-deficit/hyperactivity disorder.

Individual differences in belief updating and phasic arousal are related to psychosis proneness

Many decisions entail the updating of beliefs about the state of the environment by accumulating noisy sensory evidence. This form of probabilistic reasoning may go awry in psychosis. Computational theory shows that optimal belief updating in environments subject to hidden changes in their state requires a dynamic modulation of the evidence accumulation process. Recent empirical findings implicate transient responses of pupil-linked central arousal systems to individual evidence samples in this modulation. Here, we analyzed behavior and pupil responses during evidence accumulation in a changing environment in a community sample of human participants. We also assessed their subclinical psychotic experiences (psychosis proneness). Participants most prone to psychosis showed overall less flexible belief updating profiles, with diminished behavioral impact of evidence samples occurring late during decision formation. These same individuals also exhibited overall smaller pupil responses and less reliable pupil encoding of computational variables governing the dynamic belief updating. Our findings provide insights into the cognitive and physiological bases of psychosis proneness and open paths to unraveling the pathophysiology of psychotic disorders.

Hierarchical differences in the encoding of amplitude modulation in the subcortical auditory system of awake nonhuman primates

Sinusoidal amplitude modulation (SAM) is a key feature of complex sounds. Although psychophysical studies have characterized SAM perception, and neurophysiological studies in anesthetized animals report a transformation from the cochlear nucleus' (CN; brainstem) temporal code to the inferior colliculus' (IC; midbrain's) rate code, none have used awake animals or nonhuman primates to compare CN and IC's coding strategies to modulation-frequency perception. To address this, we recorded single-unit responses and compared derived neurometric measures in the CN and IC to psychometric measures of modulation frequency (MF) discrimination in macaques. IC and CN neurons often exhibited tuned responses to SAM in rate and spike-timing measures of modulation coding. Neurometric thresholds spanned a large range (2-200 Hz ΔMF). The lowest 40% of IC thresholds were less than or equal to psychometric thresholds, regardless of which code was used, whereas CN thresholds were greater than psychometric thresholds. Discrimination at 10-20 Hz could be explained by indiscriminately pooling 30 units in either structure, whereas discrimination at higher MFs was best explained by more selective pooling. This suggests that pooled CN activity was sufficient for AM discrimination. Psychometric and neurometric thresholds decreased as stimulus duration increased, but IC and CN thresholds were higher and more variable than behavior at short durations. This slower subcortical temporal integration compared with behavior was consistent with a drift diffusion model that reproduced individual differences in performance and can constrain future neurophysiological studies of temporal integration. These measures provide an account of AM perception at the neurophysiological, computational, and behavioral levels.NEW & NOTEWORTHY In everyday environments, the brain is tasked with extracting information from sound envelopes, which involves both sensory encoding and perceptual decision-making. Different neural codes for envelope representation have been characterized in midbrain and cortex, but studies of brainstem nuclei such as the cochlear nucleus (CN) have usually been conducted under anesthesia in nonprimate species. Here, we found that subcortical activity in awake monkeys and a biologically plausible perceptual decision-making model accounted for sound envelope discrimination behavior.

Direct serotonin release in humans shapes aversive learning and inhibition

The role of serotonin in human behaviour is informed by approaches which allow in vivo modification of synaptic serotonin. However, characterising the effects of increased serotonin signalling in human models of behaviour is challenging given the limitations of available experimental probes, notably selective serotonin reuptake inhibitors. Here we use a now-accessible approach to directly increase synaptic serotonin in humans (a selective serotonin releasing agent) and examine its influence on domains of behaviour historically considered core functions of serotonin. Computational techniques, including reinforcement learning and drift diffusion modelling, explain participant behaviour at baseline and after week-long intervention. Reinforcement learning models reveal that increasing synaptic serotonin reduces sensitivity for outcomes in aversive contexts. Furthermore, increasing synaptic serotonin enhances behavioural inhibition, and shifts bias towards impulse control during exposure to aversive emotional probes. These effects are seen in the context of overall improvements in memory for neutral verbal information. Our findings highlight the direct effects of increasing synaptic serotonin on human behaviour, underlining its role in guiding decision-making within aversive and more neutral contexts, and offering implications for longstanding theories of central serotonin function.

Pharmacological Elevation of Catecholamine Levels Improves Perceptual Decisions, But Not Metacognitive Insight

Perceptual decisions are often accompanied by a feeling of decision confidence. Where the parietal cortex is known for its crucial role in shaping such perceptual decisions, metacognitive evaluations are thought to additionally rely on the (pre)frontal cortex. Because of this supposed neural differentiation between these processes, perceptual and metacognitive decisions may be divergently affected by changes in internal (e.g., attention, arousal) and external (e.g., task and environmental demands) factors. Although intriguing, causal evidence for this hypothesis remains scarce. Here, we investigated the causal effect of two neuromodulatory systems on behavioral and neural measures of perceptual and metacognitive decision-making. Specifically, we pharmacologically elevated levels of catecholamines (with atomoxetine) and acetylcholine (with donepezil) in healthy adult human participants performing a visual discrimination task in which we gauged decision confidence, while electroencephalography was measured. Where cholinergic effects were not robust, catecholaminergic enhancement improved perceptual sensitivity, while at the same time leaving metacognitive sensitivity unaffected. Neurally, catecholaminergic elevation did not affect sensory representations of task-relevant visual stimuli but instead enhanced well-known decision signals measured over the centroparietal cortex, reflecting the accumulation of sensory evidence over time. Crucially, catecholaminergic enhancement concurrently impoverished neural markers measured over the frontal cortex linked to the formation of metacognitive evaluations. Enhanced catecholaminergic neuromodulation thus improves perceptual but not metacognitive decision-making.

Teleological reasoning bias is predicted by pupil dynamics: Evidence for the extensive integration account of bias in reasoning

Teleological reasoning is the tendency for humans to see purpose and intentionality in natural phenomena when there is none. In this study, we assess three competing theories on how bias in reasoning arises by examining performance on a teleological reasoning task while measuring pupil size and response times. We replicate that humans (N = 45) are prone to accept false teleological explanations. Further, we show that errors on the teleological reasoning task are associated with slower response times, smaller baseline pupil size, and larger pupil dilations. The results are in line with the single-process extensive integration account and directly oppose predictions from dual-processing accounts. Lastly, by modeling responses with a drift-diffusion model, we find that larger baseline pupil size is associated with lower decision threshold and higher drift rate, whereas larger pupil dilations are associated with higher decision threshold and lower drift rate. The results highlight the role of neural gain and the Locus Coeruleus-Norepinephrine system in modulating evidence integration and bias in reasoning. Thus, teleological reasoning and susceptibility to bias likely arise due to extensive processing rather than through fast and effortless processing.

Spiking activity in the visual thalamus is coupled to pupil dynamics across temporal scales

The processing of sensory information, even at early stages, is influenced by the internal state of the animal. Internal states, such as arousal, are often characterized by relating neural activity to a single "level" of arousal, defined by a behavioral indicator such as pupil size. In this study, we expand the understanding of arousal-related modulations in sensory systems by uncovering multiple timescales of pupil dynamics and their relationship to neural activity. Specifically, we observed a robust coupling between spiking activity in the mouse dorsolateral geniculate nucleus (dLGN) of the thalamus and pupil dynamics across timescales spanning a few seconds to several minutes. Throughout all these timescales, 2 distinct spiking modes-individual tonic spikes and tightly clustered bursts of spikes-preferred opposite phases of pupil dynamics. This multi-scale coupling reveals modulations distinct from those captured by pupil size per se, locomotion, and eye movements. Furthermore, coupling persisted even during viewing of a naturalistic movie, where it contributed to differences in the encoding of visual information. We conclude that dLGN spiking activity is under the simultaneous influence of multiple arousal-related processes associated with pupil dynamics occurring over a broad range of timescales.

Neural Mechanisms Determining the Duration of Task-free, Self-paced Visual Perception

Humans spend hours each day spontaneously engaging with visual content, free from specific tasks and at their own pace. Currently, the brain mechanisms determining the duration of self-paced perceptual behavior remain largely unknown. Here, participants viewed naturalistic images under task-free settings and self-paced each image's viewing duration while undergoing EEG and pupillometry recordings. Across two independent data sets, we observed large inter- and intra-individual variability in viewing duration. However, beyond an image's presentation order and category, specific image content had no consistent effects on spontaneous viewing duration across participants. Overall, longer viewing durations were associated with sustained enhanced posterior positivity and anterior negativity in the ERPs. Individual-specific variations in the spontaneous viewing duration were consistently correlated with evoked EEG activity amplitudes and pupil size changes. By contrast, presentation order was selectively correlated with baseline alpha power and baseline pupil size. Critically, spontaneous viewing duration was strongly predicted by the temporal stability in neural activity patterns starting as early as 350 msec after image onset, suggesting that early neural stability is a key predictor for sustained perceptual engagement. Interestingly, neither bottom-up nor top-down predictions about image category influenced spontaneous viewing duration. Overall, these results suggest that individual-specific factors can influence perceptual processing at a surprisingly early time point and influence the multifaceted ebb and flow of spontaneous human perceptual behavior in naturalistic settings.

From pre-processing to advanced dynamic modeling of pupil data

The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.

Changes in pupil size track self-control failure

People are more likely to perform poorly on a self-control task following a previous task requiring self-control (ego-depletion), but the mechanism for this effect remains unclear. We used pupillometry to test the role of attentional effort in ego-depletion. We hypothesized that an elevated pupil diameter (PD)-a common physiological measure of effort-during an initial task requiring self-control should be negatively associated with performance on a subsequent control task. To test this hypothesis, participants were first assigned to either a high- or low-demand attention task (manipulation; a standard ego-depletion paradigm), after which all participants completed the same Stroop task. We then separately extracted both sustained (low-frequency) and phasic (high-frequency) changes in PD from both tasks to evaluate possible associations with lapses of cognitive control on the Stroop task. We first show that in the initial task, sustained PD was larger among participants who were assigned to the demanding attention condition. Furthermore, ego-depletion effects were serially mediated by PD: an elevated PD response emerged rapidly among the experimental group during the manipulation, persisted as an elevated baseline response during the Stroop task, and predicted worse accuracy on incongruent trials, revealing a potential indirect pathway to ego-depletion via sustained attention. Secondary analyses revealed another, independent and direct pathway via high levels of transient attentional control: participants who exhibited large phasic responses during the manipulation tended to perform worse on the subsequent Stroop task. We conclude by exploring the neuroscientific implications of these results within the context of current theories of self-control.

A disinhibitory circuit mechanism explains a general principle of peak performance during mid-level arousal

Perceptual decision-making is highly dependent on the momentary arousal state of the brain, which fluctuates over time on a scale of hours, minutes, and even seconds. The textbook relationship between momentary arousal and task performance is captured by an inverted U-shape, as put forward in the Yerkes-Dodson law. This law suggests optimal performance at moderate levels of arousal and impaired performance at low or high arousal levels. However, despite its popularity, the evidence for this relationship in humans is mixed at best. Here, we use pupil-indexed arousal and performance data from various perceptual decision-making tasks to provide converging evidence for the inverted U-shaped relationship between spontaneous arousal fluctuations and performance across different decision types (discrimination, detection) and sensory modalities (visual, auditory). To further understand this relationship, we built a neurobiologically plausible mechanistic model and show that it is possible to reproduce our findings by incorporating two types of interneurons that are both modulated by an arousal signal. The model architecture produces two dynamical regimes under the influence of arousal: one regime in which performance increases with arousal and another regime in which performance decreases with arousal, together forming an inverted U-shaped arousal-performance relationship. We conclude that the inverted U-shaped arousal-performance relationship is a general and robust property of sensory processing. It might be brought about by the influence of arousal on two types of interneurons that together act as a disinhibitory pathway for the neural populations that encode the available sensory evidence used for the decision.

Optimization of Temporal Coding of Tactile Information in Rat Thalamus by Locus Coeruleus Activation

The brainstem noradrenergic nucleus, the locus coeruleus (LC), exerts heavy influences on sensory processing, perception, and cognition through its diffuse projections throughout the brain. Previous studies have demonstrated that LC activation modulates the response and feature selectivity of thalamic relay neurons. However, the extent to which LC modulates the temporal coding of sensory information in the thalamus remains mostly unknown. Here, we found that LC stimulation significantly altered the temporal structure of the responses of the thalamic relay neurons to repeated whisker stimulation. A substantial portion of events (i.e., time points where the stimulus reliably evoked spikes as evidenced by dramatic elevations in the firing rate of the spike density function) were removed during LC stimulation, but many new events emerged. Interestingly, spikes within the emerged events have a higher feature selectivity, and therefore transmit more information about a tactile stimulus, than spikes within the removed events. This suggests that LC stimulation optimized the temporal coding of tactile information to improve information transmission. We further reconstructed the original whisker stimulus from a population of thalamic relay neurons' responses and corresponding feature selectivity. As expected, we found that reconstruction from thalamic responses was more accurate using spike trains of thalamic neurons recorded during LC stimulation than without LC stimulation, functionally confirming LC optimization of the thalamic temporal code. Together, our results demonstrated that activation of the LC-NE system optimizes temporal coding of sensory stimulus in the thalamus, presumably allowing for more accurate decoding of the stimulus in the downstream brain structures.

Rapid fluctuations in functional connectivity of cortical networks encode spontaneous behavior

Experimental work across species has demonstrated that spontaneously generated behaviors are robustly coupled to variations in neural activity within the cerebral cortex. Functional magnetic resonance imaging data suggest that temporal correlations in cortical networks vary across distinct behavioral states, providing for the dynamic reorganization of patterned activity. However, these data generally lack the temporal resolution to establish links between cortical signals and the continuously varying fluctuations in spontaneous behavior observed in awake animals. Here, we used wide-field mesoscopic calcium imaging to monitor cortical dynamics in awake mice and developed an approach to quantify rapidly time-varying functional connectivity. We show that spontaneous behaviors are represented by fast changes in both the magnitude and correlational structure of cortical network activity. Combining mesoscopic imaging with simultaneous cellular-resolution two-photon microscopy demonstrated that correlations among neighboring neurons and between local and large-scale networks also encode behavior. Finally, the dynamic functional connectivity of mesoscale signals revealed subnetworks not predicted by traditional anatomical atlas-based parcellation of the cortex. These results provide new insights into how behavioral information is represented across the neocortex and demonstrate an analytical framework for investigating time-varying functional connectivity in neural networks.

Adaptive decision-making depends on pupil-linked arousal in rats performing tactile discrimination tasks

Perceptual decision-making is a dynamic cognitive process and is shaped by many factors, including behavioral state, reward contingency, and sensory environment. To understand the extent to which adaptive behavior in decision-making is dependent on pupil-linked arousal, we trained head-fixed rats to perform perceptual decision-making tasks and systematically manipulated the probability of Go and No-go stimuli while simultaneously measuring their pupil size in the tasks. Our data demonstrated that the animals adaptively modified their behavior in response to the changes in the sensory environment. The response probability to both Go and No-go stimuli decreased as the probability of the Go stimulus being presented decreased. Analyses within the signal detection theory framework showed that while the animals' perceptual sensitivity was invariant, their decision criterion increased as the probability of the Go stimulus decreased. Simulation results indicated that the adaptive increase in the decision criterion will increase possible water rewards during the task. Moreover, the adaptive decision-making is dependent on pupil-linked arousal as the increase in the decision criterion was the largest during low pupil-linked arousal periods. Taken together, our results demonstrated that the rats were able to adjust their decision-making to maximize rewards in the tasks, and that adaptive behavior in perceptual decision-making is dependent on pupil-linked arousal.NEW & NOTEWORTHY Perceptual decision-making is a dynamic cognitive process and is shaped by many factors. However, the extent to which changes in sensory environment result in adaptive decision-making remains poorly understood. Our data provided new experimental evidence demonstrating that the rats were able to adaptively modify their decision criterion to maximize water reward in response to changes in the statistics of the sensory environment. Furthermore, the adaptive decision-making is dependent on pupil-linked arousal.

Catecholaminergic neuromodulation and selective attention jointly shape perceptual decision-making

Perceptual decisions about sensory input are influenced by fluctuations in ongoing neural activity, most prominently driven by attention and neuromodulator systems. It is currently unknown if neuromodulator activity and attention differentially modulate perceptual decision-making and/or whether neuromodulatory systems in fact control attentional processes. To investigate the effects of two distinct neuromodulatory systems and spatial attention on perceptual decisions, we pharmacologically elevated cholinergic (through donepezil) and catecholaminergic (through atomoxetine) levels in humans performing a visuo-spatial attention task, while we measured electroencephalography (EEG). Both attention and catecholaminergic enhancement improved decision-making at the behavioral and algorithmic level, as reflected in increased perceptual sensitivity and the modulation of the drift rate parameter derived from drift diffusion modeling. Univariate analyses of EEG data time-locked to the attentional cue, the target stimulus, and the motor response further revealed that attention and catecholaminergic enhancement both modulated pre-stimulus cortical excitability, cue- and stimulus-evoked sensory activity, as well as parietal evidence accumulation signals. Interestingly, we observed both similar, unique, and interactive effects of attention and catecholaminergic neuromodulation on these behavioral, algorithmic, and neural markers of the decision-making process. Thereby, this study reveals an intricate relationship between attentional and catecholaminergic systems and advances our understanding about how these systems jointly shape various stages of perceptual decision-making.

Sensory processing in humans and mice fluctuates between external and internal modes

Perception is known to cycle through periods of enhanced and reduced sensitivity to external information. Here, we asked whether such slow fluctuations arise as a noise-related epiphenomenon of limited processing capacity or, alternatively, represent a structured mechanism of perceptual inference. Using 2 large-scale datasets, we found that humans and mice alternate between externally and internally oriented modes of sensory analysis. During external mode, perception aligns more closely with the external sensory information, whereas internal mode is characterized by enhanced biases toward perceptual history. Computational modeling indicated that dynamic changes in mode are enabled by 2 interlinked factors: (i) the integration of subsequent inputs over time and (ii) slow antiphase oscillations in the impact of external sensory information versus internal predictions that are provided by perceptual history. We propose that between-mode fluctuations generate unambiguous error signals that enable optimal inference in volatile environments.

Boundary updating as a source of history effect on decision uncertainty

When sorting a sequence of stimuli into binary classes, current choices are often negatively correlated with recent stimulus history. This phenomenon-dubbed the repulsive bias-can be explained by boundary updating, a process of shifting the class boundary to previous stimuli. This explanation implies that recent stimulus history can also influence "decision uncertainty," the probability of making incorrect decisions, because it depends on the location of the boundary. However, there have been no previous efforts to elucidate the impact of previous stimulus history on decision uncertainty. Here, from the boundary-updating process that accounts for the repulsive bias, we derived a prediction that decision uncertainty increases as current choices become more congruent with previous stimuli. We confirmed this prediction in behavioral, physiological, and neural correlates of decision uncertainty. Our work demonstrates that boundary updating offers a principled account of how previous stimulus history concurrently relates to choice bias and decision uncertainty.

Drift-diffusion modeling reveals that masked faces are preconceived as unfriendly

During the COVID-19 pandemic, the use of face masks has become a daily routine. Studies have shown that face masks increase the ambiguity of facial expressions which not only affects (the development of) emotion recognition, but also interferes with social interaction and judgement. To disambiguate facial expressions, we rely on perceptual (stimulus-driven) as well as preconceptual (top-down) processes. However, it is unknown which of these two mechanisms accounts for the misinterpretation of masked expressions. To investigate this, we asked participants (N = 136) to decide whether ambiguous (morphed) facial expressions, with or without a mask, were perceived as friendly or unfriendly. To test for the independent effects of perceptual and preconceptual biases we fitted a drift-diffusion model (DDM) to the behavioral data of each participant. Results show that face masks induce a clear loss of information leading to a slight perceptual bias towards friendly choices, but also a clear preconceptual bias towards unfriendly choices for masked faces. These results suggest that, although face masks can increase the perceptual friendliness of faces, people have the prior preconception to interpret masked faces as unfriendly.

Evidence Accumulation Rate Moderates the Relationship between Enriched Environment Exposure and Age-Related Response Speed Declines

Older adults exposed to enriched environments (EEs) maintain relatively higher levels of cognitive function, even in the face of compromised markers of brain health. Response speed (RS) is often used as a simple proxy to measure the preservation of global cognitive function in older adults. However, it is unknown which specific selection, decision, and/or motor processes provide the most specific indices of neurocognitive health. Here, using a simple decision task with electroencephalography (EEG), we found that the efficiency with which an individual accumulates sensory evidence was a critical determinant of the extent to which RS was preserved in older adults (63% female, 37% male). Moreover, the mitigating influence of EE on age-related RS declines was most pronounced when evidence accumulation rates were shallowest. These results suggest that the phenomenon of cognitive reserve, whereby high EE individuals can better tolerate suboptimal brain health to facilitate the preservation of cognitive function, is not just applicable to neuroanatomical indicators of brain aging but can be observed in markers of neurophysiology. Our results suggest that EEG metrics of evidence accumulation may index neurocognitive vulnerability of the aging brain.Significance Statement Response speed in older adults is closely linked with trajectories of cognitive aging. Here, by recording brain activity while individuals perform a simple computer task, we identify a neural metric that is a critical determinant of response speed. Older adults exposed to greater cognitive and social stimulation throughout a lifetime could maintain faster responding, even when this neural metric was impaired. This work suggests EEG is a useful technique for interrogating how a lifetime of stimulation benefits brain health in aging.

Multiple routes to enhanced memory for emotionally relevant events

Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.

Sensory representations and pupil-indexed listening effort provide complementary contributions to multi-talker speech intelligibility

Optimal speech perception in noise requires successful separation of the target speech stream from multiple competing background speech streams. The ability to segregate these competing speech streams depends on the fidelity of bottom-up neural representations of sensory information in the auditory system and top-down influences of effortful listening. Here, we use objective neurophysiological measures of bottom-up temporal processing using envelope-following responses (EFRs) to amplitude modulated tones and investigate their interactions with pupil-indexed listening effort, as it relates to performance on the Quick speech in noise (QuickSIN) test in young adult listeners with clinically normal hearing thresholds. We developed an approach using ear-canal electrodes and adjusting electrode montages for modulation rate ranges, which extended the rage of reliable EFR measurements as high as 1024Hz. Pupillary responses revealed changes in listening effort at the two most difficult signal-to-noise ratios (SNR), but behavioral deficits at the hardest SNR only. Neither pupil-indexed listening effort nor the slope of the EFR decay function independently related to QuickSIN performance. However, a linear model using the combination of EFRs and pupil metrics significantly explained variance in QuickSIN performance. These results suggest a synergistic interaction between bottom-up sensory coding and top-down measures of listening effort as it relates to speech perception in noise. These findings can inform the development of next-generation tests for hearing deficits in listeners with normal-hearing thresholds that incorporates a multi-dimensional approach to understanding speech intelligibility deficits.

Parabrachial Nucleus Activity in Nociception and Pain in Awake Mice

The parabrachial nuclear complex (PBN) is a nexus for aversion and for the sensory and affective components of pain perception. We have previously shown that during chronic pain PBN neurons in anesthetized rodents have amplified activity. We report a method to record from PBN neurons of behaving, head-restrained mice while applying reproducible noxious stimuli. We find that both spontaneous and evoked activity are higher in awake animals compared with urethane anesthetized mice. Fiber photometry of calcium responses from calcitonin-gene-related peptide-expressing PBN neurons demonstrates that these neurons respond to noxious stimuli. In both males and females with neuropathic or inflammatory pain, responses of PBN neurons remain amplified for at least 5 weeks, in parallel with increased pain metrics. We also show that PBN neurons can be rapidly conditioned to respond to innocuous stimuli after pairing with noxious stimuli. Finally, we demonstrate that changes in PBN neuronal activity are correlated with changes in arousal, measured as changes in pupil area.SIGNIFICANCE STATEMENT The parabrachial complex is a nexus of aversion, including pain. We report a method to record from parabrachial nucleus neurons of behaving mice while applying reproducible noxious stimuli. This allowed us to track parabrachial activity over time in animals with neuropathic or inflammatory pain. It also allowed us to show that the activity of these neurons correlates with arousal states and that these neurons can be conditioned to respond to innocuous stimuli.

Not everything, not everywhere, not all at once: a study of brain-wide encoding of movement

Activity related to movement is found throughout sensory and motor regions of the brain. However, it remains unclear how movement-related activity is distributed across the brain and whether systematic differences exist between brain areas. Here, we analyzed movement related activity in brain-wide recordings containing more than 50,000 neurons in mice performing a decision-making task. Using multiple techniques, from markers to deep neural networks, we find that movement-related signals were pervasive across the brain, but systematically differed across areas. Movement-related activity was stronger in areas closer to the motor or sensory periphery. Delineating activity in terms of sensory- and motor-related components revealed finer scale structures of their encodings within brain areas. We further identified activity modulation that correlates with decision-making and uninstructed movement. Our work charts out a largescale map of movement encoding and provides a roadmap for dissecting different forms of movement and decision-making related encoding across multi-regional neural circuits.

Pupil size predicts the onset of exploration in brain and behavior

In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible.

Time Scales and Moments of Listening Effort Revealed in Pupillometry

This article offers a collection of observations that highlight the value of time course data in pupillometry and points out ways in which these observations create deeper understanding of listening effort. The main message is that listening effort should be considered on a moment-to-moment basis rather than as a singular amount. A review of various studies and the reanalysis of data reveal distinct signatures of effort before a stimulus, during a stimulus, in the moments after a stimulus, and changes over whole experimental testing sessions. Collectively these observations motivate questions that extend beyond the "amount" of effort, toward understanding how long the effort lasts, and how precisely someone can allocate effort at specific points in time or reduce effort at other times. Apparent disagreements between studies are reconsidered as informative lessons about stimulus selection and the nature of pupil dilation as a reflection of decision making rather than the difficulty of sensory encoding.

Pupillometric evidence for a temporal expectations-based account of persistence under temporal uncertainty

People often quit waiting for delayed rewards when the exact timing of those rewards is uncertain. This behavior often has been attributed to self-control failure. Another possibility is that quitting is the result of a rational decision-making process in the face of uncertainty, based on the decision-maker's expectations about the possible arrival times of the awaited reward. There are forms of temporal expectations (e.g., heavy-tailed) under which the expected time remaining until a reward arrives actually increases as time elapses. In those cases, the rational strategy is to quit waiting when the expected reward is no longer worth the expected time remaining. To arbitrate between the "limited self-control" and "temporal expectations" accounts of persistence, we measured pupil diameter during a persistence task, as a physiological marker of surprise (phasic responses) and effort (pre-decision diameter). Phasic pupil responses were elevated in response to reward receipt. Critically, the extent to which pupils dilated following rewards depended on the delay: people showed larger pupillary surprise responses the more delayed the reward was. This result suggests that people expect the reward less the longer they wait for it-a form of temporal expectations under which limiting persistence is rational. Moreover, predecision pupil diameter before quit events was not associated with how long the participant had been waiting, but rather, depended on how atypical the quit decision was compared with the participant's usual behavior. These data provide physiological evidence for a temporal expectations account of persistence under temporal uncertainty.

Sex differences in the role of parabrachial in nociception and pain in awake mice.. [PMID: 36993729 PMCID: PMC10055376 DOI: 10.1101/2023.03.22.533230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

The parabrachial nucleus is a nexus for aversion, and for the sensory and affective components of pain. In anesthetized rodents with chronic pain, parabrachial neurons have amplified activity. Both spontaneous and evoked activity are considerably higher in awake, compared to anesthetized animals. Parabrachial neurons are rapidly conditioned to respond to innocuous stimuli, after pairing with nociceptive stimuli. In neuropathic or inflammatory pain, parabrachial responses remain amplified for at least 6 weeks, in parallel with increased pain metrics. Calcium responses from CGRP- expressing parabrachial neurons in males demonstrate responses to nociceptive stimuli, and amplified activity in inflammatory pain. In females these neurons evoke smaller responses at baseline, and only small increases in neuropathic pain. This sex difference may relate to our finding that, in females, a small percentage of neurons expresses CGRP RNA. Finally, we show that changes in parabrachial activity are correlated with in arousal, measured as changes in pupil size.

Neuromodulation of Neural Oscillations in Health and Disease

Using EEG and local field potentials (LFPs) as an index of large-scale neural activities, research has been able to associate neural oscillations in different frequency bands with markers of cognitive functions, goal-directed behavior, and various neurological disorders. While this gives us a glimpse into how neurons communicate throughout the brain, the causality of these synchronized network activities remains poorly understood. Moreover, the effect of the major neuromodulatory systems (e.g., noradrenergic, cholinergic, and dopaminergic) on brain oscillations has drawn much attention. More recent studies have suggested that cross-frequency coupling (CFC) is heavily responsible for mediating network-wide communication across subcortical and cortical brain structures, implicating the importance of neurotransmitters in shaping coordinated actions. By bringing to light the role each neuromodulatory system plays in regulating brain-wide neural oscillations, we hope to paint a clearer picture of the pivotal role neural oscillations play in a variety of cognitive functions and neurological disorders, and how neuromodulation techniques can be optimized as a means of controlling neural network dynamics. The aim of this review is to showcase the important role that neuromodulatory systems play in large-scale neural network dynamics, informing future studies to pay close attention to their involvement in specific features of neural oscillations and associated behaviors.

Pupillary dynamics reflect the impact of temporal expectation on detection strategy

Everyday life's perceptual decision-making is informed by experience. In particular, temporal expectation can ease the detection of relevant events in noisy sensory streams. Here, we investigated if humans can extract hidden temporal cues from the occurrences of probabilistic targets and utilize them to inform target detection in a complex acoustic stream. To understand what neural mechanisms implement temporal expectation influence on decision-making, we used pupillometry as a proxy for underlying neuromodulatory activity. We found that participants' detection strategy was influenced by the hidden temporal context and correlated with sound-evoked pupil dilation. A model of urgency fitted on false alarms predicted detection reaction time. Altogether, these findings suggest that temporal expectation informs decision-making and could be implemented through neuromodulatory-mediated urgency signals.

Vocalization categorization behavior explained by a feature-based auditory categorization model

Vocal animals produce multiple categories of calls with high between- and within-subject variability, over which listeners must generalize to accomplish call categorization. The behavioral strategies and neural mechanisms that support this ability to generalize are largely unexplored. We previously proposed a theoretical model that accomplished call categorization by detecting features of intermediate complexity that best contrasted each call category from all other categories. We further demonstrated that some neural responses in the primary auditory cortex were consistent with such a model. Here, we asked whether a feature-based model could predict call categorization behavior. We trained both the model and guinea pigs (GPs) on call categorization tasks using natural calls. We then tested categorization by the model and GPs using temporally and spectrally altered calls. Both the model and GPs were surprisingly resilient to temporal manipulations, but sensitive to moderate frequency shifts. Critically, the model predicted about 50% of the variance in GP behavior. By adopting different model training strategies and examining features that contributed to solving specific tasks, we could gain insight into possible strategies used by animals to categorize calls. Our results validate a model that uses the detection of intermediate-complexity contrastive features to accomplish call categorization.

Rule-based and stimulus-based cues bias auditory decisions via different computational and physiological mechanisms

Expectations, such as those arising from either learned rules or recent stimulus regularities, can bias subsequent auditory perception in diverse ways. However, it is not well understood if and how these diverse effects depend on the source of the expectations. Further, it is unknown whether different sources of bias use the same or different computational and physiological mechanisms. We examined how rule-based and stimulus-based expectations influenced behavior and pupil-linked arousal, a marker of certain forms of expectation-based processing, of human subjects performing an auditory frequency-discrimination task. Rule-based cues consistently biased choices and response times (RTs) toward the more-probable stimulus. In contrast, stimulus-based cues had a complex combination of effects, including choice and RT biases toward and away from the frequency of recently presented stimuli. These different behavioral patterns also had: 1) distinct computational signatures, including different modulations of key components of a novel form of a drift-diffusion decision model and 2) distinct physiological signatures, including substantial bias-dependent modulations of pupil size in response to rule-based but not stimulus-based cues. These results imply that different sources of expectations can modulate auditory processing via distinct mechanisms: one that uses arousal-linked, rule-based information and another that uses arousal-independent, stimulus-based information to bias the speed and accuracy of auditory perceptual decisions.

Prior information about upcoming stimuli can bias our perception of those stimuli. Whether different sources of prior information bias perception in similar or distinct ways is not well understood. We compared the influence of two kinds of prior information on tone-frequency discrimination: rule-based cues, in the form of explicit information about the most-likely identity of the upcoming tone; and stimulus-based cues, in the form of sequences of tones presented before the to-be-discriminated tone. Although both types of prior information biased auditory decision-making, they demonstrated distinct behavioral, computational, and physiological signatures. Our results suggest that the brain processes prior information in a form-specific manner rather than utilizing a general-purpose prior. Such form-specific processing has implications for understanding decision biases real-world contexts, in which prior information comes from many different sources and modalities.

Attention cueing in rivalry: insights from pupillometry

We used pupillometry to evaluate the effects of attention cueing on perceptual bi-stability, as reported by adult human observers. Perceptual alternations and pupil diameter were measured during two forms of rivalry, generated by presenting a white and a black disk to the two eyes (binocular rivalry) or splitting the disks between eyes (interocular grouping rivalry). In line with previous studies, we found that subtle pupil size modulations (about 0.05 mm) tracked alternations between exclusive dominance phases of the black or white disk. These pupil responses were larger for perceptually stronger stimuli: presented to the dominant eye or with physically higher luminance contrast. However, cueing of endogenous attention to one of the rivaling percepts did not affect pupil modulations during exclusive dominance phases. This was observed despite the reliable effects of endogenous attention on perceptual dominance, which shifted in favor of the cued percept by about 10%. The results were comparable for binocular and interocular grouping rivalry. Cueing only had a marginal modulatory effect on pupil size during mixed percepts in binocular rivalry. This may suggest that, rather than acting by modulating perceptual strength, endogenous attention primarily acts during periods of unresolved competition, which is compatible with attention being automatically directed to the rivaling stimuli during periods of exclusive dominance and thereby sustaining perceptual alternations.Significance StatementBinocular rivalry depends on attention. When it is diverted away from the stimuli, perceptual alternations slow down; when it is preferentially directed to one stimulus, perception lingers more on it, consistent with attention enhancing the effective strength of the rivaling stimuli. Here we introduce pupillometry as a means to indirectly track changes in effective stimulus strength. We find that pupil size accurately tracks perceived luminance during two forms of rivalry: binocular rivalry and interocular grouping rivalry. Both show robust effects of attention cueing on perceptual dominance, but pupil modulations during exclusive dominance are unaffected by cueing. This suggests that endogenous attention does not affect perceptual strength during exclusive dominance, though it might do so during transition phases.

Task-related hemodynamic responses in human early visual cortex are modulated by task difficulty and behavioral performance

Early visual cortex exhibits widespread hemodynamic responses in the absence of visual stimulation, which are entrained to the timing of a task and not predicted by local spiking or local field potential. Such task-related responses (TRRs) covary with reward magnitude and physiological signatures of arousal. It is unknown, however, if TRRs change on a trial-to-trial basis according to behavioral performance and task difficulty. If so, this would suggest that TRRs reflect arousal on a trial-to-trial timescale and covary with critical task and behavioral variables. We measured functional magnetic resonance imaging blood-oxygen-level-dependent (fMRI-BOLD) responses in the early visual cortex of human observers performing an orientation discrimination task consisting of separate easy and hard runs of trials. Stimuli were presented in a small portion of one hemifield, but the fMRI response was measured in the ipsilateral hemisphere, far from the stimulus representation and focus of spatial attention. TRRs scaled in amplitude with task difficulty, behavioral accuracy, reaction time, and lapses across trials. These modulations were not explained by the influence of respiration, cardiac activity, or head movement on the fMRI signal. Similar modulations with task difficulty and behavior were observed in pupil size. These results suggest that TRRs reflect arousal and behavior on the timescale of individual trials.

Arousal state affects perceptual decision-making by modulating hierarchical sensory processing in a large-scale visual system model

Arousal levels strongly affect task performance. Yet, what arousal level is optimal for a task depends on its difficulty. Easy task performance peaks at higher arousal levels, whereas performance on difficult tasks displays an inverted U-shape relationship with arousal, peaking at medium arousal levels, an observation first made by Yerkes and Dodson in 1908. It is commonly proposed that the noradrenergic locus coeruleus system regulates these effects on performance through a widespread release of noradrenaline resulting in changes of cortical gain. This account, however, does not explain why performance decays with high arousal levels only in difficult, but not in simple tasks. Here, we present a mechanistic model that revisits the Yerkes-Dodson effect from a sensory perspective: a deep convolutional neural network augmented with a global gain mechanism reproduced the same interaction between arousal state and task difficulty in its performance. Investigating this model revealed that global gain states differentially modulated sensory information encoding across the processing hierarchy, which explained their differential effects on performance on simple versus difficult tasks. These findings offer a novel hierarchical sensory processing account of how, and why, arousal state affects task performance.

Different Forms of Variability Could Explain a Difference Between Human and Rat Decision Making

When observers make rapid, difficult perceptual decisions, their response time is highly variable from trial to trial. In a visual motion discrimination task, it has been reported that human accuracy declines with increasing response time, whereas rat accuracy increases with response time. This is of interest because different mathematical theories of decision-making differ in their predictions regarding the correlation of accuracy with response time. On the premise that perceptual decision-making mechanisms are likely to be conserved among mammals, we seek to unify the rodent and primate results in a common theoretical framework. We show that a bounded drift diffusion model (DDM) can explain both effects with variable parameters: trial-to-trial variability in the starting point of the diffusion process produces the pattern typically observed in rats, whereas variability in the drift rate produces the pattern typically observed in humans. We further show that the same effects can be produced by deterministic biases, even in the absence of parameter stochasticity or parameter change within a trial.

Pupil diameter is not an accurate real-time readout of locus coeruleus activity

Pupil diameter is often treated as a noninvasive readout of activity in the locus coeruleus (LC). However, how accurately it can be used to index LC activity is not known. To address this question, we established a graded relationship between pupil size changes and LC spiking activity in mice, where pupil dilation increased monotonically with the number of LC spikes. However, this relationship exists with substantial variability such that pupil diameter can only be used to accurately predict a small fraction of LC activity on a moment-by-moment basis. In addition, pupil exhibited large session-to-session fluctuations in response to identical optical stimulation in the LC. The variations in the pupil–LC relationship were strongly correlated with decision bias-related behavioral variables. Together, our data show that substantial variability exists in an overall graded relationship between pupil diameter and LC activity, and further suggest that the pupil–LC relationship is dynamically modulated by brain states, supporting and extending our previous findings (Yang et al., 2021).

Arousal States as a Key Source of Variability in Speech Perception and Learning

The human brain exhibits the remarkable ability to categorize speech sounds into distinct, meaningful percepts, even in challenging tasks like learning non-native speech categories in adulthood and hearing speech in noisy listening conditions. In these scenarios, there is substantial variability in perception and behavior, both across individual listeners and individual trials. While there has been extensive work characterizing stimulus-related and contextual factors that contribute to variability, recent advances in neuroscience are beginning to shed light on another potential source of variability that has not been explored in speech processing. Specifically, there are task-independent, moment-to-moment variations in neural activity in broadly-distributed cortical and subcortical networks that affect how a stimulus is perceived on a trial-by-trial basis. In this review, we discuss factors that affect speech sound learning and moment-to-moment variability in perception, particularly arousal states—neurotransmitter-dependent modulations of cortical activity. We propose that a more complete model of speech perception and learning should incorporate subcortically-mediated arousal states that alter behavior in ways that are distinct from, yet complementary to, top-down cognitive modulations. Finally, we discuss a novel neuromodulation technique, transcutaneous auricular vagus nerve stimulation (taVNS), which is particularly well-suited to investigating causal relationships between arousal mechanisms and performance in a variety of perceptual tasks. Together, these approaches provide novel testable hypotheses for explaining variability in classically challenging tasks, including non-native speech sound learning.