Collective Activity of Many Bistable Assemblies Reproduces Characteristic Dynamics of Multistable Perception

Co-existence of synaptic plasticity and metastable dynamics in a spiking model of cortical circuits

Evidence for metastable dynamics and its role in brain function is emerging at a fast pace and is changing our understanding of neural coding by putting an emphasis on hidden states of transient activity. Clustered networks of spiking neurons have enhanced synaptic connections among groups of neurons forming structures called cell assemblies; such networks are capable of producing metastable dynamics that is in agreement with many experimental results. However, it is unclear how a clustered network structure producing metastable dynamics may emerge from a fully local plasticity rule, i.e., a plasticity rule where each synapse has only access to the activity of the neurons it connects (as opposed to the activity of other neurons or other synapses). Here, we propose a local plasticity rule producing ongoing metastable dynamics in a deterministic, recurrent network of spiking neurons. The metastable dynamics co-exists with ongoing plasticity and is the consequence of a self-tuning mechanism that keeps the synaptic weights close to the instability line where memories are spontaneously reactivated. In turn, the synaptic structure is stable to ongoing dynamics and random perturbations, yet it remains sufficiently plastic to remap sensory representations to encode new sets of stimuli. Both the plasticity rule and the metastable dynamics scale well with network size, with synaptic stability increasing with the number of neurons. Overall, our results show that it is possible to generate metastable dynamics over meaningful hidden states using a simple but biologically plausible plasticity rule which co-exists with ongoing neural dynamics.

Functional coupling between auditory memory and verbal transformations

The ability to parse sound mixtures into coherent auditory objects is fundamental to cognitive functions, such as speech comprehension and language acquisition. Yet, we still lack a clear understanding of how auditory objects are formed. To address this question, we studied a speech-specific case of perceptual multistability, called verbal transformations (VTs), in which a variety of verbal forms is induced by continuous repetition of a physically unchanging word. Here, we investigated the degree to which auditory memory through sensory adaptation influences VTs. Specifically, we hypothesized that when memory persistence is longer, participants are able to retain the current verbal form longer, resulting in sensory adaptation, which in turn, affects auditory perception. Participants performed VT and auditory memory tasks on different days. In the VT task, Japanese participants continuously reported their perception while listening to a Japanese word (2- or 3-mora in length) played repeatedly for 5 min. In the auditory memory task, a different sequence of three morae, e.g., /ka/, /hi/, and /su/, was presented to each ear simultaneously. After some period (0-4 s), participants were visually cued to recall one of the sequences, i.e., in the left or right ear. We found that delayed recall accuracy was negatively correlated with the number of VTs, particularly under 2-mora conditions. This suggests that memory persistence is important for formation and selection of perceptual objects.

Coalescence of limit cycles in the presence of noise

Complex dynamical systems may exhibit multiple steady states, including time-periodic limit cycles, where the final trajectory depends on initial conditions. With tuning of parameters, limit cycles can proliferate or merge at an exceptional point. Here we ask how dynamics in the vicinity of such a bifurcation are influenced by noise. A pitchfork bifurcation can be used to induce bifurcation behavior. We model a limit cycle with the normal form of the Hopf oscillator, couple it to the pitchfork, and investigate the resulting dynamical system in the presence of noise. We show that the generating functional for the averages of the dynamical variables factorizes between the pitchfork and the oscillator. The statistical properties of the pitchfork in the presence of noise in its various regimes are investigated and a scaling theory is developed for the correlation and response functions, including a possible symmetry-breaking field. The analysis is done by perturbative calculations as well as numerical means. Finally, observables illustrating the coupling of a system with a limit cycle to a pitchfork are discussed and the phase-phase correlations are shown to exhibit nondiffusive behavior with universal scaling.

An Accumulating Neural Signal Underlying Binocular Rivalry Dynamics

During binocular rivalry, conflicting images are presented one to each eye and perception alternates stochastically between them. Despite stable percepts between alternations, modeling suggests that neural signals representing the two images change gradually, and that the duration of stable percepts are determined by the time required for these signals to reach a threshold that triggers an alternation. However, direct physiological evidence for such signals has been lacking. Here, we identify a neural signal in the human visual cortex that shows these predicted properties. We measured steady-state visual evoked potentials (SSVEPs) in 84 human participants (62 females, 22 males) who were presented with orthogonal gratings, one to each eye, flickering at different frequencies. Participants indicated their percept while EEG data were collected. The time courses of the SSVEP amplitudes at the two frequencies were then compared across different percept durations, within participants. For all durations, the amplitude of signals corresponding to the suppressed stimulus increased and the amplitude corresponding to the dominant stimulus decreased throughout the percept. Critically, longer percepts were characterized by more gradual increases in the suppressed signal and more gradual decreases of the dominant signal. Changes in signals were similar and rapid at the end of all percepts, presumably reflecting perceptual transitions. These features of the SSVEP time courses are well predicted by a model in which perceptual transitions are produced by the accumulation of noisy signals. Identification of this signal underlying binocular rivalry should allow strong tests of neural models of rivalry, bistable perception, and neural suppression.SIGNIFICANCE STATEMENT During binocular rivalry, two conflicting images are presented to the two eyes and perception alternates between them, with switches occurring at seemingly random times. Rivalry is an important and longstanding model system in neuroscience, used for understanding neural suppression, intrinsic neural dynamics, and even the neural correlates of consciousness. All models of rivalry propose that it depends on gradually changing neural activity that on reaching some threshold triggers the perceptual switches. This manuscript reports the first physiological measurement of neural signals with that set of properties in human participants. The signals, measured with EEG in human observers, closely match the predictions of recent models of rivalry, and should pave the way for much future work.

Adaptation in the sensory cortex drives bistable switching during auditory stream segregation

Current theories of perception emphasize the role of neural adaptation, inhibitory competition, and noise as key components that lead to switches in perception. Supporting evidence comes from neurophysiological findings of specific neural signatures in modality-specific and supramodal brain areas that appear to be critical to switches in perception. We used functional magnetic resonance imaging to study brain activity around the time of switches in perception while participants listened to a bistable auditory stream segregation stimulus, which can be heard as one integrated stream of tones or two segregated streams of tones. The auditory thalamus showed more activity around the time of a switch from segregated to integrated compared to time periods of stable perception of integrated; in contrast, the rostral anterior cingulate cortex and the inferior parietal lobule showed more activity around the time of a switch from integrated to segregated compared to time periods of stable perception of segregated streams, consistent with prior findings of asymmetries in brain activity depending on the switch direction. In sound-responsive areas in the auditory cortex, neural activity increased in strength preceding switches in perception and declined in strength over time following switches in perception. Such dynamics in the auditory cortex are consistent with the role of adaptation proposed by computational models of visual and auditory bistable switching, whereby the strength of neural activity decreases following a switch in perception, which eventually destabilizes the current percept enough to lead to a switch to an alternative percept.

Mesoscopic description of hippocampal replay and metastability in spiking neural networks with short-term plasticity

Bottom-up models of functionally relevant patterns of neural activity provide an explicit link between neuronal dynamics and computation. A prime example of functional activity patterns are propagating bursts of place-cell activities called hippocampal replay, which is critical for memory consolidation. The sudden and repeated occurrences of these burst states during ongoing neural activity suggest metastable neural circuit dynamics. As metastability has been attributed to noise and/or slow fatigue mechanisms, we propose a concise mesoscopic model which accounts for both. Crucially, our model is bottom-up: it is analytically derived from the dynamics of finite-size networks of Linear-Nonlinear Poisson neurons with short-term synaptic depression. As such, noise is explicitly linked to stochastic spiking and network size, and fatigue is explicitly linked to synaptic dynamics. To derive the mesoscopic model, we first consider a homogeneous spiking neural network and follow the temporal coarse-graining approach of Gillespie to obtain a "chemical Langevin equation", which can be naturally interpreted as a stochastic neural mass model. The Langevin equation is computationally inexpensive to simulate and enables a thorough study of metastable dynamics in classical setups (population spikes and Up-Down-states dynamics) by means of phase-plane analysis. An extension of the Langevin equation for small network sizes is also presented. The stochastic neural mass model constitutes the basic component of our mesoscopic model for replay. We show that the mesoscopic model faithfully captures the statistical structure of individual replayed trajectories in microscopic simulations and in previously reported experimental data. Moreover, compared to the deterministic Romani-Tsodyks model of place-cell dynamics, it exhibits a higher level of variability regarding order, direction and timing of replayed trajectories, which seems biologically more plausible and could be functionally desirable. This variability is the product of a new dynamical regime where metastability emerges from a complex interplay between finite-size fluctuations and local fatigue.

Signal neutrality, scalar property, and collapsing boundaries as consequences of a learned multi-timescale strategy

We postulate that three fundamental elements underlie a decision making process: perception of time passing, information processing in multiple timescales and reward maximisation. We build a simple reinforcement learning agent upon these principles that we train on a random dot-like task. Our results, similar to the experimental data, demonstrate three emerging signatures. (1) signal neutrality: insensitivity to the signal coherence in the interval preceding the decision. (2) Scalar property: the mean of the response times varies widely for different signal coherences, yet the shape of the distributions stays almost unchanged. (3) Collapsing boundaries: the “effective” decision-making boundary changes over time in a manner reminiscent of the theoretical optimal. Removing the perception of time or the multiple timescales from the model does not preserve the distinguishing signatures. Our results suggest an alternative explanation for signal neutrality. We propose that it is not part of motor planning. It is part of the decision-making process and emerges from information processing on multiple timescales.

Hierarchical processing underpins competition in tactile perceptual bistability

Ambiguous sensory information can lead to spontaneous alternations between perceptual states, recently shown to extend to tactile perception. The authors recently proposed a simplified form of tactile rivalry which evokes two competing percepts for a fixed difference in input amplitudes across antiphase, pulsatile stimulation of the left and right fingers. This study addresses the need for a tactile rivalry model that captures the dynamics of perceptual alternations and that incorporates the structure of the somatosensory system. The model features hierarchical processing with two stages. The first and the second stages of model could be located at the secondary somatosensory cortex (area S2), or in higher areas driven by S2. The model captures dynamical features specific to the tactile rivalry percepts and produces general characteristics of perceptual rivalry: input strength dependence of dominance times (Levelt's proposition II), short-tailed skewness of dominance time distributions and the ratio of distribution moments. The presented modelling work leads to experimentally testable predictions. The same hierarchical model could generalise to account for percept formation, competition and alternations for bistable stimuli that involve pulsatile inputs from the visual and auditory domains.

Neuronal population dynamics during motor plan cancellation in nonhuman primates

To understand the cortical neuronal dynamics behind movement generation and control, most studies have focused on tasks where actions were planned and then executed using different instances of visuomotor transformations. However, to fully understand the dynamics related to movement control, one must also study how movements are actively inhibited. Inhibition, indeed, represents the first level of control both when different alternatives are available and only one solution could be adopted and when it is necessary to maintain the current position. We recorded neuronal activity from a multielectrode array in the dorsal premotor cortex (PMd) of monkeys performing a countermanding reaching task that requires, in a subset of trials, them to cancel a planned movement before its onset. In the analysis of the neuronal state space of PMd, we found a subspace in which activities conveying temporal information were confined during active inhibition and position holding. Movement execution required activities to escape from this subspace toward an orthogonal subspace and, furthermore, surpass a threshold associated with the maturation of the motor plan. These results revealed further details in the neuronal dynamics underlying movement control, extending the hypothesis that neuronal computation confined in an "output-null" subspace does not produce movements.

Neural mechanisms underlying the temporal organization of naturalistic animal behavior

Naturalistic animal behavior exhibits a strikingly complex organization in the temporal domain, with variability arising from at least three sources: hierarchical, contextual, and stochastic. What neural mechanisms and computational principles underlie such intricate temporal features? In this review, we provide a critical assessment of the existing behavioral and neurophysiological evidence for these sources of temporal variability in naturalistic behavior. Recent research converges on an emergent mechanistic theory of temporal variability based on attractor neural networks and metastable dynamics, arising via coordinated interactions between mesoscopic neural circuits. We highlight the crucial role played by structural heterogeneities as well as noise from mesoscopic feedback loops in regulating flexible behavior. We assess the shortcomings and missing links in the current theoretical and experimental literature and propose new directions of investigation to fill these gaps.

Metastable dynamics of neural circuits and networks

Cortical neurons emit seemingly erratic trains of action potentials or "spikes," and neural network dynamics emerge from the coordinated spiking activity within neural circuits. These rich dynamics manifest themselves in a variety of patterns, which emerge spontaneously or in response to incoming activity produced by sensory inputs. In this Review, we focus on neural dynamics that is best understood as a sequence of repeated activations of a number of discrete hidden states. These transiently occupied states are termed "metastable" and have been linked to important sensory and cognitive functions. In the rodent gustatory cortex, for instance, metastable dynamics have been associated with stimulus coding, with states of expectation, and with decision making. In frontal, parietal, and motor areas of macaques, metastable activity has been related to behavioral performance, choice behavior, task difficulty, and attention. In this article, we review the experimental evidence for neural metastable dynamics together with theoretical approaches to the study of metastable activity in neural circuits. These approaches include (i) a theoretical framework based on non-equilibrium statistical physics for network dynamics; (ii) statistical approaches to extract information about metastable states from a variety of neural signals; and (iii) recent neural network approaches, informed by experimental results, to model the emergence of metastable dynamics. By discussing these topics, we aim to provide a cohesive view of how transitions between different states of activity may provide the neural underpinnings for essential functions such as perception, memory, expectation, or decision making, and more generally, how the study of metastable neural activity may advance our understanding of neural circuit function in health and disease.

Alternative female and male developmental trajectories in the dynamic balance of human visual perception

The numerous multistable phenomena in vision, hearing and touch attest that the inner workings of perception are prone to instability. We investigated a visual example-binocular rivalry-with an accurate no-report paradigm, and uncovered developmental and maturational lifespan trajectories that were specific for age and sex. To interpret these trajectories, we hypothesized that conflicting objectives of visual perception-such as stability of appearance, sensitivity to visual detail, and exploration of fundamental alternatives-change in relative importance over the lifespan. Computational modelling of our empirical results allowed us to estimate this putative development of stability, sensitivity, and exploration over the lifespan. Our results confirmed prior findings of developmental psychology and appear to quantify important aspects of neurocognitive phenotype. Additionally, we report atypical function of binocular rivalry in autism spectrum disorder and borderline personality disorder. Our computational approach offers new ways of quantifying neurocognitive phenotypes both in development and in dysfunction.

Bi-Stable Perception: Self-Coordinating Brain Regions to Make-Up the Mind

Bi-stable perception is a strong instance of cognitive self-organization, providing a research model for how ‘the brain makes up its mind.’ The complexity of perceptual bistability prevents a simple attribution of functions to areas, because many cognitive processes, recruiting multiple brain regions, are simultaneously involved. The functional magnetic resonance imaging (fMRI) evidence suggests the activation of a large network of distant brain areas. Concurrently, electroencephalographic and magnetoencephalographic (MEEG) literature shows sub second oscillatory activity and phase synchrony on several frequency bands. Strongly represented are beta and gamma bands, often associated with neural/cognitive integration processes. The spatial extension and short duration of brain activities suggests the need for a fast, large-scale neural coordination mechanism. To address the range of temporo-spatial scales involved, we systematize the current knowledge from mathematical models, cognitive sciences and neuroscience at large, from single-cell- to system-level research, including evidence from human and non-human primates. Surprisingly, despite evidence spanning through different organization levels, models, and experimental approaches, the scarcity of integrative studies is evident. In a final section of the review we dwell on the reasons behind such scarcity and on the need of integration in order to achieve a real understanding of the complexities underlying bi-stable perception processes.

Binocular rivalry reveals an out-of-equilibrium neural dynamics suited for decision-making

In ambiguous or conflicting sensory situations, perception is often ‘multistable’ in that it perpetually changes at irregular intervals, shifting abruptly between distinct alternatives. The interval statistics of these alternations exhibits quasi-universal characteristics, suggesting a general mechanism. Using binocular rivalry, we show that many aspects of this perceptual dynamics are reproduced by a hierarchical model operating out of equilibrium. The constitutive elements of this model idealize the metastability of cortical networks. Independent elements accumulate visual evidence at one level, while groups of coupled elements compete for dominance at another level. As soon as one group dominates perception, feedback inhibition suppresses supporting evidence. Previously unreported features in the serial dependencies of perceptual alternations compellingly corroborate this mechanism. Moreover, the proposed out-of-equilibrium dynamics satisfies normative constraints of continuous decision-making. Thus, multistable perception may reflect decision-making in a volatile world: integrating evidence over space and time, choosing categorically between hypotheses, while concurrently evaluating alternatives.

Perceptual rivalry with vibrotactile stimuli

In perceptual rivalry, ambiguous sensory information leads to dynamic changes in the perceptual interpretation of fixed stimuli. This phenomenon occurs when participants receive sensory stimuli that support two or more distinct interpretations; this results in spontaneous alternations between possible perceptual interpretations. Perceptual rivalry has been widely studied across different sensory modalities including vision, audition, and to a limited extent, in the tactile domain. Common features of perceptual rivalry across various ambiguous visual and auditory paradigms characterize the randomness of switching times and their dependence on input strength manipulations (Levelt's propositions). It is still unclear whether the general characteristics of perceptual rivalry are preserved with tactile stimuli. This study aims to introduce a simple tactile stimulus capable of generating perceptual rivalry and explores whether general features of perceptual rivalry from other modalities extend to the tactile domain. Our results confirm that Levelt's proposition II extends to tactile bistability, and that the stochastic characteristics of irregular perceptual alternations agree with non-tactile modalities. An analysis of correlations between subsequent perceptual phases reveals a significant positive correlation at lag 1 (as found in visual bistability), and a negative correlation for lag 2 (in contrast with visual bistability).

Parameter dependence in visual pattern-component rivalry at onset and during prolonged viewing

In multistability, perceptual interpretations ("percepts") of ambiguous stimuli alternate over time. There is considerable debate as to whether similar regularities govern the first percept after stimulus onset and percepts during prolonged presentation. We address this question in a visual pattern-component rivalry paradigm by presenting two overlaid drifting gratings, which participants perceived as individual gratings passing in front of each other ("segregated") or as a plaid ("integrated"). We varied the enclosed angle ("opening angle") between the gratings (experiments 1 and 2) and stimulus orientation (experiment 2). The relative number of integrated percepts increased monotonically with opening angle. The point of equality, where half of the percepts were integrated, was at a smaller opening angle at onset than during prolonged viewing. The functional dependence of the relative number of integrated percepts on opening angle showed a steeper curve at onset than during prolonged viewing. Dominance durations of integrated percepts were longer at onset than during prolonged viewing and increased with opening angle. The general pattern persisted when stimuli were rotated (experiment 2), despite some perceptual preference for cardinal motion directions over oblique directions. Analysis of eye movements, specifically the slow phase of the optokinetic nystagmus (OKN), confirmed the veridicality of participants' reports and provided a temporal characterization of percept formation after stimulus onset. Together, our results show that the first percept after stimulus onset exhibits a different dependence on stimulus parameters than percepts during prolonged viewing. This underlines the distinct role of the first percept in multistability.

Buildup and bistability in auditory streaming as an evidence accumulation process with saturation

A repeating triplet-sequence ABA- of non-overlapping brief tones, A and B, is a valued paradigm for studying auditory stream formation and the cocktail party problem. The stimulus is "heard" either as a galloping pattern (integration) or as two interleaved streams (segregation); the initial percept is typically integration then followed by spontaneous alternations between segregation and integration, each being dominant for a few seconds. The probability of segregation grows over seconds, from near-zero to a steady value, defining the buildup function, BUF. Its stationary level increases with the difference in tone frequencies, DF, and the BUF rises faster. Percept durations have DF-dependent means and are gamma-like distributed. Behavioral and computational studies usually characterize triplet streaming either during alternations or during buildup. Here, our experimental design and modeling encompass both. We propose a pseudo-neuromechanistic model that incorporates spiking activity in primary auditory cortex, A1, as input and resolves perception along two network-layers downstream of A1. Our model is straightforward and intuitive. It describes the noisy accumulation of evidence against the current percept which generates switches when reaching a threshold. Accumulation can saturate either above or below threshold; if below, the switching dynamics resemble noise-induced transitions from an attractor state. Our model accounts quantitatively for three key features of data: the BUFs, mean durations, and normalized dominance duration distributions, at various DF values. It describes perceptual alternations without competition per se, and underscores that treating triplets in the sequence independently and averaging across trials, as implemented in earlier widely cited studies, is inadequate.

A Network Activity Reconfiguration Underlies the Transition from Goal to Action

Neurons in prefrontal cortex (PF) represent mnemonic information about current goals until the action can be selected and executed. However, the neuronal dynamics underlying the transition from goal into specific actions are poorly understood. Here, we show that the goal-coding PF network is dynamically reconfigured from mnemonic to action selection states and that such reconfiguration is mediated by cell assemblies with heterogeneous excitability. We recorded neuronal activity from PF while monkeys selected their actions on the basis of memorized goals. Many PF neurons encoded the goal, but only a minority of them did so across both memory retention and action selection stages. Interestingly, about half of this minority of neurons switched their goal preference across the goal-action transition. Our computational model led us to propose a PF network composed of heterogeneous cell assemblies with single-state and bistable local dynamics able to produce both dynamical stability and input susceptibility simultaneously.

Coding Perceptual Decisions: From Single Units to Emergent Signaling Properties in Cortical Circuits

Spiking activity in single neurons of the primate visual cortex has been tightly linked to perceptual decisions. Any mechanism that reads out these perceptual signals to support behavior must respect the underlying neuroanatomy that shapes the functional properties of sensory neurons. Spatial distribution and timing of inputs to the next processing levels are critical, as conjoint activity of precursor neurons increases the spiking rate of downstream neurons and ultimately drives behavior. I set out how correlated activity might coalesce into a micropool of task-sensitive neurons signaling a particular percept to determine perceptual decision signals locally and for flexible interarea transmission depending on the task context. As data from more and more neurons and their complex interactions are analyzed, the space of computational mechanisms must be constrained based on what is plausible within neurobiological limits. This review outlines experiments to test the new perspectives offered by these extended methods.

Ensemble modeling of auditory streaming reveals potential sources of bistability across the perceptual hierarchy

Perceptual bistability-the spontaneous, irregular fluctuation of perception between two interpretations of a stimulus-occurs when observing a large variety of ambiguous stimulus configurations. This phenomenon has the potential to serve as a tool for, among other things, understanding how function varies across individuals due to the large individual differences that manifest during perceptual bistability. Yet it remains difficult to interpret the functional processes at work, without knowing where bistability arises during perception. In this study we explore the hypothesis that bistability originates from multiple sources distributed across the perceptual hierarchy. We develop a hierarchical model of auditory processing comprised of three distinct levels: a Peripheral, tonotopic analysis, a Central analysis computing features found more centrally in the auditory system, and an Object analysis, where sounds are segmented into different streams. We model bistable perception within this system by applying adaptation, inhibition and noise into one or all of the three levels of the hierarchy. We evaluate a large ensemble of variations of this hierarchical model, where each model has a different configuration of adaptation, inhibition and noise. This approach avoids the assumption that a single configuration must be invoked to explain the data. Each model is evaluated based on its ability to replicate two hallmarks of bistability during auditory streaming: the selectivity of bistability to specific stimulus configurations, and the characteristic log-normal pattern of perceptual switches. Consistent with a distributed origin, a broad range of model parameters across this hierarchy lead to a plausible form of perceptual bistability.

Intraindividual Consistency Between Auditory and Visual Multistability

We compared perceptual multistability across modalities, using a visual plaid pattern (composed of two transparently overlaid drifting gratings) and auditory streaming (elicited by a repeating “ABA_” tone sequence). Both stimuli can be perceived as integrated (one plaid pattern, one stream comprising “A” and “B” tones) or segregated (two individual gratings, two tone streams). In the segregated case, either stream or grating can be perceived in the foreground. We queried auditory and visual perception with these three response options. We found that perceptual dominance of the integrated states was correlated across modalities: Participants who were more likely to perceive the plaid were also more likely to perceive the integrated auditory stream. When presenting both stimuli simultaneously and querying the auditory percept, eye-movement data showed that perceiving auditory and visual integration is related on a moment-by-moment basis. This suggests that in part common mechanisms underlie multistability in visual and auditory perception.

Switch rates for orthogonally oriented kinetic-depth displays are correlated across observers

When continuously viewing multistable displays, which are compatible with several comparably likely interpretations, perception perpetually switches between available alternatives. Prior studies typically report the lack of consistent individual switch rates across different displays. However, this comparison is based on an assumption that neural representations of physically identical displays are consistent across observers. Yet, given how different individuals are already at the level of the retina, it is likely that the difference in other relevant factors might mask the correlation. To address this issue, we compared switch rates in two kinetic-depth displays (KDE) that rotated around orthogonal axes (45° counterclockwise vs. 45° clockwise relative to the vertical). This ensured that dynamics of multistable perception was based on highly similar, but different and independent neural representations. We also included a Necker cube (NC) display as a control. We report that switch rates were correlated between two kinetic-depth effect displays, but not between either of the KDE and NC displays. This demonstrates that the usual lack of correlation may not be evidence for the lack of a shared pacesetter mechanism of multistable perception, but reflect other factors, such as differently modulated inputs to competing representations. In addition, we asked participants to speed-up or slow-down perceptual alternations and found that only the former ability was correlated across different displays. This indicates that these two types of volitional control may differ in their use of attentional resources.

Auditory streaming and bistability paradigm extended to a dynamic environment

We explore stream segregation with temporally modulated acoustic features using behavioral experiments and modelling. The auditory streaming paradigm in which alternating high- A and low-frequency tones B appear in a repeating ABA-pattern, has been shown to be perceptually bistable for extended presentations (order of minutes). For a fixed, repeating stimulus, perception spontaneously changes (switches) at random times, every 2-15 s, between an integrated interpretation with a galloping rhythm and segregated streams. Streaming in a natural auditory environment requires segregation of auditory objects with features that evolve over time. With the relatively idealized ABA-triplet paradigm, we explore perceptual switching in a non-static environment by considering slowly and periodically varying stimulus features. Our previously published model captures the dynamics of auditory bistability and predicts here how perceptual switches are entrained, tightly locked to the rising and falling phase of modulation. In psychoacoustic experiments we find that entrainment depends on both the period of modulation and the intrinsic switch characteristics of individual listeners. The extended auditory streaming paradigm with slowly modulated stimulus features presented here will be of significant interest for future imaging and neurophysiology experiments by reducing the need for subjective perceptual reports of ongoing perception.

Dynamical modeling of multi-scale variability in neuronal competition

Variability is observed at multiple-scales in the brain and ubiquitous in perception. However, the nature of perceptual variability is an open question. We focus on variability during perceptual rivalry, a form of neuronal competition. Rivalry provides a window into neural processing since activity in many brain areas is correlated to the alternating perception rather than a constant ambiguous stimulus. It exhibits robust properties at multiple scales including conscious awareness and neuron dynamics. The prevalent theory for spiking variability is called the balanced state; whereas, the source of perceptual variability is unknown. Here we show that a single biophysical circuit model, satisfying certain mutual inhibition architectures, can explain spiking and perceptual variability during rivalry. These models adhere to a broad set of strict experimental constraints at multiple scales. As we show, the models predict how spiking and perceptual variability changes with stimulus conditions.

Neural Signatures of Auditory Perceptual Bistability Revealed by Large-Scale Human Intracranial Recordings

A key challenge in neuroscience is understanding how sensory stimuli give rise to perception, especially when the process is supported by neural activity from an extended network of brain areas. Perception is inherently subjective, so interrogating its neural signatures requires, ideally, a combination of three factors: (1) behavioral tasks that separate stimulus-driven activity from perception per se; (2) human subjects who self-report their percepts while performing those tasks; and (3) concurrent neural recordings acquired at high spatial and temporal resolution. In this study, we analyzed human electrocorticographic recordings obtained during an auditory task which supported mutually exclusive perceptual interpretations. Eight neurosurgical patients (5 male; 3 female) listened to sequences of repeated triplets where tones were separated in frequency by several semitones. Subjects reported spontaneous alternations between two auditory perceptual states, 1-stream and 2-stream, by pressing a button. We compared averaged auditory evoked potentials (AEPs) associated with 1-stream and 2-stream percepts and identified significant differences between them in primary and nonprimary auditory cortex, surrounding auditory-related temporoparietal cortex, and frontal areas. We developed classifiers to identify spatial maps of percept-related differences in the AEP, corroborating findings from statistical analysis. We used one-dimensional embedding spaces to perform the group-level analysis. Our data illustrate exemplar high temporal resolution AEP waveforms in auditory core region; explain inconsistencies in perceptual effects within auditory cortex, reported across noninvasive studies of streaming of triplets; show percept-related changes in frontoparietal areas previously highlighted by studies that focused on perceptual transitions; and demonstrate that auditory cortex encodes maintenance of percepts and switches between them.

SIGNIFICANCE STATEMENT The human brain has the remarkable ability to discern complex and ambiguous stimuli from the external world by parsing mixed inputs into interpretable segments. However, one's perception can deviate from objective reality. But how do perceptual discrepancies occur? What are their anatomical substrates? To address these questions, we performed intracranial recordings in neurosurgical patients as they reported their perception of sounds associated with two mutually exclusive interpretations. We identified signatures of subjective percepts as distinct from sound-driven brain activity in core and non-core auditory cortex and frontoparietal cortex. These findings were compared with previous studies of auditory bistable perception and suggested that perceptual transitions and maintenance of perceptual states were supported by common neural substrates.

Interneuronal correlations at longer time scales predict decision signals for bistable structure-from-motion perception

Perceptual decisions are thought to depend on the activation of task-relevant neurons, whose activity is often correlated in time. Here, we examined how the temporal structure of shared variability in neuronal firing relates to perceptual choices. We recorded stimulus-selective neurons from visual area V5/MT while two monkeys (Macaca mulatta) made perceptual decisions about the rotation direction of structure-from-motion cylinders. Interneuronal correlations for a perceptually ambiguous cylinder stimulus were significantly higher than those for unambiguous cylinders or for random 2D motion during passive viewing. Much of the difference arose from correlations at relatively long timescales (hundreds of milliseconds). Choice-related neural activity (quantified as choice probability; CP) for ambiguous cylinders was positively correlated with interneuronal correlations and was specifically associated with their long timescale component. Furthermore, the slope of the long timescale - but not the instantaneous - component of the correlation predicted higher CPs towards the end of the trial i.e. close to the decision. Our results suggest that the perceptual stability of structure-from-motion cylinders may be controlled by enhanced interneuronal correlations on longer timescales. We propose this as a potential signature of top-down influences onto V5/MT processing that shape and stabilize the appearance of 3D-motion percepts.

Computational models of auditory perception from feature extraction to stream segregation and behavior

Audition is by nature dynamic, from brainstem processing on sub-millisecond time scales, to segregating and tracking sound sources with changing features, to the pleasure of listening to music and the satisfaction of getting the beat. We review recent advances from computational models of sound localization, of auditory stream segregation and of beat perception/generation. A wealth of behavioral, electrophysiological and imaging studies shed light on these processes, typically with synthesized sounds having regular temporal structure. Computational models integrate knowledge from different experimental fields and at different levels of description. We advocate a neuromechanistic modeling approach that incorporates knowledge of the auditory system from various fields, that utilizes plausible neural mechanisms, and that bridges our understanding across disciplines.

Cortical computations via metastable activity

Metastable brain dynamics are characterized by abrupt, jump-like modulations so that the neural activity in single trials appears to unfold as a sequence of discrete, quasi-stationary 'states'. Evidence that cortical neural activity unfolds as a sequence of metastable states is accumulating at fast pace. Metastable activity occurs both in response to an external stimulus and during ongoing, self-generated activity. These spontaneous metastable states are increasingly found to subserve internal representations that are not locked to external triggers, including states of deliberations, attention and expectation. Moreover, decoding stimuli or decisions via metastable states can be carried out trial-by-trial. Focusing on metastability will allow us to shift our perspective on neural coding from traditional concepts based on trial-averaging to models based on dynamic ensemble representations. Recent theoretical work has started to characterize the mechanistic origin and potential roles of metastable representations. In this article we review recent findings on metastable activity, how it may arise in biologically realistic models, and its potential role for representing internal states as well as relevant task variables.

Perceptual reversals in binocular rivalry: Improved detection from OKN

When binocular rivalry is induced by opponent motion displays, perceptual reversals are often associated with changed oculomotor behavior (Frässle, Sommer, Jansen, Naber, & Einhäuser, 2014; Fujiwara et al., 2017). Specifically, the direction of smooth pursuit phases in optokinetic nystagmus typically corresponds to the direction of motion that dominates perceptual appearance at any given time. Here we report an improved analysis that continuously estimates perceived motion in terms of "cumulative smooth pursuit." In essence, smooth pursuit segments are identified, interpolated where necessary, and joined probabilistically into a continuous record of cumulative smooth pursuit (i.e., probability of eye position disregarding blinks, saccades, signal losses, and artefacts). The analysis is fully automated and robust in healthy, developmental, and patient populations. To validate reliability, we compare volitional reports of perceptual reversals in rivalry displays, and of physical reversals in nonrivalrous control displays. Cumulative smooth pursuit detects physical reversals and estimates eye velocity more accurately than existing methods do (Frässle et al., 2014). It also appears to distinguish dominant and transitional perceptual states, detecting changes with a precision of ±100 ms. We conclude that cumulative smooth pursuit significantly improves the monitoring of binocular rivalry by means of recording optokinetic nystagmus.

Similar but separate systems underlie perceptual bistability in vision and audition

The dynamics of perceptual bistability, the phenomenon in which perception switches between different interpretations of an unchanging stimulus, are characterised by very similar properties across a wide range of qualitatively different paradigms. This suggests that perceptual switching may be triggered by some common source. However, it is also possible that perceptual switching may arise from a distributed system, whose components vary according to the specifics of the perceptual experiences involved. Here we used a visual and an auditory task to determine whether individuals show cross-modal commonalities in perceptual switching. We found that individual perceptual switching rates were significantly correlated across modalities. We then asked whether perceptual switching arises from some central (modality-) task-independent process or from a more distributed task-specific system. We found that a log-normal distribution best explained the distribution of perceptual phases in both modalities, suggestive of a combined set of independent processes causing perceptual switching. Modality- and/or task-dependent differences in these distributions, and lack of correlation with the modality-independent central factors tested (ego-resiliency, creativity, and executive function), also point towards perceptual switching arising from a distributed system of similar but independent processes.

A hierarchical stochastic model for bistable perception

Viewing of ambiguous stimuli can lead to bistable perception alternating between the possible percepts. During continuous presentation of ambiguous stimuli, percept changes occur as single events, whereas during intermittent presentation of ambiguous stimuli, percept changes occur at more or less regular intervals either as single events or bursts. Response patterns can be highly variable and have been reported to show systematic differences between patients with schizophrenia and healthy controls. Existing models of bistable perception often use detailed assumptions and large parameter sets which make parameter estimation challenging. Here we propose a parsimonious stochastic model that provides a link between empirical data analysis of the observed response patterns and detailed models of underlying neuronal processes. Firstly, we use a Hidden Markov Model (HMM) for the times between percept changes, which assumes one single state in continuous presentation and a stable and an unstable state in intermittent presentation. The HMM captures the observed differences between patients with schizophrenia and healthy controls, but remains descriptive. Therefore, we secondly propose a hierarchical Brownian model (HBM), which produces similar response patterns but also provides a relation to potential underlying mechanisms. The main idea is that neuronal activity is described as an activity difference between two competing neuronal populations reflected in Brownian motions with drift. This differential activity generates switching between the two conflicting percepts and between stable and unstable states with similar mechanisms on different neuronal levels. With only a small number of parameters, the HBM can be fitted closely to a high variety of response patterns and captures group differences between healthy controls and patients with schizophrenia. At the same time, it provides a link to mechanistic models of bistable perception, linking the group differences to potential underlying mechanisms.

Patients suffering from schizophrenia show specific cognitive deficits. One way to study these cognitive phenomena works with the presentation of ambiguous stimuli. During viewing of an ambiguous stimulus, perception alters spontaneously between different percepts. Percept changes occur as single events during continuous presentation, whereas during intermittent presentation, percept changes occur at regular intervals either as single events or bursts. Here we investigate perceptual responses to continuous and intermittent stimulation in healthy control subjects and patients with schizophrenia. Interestingly, the response patterns can be highly variable but show systematic group differences. We propose a model that connects these perceptual responses to underlying neuronal processes. The model mainly describes the activity difference between competing neuronal populations on different neuronal levels. In a hierarchical manner, the differential activity generates switching between the conflicting percepts as well as between states of higher and lower perceptual stability. By fitting the model directly to empirical responses, a high variety of patterns can be reproduced, and group differences between healthy controls and patients with schizophrenia can be captured. This helps to link the observed group differences to potential neuronal mechanisms, suggesting that patients with schizophrenia tend to spend more time in neuronal states of lower perceptual stability.

Attention model of binocular rivalry

When the corresponding retinal locations in the two eyes are presented with incompatible images, a stable percept gives way to perceptual alternations in which the two images compete for perceptual dominance. As perceptual experience evolves dynamically under constant external inputs, binocular rivalry has been used for studying intrinsic cortical computations and for understanding how the brain regulates competing inputs. Converging behavioral and EEG results have shown that binocular rivalry and attention are intertwined: binocular rivalry ceases when attention is diverted away from the rivalry stimuli. In addition, the competing image in one eye suppresses the target in the other eye through a pattern of gain changes similar to those induced by attention. These results require a revision of the current computational theories of binocular rivalry, in which the role of attention is ignored. Here, we provide a computational model of binocular rivalry. In the model, competition between two images in rivalry is driven by both attentional modulation and mutual inhibition, which have distinct selectivity (feature vs. eye of origin) and dynamics (relatively slow vs. relatively fast). The proposed model explains a wide range of phenomena reported in rivalry, including the three hallmarks: (i) binocular rivalry requires attention; (ii) various perceptual states emerge when the two images are swapped between the eyes multiple times per second; (iii) the dominance duration as a function of input strength follows Levelt's propositions. With a bifurcation analysis, we identified the parameter space in which the model's behavior was consistent with experimental results.

Towards a theory of cortical columns: From spiking neurons to interacting neural populations of finite size

Neural population equations such as neural mass or field models are widely used to study brain activity on a large scale. However, the relation of these models to the properties of single neurons is unclear. Here we derive an equation for several interacting populations at the mesoscopic scale starting from a microscopic model of randomly connected generalized integrate-and-fire neuron models. Each population consists of 50–2000 neurons of the same type but different populations account for different neuron types. The stochastic population equations that we find reveal how spike-history effects in single-neuron dynamics such as refractoriness and adaptation interact with finite-size fluctuations on the population level. Efficient integration of the stochastic mesoscopic equations reproduces the statistical behavior of the population activities obtained from microscopic simulations of a full spiking neural network model. The theory describes nonlinear emergent dynamics such as finite-size-induced stochastic transitions in multistable networks and synchronization in balanced networks of excitatory and inhibitory neurons. The mesoscopic equations are employed to rapidly integrate a model of a cortical microcircuit consisting of eight neuron types, which allows us to predict spontaneous population activities as well as evoked responses to thalamic input. Our theory establishes a general framework for modeling finite-size neural population dynamics based on single cell and synapse parameters and offers an efficient approach to analyzing cortical circuits and computations.

Understanding the brain requires mathematical models on different spatial scales. On the “microscopic” level of nerve cells, neural spike trains can be well predicted by phenomenological spiking neuron models. On a coarse scale, neural activity can be modeled by phenomenological equations that summarize the total activity of many thousands of neurons. Such population models are widely used to model neuroimaging data such as EEG, MEG or fMRI data. However, it is largely unknown how large-scale models are connected to an underlying microscale model. Linking the scales is vital for a correct description of rapid changes and fluctuations of the population activity, and is crucial for multiscale brain models. The challenge is to treat realistic spiking dynamics as well as fluctuations arising from the finite number of neurons. We obtained such a link by deriving stochastic population equations on the mesoscopic scale of 100–1000 neurons from an underlying microscopic model. These equations can be efficiently integrated and reproduce results of a microscopic simulation while achieving a high speed-up factor. We expect that our novel population theory on the mesoscopic scale will be instrumental for understanding experimental data on information processing in the brain, and ultimately link microscopic and macroscopic activity patterns.