Natural stimuli evoke dynamic sequences of states in sensory cortical ensembles

Toward a theoretical role for tonic norepinephrine in the orbitofrontal cortex in facilitating flexible learning

To adaptively respond in a complex, changing world, animals need to flexibly update their understanding of the world when their expectations are violated. Though several brain regions in rodents and primates have been implicated in aspects of this updating, current models of orbitofrontal cortex (OFC) and norepinephrine neurons of the locus coeruleus (LC-NE) suggest that each plays a role in responding to environmental change, where the OFC allows updating of prior learning to occur without overwriting or unlearning one's previous understanding of the world that changed, while elevated tonic NE allows for increased flexibility in behavior that tracks an animal's uncertainty. In light of recent studies highlighting a specific LC-NE projection to the OFC, in this review we discuss current models of OFC and NE function, and their potential synergy in the updating of associations following environmental change.

Large-Scale Simulations of Plastic Neural Networks on Neuromorphic Hardware

SpiNNaker is a digital, neuromorphic architecture designed for simulating large-scale spiking neural networks at speeds close to biological real-time. Rather than using bespoke analog or digital hardware, the basic computational unit of a SpiNNaker system is a general-purpose ARM processor, allowing it to be programmed to simulate a wide variety of neuron and synapse models. This flexibility is particularly valuable in the study of biological plasticity phenomena. A recently proposed learning rule based on the Bayesian Confidence Propagation Neural Network (BCPNN) paradigm offers a generic framework for modeling the interaction of different plasticity mechanisms using spiking neurons. However, it can be computationally expensive to simulate large networks with BCPNN learning since it requires multiple state variables for each synapse, each of which needs to be updated every simulation time-step. We discuss the trade-offs in efficiency and accuracy involved in developing an event-based BCPNN implementation for SpiNNaker based on an analytical solution to the BCPNN equations, and detail the steps taken to fit this within the limited computational and memory resources of the SpiNNaker architecture. We demonstrate this learning rule by learning temporal sequences of neural activity within a recurrent attractor network which we simulate at scales of up to 2.0 × 104 neurons and 5.1 × 107 plastic synapses: the largest plastic neural network ever to be simulated on neuromorphic hardware. We also run a comparable simulation on a Cray XC-30 supercomputer system and find that, if it is to match the run-time of our SpiNNaker simulation, the super computer system uses approximately 45× more power. This suggests that cheaper, more power efficient neuromorphic systems are becoming useful discovery tools in the study of plasticity in large-scale brain models.

Recurrent Spiking Networks Solve Planning Tasks

A recurrent spiking neural network is proposed that implements planning as probabilistic inference for finite and infinite horizon tasks. The architecture splits this problem into two parts: The stochastic transient firing of the network embodies the dynamics of the planning task. With appropriate injected input this dynamics is shaped to generate high-reward state trajectories. A general class of reward-modulated plasticity rules for these afferent synapses is presented. The updates optimize the likelihood of getting a reward through a variant of an Expectation Maximization algorithm and learning is guaranteed to convergence to a local maximum. We find that the network dynamics are qualitatively similar to transient firing patterns during planning and foraging in the hippocampus of awake behaving rats. The model extends classical attractor models and provides a testable prediction on identifying modulating contextual information. In a real robot arm reaching and obstacle avoidance task the ability to represent multiple task solutions is investigated. The neural planning method with its local update rules provides the basis for future neuromorphic hardware implementations with promising potentials like large data processing abilities and early initiation of strategies to avoid dangerous situations in robot co-worker scenarios.

Stimuli Reduce the Dimensionality of Cortical Activity

The activity of ensembles of simultaneously recorded neurons can be represented as a set of points in the space of firing rates. Even though the dimension of this space is equal to the ensemble size, neural activity can be effectively localized on smaller subspaces. The dimensionality of the neural space is an important determinant of the computational tasks supported by the neural activity. Here, we investigate the dimensionality of neural ensembles from the sensory cortex of alert rats during periods of ongoing (inter-trial) and stimulus-evoked activity. We find that dimensionality grows linearly with ensemble size, and grows significantly faster during ongoing activity compared to evoked activity. We explain these results using a spiking network model based on a clustered architecture. The model captures the difference in growth rate between ongoing and evoked activity and predicts a characteristic scaling with ensemble size that could be tested in high-density multi-electrode recordings. Moreover, we present a simple theory that predicts the existence of an upper bound on dimensionality. This upper bound is inversely proportional to the amount of pair-wise correlations and, compared to a homogeneous network without clusters, it is larger by a factor equal to the number of clusters. The empirical estimation of such bounds depends on the number and duration of trials and is well predicted by the theory. Together, these results provide a framework to analyze neural dimensionality in alert animals, its behavior under stimulus presentation, and its theoretical dependence on ensemble size, number of clusters, and correlations in spiking network models.

Visually Evoked Spiking Evolves While Spontaneous Ongoing Dynamics Persist

Neurons in the primary visual cortex spontaneously spike even when there are no visual stimuli. It is unknown whether the spiking evoked by visual stimuli is just a modification of the spontaneous ongoing cortical spiking dynamics or whether the spontaneous spiking state disappears and is replaced by evoked spiking. This study of laminar recordings of spontaneous spiking and visually evoked spiking of neurons in the ferret primary visual cortex shows that the spiking dynamics does not change: the spontaneous spiking as well as evoked spiking is controlled by a stable and persisting fixed point attractor. Its existence guarantees that evoked spiking return to the spontaneous state. However, the spontaneous ongoing spiking state and the visual evoked spiking states are qualitatively different and are separated by a threshold (separatrix). The functional advantage of this organization is that it avoids the need for a system reorganization following visual stimulation, and impedes the transition of spontaneous spiking to evoked spiking and the propagation of spontaneous spiking from layer 4 to layers 2-3.

A neural coding scheme reproducing foraging trajectories

The movement of many animals may follow Lévy patterns. The underlying generating neuronal dynamics of such a behavior is unknown. In this paper we show that a novel discovery of multifractality in winnerless competition (WLC) systems reveals a potential encoding mechanism that is translatable into two dimensional superdiffusive Lévy movements. The validity of our approach is tested on a conductance based neuronal model showing WLC and through the extraction of Lévy flights inducing fractals from recordings of rat hippocampus during open field foraging. Further insights are gained analyzing mice motor cortex neurons and non motor cell signals. The proposed mechanism provides a plausible explanation for the neuro-dynamical fundamentals of spatial searching patterns observed in animals (including humans) and illustrates an until now unknown way to encode information in neuronal temporal series.

State Dependency of Chemosensory Coding in the Gustatory Thalamus (VPMpc) of Alert Rats

The parvicellular portion of the ventroposteromedial nucleus (VPMpc) is the part of the thalamus that processes gustatory information. Anatomical evidence shows that the VPMpc receives ascending gustatory inputs from the parabrachial nucleus (PbN) in the brainstem and sends projections to the gustatory cortex (GC). Although taste processing in PbN and GC has been the subject of intense investigation in behaving rodents, much less is known on how VPMpc neurons encode gustatory information. Here we present results from single-unit recordings in the VPMpc of alert rats receiving multiple tastants. Thalamic neurons respond to taste with time-varying modulations of firing rates, consistent with those observed in GC and PbN. These responses encode taste quality as well as palatability. Comparing responses to tastants either passively delivered, or self-administered after a cue, unveiled the effects of general expectation on taste processing in VPMpc. General expectation led to an improvement of taste coding by modulating response dynamics, and single neuron ability to encode multiple tastants. Our results demonstrate that the time course of taste coding as well as single neurons' ability to encode for multiple qualities are not fixed but rather can be altered by the state of the animal. Together, the data presented here provide the first description that taste coding in VPMpc is dynamic and state-dependent.

SIGNIFICANCE STATEMENT

Over the past years, a great deal of attention has been devoted to understanding taste coding in the brainstem and cortex of alert rodents. Thanks to this research, we now know that taste coding is dynamic, distributed, and context-dependent. Alas, virtually nothing is known on how the gustatory thalamus (VPMpc) processes gustatory information in behaving rats. This manuscript investigates taste processing in the VPMpc of behaving rats. Our results show that thalamic neurons encode taste and palatability with time-varying patterns of activity and that thalamic coding of taste is modulated by general expectation. Our data will appeal not only to researchers interested in taste, but also to a broader audience of sensory and systems neuroscientists interested in the thalamocortical system.

State Dependency of Chemosensory Coding in the Gustatory Thalamus (VPMpc) of Alert Rats

The parvicellular portion of the ventroposteromedial nucleus (VPMpc) is the part of the thalamus that processes gustatory information. Anatomical evidence shows that the VPMpc receives ascending gustatory inputs from the parabrachial nucleus (PbN) in the brainstem and sends projections to the gustatory cortex (GC). Although taste processing in PbN and GC has been the subject of intense investigation in behaving rodents, much less is known on how VPMpc neurons encode gustatory information. Here we present results from single-unit recordings in the VPMpc of alert rats receiving multiple tastants. Thalamic neurons respond to taste with time-varying modulations of firing rates, consistent with those observed in GC and PbN. These responses encode taste quality as well as palatability. Comparing responses to tastants either passively delivered, or self-administered after a cue, unveiled the effects of general expectation on taste processing in VPMpc. General expectation led to an improvement of taste coding by modulating response dynamics, and single neuron ability to encode multiple tastants. Our results demonstrate that the time course of taste coding as well as single neurons' ability to encode for multiple qualities are not fixed but rather can be altered by the state of the animal. Together, the data presented here provide the first description that taste coding in VPMpc is dynamic and state-dependent.

SIGNIFICANCE STATEMENT

Over the past years, a great deal of attention has been devoted to understanding taste coding in the brainstem and cortex of alert rodents. Thanks to this research, we now know that taste coding is dynamic, distributed, and context-dependent. Alas, virtually nothing is known on how the gustatory thalamus (VPMpc) processes gustatory information in behaving rats. This manuscript investigates taste processing in the VPMpc of behaving rats. Our results show that thalamic neurons encode taste and palatability with time-varying patterns of activity and that thalamic coding of taste is modulated by general expectation. Our data will appeal not only to researchers interested in taste, but also to a broader audience of sensory and systems neuroscientists interested in the thalamocortical system.

The Encoding of Decision Difficulty and Movement Time in the Primate Premotor Cortex

Estimating the difficulty of a decision is a fundamental process to elaborate complex and adaptive behaviour. In this paper, we show that the movement time of behaving monkeys performing a decision-making task is correlated with decision difficulty and that the activity of a population of neurons in ventral Premotor cortex correlates with the movement time. Moreover, we found another population of neurons that encodes the discriminability of the stimulus, thereby supplying another source of information about the difficulty of the decision. The activity of neurons encoding the difficulty can be produced by very different computations. Therefore, we show that decision difficulty can be encoded through three different mechanisms: 1. Switch time coding, 2. rate coding and 3. binary coding. This rich representation reflects the basis of different functional aspects of difficulty in the making of a decision and the possible role of difficulty estimation in complex decision scenarios.

Understanding how the brain produces complex cognitive functions has been a crucial question since ancient philosophical inquiries. The encoding of decision difficulty in the brain is fundamental for complex and adaptive behaviour, and can provide valuable information in uncertain environments where the future outcome of a choice must be evaluated beforehand. Here we show that neurons in premotor cortex represent the difficulty of a decision using at least three different variables: 1) the time of the neuronal response, 2) the intensity of the neuronal response, 3) the probability of switching from a low activity to a high activity profile. Moreover, we show that, by encoding the time elapsed from the end of the stimulus and commitment to a choice, another set of premotor neurons is able to provide information about the difficulty of the decision. These results show that the brain is implementing heterogeneous neural mechanisms to fulfill a complex cognitive function.

Modelling Odor Decoding in the Antennal Lobe by Combining Sequential Firing Rate Models with Bayesian Inference

The olfactory information that is received by the insect brain is encoded in the form of spatiotemporal patterns in the projection neurons of the antennal lobe. These dense and overlapping patterns are transformed into a sparse code in Kenyon cells in the mushroom body. Although it is clear that this sparse code is the basis for rapid categorization of odors, it is yet unclear how the sparse code in Kenyon cells is computed and what information it represents. Here we show that this computation can be modeled by sequential firing rate patterns using Lotka-Volterra equations and Bayesian online inference. This new model can be understood as an ‘intelligent coincidence detector’, which robustly and dynamically encodes the presence of specific odor features. We found that the model is able to qualitatively reproduce experimentally observed activity in both the projection neurons and the Kenyon cells. In particular, the model explains mechanistically how sparse activity in the Kenyon cells arises from the dense code in the projection neurons. The odor classification performance of the model proved to be robust against noise and time jitter in the observed input sequences. As in recent experimental results, we found that recognition of an odor happened very early during stimulus presentation in the model. Critically, by using the model, we found surprising but simple computational explanations for several experimental phenomena.

Odor recognition in the insect brain is amazingly fast but still not fully understood. It is known that recognition is performed in three stages. In the first stage, the sensors respond to an odor by displaying a reproducible neuronal pattern. This code is turned, in the second and third stages, into a sparse code, that is, only relatively few neurons activate over hundreds of milliseconds. It is generally assumed that the insect brain uses this temporal code to recognize an odor. We propose a new model of how this temporal code emerges using sequential activation of groups of neurons. We show that these sequential activations underlie a fast and accurate recognition which is highly robust against neuronal or sensory noise. This model replicates several key experimental findings and explains how the insect brain achieves both speed and robustness of odor recognition as observed in experiments.

Towards a systems view of IBS

Despite an extensive body of reported information about peripheral and central mechanisms involved in the pathophysiology of IBS symptoms, no comprehensive disease model has emerged that would guide the development of novel, effective therapies. In this Review, we will first describe novel insights into some key components of brain-gut interactions, starting with the emerging findings of distinct functional and structural brain signatures of IBS. We will then point out emerging correlations between these brain networks and genomic, gastrointestinal, immune and gut-microbiome-related parameters. We will incorporate this new information, as well as the reported extensive literature on various peripheral mechanisms, into a systems-based disease model of IBS, and discuss the implications of such a model for improved understanding of the disorder, and for the development of more-effective treatment approaches in the future.

Nonlinear model reduction for dynamical systems using sparse sensor locations from learned libraries

We demonstrate the synthesis of sparse sampling and dimensionality reduction to characterize and model nonlinear dynamical systems over a range of bifurcation parameters. First, we construct modal libraries using the classical proper orthogonal decomposition in order to expose the dominant low-rank coherent structures. Here, libraries of the nonlinear terms are also constructed in order to take advantage of the discrete empirical interpolation method and projection that allows for the approximation of nonlinear terms from a sparse number of grid points. The selected grid points are shown to be effective sensing and measurement locations for characterizing the underlying dynamics, stability, and bifurcations of nonlinear dynamical systems. The use of empirical interpolation points and sparse representation facilitates a family of local reduced-order models for each physical regime, rather than a higher-order global model, which has the benefit of physical interpretability of energy transfer between coherent structures. The method advocated also allows for orders-of-magnitude improvement in computational speed and memory requirements. To illustrate the method, the discrete interpolation points and nonlinear modal libraries are used for sparse representation in order to classify and reconstruct the dynamic bifurcation regimes in the complex Ginzburg-Landau equation. It is also shown that point measurements of the nonlinearity are more effective than linear measurements when sensor noise is present.

Dynamics of multistable states during ongoing and evoked cortical activity

Single-trial analyses of ensemble activity in alert animals demonstrate that cortical circuits dynamics evolve through temporal sequences of metastable states. Metastability has been studied for its potential role in sensory coding, memory, and decision-making. Yet, very little is known about the network mechanisms responsible for its genesis. It is often assumed that the onset of state sequences is triggered by an external stimulus. Here we show that state sequences can be observed also in the absence of overt sensory stimulation. Analysis of multielectrode recordings from the gustatory cortex of alert rats revealed ongoing sequences of states, where single neurons spontaneously attain several firing rates across different states. This single-neuron multistability represents a challenge to existing spiking network models, where typically each neuron is at most bistable. We present a recurrent spiking network model that accounts for both the spontaneous generation of state sequences and the multistability in single-neuron firing rates. Each state results from the activation of neural clusters with potentiated intracluster connections, with the firing rate in each cluster depending on the number of active clusters. Simulations show that the model's ensemble activity hops among the different states, reproducing the ongoing dynamics observed in the data. When probed with external stimuli, the model predicts the quenching of single-neuron multistability into bistability and the reduction of trial-by-trial variability. Both predictions were confirmed in the data. Together, these results provide a theoretical framework that captures both ongoing and evoked network dynamics in a single mechanistic model.

Single-neuron activity and eye movements during human REM sleep and awake vision

Are rapid eye movements (REMs) in sleep associated with visual-like activity, as during wakefulness? Here we examine single-unit activities (n=2,057) and intracranial electroencephalography across the human medial temporal lobe (MTL) and neocortex during sleep and wakefulness, and during visual stimulation with fixation. During sleep and wakefulness, REM onsets are associated with distinct intracranial potentials, reminiscent of ponto-geniculate-occipital waves. Individual neurons, especially in the MTL, exhibit reduced firing rates before REMs as well as transient increases in firing rate immediately after, similar to activity patterns observed upon image presentation during fixation without eye movements. Moreover, the selectivity of individual units is correlated with their response latency, such that units activated after a small number of images or REMs exhibit delayed increases in firing rates. Finally, the phase of theta oscillations is similarly reset following REMs in sleep and wakefulness, and after controlled visual stimulation. Our results suggest that REMs during sleep rearrange discrete epochs of visual-like processing as during wakefulness.

Since the discovery of rapid eye movements (REMs), a critical question endures as to whether they represent time points at which visual-like processing is updated. Here the authors demonstrate that cortical activity during sleep REMs shares many properties with that observed during saccades and vision.

Interplay of inhibition and excitation shapes a premotor neural sequence

In the zebra finch, singing behavior is driven by a sequence of bursts within premotor neurons located in the forebrain nucleus HVC (proper name). In addition to these excitatory projection neurons, HVC also contains inhibitory interneurons with a role in premotor patterning that is unclear. Here, we used a range of electrophysiological and behavioral observations to test previously described models suggesting discrete functional roles for inhibitory interneurons in song production. We show that single HVC premotor neuron bursts are sufficient to drive structured activity within the interneuron network because of pervasive and facilitating synaptic connections. We characterize interneuron activity during singing and describe reliable pauses in the firing of those neurons. We then demonstrate that these gaps in inhibition are likely to be necessary for driving normal bursting behavior in HVC premotor neurons and suggest that structured inhibition and excitation may be a general mechanism enabling sequence generation in other circuits.

Maximally informative foraging by Caenorhabditis elegans

Animals have evolved intricate search strategies to find new sources of food. Here, we analyze a complex food seeking behavior in the nematode Caenorhabditis elegans (C. elegans) to derive a general theory describing different searches. We show that C. elegans, like many other animals, uses a multi-stage search for food, where they initially explore a small area intensively ('local search') before switching to explore a much larger area ('global search'). We demonstrate that these search strategies as well as the transition between them can be quantitatively explained by a maximally informative search strategy, where the searcher seeks to continuously maximize information about the target. Although performing maximally informative search is computationally demanding, we show that a drift-diffusion model can approximate it successfully with just three neurons. Our study reveals how the maximally informative search strategy can be implemented and adopted to different search conditions.

Seeing the whole picture: A comprehensive imaging approach to functional mapping of circuits in behaving zebrafish

In recent years, the zebrafish has emerged as an appealing model system to tackle questions relating to the neural circuit basis of behavior. This can be attributed not just to the growing use of genetically tractable model organisms, but also in large part to the rapid advances in optical techniques for neuroscience, which are ideally suited for application to the small, transparent brain of the larval fish. Many characteristic features of vertebrate brains, from gross anatomy down to particular circuit motifs and cell-types, as well as conserved behaviors, can be found in zebrafish even just a few days post fertilization, and, at this early stage, the physical size of the brain makes it possible to analyze neural activity in a comprehensive fashion. In a recent study, we used a systematic and unbiased imaging method to record the pattern of activity dynamics throughout the whole brain of larval zebrafish during a simple visual behavior, the optokinetic response (OKR). This approach revealed the broadly distributed network of neurons that were active during the behavior and provided insights into the fine-scale functional architecture in the brain, inter-individual variability, and the spatial distribution of behaviorally relevant signals. Combined with mapping anatomical and functional connectivity, targeted electrophysiological recordings, and genetic labeling of specific populations, this comprehensive approach in zebrafish provides an unparalleled opportunity to study complete circuits in a behaving vertebrate animal.

Encoding and tracking of outcome-specific expectancy in the gustatory cortex of alert rats

In natural conditions, gustatory stimuli are typically expected. Anticipatory and contextual cues provide information that allows animals to predict the availability and the identity of the substance to be ingested. Recording in alert rats trained to self-administer tastants following a go signal revealed that neurons in the primary gustatory cortex (GC) can respond to anticipatory cues. These experiments were optimized to demonstrate that even the most general form of expectation can activate neurons in GC, and did not provide indications on whether cues predicting different tastants could be encoded selectively by GC neurons. Here we recorded single-neuron activity in GC of rats engaged in a task where one auditory cue predicted sucrose, while another predicted quinine. We found that GC neurons respond differentially to the two cues. Cue-selective responses develop in parallel with learning. Comparison between cue and sucrose responses revealed that cues could trigger the activation of anticipatory representations. Additional experiments showed that an expectation of sucrose leads a subset of neurons to produce sucrose-like responses even when the tastant was omitted. Altogether, the data show that primary sensory cortices can encode for cues predicting different outcomes, and that specific expectations result in the activation of anticipatory representations.

Robust sequential working memory recall in heterogeneous cognitive networks

Psychiatric disorders are often caused by partial heterogeneous disinhibition in cognitive networks, controlling sequential and spatial working memory (SWM). Such dynamic connectivity changes suggest that the normal relationship between the neuronal components within the network deteriorates. As a result, competitive network dynamics is qualitatively altered. This dynamics defines the robust recall of the sequential information from memory and, thus, the SWM capacity. To understand pathological and non-pathological bifurcations of the sequential memory dynamics, here we investigate the model of recurrent inhibitory-excitatory networks with heterogeneous inhibition. We consider the ensemble of units with all-to-all inhibitory connections, in which the connection strengths are monotonically distributed at some interval. Based on computer experiments and studying the Lyapunov exponents, we observed and analyzed the new phenomenon—clustered sequential dynamics. The results are interpreted in the context of the winnerless competition principle. Accordingly, clustered sequential dynamics is represented in the phase space of the model by two weakly interacting quasi-attractors. One of them is similar to the sequential heteroclinic chain—the regular image of SWM, while the other is a quasi-chaotic attractor. Coexistence of these quasi-attractors means that the recall of the normal information sequence is intermittently interrupted by episodes with chaotic dynamics. We indicate potential dynamic ways for augmenting damaged working memory and other cognitive functions.

Dimensionality reduction for large-scale neural recordings

Most sensory, cognitive and motor functions depend on the interactions of many neurons. In recent years, there has been rapid development and increasing use of technologies for recording from large numbers of neurons, either sequentially or simultaneously. A key question is what scientific insight can be gained by studying a population of recorded neurons beyond studying each neuron individually. Here, we examine three important motivations for population studies: single-trial hypotheses requiring statistical power, hypotheses of population response structure and exploratory analyses of large data sets. Many recent studies have adopted dimensionality reduction to analyze these populations and to find features that are not apparent at the level of individual neurons. We describe the dimensionality reduction methods commonly applied to population activity and offer practical advice about selecting methods and interpreting their outputs. This review is intended for experimental and computational researchers who seek to understand the role dimensionality reduction has had and can have in systems neuroscience, and who seek to apply these methods to their own data.

Sensory cortical population dynamics uniquely track behavior across learning and extinction

Neural responses in many cortical regions encode information relevant to behavior: information that necessarily changes as that behavior changes with learning. Although such responses are reasonably theorized to be related to behavior causation, the true nature of that relationship cannot be clarified by simple learning studies, which show primarily that responses change with experience. Neural activity that truly tracks behavior (as opposed to simply changing with experience) will not only change with learning but also change back when that learning is extinguished. Here, we directly probed for this pattern, recording the activity of ensembles of gustatory cortical single neurons as rats that normally consumed sucrose avidly were trained first to reject it (i.e., conditioned taste aversion learning) and then to enjoy it again (i.e., extinction), all within 49 h. Both learning and extinction altered cortical responses, consistent with the suggestion (based on indirect evidence) that extinction is a novel form of learning. But despite the fact that, as expected, postextinction single-neuron responses did not resemble "naive responses," ensemble response dynamics changed with learning and reverted with extinction: both the speed of stimulus processing and the relationships among ensemble responses to the different stimuli tracked behavioral relevance. These data suggest that population coding is linked to behavior with a fidelity that single-neuron coding is not.

Neural circuits as computational dynamical systems

Many recent studies of neurons recorded from cortex reveal complex temporal dynamics. How such dynamics embody the computations that ultimately lead to behavior remains a mystery. Approaching this issue requires developing plausible hypotheses couched in terms of neural dynamics. A tool ideally suited to aid in this question is the recurrent neural network (RNN). RNNs straddle the fields of nonlinear dynamical systems and machine learning and have recently seen great advances in both theory and application. I summarize recent theoretical and technological advances and highlight an example of how RNNs helped to explain perplexing high-dimensional neurophysiological data in the prefrontal cortex.

Firing rate homeostasis in visual cortex of freely behaving rodents

It has been postulated that homeostatic mechanisms maintain stable circuit function by keeping neuronal firing within a set point range, but such firing rate homeostasis has never been demonstrated in vivo. Here we use chronic multielectrode recordings to monitor firing rates in visual cortex of freely behaving rats during chronic monocular visual deprivation (MD). Firing rates in V1 were suppressed over the first 2 day of MD but then rebounded to baseline over the next 2-3 days despite continued MD. This drop and rebound in firing was accompanied by bidirectional changes in mEPSC amplitude measured ex vivo. The rebound in firing was independent of sleep-wake state but was cell type specific, as putative FS and regular spiking neurons responded to MD with different time courses. These data establish that homeostatic mechanisms within the intact CNS act to stabilize neuronal firing rates in the face of sustained sensory perturbations.

Neural representation of spatial topology in the rodent hippocampus

Pyramidal cells in the rodent hippocampus often exhibit clear spatial tuning in navigation. Although it has been long suggested that pyramidal cell activity may underlie a topological code rather than a topographic code, it remains unclear whether an abstract spatial topology can be encoded in the ensemble spiking activity of hippocampal place cells. Using a statistical approach developed previously, we investigate this question and related issues in greater detail. We recorded ensembles of hippocampal neurons as rodents freely foraged in one- and two-dimensional spatial environments and used a "decode-to-uncover" strategy to examine the temporally structured patterns embedded in the ensemble spiking activity in the absence of observed spatial correlates during periods of rodent navigation or awake immobility. Specifically, the spatial environment was represented by a finite discrete state space. Trajectories across spatial locations ("states") were associated with consistent hippocampal ensemble spiking patterns, which were characterized by a state transition matrix. From this state transition matrix, we inferred a topology graph that defined the connectivity in the state space. In both one- and two-dimensional environments, the extracted behavior patterns from the rodent hippocampal population codes were compared against randomly shuffled spike data. In contrast to a topographic code, our results support the efficiency of topological coding in the presence of sparse sample size and fuzzy space mapping. This computational approach allows us to quantify the variability of ensemble spiking activity, examine hippocampal population codes during off-line states, and quantify the topological complexity of the environment.

Stochastic computations in cortical microcircuit models

Experimental data from neuroscience suggest that a substantial amount of knowledge is stored in the brain in the form of probability distributions over network states and trajectories of network states. We provide a theoretical foundation for this hypothesis by showing that even very detailed models for cortical microcircuits, with data-based diverse nonlinear neurons and synapses, have a stationary distribution of network states and trajectories of network states to which they converge exponentially fast from any initial state. We demonstrate that this convergence holds in spite of the non-reversibility of the stochastic dynamics of cortical microcircuits. We further show that, in the presence of background network oscillations, separate stationary distributions emerge for different phases of the oscillation, in accordance with experimentally reported phase-specific codes. We complement these theoretical results by computer simulations that investigate resulting computation times for typical probabilistic inference tasks on these internally stored distributions, such as marginalization or marginal maximum-a-posteriori estimation. Furthermore, we show that the inherent stochastic dynamics of generic cortical microcircuits enables them to quickly generate approximate solutions to difficult constraint satisfaction problems, where stored knowledge and current inputs jointly constrain possible solutions. This provides a powerful new computing paradigm for networks of spiking neurons, that also throws new light on how networks of neurons in the brain could carry out complex computational tasks such as prediction, imagination, memory recall and problem solving.

A spatiotemporal coding mechanism for background-invariant odor recognition

Sensory stimuli evoke neural activity that evolves over time. What features of these spatiotemporal responses allow the robust encoding of stimulus identity in a multistimulus environment? Here we examined this issue in the locust (Schistocerca americana) olfactory system. We found that sensory responses evoked by an odorant (foreground) varied when presented atop or after an ongoing stimulus (background). These inconsistent sensory inputs triggered dynamic reorganization of ensemble activity in the downstream antennal lobe. As a result, partial pattern matches between neural representations encoding the same foreground stimulus across conditions were achieved. The degree and segments of response overlaps varied; however, any overlap observed was sufficient to drive background-independent responses in the downstream neural population. Notably, recognition performance of locusts in behavioral assays correlated well with our physiological findings. Hence, our results reveal how background-independent recognition of odors can be achieved using spatiotemporal patterns of neural activity.

Emergence of dynamic memory traces in cortical microcircuit models through STDP

Numerous experimental data suggest that simultaneously or sequentially activated assemblies of neurons play a key role in the storage and computational use of long-term memory in the brain. However, a model that elucidates how these memory traces could emerge through spike-timing-dependent plasticity (STDP) has been missing. We show here that stimulus-specific assemblies of neurons emerge automatically through STDP in a simple cortical microcircuit model. The model that we consider is a randomly connected network of well known microcircuit motifs: pyramidal cells with lateral inhibition. We show that the emergent assembly codes for repeatedly occurring spatiotemporal input patterns tend to fire in some loose, sequential manner that is reminiscent of experimentally observed stereotypical trajectories of network states. We also show that the emergent assembly codes add an important computational capability to standard models for online computations in cortical microcircuits: the capability to integrate information from long-term memory with information from novel spike inputs.

Synthesizing cognition in neuromorphic electronic systems

The quest to implement intelligent processing in electronic neuromorphic systems lacks methods for achieving reliable behavioral dynamics on substrates of inherently imprecise and noisy neurons. Here we report a solution to this problem that involves first mapping an unreliable hardware layer of spiking silicon neurons into an abstract computational layer composed of generic reliable subnetworks of model neurons and then composing the target behavioral dynamics as a "soft state machine" running on these reliable subnets. In the first step, the neural networks of the abstract layer are realized on the hardware substrate by mapping the neuron circuit bias voltages to the model parameters. This mapping is obtained by an automatic method in which the electronic circuit biases are calibrated against the model parameters by a series of population activity measurements. The abstract computational layer is formed by configuring neural networks as generic soft winner-take-all subnetworks that provide reliable processing by virtue of their active gain, signal restoration, and multistability. The necessary states and transitions of the desired high-level behavior are then easily embedded in the computational layer by introducing only sparse connections between some neurons of the various subnets. We demonstrate this synthesis method for a neuromorphic sensory agent that performs real-time context-dependent classification of motion patterns observed by a silicon retina.

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device

Animal behavior often involves a temporally ordered sequence of actions learned from experience. Here we describe simulations of interconnected networks of spiking neurons that learn to generate patterns of activity in correct temporal order. The simulation consists of large-scale networks of thousands of excitatory and inhibitory neurons that exhibit short-term synaptic plasticity and spike-timing dependent synaptic plasticity. The neural architecture within each area is arranged to evoke winner-take-all (WTA) patterns of neural activity that persist for tens of milliseconds. In order to generate and switch between consecutive firing patterns in correct temporal order, a reentrant exchange of signals between these areas was necessary. To demonstrate the capacity of this arrangement, we used the simulation to train a brain-based device responding to visual input by autonomously generating temporal sequences of motor actions.

Abrupt changes in the patterns and complexity of anterior cingulate cortex activity when food is introduced into an environment

Foraging typically involves two distinct phases, an exploration phase where an organism explores its local environment in search of needed resources and an exploitation phase where a discovered resource is consumed. The behavior and cognitive requirements of exploration and exploitation are quite different and yet organisms can quickly and efficiently switch between them many times during a foraging bout. The present study investigated neural activity state dynamics in the anterior cingulate sub-region of the rat medial prefrontal cortex (mPFC) when a reliable food source was introduced into an environment. Distinct and largely independent states were detected using a Hidden Markov Model (HMM) when food was present or absent in the environment. Measures of neural entropy or complexity decreased when rats went from exploring the environment to exploiting a reliable food source. Exploration in the absence of food was associated with many weak activity states, while bouts of food consumption were characterized by fewer stronger states. Widespread activity state changes in the mPFC may help to inform foraging decisions and focus behavior on what is currently most prominent or valuable in the environment.

Neural dynamics of attentional cross-modality control

Attentional networks that integrate many cortical and subcortical elements dynamically control mental processes to focus on specific events and make a decision. The resources of attentional processing are finite. Nevertheless, we often face situations in which it is necessary to simultaneously process several modalities, for example, to switch attention between players in a soccer field. Here we use a global brain mode description to build a model of attentional control dynamics. This model is based on sequential information processing stability conditions that are realized through nonsymmetric inhibition in cortical circuits. In particular, we analyze the dynamics of attentional switching and focus in the case of parallel processing of three interacting mental modalities. Using an excitatory-inhibitory network, we investigate how the bifurcations between different attentional control strategies depend on the stimuli and analyze the relationship between the time of attention focus and the strength of the stimuli. We discuss the interplay between attention and decision-making: in this context, a decision-making process is a controllable bifurcation of the attention strategy. We also suggest the dynamical evaluation of attentional resources in neural sequence processing.

Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits

Animals choose actions based on imperfect, ambiguous data. “Noise” inherent in neural processing adds further variability to this already-noisy input signal. Mathematical analysis has suggested that the optimal apparatus (in terms of the speed/accuracy trade-off) for reaching decisions about such noisy inputs is perfect accumulation of the inputs by a temporal integrator. Thus, most highly cited models of neural circuitry underlying decision-making have been instantiations of a perfect integrator. Here, in accordance with a growing mathematical and empirical literature, we describe circumstances in which perfect integration is rendered suboptimal. In particular we highlight the impact of three biological constraints: (1) significant noise arising within the decision-making circuitry itself; (2) bounding of integration by maximal neural firing rates; and (3) time limitations on making a decision. Under conditions (1) and (2), an attractor system with stable attractor states can easily best an integrator when accuracy is more important than speed. Moreover, under conditions in which such stable attractor networks do not best the perfect integrator, a system with unstable initial states can do so if readout of the system’s final state is imperfect. Ubiquitously, an attractor system with a nonselective time-dependent input current is both more accurate and more robust to imprecise tuning of parameters than an integrator with such input. Given that neural responses that switch stochastically between discrete states can “masquerade” as integration in single-neuron and trial-averaged data, our results suggest that such networks should be considered as plausible alternatives to the integrator model.

Thalamic contribution to cortical processing of taste and expectation

Taste-related information reaches the gustatory cortex (GC) through two routes: a thalamic and a limbic pathway. While evidence is accumulating on limbic-cortical interactions in taste, very little information is available on the function of the gustatory thalamus in shaping GC activity. Here we rely on behavioral electrophysiological techniques to study taste-evoked activity in GC before and after inactivation of the parvicellular portion of the ventroposteromedial nucleus of thalamus (VPMpc; i.e., the gustatory thalamus). Gustatory stimuli were presented to rats either alone or preceded by an anticipatory cue. The reliance on two different behavioral contexts allowed us to investigate how the VPMpc mediates GC responses to uncued tastants, cued tastants, and anticipatory cues. Inactivation of the thalamus resulted in a dramatic reduction of taste processing in GC. However, responses to anticipatory cues were unaffected by this manipulation. The use of a cue-taste association paradigm also allowed for the identification of two subpopulations of taste-specific neurons: those that responded to gustatory stimulation and to the cue (i.e., cue-and-taste) and those that responded to tastants only (i.e., taste-only). Analyses of these two populations revealed differences in response dynamics and connectivity with the VPMpc. The results provide novel evidence for the role of VPMpc in shaping GC activity and demonstrate a previously unknown association between responsiveness to behavioral events, temporal dynamics, and thalamic connectivity in GC.

Quantitative examination of stimulus-response relations in cortical networks in vitro

Variable responses of neuronal networks to repeated sensory or electrical stimuli reflect the interaction of the stimulus' response with ongoing activity in the brain and its modulation by adaptive mechanisms, such as cognitive context, network state, or cellular excitability and synaptic transmission capability. Here, we focus on reliability, length, delays, and variability of evoked responses with respect to their spatial distribution, interaction with spontaneous activity in the networks, and the contribution of GABAergic inhibition. We identified network-intrinsic principles that underlie the formation and modulation of spontaneous activity and stimulus-response relations with the use of state-dependent stimulation in generic neuronal networks in vitro. The duration of spontaneously recurring network-wide bursts of spikes was best predicted by the length of the preceding interval. Length, delay, and structure of responses to identical stimuli systematically depended on stimulus timing and distance to the stimulation site, which were described by a set of simple functions of spontaneous activity. Response length at proximal recording sites increased with the duration of prestimulus inactivity and was best described by a saturation function y( t) = A( 1 − e−α t). Concomitantly, the delays of polysynaptic late responses at distant sites followed an exponential decay y( t) = Be−β t + C. In addition, the speed of propagation was determined by the overall state of the network at the moment of stimulation. Disinhibition increased the number of spikes/network burst and interburst interval length at unchanged gross firing rate, whereas the response modulation by the duration of prestimulus inactivity was preserved. Our data suggest a process of network depression during bursts and subsequent recovery that limit evoked responses following distinct rules. We discuss short-term synaptic depression due to depletion of neurotransmitter vesicles as an underlying mechanism. The seemingly unreliable patterns of spontaneous activity and stimulus-response relations thus follow a predictable structure determined by the interdependencies of network structures and activity states.

Heteroclinic contours in oscillatory ensembles

In this work, we study the onset of sequential activity in ensembles of neuronlike oscillators with inhibitorylike coupling between them. The winnerless competition (WLC) principle is a dynamical concept underlying sequential activity generation. According to the WLC principle, stable heteroclinic sequences in the phase space of a network model represent sequential metastable dynamics. We show that stable heteroclinic sequences and stable heteroclinic channels, connecting saddle limit cycles, can appear in oscillatory models of neural activity. We find the key bifurcations which lead to the occurrence of sequential activity as well as heteroclinic sequences and channels.

Thalamic contribution to cortical processing of taste and expectation

Taste-related information reaches the gustatory cortex (GC) through two routes: a thalamic and a limbic pathway. While evidence is accumulating on limbic-cortical interactions in taste, very little information is available on the function of the gustatory thalamus in shaping GC activity. Here we rely on behavioral electrophysiological techniques to study taste-evoked activity in GC before and after inactivation of the parvicellular portion of the ventroposteromedial nucleus of thalamus (VPMpc; i.e., the gustatory thalamus). Gustatory stimuli were presented to rats either alone or preceded by an anticipatory cue. The reliance on two different behavioral contexts allowed us to investigate how the VPMpc mediates GC responses to uncued tastants, cued tastants, and anticipatory cues. Inactivation of the thalamus resulted in a dramatic reduction of taste processing in GC. However, responses to anticipatory cues were unaffected by this manipulation. The use of a cue-taste association paradigm also allowed for the identification of two subpopulations of taste-specific neurons: those that responded to gustatory stimulation and to the cue (i.e., cue-and-taste) and those that responded to tastants only (i.e., taste-only). Analyses of these two populations revealed differences in response dynamics and connectivity with the VPMpc. The results provide novel evidence for the role of VPMpc in shaping GC activity and demonstrate a previously unknown association between responsiveness to behavioral events, temporal dynamics, and thalamic connectivity in GC.

Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks

Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships between time-varying inputs and outputs with complex temporal dependencies. Recently developed algorithms have been successful at training RNNs to perform a wide variety of tasks, but the resulting networks have been treated as black boxes: their mechanism of operation remains unknown. Here we explore the hypothesis that fixed points, both stable and unstable, and the linearized dynamics around them, can reveal crucial aspects of how RNNs implement their computations. Further, we explore the utility of linearization in areas of phase space that are not true fixed points but merely points of very slow movement. We present a simple optimization technique that is applied to trained RNNs to find the fixed and slow points of their dynamics. Linearization around these slow regions can be used to explore, or reverse-engineer, the behavior of the RNN. We describe the technique, illustrate it using simple examples, and finally showcase it on three high-dimensional RNN examples: a 3-bit flip-flop device, an input-dependent sine wave generator, and a two-point moving average. In all cases, the mechanisms of trained networks could be inferred from the sets of fixed and slow points and the linearized dynamics around them.

Neural dynamics of choice: single-trial analysis of decision-related activity in parietal cortex

Previous neurophysiological studies of perceptual decision-making have focused on single-unit activity, providing insufficient information about how individual decisions are accomplished. For the first time, we recorded simultaneously from multiple decision-related neurons in parietal cortex of monkeys performing a perceptual decision task and used these recordings to analyze the neural dynamics during single trials. We demonstrate that decision-related lateral intraparietal area neurons typically undergo gradual changes in firing rate during individual decisions, as predicted by mechanisms based on continuous integration of sensory evidence. Furthermore, we identify individual decisions that can be described as a change of mind: the decision circuitry was transiently in a state associated with a different choice before transitioning into a state associated with the final choice. These changes of mind reflected in monkey neural activity share similarities with previously reported changes of mind reflected in human behavior.

Dynamics of cortical neuronal ensembles transit from decision making to storage for later report

Decisions based on sensory evaluation during single trials may depend on the collective activity of neurons distributed across brain circuits. Previous studies have deepened our understanding of how the activity of individual neurons relates to the formation of a decision and its storage for later report. However, little is known about how decision-making and decision maintenance processes evolve in single trials. We addressed this problem by studying the activity of simultaneously recorded neurons from different somatosensory and frontal lobe cortices of monkeys performing a vibrotactile discrimination task. We used the hidden Markov model to describe the spatiotemporal pattern of activity in single trials as a sequence of firing rate states. We show that the animal's decision was reliably maintained in frontal lobe activity through a selective state sequence, initiated by an abrupt state transition, during which many neurons changed their activity in a concomitant way, and for which both latency and variability depended on task difficulty. Indeed, transitions were more delayed and more variable for difficult trials compared with easy trials. In contrast, state sequences in somatosensory cortices were weakly decision related, had less variable transitions, and were not affected by the difficulty of the task. In summary, our results suggest that the decision process and its subsequent maintenance are dynamically linked by a cascade of transient events in frontal lobe cortices.

Inactivation of basolateral amygdala specifically eliminates palatability-related information in cortical sensory responses

Evidence indirectly implicates the amygdala as the primary processor of emotional information used by cortex to drive appropriate behavioral responses to stimuli. Taste provides an ideal system with which to test this hypothesis directly, as neurons in both basolateral amygdala (BLA) and gustatory cortex (GC)-anatomically interconnected nodes of the gustatory system-code the emotional valence of taste stimuli (i.e., palatability), in firing rate responses that progress similarly through "epochs." The fact that palatability-related firing appears one epoch earlier in BLA than GC is broadly consistent with the hypothesis that such information may propagate from the former to the latter. Here, we provide evidence supporting this hypothesis, assaying taste responses in small GC single-neuron ensembles before, during, and after temporarily inactivating BLA in awake rats. BLA inactivation (BLAx) changed responses in 98% of taste-responsive GC neurons, altering the entirety of every taste response in many neurons. Most changes involved reductions in firing rate, but regardless of the direction of change, the effect of BLAx was epoch-specific: while firing rates were changed, the taste specificity of responses remained stable; information about taste palatability, however, which normally resides in the "Late" epoch, was reduced in magnitude across the entire GC sample and outright eliminated in most neurons. Only in the specific minority of neurons for which BLAx enhanced responses did palatability specificity survive undiminished. Our data therefore provide direct evidence that BLA is a necessary component of GC gustatory processing, and that cortical palatability processing in particular is, in part, a function of BLA activity.

Sodium concentration coding gives way to evaluative coding in cortex and amygdala

Typically, stimulus batteries used to characterize sensory neural coding span physical parameter spaces (e.g., concentration: from low to high). For awake animals, however, psychological variables (e.g., pleasantness/palatability) with complicated relationships to the physical often dominate neural responses. Here we pit physical and psychological axes against one another, presenting awake rats with a stimulus set including 4 NaCl concentrations (0.01, 0.1, 0.3, and 1.0 m) plus palatable (0.3 m sucrose) and aversive (0.001 m quinine) benchmarks, while recording the activity of neurons in two sites vital for NaCl taste processing, gustatory cortex (GC) and central amygdala (CeA). Since NaCl palatability (i.e., preference) follows a non-monotonic, "inverted-U-shaped" curve while concentration increases monotonically, this stimulus battery allowed us to test whether GC and CeA responses better reflect external or internal variables. As predicted, GC single-neuron and population responses reflected both parameters in separate response epochs: sodium concentration-related information appeared with the earliest taste-specific responses, giving way to palatability-related information, in an overlapping subset of neurons, several hundred milliseconds later. CeA single-neuron and population responses, meanwhile, contained only a brief period of concentration specificity, occurring just before palatability-related information emerged (simultaneously with, or slightly later than, in GC). Thus, cortex and amygdala both prominently reflect NaCl palatability late in their responses; CeA neurons largely respond to either palatable or aversive stimuli, while GC responses tend to reflect the entire palatability spectrum in a graded fashion.

Sodium concentration coding gives way to evaluative coding in cortex and amygdala

Typically, stimulus batteries used to characterize sensory neural coding span physical parameter spaces (e.g., concentration: from low to high). For awake animals, however, psychological variables (e.g., pleasantness/palatability) with complicated relationships to the physical often dominate neural responses. Here we pit physical and psychological axes against one another, presenting awake rats with a stimulus set including 4 NaCl concentrations (0.01, 0.1, 0.3, and 1.0 m) plus palatable (0.3 m sucrose) and aversive (0.001 m quinine) benchmarks, while recording the activity of neurons in two sites vital for NaCl taste processing, gustatory cortex (GC) and central amygdala (CeA). Since NaCl palatability (i.e., preference) follows a non-monotonic, "inverted-U-shaped" curve while concentration increases monotonically, this stimulus battery allowed us to test whether GC and CeA responses better reflect external or internal variables. As predicted, GC single-neuron and population responses reflected both parameters in separate response epochs: sodium concentration-related information appeared with the earliest taste-specific responses, giving way to palatability-related information, in an overlapping subset of neurons, several hundred milliseconds later. CeA single-neuron and population responses, meanwhile, contained only a brief period of concentration specificity, occurring just before palatability-related information emerged (simultaneously with, or slightly later than, in GC). Thus, cortex and amygdala both prominently reflect NaCl palatability late in their responses; CeA neurons largely respond to either palatable or aversive stimuli, while GC responses tend to reflect the entire palatability spectrum in a graded fashion.

Effects of cue-triggered expectation on cortical processing of taste

Animals are not passive spectators of the sensory world in which they live. In natural conditions they often sense objects on the bases of expectations initiated by predictive cues. Expectation profoundly modulates neural activity by altering the background state of cortical networks and modulating sensory processing. The link between these two effects is not known. Here, we studied how cue-triggered expectation of stimulus availability influences processing of sensory stimuli in the gustatory cortex (GC). We found that expected tastants were coded more rapidly than unexpected stimuli. The faster onset of sensory coding related to anticipatory priming of GC by associative auditory cues. Simultaneous recordings and pharmacological manipulations of GC and basolateral amygdala revealed the role of top-down inputs in mediating the effects of anticipatory cues. Altogether, these data provide a model for how cue-triggered expectation changes the state of sensory cortices to achieve rapid processing of natural stimuli.

Neural processing of gustatory information in insular circuits

The insular cortex is the primary cortical site devoted to taste processing. A large body of evidence is available for how insular neurons respond to gustatory stimulation in both anesthetized and behaving animals. Most of the reports describe broadly tuned neurons that are involved in processing the chemosensory, physiological and psychological aspects of gustatory experience. However little is known about how these neural responses map onto insular circuits. Particularly mysterious is the functional role of the three subdivisions of the insular cortex: the granular, the dysgranular and the agranular insular cortices. In this article we review data on the organization of the local and long-distance circuits in the three subdivisions. The functional significance of these results is discussed in light of the latest electrophysiological data. A view of the insular cortex as a functionally integrated system devoted to processing gustatory, multimodal, cognitive and affective information is proposed.

Cortical free-association dynamics: distinct phases of a latching network

A Potts associative memory network has been proposed as a simplified model of macroscopic cortical dynamics, in which each Potts unit stands for a patch of cortex, which can be activated in one of S local attractor states. The internal neuronal dynamics of the patch is not described by the model, rather it is subsumed into an effective description in terms of graded Potts units, with adaptation effects both specific to each attractor state and generic to the patch. If each unit, or patch, receives effective (tensor) connections from C other units, the network has been shown to be able to store a large number p of global patterns, or network attractors, each with a fraction a of the units active, where the critical load p_{c} scales roughly like p_{c}≈CS^{2}/aln(1/a) (if the patterns are randomly correlated). Interestingly, after retrieving an externally cued attractor, the network can continue jumping, or latching, from attractor to attractor, driven by adaptation effects. The occurrence and duration of latching dynamics is found through simulations to depend critically on the strength of local attractor states, expressed in the Potts model by a parameter w. Here we describe with simulations and then analytically the boundaries between distinct phases of no latching, of transient and sustained latching, deriving a phase diagram in the plane w-T, where T parametrizes thermal noise effects. Implications for real cortical dynamics are briefly reviewed in the conclusions.