Striatal dynamics explain duration judgments

Oscillation patterns of local field potentials in the dorsal striatum and sensorimotor cortex during the encoding, maintenance, and decision stages for the ordinal comparison of sub- and supra-second signal durations

Ordinal comparison of successively presented signal durations requires (a) the encoding of the first signal duration (standard), (b) maintenance of temporal information specific to the standard duration in memory, and (c) timing of the second signal duration (comparison) during which a comparison is made of the first and second durations. Rats were first trained to make ordinal comparisons of signal durations within three time ranges using 0.5, 1.0, and 3.0-s standard durations. Local field potentials were then recorded from the dorsal striatum and sensorimotor cortex in order to investigate the pattern of neural oscillations during each phase of the ordinal-comparison process. Increased power in delta and theta frequency ranges was observed during both the encoding and comparison stages. Active maintenance of a selected response, "shorter" or "longer" (counter-balanced across left and right levers), was represented by an increase of theta and delta oscillations in the contralateral striatum and cortex. Taken together, these data suggest that neural oscillations in the delta-theta range play an important role in the encoding, maintenance, and comparison of signal durations.

Neural basis for categorical boundaries in the primate pre-SMA during relative categorization of time intervals

Perceptual categorization depends on the assignment of different stimuli to specific groups based, in principle, on the notion of flexible categorical boundaries. To determine the neural basis of categorical boundaries, we record the activity of pre-SMA neurons of monkeys executing an interval categorization task in which the limit between short and long categories changes between blocks of trials within a session. A large population of cells encodes this boundary by reaching a constant peak of activity close to the corresponding subjective limit. Notably, the time at which this peak is reached changes according to the categorical boundary of the current block, predicting the monkeys’ categorical decision on a trial-by-trial basis. In addition, pre-SMA cells also represent the category selected by the monkeys and the outcome of the decision. These results suggest that the pre-SMA adaptively encodes subjective duration boundaries between short and long durations and contains crucial neural information to categorize intervals and evaluate the outcome of such perceptual decisions.

Grouping stimuli into categories often depends on a subjective determination of category boundaries. Here the authors report a neuronal population in pre-supplementary motor area whose peak activity predicts the categorical decision boundary between long and short time intervals on a trial-by-trial basis.

Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model

Brain activity evolves through time, creating trajectories of activity that underlie sensorimotor processing, behavior, and learning and memory. Therefore, understanding the temporal nature of neural dynamics is essential to understanding brain function and behavior. In vivo studies have demonstrated that sequential transient activation of neurons can encode time. However, it remains unclear whether these patterns emerge from feedforward network architectures or from recurrent networks and, furthermore, what role network structure plays in timing. We address these issues using a recurrent neural network (RNN) model with distinct populations of excitatory and inhibitory units. Consistent with experimental data, a single RNN could autonomously produce multiple functionally feedforward trajectories, thus potentially encoding multiple timed motor patterns lasting up to several seconds. Importantly, the model accounted for Weber's law, a hallmark of timing behavior. Analysis of network connectivity revealed that efficiency-a measure of network interconnectedness-decreased as the number of stored trajectories increased. Additionally, the balance of excitation (E) and inhibition (I) shifted toward excitation during each unit's activation time, generating the prediction that observed sequential activity relies on dynamic control of the E/I balance. Our results establish for the first time that the same RNN can generate multiple functionally feedforward patterns of activity as a result of dynamic shifts in the E/I balance imposed by the connectome of the RNN. We conclude that recurrent network architectures account for sequential neural activity, as well as for a fundamental signature of timing behavior: Weber's law.

The Fast Spiking Subpopulation of Striatal Neurons Coding for Temporal Cognition of Movements

Background: Timing dysfunctions occur in a number of neurological and psychiatric disorders such as Parkinson's disease, obsessive-compulsive disorder, autism and attention-deficit-hyperactivity disorder. Several lines of evidence show that disrupted timing processing is involved in specific fronto-striatal abnormalities. The striatum encodes reinforcement learning and procedural motion, and consequently is required to represent temporal information precisely, which then guides actions in proper sequence. Previous studies highlighted the temporal scaling property of timing-relevant striatal neurons; however, it is still unknown how this is accomplished over short temporal latencies, such as the sub-seconds to seconds range. Methods: We designed a task with a series of timing behaviors that required rats to reproduce a fixed duration with robust action. Using chronic multichannel electrode arrays, we recorded neural activity from dorso-medial striatum in 4 rats performing the task and identified modulation response of each neuron to different events. Cell type classification was performed according to a multi-criteria clustering analysis. Results: Dorso-medial striatal neurons (n = 557) were recorded, of which 113 single units were considered as timing-relevant neurons, especially the fast-spiking subpopulation that had trial-to-trial ramping up or ramping down firing modulation during the time estimation period. Furthermore, these timing-relevant striatal neurons had to calibrate the spread of their firing pattern by rewarded experience to express the timing behavior accurately. Conclusion: Our data suggests that the dynamic activities of timing-relevant units encode information about the current duration and recent outcomes, which is needed to predict and drive the following action. These results reveal the potential mechanism of time calibration in a short temporal resolution, which may help to explain the neural basis of motor coordination affected by certain physiological or pathological conditions.

Flexible timing by temporal scaling of cortical responses

Musicians can perform at different tempos, speakers can control the cadence of their speech, and children can flexibly vary their temporal expectations of events. To understand the neural basis of such flexibility, we recorded from the medial frontal cortex of nonhuman primates trained to produce different time intervals with different effectors. Neural responses were heterogeneous, nonlinear and complex, and exhibited a remarkable form of temporal invariance: firing rate profiles were temporally scaled to match the produced intervals. Recording from downstream neurons in the caudate and thalamic neurons projecting to the medial frontal cortex indicated that this phenomenon originates within cortical networks. Recurrent neural network models trained to perform the task revealed that temporal scaling emerges from nonlinearities in the network and degree of scaling is controlled by the strength of external input. These findings demonstrate a simple and general mechanism for conferring temporal flexibility upon sensorimotor and cognitive functions.

Motor origin of temporal predictions in auditory attention

In behavior, action and perception are inherently interdependent. However, the actual mechanistic contributions of the motor system to sensory processing are unknown. We present neurophysiological evidence that the motor system is involved in predictive timing, a brain function that aligns temporal fluctuations of attention with the timing of events in a task-relevant stream, thus facilitating sensory selection and optimizing behavior. In a magnetoencephalography experiment involving auditory temporal attention, participants had to disentangle two streams of sound on the unique basis of endogenous temporal cues. We show that temporal predictions are encoded by interdependent delta and beta neural oscillations originating from the left sensorimotor cortex, and directed toward auditory regions. We also found that overt rhythmic movements improved the quality of temporal predictions and sharpened the temporal selection of relevant auditory information. This latter behavioral and functional benefit was associated with increased signaling of temporal predictions in right-lateralized frontoparietal associative regions. In sum, this study points at a covert form of auditory active sensing. Our results emphasize the key role of motor brain areas in providing contextual temporal information to sensory regions, driving perceptual and behavioral selection.

Rodent Medial Frontal Control of Temporal Processing in the Dorsomedial Striatum

Although frontostriatal circuits are critical for the temporal control of action, how time is encoded in frontostriatal circuits is unknown. We recorded from frontal and striatal neurons while rats engaged in interval timing, an elementary cognitive function that engages both areas. We report four main results. First, "ramping" activity, a monotonic change in neuronal firing rate across time, is observed throughout frontostriatal ensembles. Second, frontostriatal activity scales across multiple intervals. Third, striatal ramping neurons are correlated with activity of the medial frontal cortex. Finally, interval timing and striatal ramping activity are disrupted when the medial frontal cortex is inactivated. Our results support the view that striatal neurons integrate medial frontal activity and are consistent with drift-diffusion models of interval timing. This principle elucidates temporal processing in frontostriatal circuits and provides insight into how the medial frontal cortex exerts top-down control of cognitive processing in the striatum.SIGNIFICANCE STATEMENT The ability to guide actions in time is essential to mammalian behavior from rodents to humans. The prefrontal cortex and striatum are critically involved in temporal processing and share extensive neuronal connections, yet it remains unclear how these structures represent time. We studied these two brain areas in rodents performing interval-timing tasks and found that time-dependent "ramping" activity, a monotonic increase or decrease in neuronal activity, was a key temporal signal. Furthermore, we found that striatal ramping activity was correlated with and dependent upon medial frontal activity. These results provide insight into information-processing principles in frontostriatal circuits.

On the Integration of Space, Time, and Memory

The hippocampus is famous for mapping locations in spatially organized environments, and several recent studies have shown that hippocampal networks also map moments in temporally organized experiences. Here I consider how space and time are integrated in the representation of memories. The brain pathways for spatial and temporal cognition involve overlapping and interacting systems that converge on the hippocampal region. There is evidence that spatial and temporal aspects of memory are processed somewhat differently in the circuitry of hippocampal subregions but become fully integrated within CA1 neuronal networks as independent, multiplexed representations of space and time. Hippocampal networks also map memories across a broad range of abstract relations among events, suggesting that the findings on spatial and temporal organization reflect a generalized mechanism for organizing memories.

Differential Encoding of Time by Prefrontal and Striatal Network Dynamics

Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas.

SIGNIFICANCE STATEMENT

The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs.

Learning multiple variable-speed sequences in striatum via cortical tutoring

Sparse, sequential patterns of neural activity have been observed in numerous brain areas during timekeeping and motor sequence tasks. Inspired by such observations, we construct a model of the striatum, an all-inhibitory circuit where sequential activity patterns are prominent, addressing the following key challenges: (i) obtaining control over temporal rescaling of the sequence speed, with the ability to generalize to new speeds; (ii) facilitating flexible expression of distinct sequences via selective activation, concatenation, and recycling of specific subsequences; and (iii) enabling the biologically plausible learning of sequences, consistent with the decoupling of learning and execution suggested by lesion studies showing that cortical circuits are necessary for learning, but that subcortical circuits are sufficient to drive learned behaviors. The same mechanisms that we describe can also be applied to circuits with both excitatory and inhibitory populations, and hence may underlie general features of sequential neural activity pattern generation in the brain.

DOI:http://dx.doi.org/10.7554/eLife.26084.001

Dynamic representation of time in brain states

The ability to process time on the scale of milliseconds and seconds is essential for behaviour. A growing number of studies have started to focus on brain dynamics as a mechanism for temporal encoding. Although there is growing evidence in favour of this view from computational and in vitro studies, there is still a lack of results from experiments in humans. We show that high-dimensional brain states revealed by multivariate pattern analysis of human EEG are correlated to temporal judgements. First, we show that, as participants estimate temporal intervals, the spatiotemporal dynamics of their brain activity are consistent across trials. Second, we present evidence that these dynamics exhibit properties of temporal perception, such as scale invariance. Lastly, we show that it is possible to predict temporal judgements based on brain states. These results show how scalp recordings can reveal the spatiotemporal dynamics of human brain activity related to temporal processing.

The Computational and Neural Basis of Rhythmic Timing in Medial Premotor Cortex

The neural underpinnings of rhythmic behavior, including music and dance, have been studied using the synchronization-continuation task (SCT), where subjects initially tap in synchrony with an isochronous metronome and then keep tapping at a similar rate via an internal beat mechanism. Here, we provide behavioral and neural evidence that supports a resetting drift-diffusion model (DDM) during SCT. Behaviorally, we show the model replicates the linear relation between the mean and standard-deviation of the intervals produced by monkeys in SCT. We then show that neural populations in the medial premotor cortex (MPC) contain an accurate trial-by-trial representation of elapsed-time between taps. Interestingly, the autocorrelation structure of the elapsed-time representation is consistent with a DDM. These results indicate that MPC has an orderly representation of time with features characteristic of concatenated DDMs and that this population signal can be used to orchestrate the rhythmic structure of the internally timed elements of SCT.SIGNIFICANCE STATEMENT The present study used behavioral data, ensemble recordings from medial premotor cortex (MPC) in macaque monkeys, and computational modeling, to establish evidence in favor of a class of drift-diffusion models of rhythmic timing during a synchronization-continuation tapping task (SCT). The linear relation between the mean and standard-deviation of the intervals produced by monkeys in SCT is replicated by the model. Populations of MPC cells faithfully represent the elapsed time between taps, and there is significant trial-by-trial relation between decoded times and the timing behavior of the monkeys. Notably, the neural decoding properties, including its autocorrelation structure are consistent with a set of drift-diffusion models that are arranged sequentially and that are resetting in each SCT tap.

Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation

It has been suggested that dopamine (DA) represents reward-prediction-error (RPE) defined in reinforcement learning and therefore DA responds to unpredicted but not predicted reward. However, recent studies have found DA response sustained towards predictable reward in tasks involving self-paced behavior, and suggested that this response represents a motivational signal. We have previously shown that RPE can sustain if there is decay/forgetting of learned-values, which can be implemented as decay of synaptic strengths storing learned-values. This account, however, did not explain the suggested link between tonic/sustained DA and motivation. In the present work, we explored the motivational effects of the value-decay in self-paced approach behavior, modeled as a series of ‘Go’ or ‘No-Go’ selections towards a goal. Through simulations, we found that the value-decay can enhance motivation, specifically, facilitate fast goal-reaching, albeit counterintuitively. Mathematical analyses revealed that underlying potential mechanisms are twofold: (1) decay-induced sustained RPE creates a gradient of ‘Go’ values towards a goal, and (2) value-contrasts between ‘Go’ and ‘No-Go’ are generated because while chosen values are continually updated, unchosen values simply decay. Our model provides potential explanations for the key experimental findings that suggest DA's roles in motivation: (i) slowdown of behavior by post-training blockade of DA signaling, (ii) observations that DA blockade severely impairs effortful actions to obtain rewards while largely sparing seeking of easily obtainable rewards, and (iii) relationships between the reward amount, the level of motivation reflected in the speed of behavior, and the average level of DA. These results indicate that reinforcement learning with value-decay, or forgetting, provides a parsimonious mechanistic account for the DA's roles in value-learning and motivation. Our results also suggest that when biological systems for value-learning are active even though learning has apparently converged, the systems might be in a state of dynamic equilibrium, where learning and forgetting are balanced.

Dopamine (DA) has been suggested to have two reward-related roles: (1) representing reward-prediction-error (RPE), and (2) providing motivational drive. Role(1) is based on the physiological results that DA responds to unpredicted but not predicted reward, whereas role(2) is supported by the pharmacological results that blockade of DA signaling causes motivational impairments such as slowdown of self-paced behavior. So far, these two roles are considered to be played by two different temporal patterns of DA signals: role(1) by phasic signals and role(2) by tonic/sustained signals. However, recent studies have found sustained DA signals with features indicative of both roles (1) and (2), complicating this picture. Meanwhile, whereas synaptic/circuit mechanisms for role(1), i.e., how RPE is calculated in the upstream of DA neurons and how RPE-dependent update of learned-values occurs through DA-dependent synaptic plasticity, have now become clarified, mechanisms for role(2) remain unclear. In this work, we modeled self-paced behavior by a series of ‘Go’ or ‘No-Go’ selections in the framework of reinforcement-learning assuming DA's role(1), and demonstrated that incorporation of decay/forgetting of learned-values, which is presumably implemented as decay of synaptic strengths storing learned-values, provides a potential unified mechanistic account for the DA's two roles, together with its various temporal patterns.

Reinforcement learning with Marr

To many, the poster child for David Marr's famous three levels of scientific inquiry is reinforcement learning-a computational theory of reward optimization, which readily prescribes algorithmic solutions that evidence striking resemblance to signals found in the brain, suggesting a straightforward neural implementation. Here we review questions that remain open at each level of analysis, concluding that the path forward to their resolution calls for inspiration across levels, rather than a focus on mutual constraints.

A Citation-Based Analysis and Review of Significant Papers on Timing and Time Perception

Time is an important dimension of brain function, but little is yet known about the underlying cognitive principles and neurobiological mechanisms. The field of timing and time perception has witnessed tremendous growth and multidisciplinary interest in the recent years with the advent of modern neuroimaging and neurophysiological approaches. In this article, I used a data mining approach to analyze the timing literature published by a select group of researchers (n = 202) during the period 2000–2015 and highlight important reviews as well as empirical articles that meet the criterion of a minimum of 100 citations. The qualifying articles (n = 150) are listed in a table along with key details such as number of citations, names of authors, year and journal of publication as well as a short summary of the findings of each study. The results of such a data-driven approach to literature review not only serve as a useful resource to any researcher interested in timing, but also provides a means to evaluate key papers that have significantly influenced the field and summarize recent progress and popular research trends in the field. Additionally, such analyses provides food for thought about future scientific directions and raises important questions about improving organizational structures to boost open science and progress in the field. I discuss exciting avenues for future research that have the potential to significantly advance our understanding of the neurobiology of timing, and propose the establishment of a new society, the Timing Research Forum, to promote open science and collaborative work within the highly diverse and multidisciplinary community of researchers in the field of timing and time perception.