Measuring time with different neural chronometers during a synchronization-continuation task

Reconfiguration of striatal connectivity for timing and action

The medial cortico-striatal-thalamo-cortical (CSTC) motor circuit is a core system that exerts control over interval timing and action. A common network generates these behaviors possibly owing to cellular coding of temporal and non-temporal information, which in turn promotes reconfiguration of functional connectivity in accord with behavioral goals. At the neuroanatomical level, support for flexible CSTC reconfiguration comes from studies of temporal illusions demonstrating that this system calibrates the experience of time through functional interactions with various context-sensitive brain regions. Revelations that CSTC effective connectivity is pivotal for context-dependent facets of voluntary actions, namely action planning, complement its role in predictive processes such as timing. These observations suggest that the CSTC is positioned to represent high-level information about 'what to do' and 'when to do it' by dynamically reconfiguring effective connectivity as circumstances arise.

Time in Cortical Circuits

Time is central to cognition. However, the neural basis for time-dependent cognition remains poorly understood. We explore how the temporal features of neural activity in cortical circuits and their capacity for plasticity can contribute to time-dependent cognition over short time scales. This neural activity is linked to cognition that operates in the present or anticipates events or stimuli in the near future. We focus on deliberation and planning in the context of decision making as a cognitive process that integrates information across time. We progress to consider how temporal expectations of the future modulate perception. We propose that understanding the neural basis for how the brain tells time and operates in time will be necessary to develop general models of cognition.

SIGNIFICANCE STATEMENT

Time is central to cognition. However, the neural basis for time-dependent cognition remains poorly understood. We explore how the temporal features of neural activity in cortical circuits and their capacity for plasticity can contribute to time-dependent cognition over short time scales. We propose that understanding the neural basis for how the brain tells time and operates in time will be necessary to develop general models of cognition.

Sensorimotor synchronization: neurophysiological markers of the asynchrony in a finger-tapping task

Sensorimotor synchronization (SMS) is a form of referential behavior in which an action is coordinated with a predictable external stimulus. The neural bases of the synchronization ability remain unknown, even in the simpler, paradigmatic task of finger tapping to a metronome. In this task the subject is instructed to tap in synchrony with a periodic sequence of brief tones, and the time difference between each response and the corresponding stimulus tone (asynchrony) is recorded. We make a step towards the identification of the neurophysiological markers of SMS by recording high-density EEG event-related potentials and the concurrent behavioral response-stimulus asynchronies during an isochronous paced finger-tapping task. Using principal component analysis, we found an asymmetry between the traces for advanced and delayed responses to the stimulus, in accordance with previous behavioral observations from perturbation studies. We also found that the amplitude of the second component encodes the higher-level percept of asynchrony 100 ms after the current stimulus. Furthermore, its amplitude predicts the asynchrony of the next step, past 300 ms from the previous stimulus, independently of the period length. Moreover, the neurophysiological processing of synchronization errors is performed within a fixed-duration interval after the stimulus. Our results suggest that the correction of a large asynchrony in a periodic task and the recovery of synchrony after a perturbation could be driven by similar neural processes.

A Neural Mechanism for Sensing and Reproducing a Time Interval

Timing plays a crucial role in sensorimotor function. However, the neural mechanisms that enable the brain to flexibly measure and reproduce time intervals are not known. We recorded neural activity in parietal cortex of monkeys in a time reproduction task. Monkeys were trained to measure and immediately afterward reproduce different sample intervals. While measuring an interval, neural responses had a nonlinear profile that increased with the duration of the sample interval. Activity was reset during the transition from measurement to production and was followed by a ramping activity whose slope encoded the previously measured sample interval. We found that firing rates at the end of the measurement epoch were correlated with both the slope of the ramp and the monkey's corresponding production interval on a trial-by-trial basis. Analysis of response dynamics further linked the rate of change of firing rates in the measurement epoch to the slope of the ramp in the production epoch. These observations suggest that, during time reproduction, an interval is measured prospectively in relation to the desired motor plan to reproduce that interval.

SMA Selectively Codes the Active Accumulation of Temporal, Not Spatial, Magnitude

Estimating duration depends on the sequential integration (accumulation) of temporal information in working memory. Using fMRI, we directly compared the accumulation of information in temporal versus spatial domains. Participants estimated either the duration or distance of the dynamic trajectory of a moving dot or, in a control condition, a static line stimulus. Comparing the duration versus distance of static lines activated an extensive cortico-striatal network. By contrast, comparing the duration versus distance of dynamic trajectories, both of which required sequential integration of information, activated SMA alone. Indeed, activity in SMA, as well as right inferior occipital cortex, increased parametrically as a function of stimulus duration and also correlated with individual differences in the propensity to overestimate stimulus duration. By contrast, activity in primary visual cortex increased parametrically as a function of stimulus distance. Crucially, a direct comparison of the parametric responses to duration versus distance revealed that activity in SMA increased incrementally as a function of stimulus duration but not as a function of stimulus distance. Collectively, our results indicate that SMA responds to the active accumulation of information selectively in the temporal domain.

Single trial beta oscillations index time estimation

Recent work shows that putamen-originating beta power oscillations serve as a carrier for temporal information during tapping tasks, with higher beta power associated with longer temporal reproductions. However, given the nature of tapping tasks, it is difficult to determine whether beta power dynamics observed in these tasks are linked to the generation or execution of motor programs or to the internal representation of time. To assess whether recent findings in animals generalize to human studies we reanalyzed existing EEG data of participants who estimated a 2.5s time interval with self-paced onset and offset keypresses. The results showed that the trial-to-trial beta power measured after the onset predicts the produced duration, such that higher beta power indexes longer produced durations. Moreover, although beta power measured before the first key-press also influenced the estimated interval, it did so independently from post-first-keypress beta power. These results suggest that initial motor inhibition plays an important role in interval production, and that this inhibition can be interpreted as a biased starting point of the decision processes involved in time estimation.

β oscillations are linked to the initiation of sensory-cued movement sequences and the internal guidance of regular tapping in the monkey

β oscillations in the basal ganglia have been associated with interval timing. We recorded the putaminal local field potentials (LFPs) from monkeys performing a synchronization-continuation task (SCT) and a serial reaction-time task (RTT), where the animals produced regularly and irregularly paced tapping sequences, respectively. We compared the activation profile of β oscillations between tasks and found transient bursts of β activity in both the RTT and SCT. During the RTT, β power was higher at the beginning of the task, especially when LFPs were aligned to the stimuli. During the SCT, β was higher during the internally driven continuation phase, especially for tap-aligned LFPs. Interestingly, a set of LFPs showed an initial burst of β at the beginning of the SCT, similar to the RTT, followed by a decrease in β oscillations during the synchronization phase, to finally rebound during the continuation phase. The rebound during the continuation phase of the SCT suggests that the corticostriatal circuit is involved in the control of internally driven motor sequences. In turn, the transient bursts of β activity at the beginning of both tasks suggest that the basal ganglia produce a general initiation signal that engages the motor system in different sequential behaviors.

Keeping an eye on the conductor: neural correlates of visuo-motor synchronization and musical experience

For orchestra musicians, synchronized playing under a conductor’s direction is necessary to achieve optimal performance. Previous studies using simple auditory/visual stimuli have reported cortico-subcortical networks underlying synchronization and that training improves the accuracy of synchronization. However, it is unclear whether people who played regularly under a conductor and non-musicians activate the same networks when synchronizing with a conductor’s gestures. We conducted a functional magnetic resonance imaging (fMRI) experiment testing nonmusicians and musicians who regularly play music under a conductor. Participants were required to tap the rhythm they perceived from silent movies displaying either conductor’s gestures or a swinging metronome. Musicians performed tapping under a conductor with more precision than nonmusicians. Results from fMRI measurement showed greater activity in the anterior part of the left superior frontal gyrus (SFG) in musicians with more frequent practice under a conductor. Conversely, tapping with the metronome did not show any difference between musicians and nonmusicians, indicating that the expertize effect in tapping under the conductor does not result in a general increase in tapping performance for musicians. These results suggest that orchestra musicians have developed an advanced ability to predict conductor’s next action from the gestures.

Signs of timing in motor cortex during movement preparation and cue anticipation

The capacity to accurately anticipate the timing of predictable events is essential for sensorimotor behavior. Motor cortex holds an established role in movement preparation and execution. In this chapter we review the different ways in which motor cortical activity is modulated by event timing in sensorimotor delay tasks. During movement preparation, both single neuron and population responses reflect the temporal constraints of the task. Anticipatory modulations prior to sensory cues are also observed in motor cortex when the cue timing is predictable. We propose that the motor cortical activity during cue anticipation and movement preparation is embedded in a timing network that facilitates sensorimotor processing. In this context, the pre-cue and post-cue activity may reflect a presetting mechanism, complementing processing during movement execution, while prohibiting premature responses in situations requiring delayed motor output.

Finding the beat: a neural perspective across humans and non-human primates

Humans possess an ability to perceive and synchronize movements to the beat in music ('beat perception and synchronization'), and recent neuroscientific data have offered new insights into this beat-finding capacity at multiple neural levels. Here, we review and compare behavioural and neural data on temporal and sequential processing during beat perception and entrainment tasks in macaques (including direct neural recording and local field potential (LFP)) and humans (including fMRI, EEG and MEG). These abilities rest upon a distributed set of circuits that include the motor cortico-basal-ganglia-thalamo-cortical (mCBGT) circuit, where the supplementary motor cortex (SMA) and the putamen are critical cortical and subcortical nodes, respectively. In addition, a cortical loop between motor and auditory areas, connected through delta and beta oscillatory activity, is deeply involved in these behaviours, with motor regions providing the predictive timing needed for the perception of, and entrainment to, musical rhythms. The neural discharge rate and the LFP oscillatory activity in the gamma- and beta-bands in the putamen and SMA of monkeys are tuned to the duration of intervals produced during a beat synchronization-continuation task (SCT). Hence, the tempo during beat synchronization is represented by different interval-tuned cells that are activated depending on the produced interval. In addition, cells in these areas are tuned to the serial-order elements of the SCT. Thus, the underpinnings of beat synchronization are intrinsically linked to the dynamics of cell populations tuned for duration and serial order throughout the mCBGT. We suggest that a cross-species comparison of behaviours and the neural circuits supporting them sets the stage for a new generation of neurally grounded computational models for beat perception and synchronization.

Four principles of bio-musicology

As a species-typical trait of Homo sapiens, musicality represents a cognitively complex and biologically grounded capacity worthy of intensive empirical investigation. Four principles are suggested here as prerequisites for a successful future discipline of bio-musicology. These involve adopting: (i) a multicomponent approach which recognizes that musicality is built upon a suite of interconnected capacities, of which none is primary; (ii) a pluralistic Tinbergian perspective that addresses and places equal weight on questions of mechanism, ontogeny, phylogeny and function; (iii) a comparative approach, which seeks and investigates animal homologues or analogues of specific components of musicality, wherever they can be found; and (iv) an ecologically motivated perspective, which recognizes the need to study widespread musical behaviours across a range of human cultures (and not focus solely on Western art music or skilled musicians). Given their pervasiveness, dance and music created for dancing should be considered central subcomponents of music, as should folk tunes, work songs, lullabies and children's songs. Although the precise breakdown of capacities required by the multicomponent approach remains open to debate, and different breakdowns may be appropriate to different purposes, I highlight four core components of human musicality--song, drumming, social synchronization and dance--as widespread and pervasive human abilities spanning across cultures, ages and levels of expertise. Each of these has interesting parallels in the animal kingdom (often analogies but in some cases apparent homologies also). Finally, I suggest that the search for universal capacities underlying human musicality, neglected for many years, should be renewed. The broad framework presented here illustrates the potential for a future discipline of bio-musicology as a rich field for interdisciplinary and comparative research.

Without it no music: cognition, biology and evolution of musicality

Musicality can be defined as a natural, spontaneously developing trait based on and constrained by biology and cognition. Music, by contrast, can be defined as a social and cultural construct based on that very musicality. One critical challenge is to delineate the constituent elements of musicality. What biological and cognitive mechanisms are essential for perceiving, appreciating and making music? Progress in understanding the evolution of music cognition depends upon adequate characterization of the constituent mechanisms of musicality and the extent to which they are present in non-human species. We argue for the importance of identifying these mechanisms and delineating their functions and developmental course, as well as suggesting effective means of studying them in human and non-human animals. It is virtually impossible to underpin the evolutionary role of musicality as a whole, but a multicomponent perspective on musicality that emphasizes its constituent capacities, development and neural cognitive specificity is an excellent starting point for a research programme aimed at illuminating the origins and evolution of musical behaviour as an autonomous trait.

Searching for the origins of musicality across species

In the introduction to this theme issue, Honing et al. suggest that the origins of musicality--the capacity that makes it possible for us to perceive, appreciate and produce music--can be pursued productively by searching for components of musicality in other species. Recent studies have highlighted that the behavioural relevance of stimuli to animals and the relation of experimental procedures to their natural behaviour can have a large impact on the type of results that can be obtained for a given species. Through reviewing laboratory findings on animal auditory perception and behaviour, as well as relevant findings on natural behaviour, we provide evidence that both traditional laboratory studies and studies relating to natural behaviour are needed to answer the problem of musicality. Traditional laboratory studies use synthetic stimuli that provide more control than more naturalistic studies, and are in many ways suitable to test the perceptual abilities of animals. However, naturalistic studies are essential to inform us as to what might constitute relevant stimuli and parameters to test with laboratory studies, or why we may or may not expect certain stimulus manipulations to be relevant. These two approaches are both vital in the comparative study of musicality.

Neuronal representation of duration discrimination in the monkey striatum

Functional imaging and lesion studies in humans and animals suggest that the basal ganglia are crucial for temporal information processing. To elucidate neuronal mechanisms of interval timing in the basal ganglia, we recorded single-unit activity from the striatum of two monkeys while they performed a visual duration discrimination task. In the task, blue and red cues of different durations (0.2-2.0 sec) were successively presented. Each of the two cues was followed by a 1.0 sec delay period. The animals were instructed to choose the longer presented colored stimulus after the second delay period. A total of 498 phasically active neurons were recorded from the striatum, and 269 neurons were defined as task related. Two types of neuronal activity were distinguished during the delay periods. First, the activity gradually changed depending on the duration of the cue presented just before. This activity may represent the signal duration for later comparison between two cue durations. The activity during the second cue period also represented duration of the first cue. Second, the activity changed differently depending on whether the first or second cue was presented longer. This activity may represent discrimination results after the comparison between the two cue durations. These findings support the assumption that striatal neurons represent timing information of sensory signals for duration discrimination.

Dissociating movement from movement timing in the rat primary motor cortex

Neural encoding of the passage of time to produce temporally precise movements remains an open question. Neurons in several brain regions across different experimental contexts encode estimates of temporal intervals by scaling their activity in proportion to the interval duration. In motor cortex the degree to which this scaled activity relies upon afferent feedback and is guided by motor output remains unclear. Using a neural reward paradigm to dissociate neural activity from motor output before and after complete spinal transection, we show that temporally scaled activity occurs in the rat hindlimb motor cortex in the absence of motor output and after transection. Context-dependent changes in the encoding are plastic, reversible, and re-established following injury. Therefore, in the absence of motor output and despite a loss of afferent feedback, thought necessary for timed movements, the rat motor cortex displays scaled activity during a broad range of temporally demanding tasks similar to that identified in other brain regions.

Oscillatory multiplexing of neural population codes for interval timing and working memory

Interval timing and working memory are critical components of cognition that are supported by neural oscillations in prefrontal-striatal-hippocampal circuits. In this review, the properties of interval timing and working memory are explored in terms of behavioral, anatomical, pharmacological, and neurophysiological findings. We then describe the various neurobiological theories that have been developed to explain these cognitive processes - largely independent of each other. Following this, a coupled excitatory - inhibitory oscillation (EIO) model of temporal processing is proposed to address the shared oscillatory properties of interval timing and working memory. Using this integrative approach, we describe a hybrid model explaining how interval timing and working memory can originate from the same oscillatory processes, but differ in terms of which dimension of the neural oscillation is utilized for the extraction of item, temporal order, and duration information. This extension of the striatal beat-frequency (SBF) model of interval timing (Matell and Meck, 2000, 2004) is based on prefrontal-striatal-hippocampal circuit dynamics and has direct relevance to the pathophysiological distortions observed in time perception and working memory in a variety of psychiatric and neurological conditions.

Motor system evolution and the emergence of high cognitive functions

In human and nonhuman primates, the cortical motor system comprises a collection of brain areas primarily related to motor control. Existing evidence suggests that no other mammalian group has the number, extension, and complexity of motor-related areas observed in the frontal lobe of primates. Such diversity is probably related to the wide behavioral flexibility that primates display. Indeed, recent comparative anatomical, psychophysical, and neurophysiological studies suggest that the evolution of the motor cortical areas closely correlates with the emergence of high cognitive abilities. Advances in understanding the cortical motor system have shown that these areas are also related to functions previously linked to higher-order associative areas. In addition, experimental observations have shown that the classical distinction between perceptual and motor functions is not strictly followed across cortical areas. In this paper, we review evidence suggesting that evolution of the motor system had a role in the shaping of different cognitive functions in primates. We argue that the increase in the complexity of the motor system has contributed to the emergence of new abilities observed in human and nonhuman primates, including the recognition and imitation of the actions of others, speech perception and production, and the execution and appreciation of the rhythmic structure of music.

Dynamic representation of the temporal and sequential structure of rhythmic movements in the primate medial premotor cortex

We determined the encoding properties of single cells and the decoding accuracy of cell populations in the medial premotor cortex (MPC) of Rhesus monkeys to represent in a time-varying fashion the duration and serial order of six intervals produced rhythmically during a synchronization-continuation tapping task. We found that MPC represented the temporal and sequential structure of rhythmic movements by activating small ensembles of neurons that encoded the duration or the serial order in rapid succession, so that the pattern of active neurons changed dramatically within each interval. Interestingly, the width of the encoding or decoding function for serial order increased as a function of duration. Finally, we found that the strength of correlation in spontaneous activity of the individual cells varied as a function of the timing of their recruitment. These results demonstrate the existence of dynamic representations in MPC for the duration and serial order of intervals produced rhythmically and suggest that this dynamic code depends on ensembles of interconnected neurons that provide a strong synaptic drive to the next ensemble in a consecutive chain of neural events.

Attention and working memory: two basic mechanisms for constructing temporal experiences

Various kinds of observations show that the ability of human beings to both consciously relive past events – episodic memory – and conceive future events, entails an active process of construction. This construction process also underpins many other important aspects of conscious human life, such as perceptions, language, and conscious thinking. This article provides an explanation of what makes the constructive process possible and how it works. The process mainly relies on attentional activity, which has a discrete and periodic nature, and working memory, which allows for the combination of discrete attentional operations. An explanation is also provided of how past and future events are constructed.

Linking perception, cognition, and action: psychophysical observations and neural network modelling

It has been argued that perception, decision making, and movement planning are in reality tightly interwoven brain processes. However, how they are implemented in neural circuits is still a matter of debate. We tested human subjects in a temporal categorization task in which intervals had to be categorized as short or long. Subjects communicated their decision by moving a cursor into one of two possible targets, which appeared separated by different angles from trial to trial. Even though there was a 1 second-long delay between interval presentation and decision communication, categorization difficulty affected subjects’ performance, reaction (RT) and movement time (MT). In addition, reaction and movement times were also influenced by the distance between the targets. This implies that not only perceptual, but also movement-related considerations were incorporated into the decision process. Therefore, we searched for a model that could use categorization difficulty and target separation to describe subjects’ performance, RT, and MT. We developed a network consisting of two mutually inhibiting neural populations, each tuned to one of the possible categories and composed of an accumulation and a memory node. This network sequentially acquired interval information, maintained it in working memory and was then attracted to one of two possible states, corresponding to a categorical decision. It faithfully replicated subjects’ RT and MT as a function of categorization difficulty and target distance; it also replicated performance as a function of categorization difficulty. Furthermore, this model was used to make new predictions about the effect of untested durations, target distances and delay durations. To our knowledge, this is the first biologically plausible model that has been proposed to account for decision making and communication by integrating both sensory and motor planning information.

Synchrony and motor mimicking in chimpanzee observational learning

Cumulative tool-based culture underwrote our species' evolutionary success, and tool-based nut-cracking is one of the strongest candidates for cultural transmission in our closest relatives, chimpanzees. However the social learning processes that may explain both the similarities and differences between the species remain unclear. A previous study of nut-cracking by initially naïve chimpanzees suggested that a learning chimpanzee holding no hammer nevertheless replicated hammering actions it witnessed. This observation has potentially important implications for the nature of the social learning processes and underlying motor coding involved. In the present study, model and observer actions were quantified frame-by-frame and analysed with stringent statistical methods, demonstrating synchrony between the observer's and model's movements, cross-correlation of these movements above chance level and a unidirectional transmission process from model to observer. These results provide the first quantitative evidence for motor mimicking underlain by motor coding in apes, with implications for mirror neuron function.

Information processing in the primate basal ganglia during sensory-guided and internally driven rhythmic tapping

Gamma (γ) and beta (β) oscillations seem to play complementary functions in the cortico-basal ganglia-thalamo-cortical circuit (CBGT) during motor behavior. We investigated the time-varying changes of the putaminal spiking activity and the spectral power of local field potentials (LFPs) during a task where the rhythmic tapping of monkeys was guided by isochronous stimuli separated by a fixed duration (synchronization phase), followed by a period of internally timed movements (continuation phase). We found that the power of both bands and the discharge rate of cells showed an orderly change in magnitude as a function of the duration and/or the serial order of the intervals executed rhythmically. More LFPs were tuned to duration and/or serial order in the β- than the γ-band, although different values of preferred features were represented by single cells and by both bands. Importantly, in the LFPs tuned to serial order, there was a strong bias toward the continuation phase for the β-band when aligned to movements, and a bias toward the synchronization phase for the γ-band when aligned to the stimuli. Our results suggest that γ-oscillations reflect local computations associated with stimulus processing, whereas β-activity involves the entrainment of large putaminal circuits, probably in conjunction with other elements of CBGT, during internally driven rhythmic tapping.

Monkeys time their pauses of movement and not their movement-kinematics during a synchronization-continuation rhythmic task

A critical question in tapping behavior is to understand whether the temporal control is exerted on the duration and trajectory of the downward-upward hand movement or on the pause between hand movements. In the present study, we determined the duration of both the movement execution and pauses of monkeys performing a synchronization-continuation task (SCT), using the speed profile of their tapping behavior. We found a linear increase in the variance of pause-duration as a function of interval, while the variance of the motor implementation was relatively constant across intervals. In fact, 96% of the variability of the duration of a complete tapping cycle (pause + movement) was due to the variability of the pause duration. In addition, we performed a Bayesian model selection to determine the effect of interval duration (450–1,000 ms), serial-order (1–6 produced intervals), task phase (sensory cued or internally driven), and marker modality (auditory or visual) on the duration of the movement-pause and tapping movement. The results showed that the most important parameter used to successfully perform the SCT was the control of the pause duration. We also found that the kinematics of the tapping movements was concordant with a stereotyped ballistic control of the hand pressing the push-button. The present findings support the idea that monkeys used an explicit timing strategy to perform the SCT, where a dedicated timing mechanism controlled the duration of the pauses of movement, while also triggered the execution of fixed movements across each interval of the rhythmic sequence.

The evolutionary neuroscience of musical beat perception: the Action Simulation for Auditory Prediction (ASAP) hypothesis

EVERY HUMAN CULTURE HAS SOME FORM OF MUSIC WITH A BEAT

a perceived periodic pulse that structures the perception of musical rhythm and which serves as a framework for synchronized movement to music. What are the neural mechanisms of musical beat perception, and how did they evolve? One view, which dates back to Darwin and implicitly informs some current models of beat perception, is that the relevant neural mechanisms are relatively general and are widespread among animal species. On the basis of recent neural and cross-species data on musical beat processing, this paper argues for a different view. Here we argue that beat perception is a complex brain function involving temporally-precise communication between auditory regions and motor planning regions of the cortex (even in the absence of overt movement). More specifically, we propose that simulation of periodic movement in motor planning regions provides a neural signal that helps the auditory system predict the timing of upcoming beats. This "action simulation for auditory prediction" (ASAP) hypothesis leads to testable predictions. We further suggest that ASAP relies on dorsal auditory pathway connections between auditory regions and motor planning regions via the parietal cortex, and suggest that these connections may be stronger in humans than in non-human primates due to the evolution of vocal learning in our lineage. This suggestion motivates cross-species research to determine which species are capable of human-like beat perception, i.e., beat perception that involves accurate temporal prediction of beat times across a fairly broad range of tempi.

Sensorimotor synchronization: a review of recent research (2006-2012)

Sensorimotor synchronization (SMS) is the coordination of rhythmic movement with an external rhythm, ranging from finger tapping in time with a metronome to musical ensemble performance. An earlier review (Repp, 2005) covered tapping studies; two additional reviews (Repp, 2006a, b) focused on music performance and on rate limits of SMS, respectively. The present article supplements and extends these earlier reviews by surveying more recent research in what appears to be a burgeoning field. The article comprises four parts, dealing with (1) conventional tapping studies, (2) other forms of moving in synchrony with external rhythms (including dance and nonhuman animals' synchronization abilities), (3) interpersonal synchronization (including musical ensemble performance), and (4) the neuroscience of SMS. It is evident that much new knowledge about SMS has been acquired in the last 7 years.

Are non-human primates capable of rhythmic entrainment? Evidence for the gradual audiomotor evolution hypothesis

We propose a decomposition of the neurocognitive mechanisms that might underlie interval-based timing and rhythmic entrainment. Next to reviewing the concepts central to the definition of rhythmic entrainment, we discuss recent studies that suggest rhythmic entrainment to be specific to humans and a selected group of bird species, but, surprisingly, is not obvious in non-human primates. On the basis of these studies we propose the gradual audiomotor evolution hypothesis that suggests that humans fully share interval-based timing with other primates, but only partially share the ability of rhythmic entrainment (or beat-based timing). This hypothesis accommodates the fact that non-human primates (i.e., macaques) performance is comparable to humans in single interval tasks (such as interval reproduction, categorization, and interception), but show differences in multiple interval tasks (such as rhythmic entrainment, synchronization, and continuation). Furthermore, it is in line with the observation that macaques can, apparently, synchronize in the visual domain, but show less sensitivity in the auditory domain. And finally, while macaques are sensitive to interval-based timing and rhythmic grouping, the absence of a strong coupling between the auditory and motor system of non-human primates might be the reason why macaques cannot rhythmically entrain in the way humans do.

Neural mechanisms of rhythm perception: present findings and future directions

The capacity to synchronize movements to the beat in music is a complex, and apparently uniquely human characteristic. Synchronizing movements to the beat requires beat perception, which entails prediction of future beats in rhythmic sequences of temporal intervals. Absolute timing mechanisms, where patterns of temporal intervals are encoded as a series of absolute durations, cannot fully explain beat perception. Beat perception seems better accounted for by relative timing mechanisms, where temporal intervals of a pattern are coded relative to a periodic beat interval. Evidence from behavioral, neuroimaging, brain stimulation and neuronal cell recording studies suggests a functional dissociation between the neural substrates of absolute and relative timing. This chapter reviews current findings on relative timing in the context of rhythm and beat perception.

Searching for the holy grail: temporally informative firing patterns in the rat

This chapter reviews our work from the past decade investigating cortical and striatal firing patterns in rats while they time intervals in the multi-seconds range. We have found that both cortical and striatal firing rates contain information that the rat can use to identify how much time has elapsed both from trial onset and from the onset of an active response state. I describe findings showing that the striatal neurons that are modulated by time are also modulated by overt behaviors, suggesting that time modulates the strength of motor coding in the striatum, rather than being represented as an abstract quantity in isolation. I also describe work showing that there are a variety of temporally informative activity patterns in pre-motor cortex, and argue that the heterogeneity of these patterns can enhance an organism's temporal estimate. Finally, I describe recent behavioral work from my lab in which the simultaneous cueing of multiple durations leads to a scalar temporal expectation at an intermediate time, providing strong support for a monotonic representation of time.

Neurophysiology of timing in the hundreds of milliseconds: multiple layers of neuronal clocks in the medial premotor areas

The precise quantification of time in the subsecond scale is critical for many complex behaviors including music and dance appreciation/execution, speech comprehension/articulation, and the performance of many sports. Nevertheless, its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the cell activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing during a synchronization-continuation tapping task (SCT). In this task the rhythmic behavior of monkeys was synchronized to a metronome of isochronous stimuli in the hundreds of milliseconds range (synchronization phase), followed by a period where animals internally temporalized their movements (continuation phase). Overall, we found that the time-keeping mechanism in MPC is governed by different layers of neural clocks. Close to the temporal control of movements are two separate populations of ramping cells that code for elapsed or remaining time for a tapping movement during the SCT. Thus, the sensorimotor loops engaged during the task may depend on the cyclic interplay between two neuronal chronometers that quantify in their instantaneous discharge rate the time passed and the remaining time for an action. In addition, we found MPC neurons that are tuned to the duration of produced intervals during the rhythmic task, showing an orderly variation in the average discharge rate as a function of duration. All the tested durations in the subsecond scale were represented in the preferred intervals of the cell population. Most of the interval-tuned cells were also tuned to the ordinal structure of the six intervals produced sequentially in the SCT. Hence, this next level of temporal processing may work as the notes of a musical score, providing information to the timing network about what duration and ordinal element of the sequence are being executed. Finally, we describe how the timing circuit can use a dynamic neural representation of the passage of time and the context in which the intervals are executed by integrating the time-varying activity of populations of cells. These neural population clocks can be defined as distinct trajectories in the multidimensional cell response-space. We provide a hypothesis of how these different levels of neural clocks can interact to constitute a coherent timing machine that controls the rhythmic behavior during the SCT.

A neural hierarchy for illusions of time: duration adaptation precedes multisensory integration

Perceived time is inherently malleable. For example, adaptation to relatively long or short sensory events leads to a repulsive aftereffect such that subsequent events appear to be contracted or expanded (duration adaptation). Perceived visual duration can also be distorted via concurrent presentation of discrepant auditory durations (multisensory integration). The neural loci of both distortions remain unknown. In the current study we use a psychophysical approach to establish their relative positioning within the sensory processing hierarchy. We show that audiovisual integration induces marked distortions of perceived visual duration. We proceed to use these distorted durations as visual adapting stimuli yet find subsequent visual duration aftereffects to be consistent with physical rather than perceived visual duration. Conversely, the concurrent presentation of adapted auditory durations with nonadapted visual durations results in multisensory integration patterns consistent with perceived, rather than physical, auditory duration. These results demonstrate that recent sensory history modifies human duration perception prior to the combination of temporal information across sensory modalities and provides support for adaptation mechanisms mediated by duration selective neurons situated in early areas of the visual and auditory nervous system (Aubie, Sayegh, & Faure, 2012; Duysens, Schaafsma, & Orban, 1996; Leary, Edwards, & Rose, 2008).

Temporally specific sensory signals for the detection of stimulus omission in the primate deep cerebellar nuclei

The cerebellum is implicated in sensory prediction in the subsecond range. To explore how neurons in the cerebellum encode temporal information for the prediction of sensory events, we trained monkeys to make a saccade in response to either a single omission or deviation of isochronous repetitive stimuli. We found that neurons in the cerebellar dentate nucleus exhibited a gradual elevation of the baseline firing rate as the repetition progressed. Most neurons showed a transient suppression for each stimulus, and this firing modulation also increased gradually, opposed to the sensory adaptation. The magnitude of the enhanced sensory response positively correlated with interstimulus interval. Furthermore, when stimuli appeared unexpectedly earlier than the regular timing, the neuronal modulation became smaller, suggesting that the sensory response depended on the time elapsed since the previous stimulus. The enhancement of neuronal modulation was context dependent and was reduced or even absent when monkeys were unmotivated to detect stimulus omission. A significant negative correlation between neuronal activity at stimulus omission and saccade latency suggested that the timing of each stimulus was predicted by the amount of recovery from the transient response. Because inactivation of the recording sites delayed the detection of stimulus omission but only slightly altered the detection of stimulus deviation, these signals might be necessary for the prediction of stimulus timing but may not be involved only in the generation of saccades. Our results demonstrate a novel mechanism for temporal prediction of upcoming stimuli that accompanies the time-dependent modification of sensory gain in the cerebellum.

Rhythmic cognition in humans and animals: distinguishing meter and pulse perception

This paper outlines a cognitive and comparative perspective on human rhythmic cognition that emphasizes a key distinction between pulse perception and meter perception. Pulse perception involves the extraction of a regular pulse or "tactus" from a stream of events. Meter perception involves grouping of events into hierarchical trees with differing levels of "strength", or perceptual prominence. I argue that metrically-structured rhythms are required to either perform or move appropriately to music (e.g., to dance). Rhythms, from this metrical perspective, constitute "trees in time." Rhythmic syntax represents a neglected form of musical syntax, and warrants more thorough neuroscientific investigation. The recent literature on animal entrainment clearly demonstrates the capacity to extract the pulse from rhythmic music, and to entrain periodic movements to this pulse, in several parrot species and a California sea lion, and a more limited ability to do so in one chimpanzee. However, the ability of these or other species to infer hierarchical rhythmic trees remains, for the most part, unexplored (with some apparent negative results from macaques). The results from this animal comparative research, combined with new methods to explore rhythmic cognition neurally, provide exciting new routes for understanding not just rhythmic cognition, but hierarchical cognition more generally, from a biological and neural perspective.

Rhythm sensitivity in macaque monkeys

This study provides evidence that monkeys are rhythm sensitive. We composed isochronous tone sequences consisting of repeating triplets of two short tones and one long tone which humans perceive as repeating triplets of two weak and one strong beat. This regular sequence was compared to an irregular sequence with the same number of randomly arranged short and long tones with no such beat structure. To search for indication of rhythm sensitivity we employed an oddball paradigm in which occasional duration deviants were introduced in the sequences. In a pilot study on humans we showed that subjects more easily detected these deviants when they occurred in a regular sequence. In the monkeys we searched for spontaneous behaviors the animals executed concomitant with the deviants. We found that monkeys more frequently exhibited changes of gaze and facial expressions to the deviants when they occurred in the regular sequence compared to the irregular sequence. In addition we recorded neuronal firing and local field potentials from 175 sites of the primary auditory cortex during sequence presentation. We found that both types of neuronal signals differentiated regular from irregular sequences. Both signals were stronger in regular sequences and occurred after the onset of the long tones, i.e., at the position of the strong beat. Local field potential responses were also significantly larger for the durational deviants in regular sequences, yet in a later time window. We speculate that these temporal pattern-selective mechanisms with a focus on strong beats and their deviants underlie the perception of rhythm in the chosen sequences.

Interval tuning in the primate medial premotor cortex as a general timing mechanism

The precise quantification of time during motor performance is critical for many complex behaviors, including musical execution, speech articulation, and sports; however, its neural mechanisms are primarily unknown. We found that neurons in the medial premotor cortex (MPC) of behaving monkeys are tuned to the duration of produced intervals during rhythmic tapping tasks. Interval-tuned neurons showed similar preferred intervals across tapping behaviors that varied in the number of produced intervals and the modality used to drive temporal processing. In addition, we found that the same population of neurons is able to multiplex the ordinal structure of a sequence of rhythmic movements and a wide range of durations in the range of hundreds of milliseconds. Our results also revealed a possible gain mechanism for encoding the total number of intervals in a sequence of temporalized movements, where interval-tuned cells show a multiplicative effect of their activity for longer sequences of intervals. These data suggest that MPC is part of a core timing network that uses interval tuning as a signal to represent temporal processing in a variety of behavioral contexts where time is explicitly quantified.

Robust timing and motor patterns by taming chaos in recurrent neural networks

The brain's ability to tell time and produce complex spatiotemporal motor patterns is critical for anticipating the next ring of a telephone or playing a musical instrument. One class of models proposes that these abilities emerge from dynamically changing patterns of neural activity generated in recurrent neural networks. However, the relevant dynamic regimes of recurrent networks are highly sensitive to noise; that is, chaotic. We developed a firing rate model that tells time on the order of seconds and generates complex spatiotemporal patterns in the presence of high levels of noise. This is achieved through the tuning of the recurrent connections. The network operates in a dynamic regime that exhibits coexisting chaotic and locally stable trajectories. These stable patterns function as 'dynamic attractors' and provide a feature that is characteristic of biological systems: the ability to 'return' to the pattern being generated in the face of perturbations.

The inner sense of time: how the brain creates a representation of duration

A large number of competing models exist for how the brain creates a representation of time. However, several human and animal studies point to 'climbing neural activation' as a potential neural mechanism for the representation of duration. Neurophysiological recordings in animals have revealed how climbing neural activation that peaks at the end of a timed interval underlies the processing of duration, and, in humans, climbing neural activity in the insular cortex, which is associated with feeling states of the body and emotions, may be related to the cumulative representation of time.

Spontaneous synchronization of arm motion between Japanese macaques

Humans show spontaneous synchronization of movements during social interactions; this coordination has been shown to facilitate smooth communication. Although human studies exploring spontaneous synchronization are increasing in number, little is known about this phenomenon in other species. In this study, we examined spontaneous behavioural synchronization between monkeys in a laboratory setting. Synchronization was quantified by changes in button-pressing behaviour while pairs of monkeys were facing one another. Synchronization between the monkeys was duly observed and it was participant-partner dependent. Further tests confirmed that the speed of button pressing changed to harmonic or sub-harmonic levels in relation to the partner's speed. In addition, the visual information from the partner induced a higher degree of synchronization than auditory information. This study establishes advanced tasks for testing social coordination in monkeys, and illustrates ways in which monkeys coordinate their actions to establish synchronization.

Rhesus monkeys (Macaca mulatta) detect rhythmic groups in music, but not the beat

It was recently shown that rhythmic entrainment, long considered a human-specific mechanism, can be demonstrated in a selected group of bird species, and, somewhat surprisingly, not in more closely related species such as nonhuman primates. This observation supports the vocal learning hypothesis that suggests rhythmic entrainment to be a by-product of the vocal learning mechanisms that are shared by several bird and mammal species, including humans, but that are only weakly developed, or missing entirely, in nonhuman primates. To test this hypothesis we measured auditory event-related potentials (ERPs) in two rhesus monkeys (Macaca mulatta), probing a well-documented component in humans, the mismatch negativity (MMN) to study rhythmic expectation. We demonstrate for the first time in rhesus monkeys that, in response to infrequent deviants in pitch that were presented in a continuous sound stream using an oddball paradigm, a comparable ERP component can be detected with negative deflections in early latencies (Experiment 1). Subsequently we tested whether rhesus monkeys can detect gaps (omissions at random positions in the sound stream; Experiment 2) and, using more complex stimuli, also the beat (omissions at the first position of a musical unit, i.e. the ‘downbeat’; Experiment 3). In contrast to what has been shown in human adults and newborns (using identical stimuli and experimental paradigm), the results suggest that rhesus monkeys are not able to detect the beat in music. These findings are in support of the hypothesis that beat induction (the cognitive mechanism that supports the perception of a regular pulse from a varying rhythm) is species-specific and absent in nonhuman primates. In addition, the findings support the auditory timing dissociation hypothesis, with rhesus monkeys being sensitive to rhythmic grouping (detecting the start of a rhythmic group), but not to the induced beat (detecting a regularity from a varying rhythm).

Trial time warping to discriminate stimulus-related from movement-related neural activity

In tasks where different sensory, cognitive, and motor events are mixed in a sequence it is difficult to determine whether neural activity is related to any behavioral parameter. Here, we consider the case in which two alternative trial-alignment schemes correspond to two different neural representations, one stimulus-related and the other movement-related, using both simulations of neural activity and real recordings in the medial premotor areas during a multiple-interval tapping task called synchronization-continuation task (SCT). To discover whether neural responses are better aligned to sensory or motor events we introduce a family of trial-alignment time-warping functions indexed by a single parameter such that when the parameter takes the value 0 the trials are aligned to the stimulus and when the parameter takes the value 1 they are aligned to the movement. We then characterize neurons by the best-fitting alignment scheme (in the sense of maximum likelihood) under the assumption that the correct alignment would produce homogeneous trials without excess trial-to-trial variation. We use Bayes factors to determine the evidence in favor of sensory or motor neural alignments. The simulations revealed that the variability in neural responses and sequential motor outputs are key parameters to obtain appropriate warping results. In addition, the analysis on the activity of 500 neurons in the medial premotor areas of monkeys executing the SCT showed that most of the neural responses (54.2%) were aligned to the tapping movements instead of the stimuli used to drive the temporal behavior.

A unified model of time perception accounts for duration-based and beat-based timing mechanisms

Accurate timing is an integral aspect of sensory and motor processes such as the perception of speech and music and the execution of skilled movement. Neuropsychological studies of time perception in patient groups and functional neuroimaging studies of timing in normal participants suggest common neural substrates for perceptual and motor timing. A timing system is implicated in core regions of the motor network such as the cerebellum, inferior olive, basal ganglia, pre-supplementary, and supplementary motor area, pre-motor cortex as well as higher-level areas such as the prefrontal cortex. In this article, we assess how distinct parts of the timing system subserve different aspects of perceptual timing. We previously established brain bases for absolute, duration-based timing and relative, beat-based timing in the olivocerebellar and striato-thalamo-cortical circuits respectively (Teki et al., 2011). However, neurophysiological and neuroanatomical studies provide a basis to suggest that timing functions of these circuits may not be independent. Here, we propose a unified model of time perception based on coordinated activity in the core striatal and olivocerebellar networks that are interconnected with each other and the cerebral cortex through multiple synaptic pathways. Timing in this unified model is proposed to involve serial beat-based striatal activation followed by absolute olivocerebellar timing mechanisms.