Facial muscle coordination in monkeys during rhythmic facial expressions and ingestive movements

Facial gestures are enacted via a cortical hierarchy of dynamic and stable codes

Successful communication requires the generation and perception of a shared set of signals. Facial gestures are one fundamental set of communicative behaviors in primates, generated through the dynamic arrangement of dozens of fine muscles. While much progress has been made uncovering the neural mechanisms of face perception, little is known about those controlling facial gesture production. Commensurate with the importance of facial gestures in daily social life, anatomical work has shown that facial muscles are under direct control from multiple cortical regions, including primary and premotor in lateral frontal cortex, and cingulate in medial frontal cortex. Furthermore, neuropsychological evidence from focal lesion patients has suggested that lateral cortex controls voluntary movements, and medial emotional expressions. Here we show that lateral and medial cortical face motor regions encode both types of gestures. They do so through unique temporal activity patterns, distinguishable well-prior to movement onset. During gesture production, cortical regions encoded facial kinematics in a context-dependent manner. Our results show how cortical regions projecting in parallel downstream, but each situated at a different level of a posterior-anterior hierarchy form a continuum of gesture coding from dynamic to temporally stable, in order to produce context-related, coherent motor outputs during social communication.

Beyond neonatal imitation: Aerodigestive stereotypies, speech development, and social interaction in the extended perinatal period

In our target article, we argued that the positive results of neonatal imitation are likely to be by-products of normal aerodigestive development. Our hypothesis elicited various responses on the role of social interaction in infancy, the methodological issues about imitation experiments, and the relation between the aerodigestive theory and the development of speech. Here we respond to the commentaries.

From hand to mouth: monkeys require greater effort in motor preparation for voluntary control of vocalization than for manual actions

Voluntary control of vocal production is an essential component of the language faculty, which is thought to distinguish humans from other primates. Recent experiments have begun to reveal the capability of non-human primates to perform vocal control; however, the mechanisms underlying this ability remain unclear. Here, we revealed that Japanese macaque monkeys can learn to vocalize voluntarily through a different mechanism than that used for manual actions. The monkeys rapidly learned to touch a computer monitor when a visual stimulus was presented and showed a capacity for flexible adaptation, such that they reacted when the visual stimulus was shown at an unexpected time. By contrast, successful vocal training required additional time, and the monkeys exhibited difficulty with vocal execution when the visual stimulus appeared earlier than expected; this occurred regardless of extensive training. Thus, motor preparation before execution of an action may be a key factor in distinguishing vocalization from manual actions in monkeys; they do not exhibit a similar ability to perform motor preparation in the vocal domains. By performing direct comparisons, this study provides novel evidence regarding differences in motor control abilities between vocal and manual actions. Our findings support the suggestion that the functional expansion from hand to mouth might be a critical evolutionary event for the acquisition of voluntary control of vocalizations.

Functional Networks for Social Communication in the Macaque Monkey

All primates communicate. To dissect the neural circuits of social communication, we used fMRI to map non-human primate brain regions for social perception, second-person (interactive) social cognition, and orofacial movement generation. Face perception, second-person cognition, and face motor networks were largely non-overlapping and acted as distinct functional units rather than an integrated feedforward-processing pipeline. Whereas second-person context selectively engaged a region of medial prefrontal cortex, production of orofacial movements recruited distributed subcortical and cortical areas in medial and lateral frontal and insular cortex. These areas exhibited some specialization, but not dissociation, of function along the medio-lateral axis. Production of lipsmack movements recruited areas including putative homologs of Broca's area. These findings provide a new view of the neural architecture for social communication and suggest expressive orofacial movements generated by lateral premotor cortex as a putative evolutionary precursor to human speech.

Neuronal Encoding of Self and Others' Head Rotation in the Macaque Dorsal Prefrontal Cortex

Following gaze is a crucial skill, in primates, for understanding where and at what others are looking, and often requires head rotation. The neural basis underlying head rotation are deemed to overlap with the parieto-frontal attention/gaze-shift network. Here, we show that a set of neurons in monkey’s Brodmann area 9/46dr (BA 9/46dr), which is involved in orienting processes and joint attention, becomes active during self head rotation and that the activity of these neurons cannot be accounted for by saccade-related activity (head-rotation neurons). Another set of BA 9/46dr neurons encodes head rotation performed by an observed agent facing the monkey (visually triggered neurons). Among these latter neurons, almost half exhibit the intriguing property of encoding both execution and observation of head rotation (mirror-like neurons). Finally, by means of neuronal tracing techniques, we showed that BA 9/46dr takes part into two distinct networks: a dorso/mesial network, playing a role in spatial head/gaze orientation, and a ventrolateral network, likely involved in processing social stimuli and mirroring others’ head. The overall results of this study provide a new, comprehensive picture of the role of BA 9/46dr in encoding self and others’ head rotation, likely playing a role in head-following behaviors.

Reading What the Mind Thinks From How the Eye Sees

Human eyes convey a remarkable variety of complex social and emotional information. However, it is unknown which physical eye features convey mental states and how that came about. In the current experiments, we tested the hypothesis that the receiver's perception of mental states is grounded in expressive eye appearance that serves an optical function for the sender. Specifically, opposing features of eye widening versus eye narrowing that regulate sensitivity versus discrimination not only conveyed their associated basic emotions (e.g., fear vs. disgust, respectively) but also conveyed opposing clusters of complex mental states that communicate sensitivity versus discrimination (e.g., awe vs. suspicion). This sensitivity-discrimination dimension accounted for the majority of variance in perceived mental states (61.7%). Further, these eye features remained diagnostic of these complex mental states even in the context of competing information from the lower face. These results demonstrate that how humans read complex mental states may be derived from a basic optical principle of how people see.

A multimodal spectral approach to characterize rhythm in natural speech

Human utterances demonstrate temporal patterning, also referred to as rhythm. While simple oromotor behaviors (e.g., chewing) feature a salient periodical structure, conversational speech displays a time-varying quasi-rhythmic pattern. Quantification of periodicity in speech is challenging. Unimodal spectral approaches have highlighted rhythmic aspects of speech. However, speech is a complex multimodal phenomenon that arises from the interplay of articulatory, respiratory, and vocal systems. The present study addressed the question of whether a multimodal spectral approach, in the form of coherence analysis between electromyographic (EMG) and acoustic signals, would allow one to characterize rhythm in natural speech more efficiently than a unimodal analysis. The main experimental task consisted of speech production at three speaking rates; a simple oromotor task served as control. The EMG-acoustic coherence emerged as a sensitive means of tracking speech rhythm, whereas spectral analysis of either EMG or acoustic amplitude envelope alone was less informative. Coherence metrics seem to distinguish and highlight rhythmic structure in natural speech.

Frequency-specific hippocampal-prefrontal interactions during associative learning

Much of our knowledge of the world depends on learning associations (for example, face-name), for which the hippocampus (HPC) and prefrontal cortex (PFC) are critical. HPC-PFC interactions have rarely been studied in monkeys, whose cognitive and mnemonic abilities are akin to those of humans. We found functional differences and frequency-specific interactions between HPC and PFC of monkeys learning object pair associations, an animal model of human explicit memory. PFC spiking activity reflected learning in parallel with behavioral performance, whereas HPC neurons reflected feedback about whether trial-and-error guesses were correct or incorrect. Theta-band HPC-PFC synchrony was stronger after errors, was driven primarily by PFC to HPC directional influences and decreased with learning. In contrast, alpha/beta-band synchrony was stronger after correct trials, was driven more by HPC and increased with learning. Rapid object associative learning may occur in PFC, whereas HPC may guide neocortical plasticity by signaling success or failure via oscillatory synchrony in different frequency bands.

The evolution of speech: vision, rhythm, cooperation

A full account of human speech evolution must consider its multisensory, rhythmic, and cooperative characteristics. Humans, apes, and monkeys recognize the correspondence between vocalizations and their associated facial postures, and gain behavioral benefits from them. Some monkey vocalizations even have a speech-like acoustic rhythmicity but lack the concomitant rhythmic facial motion that speech exhibits. We review data showing that rhythmic facial expressions such as lip-smacking may have been linked to vocal output to produce an ancestral form of rhythmic audiovisual speech. Finally, we argue that human vocal cooperation (turn-taking) may have arisen through a combination of volubility and prosociality, and provide comparative evidence from one species to support this hypothesis.

Movement coordination during conversation

Behavioral coordination and synchrony contribute to a common biological mechanism that maintains communication, cooperation and bonding within many social species, such as primates and birds. Similarly, human language and social systems may also be attuned to coordination to facilitate communication and the formation of relationships. Gross similarities in movement patterns and convergence in the acoustic properties of speech have already been demonstrated between interacting individuals. In the present studies, we investigated how coordinated movements contribute to observers’ perception of affiliation (friends vs. strangers) between two conversing individuals. We used novel computational methods to quantify motor coordination and demonstrated that individuals familiar with each other coordinated their movements more frequently. Observers used coordination to judge affiliation between conversing pairs but only when the perceptual stimuli were restricted to head and face regions. These results suggest that observed movement coordination in humans might contribute to perceptual decisions based on availability of information to perceivers.

Facial expressions and the evolution of the speech rhythm

In primates, different vocalizations are produced, at least in part, by making different facial expressions. Not surprisingly, humans, apes, and monkeys all recognize the correspondence between vocalizations and the facial postures associated with them. However, one major dissimilarity between monkey vocalizations and human speech is that, in the latter, the acoustic output and associated movements of the mouth are both rhythmic (in the 3- to 8-Hz range) and tightly correlated, whereas monkey vocalizations have a similar acoustic rhythmicity but lack the concommitant rhythmic facial motion. This raises the question of how we evolved from a presumptive ancestral acoustic-only vocal rhythm to the one that is audiovisual with improved perceptual sensitivity. According to one hypothesis, this bisensory speech rhythm evolved through the rhythmic facial expressions of ancestral primates. If this hypothesis has any validity, we expect that the extant nonhuman primates produce at least some facial expressions with a speech-like rhythm in the 3- to 8-Hz frequency range. Lip smacking, an affiliative signal observed in many genera of primates, satisfies this criterion. We review a series of studies using developmental, x-ray cineradiographic, EMG, and perceptual approaches with macaque monkeys producing lip smacks to further investigate this hypothesis. We then explore its putative neural basis and remark on important differences between lip smacking and speech production. Overall, the data support the hypothesis that lip smacking may have been an ancestral expression that was linked to vocal output to produce the original rhythmic audiovisual speech-like utterances in the human lineage.

Multisensory vocal communication in primates and the evolution of rhythmic speech

The integration of the visual and auditory modalities during human speech perception is the default mode of speech processing. That is, visual speech perception is not a capacity that is "piggybacked" on to auditory-only speech perception. Visual information from the mouth and other parts of the face is used by all perceivers to enhance auditory speech. This integration is ubiquitous and automatic and is similar across all individuals across all cultures. The two modalities seem to be integrated even at the earliest stages of human cognitive development. If multisensory speech is the default mode of perception, then this should be reflected in the evolution of vocal communication. The purpose of this review is to describe the data that reveal that human speech is not uniquely multisensory. In fact, the default mode of communication is multisensory in nonhuman primates as well but perhaps emerging with a different developmental trajectory. Speech production, however, exhibits a unique bimodal rhythmic structure in that both the acoustic output and the movements of the mouth are rhythmic and tightly correlated. This structure is absent in most monkey vocalizations. One hypothesis is that the bimodal speech rhythm may have evolved through the rhythmic facial expressions of ancestral primates, as indicated by mounting comparative evidence focusing on the lip-smacking gesture.

Self-monitoring of social facial expressions in the primate amygdala and cingulate cortex

Keeping track of self-executed facial expressions is essential for the ability to correctly interpret and reciprocate social expressions. However, little is known about neural mechanisms that participate in self-monitoring of facial expression. We designed a natural paradigm for social interactions where a monkey is seated in front of a peer monkey that is concealed by an opaque liquid crystal display shutter positioned between them. Opening the shutter for short durations allowed the monkeys to see each other and encouraged facial communication. To explore neural mechanisms that participate in self-monitoring of facial expression, we simultaneously recorded the elicited natural facial interactions and the neural activity of single neurons in the amygdala and the dorsal anterior cingulate cortex (dACC), two regions that are implicated with decoding of others' gestures. Neural activity in both regions was temporally locked to distinctive facial gestures and close inspection of time lags revealed activity that either preceded (production) or lagged (monitor) initiation of facial expressions. This result indicates that single neurons in the dACC and the amygdala hold information about self-executed facial expressions and demonstrates an intimate overlap between the neural networks that participate in decoding and production of socially informative facial information.