Physiologically driven avian vocal synthesizer

Synthesizing avian dreams

During sleep, sporadically, it is possible to find neural patterns of activity in areas of the avian brain that are activated during the generation of the song. It has recently been found that in the vocal muscles of a sleeping bird, it is possible to detect activity patterns during these silent replays. In this work, we employ a dynamical systems model for song production in suboscine birds in order to translate the vocal muscles activity during sleep into synthetic songs. Besides allowing us to translate muscle activity into behavior, we argue that this approach poses the biomechanics as a unique window into the avian brain, with biophysical models as its probe.

Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions

Recently developed methods in computational neuroethology have enabled increasingly detailed and comprehensive quantification of animal movements and behavioral kinematics. Vocal communication behavior is well poised for application of similar large-scale quantification methods in the service of physiological and ethological studies. This review describes emerging techniques that can be applied to acoustic and vocal communication signals with the goal of enabling study beyond a small number of model species. We review a range of modern computational methods for bioacoustics, signal processing, and brain-behavior mapping. Along with a discussion of recent advances and techniques, we include challenges and broader goals in establishing a framework for the computational neuroethology of vocal communication.

Synthetic Birdsongs as a Tool to Induce, and Iisten to, Replay Activity in Sleeping Birds

Birdsong is a complex vocal behavior, which emerges out of the interaction between a nervous system and a highly nonlinear vocal device, the syrinx. In this work we discuss how low dimensional dynamical systems, interpretable in terms of the biomechanics involved, are capable of synthesizing realistic songs. We review the experimental and conceptual steps that lead to the formulation of low dimensional dynamical systems for the song system and describe the tests that quantify their success. In particular, we show how to evaluate computational models by comparing the responses of highly selective neurons to the bird's own song and to synthetic copies generated mathematically. Beyond testing the hypothesis behind the model's construction, these low dimensional models allow designing precise stimuli in order to explore the sensorimotor integration of acoustic signals.

Automatic reconstruction of physiological gestures used in a model of birdsong production

Highly coordinated learned behaviors are key to understanding neural processes integrating the body and the environment. Birdsong production is a widely studied example of such behavior in which numerous thoracic muscles control respiratory inspiration and expiration: the muscles of the syrinx control syringeal membrane tension, while upper vocal tract morphology controls resonances that modulate the vocal system output. All these muscles have to be coordinated in precise sequences to generate the elaborate vocalizations that characterize an individual's song. Previously we used a low-dimensional description of the biomechanics of birdsong production to investigate the associated neural codes, an approach that complements traditional spectrographic analysis. The prior study used algorithmic yet manual procedures to model singing behavior. In the present work, we present an automatic procedure to extract low-dimensional motor gestures that could predict vocal behavior. We recorded zebra finch songs and generated synthetic copies automatically, using a biomechanical model for the vocal apparatus and vocal tract. This dynamical model described song as a sequence of physiological parameters the birds control during singing. To validate this procedure, we recorded electrophysiological activity of the telencephalic nucleus HVC. HVC neurons were highly selective to the auditory presentation of the bird's own song (BOS) and gave similar selective responses to the automatically generated synthetic model of song (AUTO). Our results demonstrate meaningful dimensionality reduction in terms of physiological parameters that individual birds could actually control. Furthermore, this methodology can be extended to other vocal systems to study fine motor control.

Modeling vocalization with ECoG cortical activity recorded during vocal production in the macaque monkey

Vocal production is an example of controlled motor behavior with high temporal precision. Previous studies have decoded auditory evoked cortical activity while monkeys listened to vocalization sounds. On the other hand, there have been few attempts at decoding motor cortical activity during vocal production. Here we recorded cortical activity during vocal production in the macaque with a chronically implanted electrocorticographic (ECoG) electrode array. The array detected robust activity in motor cortex during vocal production. We used a nonlinear dynamical model of the vocal organ to reduce the dimensionality of `Coo' calls produced by the monkey. We then used linear regression to evaluate the information in motor cortical activity for this reduced representation of calls. This simple linear model accounted for circa 65% of the variance in the reduced sound representations, supporting the feasibility of using the dynamical model of the vocal organ for decoding motor cortical activity during vocal production.

Integrative physiology of fundamental frequency control in birds

One major feature of the remarkable vocal repertoires of birds is the range of fundamental frequencies across species, but also within individual species. This review discusses four variables that determine the oscillation frequency of the vibrating structures within a bird's syrinx. These are (1) viscoelastic properties of the oscillating tissue, (2) air sac pressure, (3) neuromuscular control of movements and (4) source-filter interactions. Our current understanding of morphology, biomechanics and neural control suggests that a complex interplay of these parameters can lead to multiple combinations for generating a particular fundamental frequency. An increase in the complexity of syringeal morphology from non-passeriform birds to oscines also led to a different interplay for regulating oscillation frequency by enabling control of tension that is partially independent of regulation of airflow. In addition to reviewing the available data for all different contributing variables, we point out open questions and possible approaches.

The acoustic effect of vocal tract adjustments in zebra finches

Vocal production in songbirds requires the control of the respiratory system, the syrinx as sound source and the vocal tract as acoustic filter. Vocal tract movements consist of beak, tongue and hyoid movements, which change the volume of the oropharyngeal-esophageal cavity (OEC), glottal movements and tracheal length changes. The respective contributions of each movement to filter properties are not completely understood, but the effects of this filtering are thought to be very important for acoustic communication in birds. One of the most striking movements of the upper vocal tract during vocal behavior in songbirds involves the OEC. This study measured the acoustic effect of OEC adjustments in zebra finches by comparing resonance acoustics between an utterance with OEC expansion (calls) and a similar utterance without OEC expansion (respiratory sounds induced by a bilateral syringeal denervation). X-ray cineradiography confirmed the presence of an OEC motor pattern during song and call production, and a custom-built Hall-effect collar system confirmed that OEC expansion movements were not present during respiratory sounds. The spectral emphasis during zebra finch call production ranging between 2.5 and 5 kHz was not present during respiratory sounds, indicating strongly that it can be attributed to the OEC expansion.

Prosthetic avian vocal organ controlled by a freely behaving bird based on a low dimensional model of the biomechanical periphery

Because of the parallels found with human language production and acquisition, birdsong is an ideal animal model to study general mechanisms underlying complex, learned motor behavior. The rich and diverse vocalizations of songbirds emerge as a result of the interaction between a pattern generator in the brain and a highly nontrivial nonlinear periphery. Much of the complexity of this vocal behavior has been understood by studying the physics of the avian vocal organ, particularly the syrinx. A mathematical model describing the complex periphery as a nonlinear dynamical system leads to the conclusion that nontrivial behavior emerges even when the organ is commanded by simple motor instructions: smooth paths in a low dimensional parameter space. An analysis of the model provides insight into which parameters are responsible for generating a rich variety of diverse vocalizations, and what the physiological meaning of these parameters is. By recording the physiological motor instructions elicited by a spontaneously singing muted bird and computing the model on a Digital Signal Processor in real-time, we produce realistic synthetic vocalizations that replace the bird's own auditory feedback. In this way, we build a bio-prosthetic avian vocal organ driven by a freely behaving bird via its physiologically coded motor commands. Since it is based on a low-dimensional nonlinear mathematical model of the peripheral effector, the emulation of the motor behavior requires light computation, in such a way that our bio-prosthetic device can be implemented on a portable platform.

Brain Machine Interfaces (BMIs) decode motor instructions from neuro-physiological recordings and feed them to bio-mimetic effectors. Many applications achieve high accuracy on a limited number of tasks by applying statistical methods to these data to extract features corresponding to certain motor instructions. We built a bio-prosthetic avian vocal organ. The device is based on a low-dimensional mathematical model that accounts for the dynamics of the bird's vocal organ and robustly relates smooth paths in a physiologically meaningful parameter space to complex sequences of vocalizations. The two physiological motor gestures (sub-syringeal pressure and ventral syringeal muscular activity), are reconstructed from the bird's song, and the model is implemented on a portable Digital Signal Processor to produce synthetic birdsong when driven by a freely behaving bird via the sub-syringeal pressure gesture. This exemplifies the plausibility of a type of synthetic interfacing between the brain and a complex behavior. In this type of devices, the understanding of the bio-mechanics of the periphery is key to identifying a low dimensional physiological signal coding the motor instructions, therefore enabling real-time implementation at a low computational cost.

Reconstruction of physiological instructions from Zebra finch song

We reconstruct the physiological parameters that control an avian vocal organ during birdsong production using recorded song. The procedure involves fitting the time dependent parameters of an avian vocal organ model. Computationally, the model is implemented as a dynamical system ruling the behavior of the oscillating labia that modulate the air flow during sound production, together with the equations describing the dynamics of pressure fluctuations in the vocal tract. We tested our procedure for Zebra finch song with, simultaneously recorded physiological parameters: air sac pressure and the electromyographic activity of the left and right ventral syringeal muscles. A comparison of the reconstructed instructions with measured physiological parameters during song shows a high degree of correlation. Integrating the model with reconstructed parameters leads to the synthesis of highly realistic songs.

Acoustic signatures of sound source-tract coupling

Birdsong is a complex behavior, which results from the interaction between a nervous system and a biomechanical peripheral device. While much has been learned about how complex sounds are generated in the vocal organ, little has been learned about the signature on the vocalizations of the nonlinear effects introduced by the acoustic interactions between a sound source and the vocal tract. The variety of morphologies among bird species makes birdsong a most suitable model to study phenomena associated to the production of complex vocalizations. Inspired by the sound production mechanisms of songbirds, in this work we study a mathematical model of a vocal organ, in which a simple sound source interacts with a tract, leading to a delay differential equation. We explore the system numerically, and by taking it to the weakly nonlinear limit, we are able to examine its periodic solutions analytically. By these means we are able to explore the dynamics of oscillatory solutions of a sound source-tract coupled system, which are qualitatively different from those of a sound source-filter model of a vocal organ. Nonlinear features of the solutions are proposed as the underlying mechanisms of observed phenomena in birdsong, such as unilaterally produced "frequency jumps," enhancement of resonances, and the shift of the fundamental frequency observed in heliox experiments.