1
|
Chen S, Thielk M, Gentner TQ. Auditory Feature-based Perceptual Distance. bioRxiv 2024:2024.02.28.582631. [PMID: 38464215 PMCID: PMC10925319 DOI: 10.1101/2024.02.28.582631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
Studies comparing acoustic signals often rely on pixel-wise differences between spectrograms, as in for example mean squared error (MSE). Pixel-wise errors are not representative of perceptual sensitivity, however, and such measures can be highly sensitive to small local signal changes that may be imperceptible. In computer vision, high-level visual features extracted with convolutional neural networks (CNN) can be used to calculate the fidelity of computer-generated images. Here, we propose the auditory perceptual distance (APD) metric based on acoustic features extracted with an unsupervised CNN and validated by perceptual behavior. Using complex vocal signals from songbirds, we trained a Siamese CNN on a self-supervised task using spectrograms rescaled to match the auditory frequency sensitivity of European starlings, Sturnus vulgaris. We define APD for any pair of sounds as the cosine distance between corresponding feature vectors extracted by the trained CNN. We show that APD is more robust to temporal and spectral translation than MSE, and captures the sigmoidal shape of typical behavioral psychometric functions over complex acoustic spaces. When fine-tuned using starlings' behavioral judgments of naturalistic song syllables, the APD model yields even more accurate predictions of perceptual sensitivity, discrimination, and categorization on novel complex (high-dimensional) acoustic dimensions, including diverging decisions for identical stimuli following different training conditions. Thus, the APD model outperforms MSE in robustness and perceptual accuracy, and offers tunability to match experience-dependent perceptual biases.
Collapse
Affiliation(s)
- Shukai Chen
- Department of Bioengineering, University of California, San Diego, La Jolla, CA, 92093
- Department of Psychology, University of California, San Diego, La Jolla, CA 92093
| | - Marvin Thielk
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, 92093
| | - Timothy Q. Gentner
- Department of Psychology, University of California, San Diego, La Jolla, CA 92093
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, 92093
- Neurobiology Section, Division of Biological Sciences, University of California, San Diego, La Jolla, CA, 92093
| |
Collapse
|
2
|
Mai A, Riès S, Ben-Haim S, Shih JJ, Gentner TQ. Acoustic and language-specific sources for phonemic abstraction from speech. Nat Commun 2024; 15:677. [PMID: 38263364 PMCID: PMC10805762 DOI: 10.1038/s41467-024-44844-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Accepted: 01/03/2024] [Indexed: 01/25/2024] Open
Abstract
Spoken language comprehension requires abstraction of linguistic information from speech, but the interaction between auditory and linguistic processing of speech remains poorly understood. Here, we investigate the nature of this abstraction using neural responses recorded intracranially while participants listened to conversational English speech. Capitalizing on multiple, language-specific patterns where phonological and acoustic information diverge, we demonstrate the causal efficacy of the phoneme as a unit of analysis and dissociate the unique contributions of phonemic and spectrographic information to neural responses. Quantitive higher-order response models also reveal that unique contributions of phonological information are carried in the covariance structure of the stimulus-response relationship. This suggests that linguistic abstraction is shaped by neurobiological mechanisms that involve integration across multiple spectro-temporal features and prior phonological information. These results link speech acoustics to phonology and morphosyntax, substantiating predictions about abstractness in linguistic theory and providing evidence for the acoustic features that support that abstraction.
Collapse
Affiliation(s)
- Anna Mai
- University of California, San Diego, Linguistics, 9500 Gilman Dr., La Jolla, CA, 92093, USA.
| | - Stephanie Riès
- San Diego State University, School of Speech, Language, and Hearing Sciences, 5500 Campanile Drive, San Diego, CA, 92182, USA
- San Diego State University, Center for Clinical and Cognitive Sciences, 5500 Campanile Drive, San Diego, CA, 92182, USA
| | - Sharona Ben-Haim
- University of California, San Diego, Neurological Surgery, 9500 Gilman Dr., La Jolla, CA, 92093, USA
| | - Jerry J Shih
- University of California, San Diego, Neurosciences, 9500 Gilman Dr., La Jolla, CA, 92093, USA
| | - Timothy Q Gentner
- University of California, San Diego, Psychology, 9500 Gilman Dr., La Jolla, CA, 92093, USA
- University of California, San Diego, Neurobiology, 9500 Gilman Dr., La Jolla, CA, 92093, USA
- University of California, San Diego, Kavli Institute for Brain and Mind, 9500 Gilman Dr., La Jolla, CA, 92093, USA
| |
Collapse
|
3
|
Christman KA, Finneran JJ, Mulsow J, Houser DS, Gentner TQ. The effects of range and echo-phase on range resolution in bottlenose dolphins (Tursiops truncatus) performing a successive comparison taska). J Acoust Soc Am 2024; 155:274-283. [PMID: 38215217 DOI: 10.1121/10.0024342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 12/14/2023] [Indexed: 01/14/2024]
Abstract
Echolocating bats and dolphins use biosonar to determine target range, but differences in range discrimination thresholds have been reported for the two species. Whether these differences represent a true difference in their sensory system capability is unknown. Here, the dolphin's range discrimination threshold as a function of absolute range and echo-phase was investigated. Using phantom echoes, the dolphins were trained to echo-inspect two simulated targets and indicate the closer target by pressing a paddle. One target was presented at a time, requiring the dolphin to hold the initial range in memory as they compared it to the second target. Range was simulated by manipulating echo-delay while the received echo levels, relative to the dolphins' clicks, were held constant. Range discrimination thresholds were determined at seven different ranges from 1.75 to 20 m. In contrast to bats, range discrimination thresholds increased from 4 to 75 cm, across the entire ranges tested. To investigate the acoustic features used more directly, discrimination thresholds were determined when the echo was given a random phase shift (±180°). Results for the constant-phase versus the random-phase echo were quantitatively similar, suggesting that dolphins used the envelope of the echo waveform to determine the difference in range.
Collapse
Affiliation(s)
- Katie A Christman
- Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
- Department of Biologic and Bioacoustic Research, National Marine Mammal Foundation, 3131, 2240 Shelter Island Drive, San Diego, California 92106, USA
| | - James J Finneran
- United States Navy Marine Mammal Program, Naval Information Warfare Center Pacific Code 56710, 53560 Hull Street, San Diego, California 92152, USA
| | - Jason Mulsow
- Department of Biologic and Bioacoustic Research, National Marine Mammal Foundation, 3131, 2240 Shelter Island Drive, San Diego, California 92106, USA
| | - Dorian S Houser
- Department of Biologic and Bioacoustic Research, National Marine Mammal Foundation, 3131, 2240 Shelter Island Drive, San Diego, California 92106, USA
| | - Timothy Q Gentner
- Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
- Department of Neurobiology, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| |
Collapse
|
4
|
Xing J, Sainburg T, Taylor H, Gentner TQ. Syntactic modulation of rhythm in Australian pied butcherbird song. R Soc Open Sci 2022; 9:220704. [PMID: 36177196 PMCID: PMC9515642 DOI: 10.1098/rsos.220704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 09/05/2022] [Indexed: 05/04/2023]
Abstract
The acoustic structure of birdsong is spectrally and temporally complex. Temporal complexity is often investigated in a syntactic framework focusing on the statistical features of symbolic song sequences. Alternatively, temporal patterns can be investigated in a rhythmic framework that focuses on the relative timing between song elements. Here, we investigate the merits of combining both frameworks by integrating syntactic and rhythmic analyses of Australian pied butcherbird (Cracticus nigrogularis) songs, which exhibit organized syntax and diverse rhythms. We show that rhythms of the pied butcherbird song bouts in our sample are categorically organized and predictable by the song's first-order sequential syntax. These song rhythms remain categorically distributed and strongly associated with the first-order sequential syntax even after controlling for variance in note length, suggesting that the silent intervals between notes induce a rhythmic structure on note sequences. We discuss the implication of syntactic-rhythmic relations as a relevant feature of song complexity with respect to signals such as human speech and music, and advocate for a broader conception of song complexity that takes into account syntax, rhythm, and their interaction with other acoustic and perceptual features.
Collapse
Affiliation(s)
- Jeffrey Xing
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
| | - Tim Sainburg
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
| | - Hollis Taylor
- Sydney Conservatorium of Music, University of Sydney, Sydney, New South Wales, Australia
| | - Timothy Q. Gentner
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
- Neurobiology Section, Division of Biological Sciences, University of California San Diego, La Jolla, CA, USA
- Kavli Institute for Brain and Mind, University of California San Diego, La Jolla, CA, USA
| |
Collapse
|
5
|
Xing J, Sainburg T, Taylor H, Gentner TQ. Syntactic modulation of rhythm in Australian pied butcherbird song. R Soc Open Sci 2022; 9:220704. [PMID: 36177196 DOI: 10.6084/m9.figshare.c.6197494] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 09/05/2022] [Indexed: 05/21/2023]
Abstract
The acoustic structure of birdsong is spectrally and temporally complex. Temporal complexity is often investigated in a syntactic framework focusing on the statistical features of symbolic song sequences. Alternatively, temporal patterns can be investigated in a rhythmic framework that focuses on the relative timing between song elements. Here, we investigate the merits of combining both frameworks by integrating syntactic and rhythmic analyses of Australian pied butcherbird (Cracticus nigrogularis) songs, which exhibit organized syntax and diverse rhythms. We show that rhythms of the pied butcherbird song bouts in our sample are categorically organized and predictable by the song's first-order sequential syntax. These song rhythms remain categorically distributed and strongly associated with the first-order sequential syntax even after controlling for variance in note length, suggesting that the silent intervals between notes induce a rhythmic structure on note sequences. We discuss the implication of syntactic-rhythmic relations as a relevant feature of song complexity with respect to signals such as human speech and music, and advocate for a broader conception of song complexity that takes into account syntax, rhythm, and their interaction with other acoustic and perceptual features.
Collapse
Affiliation(s)
- Jeffrey Xing
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
| | - Tim Sainburg
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
| | - Hollis Taylor
- Sydney Conservatorium of Music, University of Sydney, Sydney, New South Wales, Australia
| | - Timothy Q Gentner
- Department of Psychology, University of California San Diego, La Jolla, CA, USA
- Neurobiology Section, Division of Biological Sciences, University of California San Diego, La Jolla, CA, USA
- Kavli Institute for Brain and Mind, University of California San Diego, La Jolla, CA, USA
| |
Collapse
|
6
|
Sainburg T, Mai A, Gentner TQ. Long-range sequential dependencies precede complex syntactic production in language acquisition. Proc Biol Sci 2022; 289:20212657. [PMID: 35259983 PMCID: PMC8905171 DOI: 10.1098/rspb.2021.2657] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 02/01/2022] [Indexed: 12/27/2022] Open
Abstract
To convey meaning, human language relies on hierarchically organized, long-range relationships spanning words, phrases, sentences and discourse. As the distances between elements (e.g. phonemes, characters, words) in human language sequences increase, the strength of the long-range relationships between those elements decays following a power law. This power-law relationship has been attributed variously to long-range sequential organization present in human language syntax, semantics and discourse structure. However, non-linguistic behaviours in numerous phylogenetically distant species, ranging from humpback whale song to fruit fly motility, also demonstrate similar long-range statistical dependencies. Therefore, we hypothesized that long-range statistical dependencies in human speech may occur independently of linguistic structure. To test this hypothesis, we measured long-range dependencies in several speech corpora from children (aged 6 months-12 years). We find that adult-like power-law statistical dependencies are present in human vocalizations at the earliest detectable ages, prior to the production of complex linguistic structure. These linguistic structures cannot, therefore, be the sole cause of long-range statistical dependencies in language.
Collapse
Affiliation(s)
- Tim Sainburg
- Department of Psychology, Kavli Institute for Brain and Mind, UC San Diego, La Jolla, CA 92093, USA
- Center for Academic Research & Training in Anthropogeny, Kavli Institute for Brain and Mind, UC San Diego, La Jolla, CA 92093, USA
| | - Anna Mai
- Department of Linguistics, Kavli Institute for Brain and Mind, UC San Diego, La Jolla, CA 92093, USA
| | - Timothy Q. Gentner
- Department of Psychology, Kavli Institute for Brain and Mind, UC San Diego, La Jolla, CA 92093, USA
- Neurosciences Graduate Program, Neurobiology Section, Kavli Institute for Brain and Mind, UC San Diego, La Jolla, CA 92093, USA
| |
Collapse
|
7
|
Sainburg T, Gentner TQ. Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions. Front Behav Neurosci 2021; 15:811737. [PMID: 34987365 PMCID: PMC8721140 DOI: 10.3389/fnbeh.2021.811737] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 11/29/2021] [Indexed: 11/23/2022] Open
Abstract
Recently developed methods in computational neuroethology have enabled increasingly detailed and comprehensive quantification of animal movements and behavioral kinematics. Vocal communication behavior is well poised for application of similar large-scale quantification methods in the service of physiological and ethological studies. This review describes emerging techniques that can be applied to acoustic and vocal communication signals with the goal of enabling study beyond a small number of model species. We review a range of modern computational methods for bioacoustics, signal processing, and brain-behavior mapping. Along with a discussion of recent advances and techniques, we include challenges and broader goals in establishing a framework for the computational neuroethology of vocal communication.
Collapse
Affiliation(s)
- Tim Sainburg
- Department of Psychology, University of California, San Diego, La Jolla, CA, United States
- Center for Academic Research & Training in Anthropogeny, University of California, San Diego, La Jolla, CA, United States
| | - Timothy Q. Gentner
- Department of Psychology, University of California, San Diego, La Jolla, CA, United States
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, United States
- Neurobiology Section, Division of Biological Sciences, University of California, San Diego, La Jolla, CA, United States
- Kavli Institute for Brain and Mind, University of California, San Diego, La Jolla, CA, United States
| |
Collapse
|
8
|
Sainburg T, McInnes L, Gentner TQ. Parametric UMAP Embeddings for Representation and Semisupervised Learning. Neural Comput 2021; 33:2881-2907. [PMID: 34474477 DOI: 10.1162/neco_a_01434] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Accepted: 06/01/2021] [Indexed: 11/04/2022]
Abstract
UMAP is a nonparametric graph-based dimensionality reduction algorithm using applied Riemannian geometry and algebraic topology to find low-dimensional embeddings of structured data. The UMAP algorithm consists of two steps: (1) computing a graphical representation of a data set (fuzzy simplicial complex) and (2) through stochastic gradient descent, optimizing a low-dimensional embedding of the graph. Here, we extend the second step of UMAP to a parametric optimization over neural network weights, learning a parametric relationship between data and embedding. We first demonstrate that parametric UMAP performs comparably to its nonparametric counterpart while conferring the benefit of a learned parametric mapping (e.g., fast online embeddings for new data). We then explore UMAP as a regularization, constraining the latent distribution of autoencoders, parametrically varying global structure preservation, and improving classifier accuracy for semisupervised learning by capturing structure in unlabeled data.1.
Collapse
Affiliation(s)
- Tim Sainburg
- University of California San Diego, La Jolla, CA 92093, U.S.A.
| | - Leland McInnes
- Tutte Institute for Mathematics and Computing, Ottawa, Ontario Canada
| | | |
Collapse
|
9
|
Arneodo EM, Chen S, Brown DE, Gilja V, Gentner TQ. Neurally driven synthesis of learned, complex vocalizations. Curr Biol 2021; 31:3419-3425.e5. [PMID: 34139192 DOI: 10.1016/j.cub.2021.05.035] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2019] [Revised: 04/03/2021] [Accepted: 05/18/2021] [Indexed: 12/29/2022]
Abstract
Brain machine interfaces (BMIs) hold promise to restore impaired motor function and serve as powerful tools to study learned motor skill. While limb-based motor prosthetic systems have leveraged nonhuman primates as an important animal model,1-4 speech prostheses lack a similar animal model and are more limited in terms of neural interface technology, brain coverage, and behavioral study design.5-7 Songbirds are an attractive model for learned complex vocal behavior. Birdsong shares a number of unique similarities with human speech,8-10 and its study has yielded general insight into multiple mechanisms and circuits behind learning, execution, and maintenance of vocal motor skill.11-18 In addition, the biomechanics of song production bear similarity to those of humans and some nonhuman primates.19-23 Here, we demonstrate a vocal synthesizer for birdsong, realized by mapping neural population activity recorded from electrode arrays implanted in the premotor nucleus HVC onto low-dimensional compressed representations of song, using simple computational methods that are implementable in real time. Using a generative biomechanical model of the vocal organ (syrinx) as the low-dimensional target for these mappings allows for the synthesis of vocalizations that match the bird's own song. These results provide proof of concept that high-dimensional, complex natural behaviors can be directly synthesized from ongoing neural activity. This may inspire similar approaches to prosthetics in other species by exploiting knowledge of the peripheral systems and the temporal structure of their output.
Collapse
Affiliation(s)
- Ezequiel M Arneodo
- Biocircuits Institute, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA; Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA; IFLP-CONICET, Departamento de Física, Universidad Nacional de La Plata, CC 67, La Plata 1900, Argentina
| | - Shukai Chen
- Department of Bioengineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA
| | - Daril E Brown
- Department of Electrical and Computer Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA
| | - Vikash Gilja
- Department of Electrical and Computer Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA
| | - Timothy Q Gentner
- Biocircuits Institute, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA; Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA; Kavli Institute for Brain and Mind, 9500 Gilman Drive, La Jolla, CA 92093, USA; Neurobiology Section, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA.
| |
Collapse
|
10
|
Sainburg T, Thielk M, Gentner TQ. Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires. PLoS Comput Biol 2020; 16:e1008228. [PMID: 33057332 PMCID: PMC7591061 DOI: 10.1371/journal.pcbi.1008228] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2019] [Revised: 10/27/2020] [Accepted: 08/08/2020] [Indexed: 12/15/2022] Open
Abstract
Animals produce vocalizations that range in complexity from a single repeated call to hundreds of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex vocalizations can require considerable effort and a deep intuition about each species' vocal behavior. Even with a great deal of experience, human characterizations of animal communication can be affected by human perceptual biases. We present a set of computational methods for projecting animal vocalizations into low dimensional latent representational spaces that are directly learned from the spectrograms of vocal signals. We apply these methods to diverse datasets from over 20 species, including humans, bats, songbirds, mice, cetaceans, and nonhuman primates. Latent projections uncover complex features of data in visually intuitive and quantifiable ways, enabling high-powered comparative analyses of vocal acoustics. We introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent variables. Each method can be used to disentangle complex spectro-temporal structure and observe long-timescale organization in communication.
Collapse
Affiliation(s)
- Tim Sainburg
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA
- Center for Academic Research & Training in Anthropogeny, University of California, San Diego, La Jolla, CA, USA
| | - Marvin Thielk
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, USA
| | - Timothy Q. Gentner
- Department of Psychology, University of California, San Diego, La Jolla, CA, USA
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, USA
- Neurobiology Section, Division of Biological Sciences, University of California, San Diego, La Jolla, CA, USA
- Kavli Institute for Brain and Mind, University of California, San Diego, La Jolla, CA, USA
| |
Collapse
|
11
|
Vahidi NW, Rudraraju S, Castagnola E, Cea C, Nimbalkar S, Hanna R, Arvizu R, Dayeh SA, Gentner TQ, Kassegne S. Epi-Intra neural probes with glassy carbon microelectrodes help elucidate neural coding and stimulus encoding in 3D volume of tissue. J Neural Eng 2020; 17:046005. [PMID: 32521531 DOI: 10.1088/1741-2552/ab9b5c] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
OBJECTIVE In this study, we demonstrate practical applications of a novel 3-dimensional neural probe for simultaneous electrophysiological recordings from the surface of the brain as well as deep intra-cortical tissue. We used this 3D probe to investigate signal propagation mechanisms between neuronal cells and their responses to stimuli in a 3D fashion. APPROACH This novel probe leverage 2D thin-film microfabrication technique to combine an epi-cortical (surface) and an intra-cortical (depth) microelectrode arrays (Epi-Intra), that unfold into an origami 3D-like probe during brain implantation. The flexible epi-cortical component conforms to the brain surface while the intra-cortical array is reinforced with stiffer durimide polymer layer for ease of tissue penetration. The microelectrodes are made of glassy carbon material that is biocompatible and has low electrochemical impedance that is important for high fidelity neuronal recordings. These recordings were performed on the auditory region of anesthetized European starling songbirds during playback of conspecific songs as auditory stimuli. MAIN RESULTS The Epi-Intra probe recorded broadband activity including local field potentials (LFPs) signals as well as single-unit activity and multi-unit activity from both surface and deep brain. The majority of recorded cellular activities were stimulus-locked and exhibited low noise. Notably, while LFPs recorded on surface and depth electrodes did not exhibit strong correlation, composite receptive fields (CRFs)-extracted from individual neuron cells through a non-linear model and that are cell-dependent-were correlated. SIGNIFICANCE These findings demonstrate that CRFs extracted from Epi-Intra recordings are excellent candidates for neural coding and for understanding the relationship between sensory neuronal responses and their stimuli (stimulus encoding). Beyond CRFs, this novel neural probe may enable new spatiotemporal 3D volumetric mapping to address, with cellular resolution, how the brain coordinates function.
Collapse
Affiliation(s)
- Nasim W Vahidi
- Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, United States of America
| | | | | | | | | | | | | | | | | | | |
Collapse
|
12
|
Hermiz J, Hossain L, Arneodo EM, Ganji M, Rogers N, Vahidi N, Halgren E, Gentner TQ, Dayeh SA, Gilja V. Stimulus Driven Single Unit Activity From Micro-Electrocorticography. Front Neurosci 2020; 14:55. [PMID: 32180695 PMCID: PMC7059620 DOI: 10.3389/fnins.2020.00055] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2019] [Accepted: 01/14/2020] [Indexed: 12/18/2022] Open
Abstract
High-fidelity measurements of neural activity can enable advancements in our understanding of the neural basis of complex behaviors such as speech, audition, and language, and are critical for developing neural prostheses that address impairments to these abilities due to disease or injury. We develop a novel high resolution, thin-film micro-electrocorticography (micro-ECoG) array that enables high-fidelity surface measurements of neural activity from songbirds, a well-established animal model for studying speech behavior. With this device, we provide the first demonstration of sensory-evoked modulation of surface-recorded single unit responses. We establish that single unit activity is consistently sensed from micro-ECoG electrodes over the surface of sensorimotor nucleus HVC (used as a proper name) in anesthetized European starlings, and validate responses with correlated firing in single units recorded simultaneously at surface and depth. The results establish a platform for high-fidelity recording from the surface of subcortical structures that will accelerate neurophysiological studies, and development of novel electrode arrays and neural prostheses.
Collapse
Affiliation(s)
- John Hermiz
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, United States
| | - Lorraine Hossain
- Department of Materials Science and Engineering, University of California, San Diego, La Jolla, CA, United States
| | - Ezequiel M Arneodo
- Biocircuits Institute, University of California, San Diego, La Jolla, CA, United States
| | - Mehran Ganji
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, United States
| | - Nicholas Rogers
- Department of Physics, University of California, San Diego, La Jolla, CA, United States
| | - Nasim Vahidi
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, United States
| | - Eric Halgren
- Department of Radiology, University of California, San Diego, La Jolla, CA, United States.,Department of Neurosciences, University of California, San Diego, La Jolla, CA, United States
| | - Timothy Q Gentner
- Department of Psychology, University of California, San Diego, La Jolla, CA, United States.,Kavli Institute for Brain and Mind, La Jolla, CA, United States.,Neurobiology Section, University of California, San Diego, La Jolla, CA, United States
| | - Shadi A Dayeh
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, United States.,Department of Materials Science and Engineering, University of California, San Diego, La Jolla, CA, United States.,Department of Nanoengineering, University of California, San Diego, La Jolla, CA, United States
| | - Vikash Gilja
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, United States
| |
Collapse
|
13
|
Zuidema W, French RM, Alhama RG, Ellis K, O'Donnell TJ, Sainburg T, Gentner TQ. Five Ways in Which Computational Modeling Can Help Advance Cognitive Science: Lessons From Artificial Grammar Learning. Top Cogn Sci 2019; 12:925-941. [PMID: 31663267 PMCID: PMC7496886 DOI: 10.1111/tops.12474] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2018] [Revised: 06/17/2019] [Accepted: 06/24/2019] [Indexed: 11/28/2022]
Abstract
There is a rich tradition of building computational models in cognitive science, but modeling, theoretical, and experimental research are not as tightly integrated as they could be. In this paper, we show that computational techniques—even simple ones that are straightforward to use—can greatly facilitate designing, implementing, and analyzing experiments, and generally help lift research to a new level. We focus on the domain of artificial grammar learning, and we give five concrete examples in this domain for (a) formalizing and clarifying theories, (b) generating stimuli, (c) visualization, (d) model selection, and (e) exploring the hypothesis space. Zuidema et al. illustrate how empirical AGL studies can benefit from computational models and techniques. Computational models can help clarifying theories, and thus in delineating research questions, but also in facilitating experimental design, stimulus generation, and data analysis. The authors show, with a series of examples, how computational modeling can be integrated with empirical AGL approaches, and how model selection techniques can indicate the most likely model to explain experimental outcomes.
Collapse
Affiliation(s)
- Willem Zuidema
- Institute for Logic, Language and Computation, University of Amsterdam
| | | | | | - Kevin Ellis
- Department of Brain and Cognitive Sciences, MIT
| | | | - Tim Sainburg
- Department of Psychology, University of California San Diego
| | - Timothy Q Gentner
- Department of Psychology & Division of Biological Sciences, University of California San Diego
| |
Collapse
|
14
|
Ganji M, Paulk AC, Yang JC, Vahidi NW, Lee SH, Liu R, Hossain L, Arneodo EM, Thunemann M, Shigyo M, Tanaka A, Ryu SB, Lee SW, Tchoe Y, Marsala M, Devor A, Cleary DR, Martin JR, Oh H, Gilja V, Gentner TQ, Fried SI, Halgren E, Cash SS, Dayeh SA. Selective Formation of Porous Pt Nanorods for Highly Electrochemically Efficient Neural Electrode Interfaces. Nano Lett 2019; 19:6244-6254. [PMID: 31369283 PMCID: PMC7174248 DOI: 10.1021/acs.nanolett.9b02296] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
The enhanced electrochemical activity of nanostructured materials is readily exploited in energy devices, but their utility in scalable and human-compatible implantable neural interfaces can significantly advance the performance of clinical and research electrodes. We utilize low-temperature selective dealloying to develop scalable and biocompatible one-dimensional platinum nanorod (PtNR) arrays that exhibit superb electrochemical properties at various length scales, stability, and biocompatibility for high performance neurotechnologies. PtNR arrays record brain activity with cellular resolution from the cortical surfaces in birds and nonhuman primates. Significantly, strong modulation of surface recorded single unit activity by auditory stimuli is demonstrated in European Starling birds as well as the modulation of local field potentials in the visual cortex by light stimuli in a nonhuman primate and responses to electrical stimulation in mice. PtNRs record behaviorally and physiologically relevant neuronal dynamics from the surface of the brain with high spatiotemporal resolution, which paves the way for less invasive brain-machine interfaces.
Collapse
Affiliation(s)
- Mehran Ganji
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Angelique C. Paulk
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
| | - Jimmy C. Yang
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
| | - Nasim W. Vahidi
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Sang Heon Lee
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Ren Liu
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Lorraine Hossain
- Materials Science and Engineering Program, University of California San Diego, La Jolla, California 92093, United States
| | - Ezequiel M. Arneodo
- Department of Neurosciences, University of California San Diego, La Jolla, California 92093, United States
| | - Martin Thunemann
- Department of Neurosciences, University of California San Diego, La Jolla, California 92093, United States
| | - Michiko Shigyo
- Department of Anesthesiology, University of California, San Diego (UCSD), La Jolla, California 92037, United States
| | - Atsunori Tanaka
- Materials Science and Engineering Program, University of California San Diego, La Jolla, California 92093, United States
| | - Sang Baek Ryu
- Department of Neurosurgery, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
| | - Seung Woo Lee
- Department of Neurosurgery, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
| | - Youngbin Tchoe
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Martin Marsala
- Department of Anesthesiology, University of California, San Diego (UCSD), La Jolla, California 92037, United States
| | - Anna Devor
- Departments of Radiology and Neurosciences, University of California San Diego, La Jolla, California 92093, United States
| | - Daniel R. Cleary
- Department of Neurosurgery, University of California, San Diego (UCSD), La Jolla, California 92037, United States
| | - Joel R. Martin
- Department of Neurosurgery, University of California, San Diego (UCSD), La Jolla, California 92037, United States
| | - Hongseok Oh
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Vikash Gilja
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
| | - Timothy Q. Gentner
- Department of Neurosciences, University of California San Diego, La Jolla, California 92093, United States
| | - Shelley I. Fried
- Department of Neurosurgery, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
- Boston VA Healthcare System, 150 South Huntington Avenue, Boston, Massachusetts 02130, United States
| | - Eric Halgren
- Departments of Radiology and Neurosciences, University of California San Diego, La Jolla, California 92093, United States
| | - Sydney S. Cash
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
| | - Shadi A. Dayeh
- Department of Electrical and Computer Engineering, University of California San Diego La Jolla, California 92093, United States
- Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts 02114, United States
- Department of Nanoengineering, University of California San Diego, La Jolla, California 92093, United States
- Corresponding Author (S.A.D.)
| |
Collapse
|
15
|
Abstract
Human speech possesses a rich hierarchical structure that allows for meaning to be altered by words spaced far apart in time. Conversely, the sequential structure of nonhuman communication is thought to follow non-hierarchical Markovian dynamics operating over only short distances. Here, we show that human speech and birdsong share a similar sequential structure indicative of both hierarchical and Markovian organization. We analyze the sequential dynamics of song from multiple songbird species and speech from multiple languages by modeling the information content of signals as a function of the sequential distance between vocal elements. Across short sequence-distances, an exponential decay dominates the information in speech and birdsong, consistent with underlying Markovian processes. At longer sequence-distances, the decay in information follows a power law, consistent with underlying hierarchical processes. Thus, the sequential organization of acoustic elements in two learned vocal communication signals (speech and birdsong) shows functionally equivalent dynamics, governed by similar processes.
Collapse
Affiliation(s)
- Tim Sainburg
- Department of Psychology, University of California, UC San Diego, La Jolla, CA, 92093, USA
- Center for Academic Research & Training in Anthropogeny, UC San Diego, La Jolla, CA, 92093, USA
| | - Brad Theilman
- Neurosciences Graduate Program, University of California, UC San Diego, La Jolla, CA, 92093, USA
| | - Marvin Thielk
- Neurosciences Graduate Program, University of California, UC San Diego, La Jolla, CA, 92093, USA
| | - Timothy Q Gentner
- Department of Psychology, University of California, UC San Diego, La Jolla, CA, 92093, USA.
- Neurosciences Graduate Program, University of California, UC San Diego, La Jolla, CA, 92093, USA.
- Neurobiology Section, Division of Biological Sciences, UC San Diego, La Jolla, CA, 92093, USA.
- Kavli Institute for Brain and Mind, La Jolla, CA, 92093, USA.
| |
Collapse
|
16
|
Coffinger S, Houser D, Finneran JJ, Mulsow J, Gentner TQ, Burkard R. Stimulus bandwidth impact on auditory evoked potential thresholds and estimated upper-frequency limits of hearing in dolphins. J Acoust Soc Am 2018; 144:3575. [PMID: 30599667 DOI: 10.1121/1.5084043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2018] [Accepted: 11/29/2018] [Indexed: 06/09/2023]
Abstract
The frequency range of hearing is important for assessing the potential impact of anthropogenic noise on marine mammals. Auditory evoked potentials (AEPs) are commonly used to assess toothed whale hearing, but measurement methods vary across researchers and laboratories. In particular, estimates of the upper-frequency limit of hearing (UFL) can vary due to interactions between the unintended spread of spectral energy to frequencies below the desired test frequency and a sharp decline in hearing sensitivity at frequencies near the UFL. To assess the impact of stimulus bandwidth on UFL measurement, AEP hearing tests were conducted in four bottlenose dolphins (Tursiops truncatus) with normal and impaired hearing ranges. Dolphins were tested at frequencies near the UFL and at a frequency 1/2-octave below the UFL, where hearing sensitivity was better (i.e., threshold was lower). Thresholds were measured using sinusoidal amplitude modulated (SAM) tones and tone-bursts of varying bandwidth. Measured thresholds varied inversely as a function of stimulus bandwidth near the UFL with narrow-band tone-bursts approximating thresholds measured using SAM tones. Bandwidth did not impact measured thresholds where hearing was more sensitive, highlighting how stimulus bandwidth and the rate of decline of hearing sensitivity interact to affect measured threshold near the UFL.
Collapse
Affiliation(s)
- Sean Coffinger
- Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| | - Dorian Houser
- Department of Biologic and Bioacoustic Research, National Marine Mammal Foundation, 3131, 2240 Shelter Island Drive, San Diego, California 92106, USA
| | - James J Finneran
- United States Navy Marine Mammal Program, Space and Naval Warfare Systems Center Pacific, 53560 Hull Street, San Diego, California 92152, USA
| | - Jason Mulsow
- Department of Biologic and Bioacoustic Research, National Marine Mammal Foundation, 3131, 2240 Shelter Island Drive, San Diego, California 92106, USA
| | - Timothy Q Gentner
- Department of Psychology, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| | - Robert Burkard
- Department of Rehabilitation Science, University at Buffalo, 501 Kimball Tower, Buffalo, New York 14214, USA
| |
Collapse
|
17
|
Castagnola E, Vahidi NW, Nimbalkar S, Rudraraju S, Thielk M, Zucchini E, Cea C, Carli S, Gentner TQ, Ricci D, Fadiga L, Kassegne S. In Vivo Dopamine Detection and Single Unit Recordings Using Intracortical Glassy Carbon Microelectrode Arrays. ACTA ACUST UNITED AC 2018; 3:1629-1634. [PMID: 29881642 DOI: 10.1557/adv.2018.98] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
In this study, we present a 4-channel intracortical glassy carbon (GC) microelectrode array on a flexible substrate for the simultaneous in vivo neural activity recording and dopamine (DA) concentration measurement at four different brain locations (220μm vertical spacing). The ability of GC microelectrodes to detect DA was firstly assessed in vitro in phosphate-buffered saline solution and then validated in vivo measuring spontaneous DA concentration in the Striatum of European Starling songbird through fast scan cyclic voltammetry (FSCV). The capability of GC microelectrode arrays and commercial penetrating metal microelectrode arrays to record neural activity from the Caudomedial Neostriatum of European starling songbird was compared. Preliminary results demonstrated the ability of GC microelectrodes in detecting neurotransmitters release and recording neural activity in vivo. GC microelectrodes array may, therefore, offer a new opportunity to understand the intimate relations linking electrophysiological parameters with neurotransmitters release.
Collapse
Affiliation(s)
- Elisa Castagnola
- MEMS Research Lab., Department of Mechanical Engineering, College of Engineering, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1323, USA.,Center for Sensorimotor Neural Engineering (CSNE), Box 37, 1414 NE 42nd St., Suite 204, Seattle, WA 98105-6271, USA
| | - Nasim Winchester Vahidi
- Dept. of Electrical Engineering, University of California San Diego, La Jolla, CA 92093, USA.,Neurosciences Graduate Program, University of California San Diego, 9500 Gilman Dr, La Jolla, CA, USA
| | - Surabhi Nimbalkar
- MEMS Research Lab., Department of Mechanical Engineering, College of Engineering, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1323, USA.,Center for Sensorimotor Neural Engineering (CSNE), Box 37, 1414 NE 42nd St., Suite 204, Seattle, WA 98105-6271, USA
| | - Srihita Rudraraju
- Department of Bioengineering, University of California San Diego, La Jolla, CA, USA
| | - Marvin Thielk
- Neurosciences Graduate Program, University of California San Diego, 9500 Gilman Dr, La Jolla, CA, USA
| | - Elena Zucchini
- Center for Translational Neurophysiology of Speech and Communication, Istituto Italiano di Tecnologia, Via Fossato di Mortara 17/19, 44121, Ferrara, Italy
| | - Claudia Cea
- MEMS Research Lab., Department of Mechanical Engineering, College of Engineering, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1323, USA.,Center for Sensorimotor Neural Engineering (CSNE), Box 37, 1414 NE 42nd St., Suite 204, Seattle, WA 98105-6271, USA
| | - Stefano Carli
- Center for Translational Neurophysiology of Speech and Communication, Istituto Italiano di Tecnologia, Via Fossato di Mortara 17/19, 44121, Ferrara, Italy
| | - Timothy Q Gentner
- Neurosciences Graduate Program, University of California San Diego, 9500 Gilman Dr, La Jolla, CA, USA
| | - Davide Ricci
- Department of Bioengineering, University of California San Diego, La Jolla, CA, USA
| | - Luciano Fadiga
- Center for Translational Neurophysiology of Speech and Communication, Istituto Italiano di Tecnologia, Via Fossato di Mortara 17/19, 44121, Ferrara, Italy.,Human Physiology, University of Ferrara, Via Fossato di Mortara 17/19, 44121, Ferrara, Italy
| | - Sam Kassegne
- MEMS Research Lab., Department of Mechanical Engineering, College of Engineering, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1323, USA.,Center for Sensorimotor Neural Engineering (CSNE), Box 37, 1414 NE 42nd St., Suite 204, Seattle, WA 98105-6271, USA
| |
Collapse
|
18
|
Perks KE, Gentner TQ. Subthreshold membrane responses underlying sparse spiking to natural vocal signals in auditory cortex. Eur J Neurosci 2015; 41:725-33. [PMID: 25728189 DOI: 10.1111/ejn.12831] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2014] [Revised: 12/07/2014] [Accepted: 12/11/2014] [Indexed: 01/31/2023]
Abstract
Natural acoustic communication signals, such as speech, are typically high-dimensional with a wide range of co-varying spectro-temporal features at multiple timescales. The synaptic and network mechanisms for encoding these complex signals are largely unknown. We are investigating these mechanisms in high-level sensory regions of the songbird auditory forebrain, where single neurons show sparse, object-selective spiking responses to conspecific songs. Using whole-cell in vivo patch clamp techniques in the caudal mesopallium and the caudal nidopallium of starlings, we examine song-driven subthreshold and spiking activity. We find that both the subthreshold and the spiking activity are reliable (i.e. the same song drives a similar response each time it is presented) and specific (i.e. responses to different songs are distinct). Surprisingly, however, the reliability and specificity of the subthreshold response was uniformly high regardless of when the cell spiked, even for song stimuli that drove no spikes. We conclude that despite a selective and sparse spiking response, high-level auditory cortical neurons are under continuous, non-selective, stimulus-specific synaptic control. To investigate the role of local network inhibition in this synaptic control, we then recorded extracellularly while pharmacologically blocking local GABAergic transmission. This manipulation modulated the strength and the reliability of stimulus-driven spiking, consistent with a role for local inhibition in regulating the reliability of network activity and the stimulus specificity of the subthreshold response in single cells. We discuss these results in the context of underlying computations that could generate sparse, stimulus-selective spiking responses, and models for hierarchical pooling.
Collapse
Affiliation(s)
- Krista E Perks
- Neurosciences Graduate Program, University of California, San Diego, 9500 Gilman Dr, La Jolla, CA, USA
| | | |
Collapse
|
19
|
Abstract
Language is uniquely human, but its acquisition may involve cognitive capacities shared with other species. During development, language experience alters speech sound (phoneme) categorization. Newborn infants distinguish the phonemes in all languages but by 10 months show adult-like greater sensitivity to native language phonemic contrasts than non-native contrasts. Distributional theories account for phonetic learning by positing that infants infer category boundaries from modal distributions of speech sounds along acoustic continua. For example, tokens of the sounds /b/ and /p/ cluster around different mean voice onset times. To disambiguate overlapping distributions, contextual theories propose that phonetic category learning is informed by higher-level patterns (e.g., words) in which phonemes normally occur. For example, the vowel sounds /Ι/ and /e/ can occupy similar perceptual spaces but can be distinguished in the context of "with" and "well." Both distributional and contextual cues appear to function in speech acquisition. Non-human species also benefit from distributional cues for category learning, but whether category learning benefits from contextual information in non-human animals is unknown. The use of higher-level patterns to guide lower-level category learning may reflect uniquely human capacities tied to language acquisition or more general learning abilities reflecting shared neurobiological mechanisms. Using songbirds, European starlings, we show that higher-level pattern learning covertly enhances categorization of the natural communication sounds. This observation mirrors the support for contextual theories of phonemic category learning in humans and demonstrates a general form of learning not unique to humans or language.
Collapse
Affiliation(s)
- Jordan A Comins
- Department of Psychology, University of California San Diego, La Jolla, CA 92093, USA
| | - Timothy Q Gentner
- Department of Psychology, University of California San Diego, La Jolla, CA 92093, USA; Section of Neurobiology, University of California San Diego, La Jolla, CA 92093, USA; Neurosciences Graduate Program, University of California San Diego, La Jolla, CA 92093, USA; Kavli Institute for Brain and Mind, University of California San Diego, La Jolla, CA 92093, USA.
| |
Collapse
|
20
|
Comins JA, Gentner TQ. Temporal pattern processing in songbirds. Curr Opin Neurobiol 2014; 28:179-87. [PMID: 25201176 DOI: 10.1016/j.conb.2014.08.003] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2014] [Revised: 08/07/2014] [Accepted: 08/20/2014] [Indexed: 10/24/2022]
Abstract
Understanding how the brain perceives, organizes and uses patterned information is directly related to the neurobiology of language. Given the present limitations, such knowledge at the scale of neurons, neural circuits and neural populations can only come from non-human models, focusing on shared capacities that are relevant to language processing. Here we review recent advances in the behavioral and neural basis of temporal pattern processing of natural auditory communication signals in songbirds, focusing on European starlings. We suggest a general inhibitory circuit for contextual modulation that can act to control sensory representations based on patterning rules.
Collapse
Affiliation(s)
- Jordan A Comins
- Department of Psychology, University of California, San Diego, United States
| | - Timothy Q Gentner
- Department of Psychology, University of California, San Diego, United States; Neurobiology Section, Division of Biological Sciences, University of California, San Diego, United States; Neurosciences Graduate Program, University of California, San Diego, United States; Kavli Institute for Brain and Mind, La Jolla, United States.
| |
Collapse
|
21
|
Abstract
Recognition of natural stimuli requires a combination of selectivity and invariance. Classical neurobiological models achieve selectivity and invariance, respectively, by assigning to each cortical neuron either a computation equivalent to the logical "AND" or a computation equivalent to the logical "OR." One powerful OR-like operation is the MAX function, which computes the maximum over input activities. The MAX function is frequently employed in computer vision to achieve invariance and considered a key operation in visual cortex. Here we explore the computations for selectivity and invariance in the auditory system of a songbird, using natural stimuli. We ask two related questions: does the MAX operation exist in auditory system? Is it implemented by specialized "MAX" neurons, as assumed in vision? By analyzing responses of individual neurons to combinations of stimuli we systematically sample the space of implemented feature recombination functions. Although we frequently observe the MAX function, we show that the same neurons that implement it also readily implement other operations, including the AND-like response. We then show that sensory adaptation, a ubiquitous property of neural circuits, causes transitions between these operations in individual neurons, violating the fixed neuron-to-computation mapping posited in the state-of-the-art object-recognition models. These transitions, however, accord with predictions of neural-circuit models incorporating divisive normalization and variable polynomial nonlinearities at the spike threshold. Because these biophysical properties are not tied to a particular sensory modality but are generic, the flexible neuron-to-computation mapping demonstrated in this study in the auditory system is likely a general property.
Collapse
Affiliation(s)
- Andrei S Kozlov
- Department of Psychology, University of California San Diego, La Jolla, California
| | | |
Collapse
|
22
|
Petkov CI, Gentner TQ, Liu RC. Editorial introduction to Hearing Research special issue on communication sounds and the brain: new directions and perspectives. Hear Res 2013; 305:1-2. [PMID: 24076210 DOI: 10.1016/j.heares.2013.09.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/02/2013] [Accepted: 09/03/2013] [Indexed: 10/26/2022]
|
23
|
Calisi RM, Knudsen DP, Krause JS, Wingfield JC, Gentner TQ. Estradiol differentially affects auditory recognition and learning according to photoperiodic state in the adult male songbird, European starling (Sturnus vulgaris). PeerJ 2013; 1:e150. [PMID: 24058881 PMCID: PMC3775630 DOI: 10.7717/peerj.150] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2013] [Accepted: 08/14/2013] [Indexed: 11/20/2022] Open
Abstract
Changes in hormones can affect many types of learning in vertebrates. Adults experience fluctuations in a multitude of hormones over a temporal scale, from local, rapid action to more long-term, seasonal changes. Endocrine changes during development can affect behavioral outcomes in adulthood, but how learning is affected in adults by hormone fluctuations experienced during adulthood is less understood. Previous reports have implicated the sex steroid hormone estradiol (E2) in both male and female vertebrate cognitive functioning. Here, we examined the effects of E2 on auditory recognition and learning in male European starlings (Sturnus vulgaris). European starlings are photoperiodic, seasonally breeding songbirds that undergo different periods of reproductive activity according to annual changes in day length. We simulated these reproductive periods, specifically 1. photosensitivity, 2. photostimulation, and 3. photorefractoriness in captive birds by altering day length. During each period, we manipulated circulating E2 and examined multiple measures of learning. To manipulate circulating E2, we used subcutaneous implants containing either 17-β E2 and/or fadrozole (FAD), a highly specific aromatase inhibitor that suppresses E2 production in the body and the brain, and measured the latency for birds to learn and respond to short, male conspecific song segments (motifs). We report that photostimulated birds given E2 had higher response rates and responded with better accuracy than those given saline controls or FAD. Conversely, photosensitive, animals treated with E2 responded with less accuracy than those given FAD. These results demonstrate how circulating E2 and photoperiod can interact to shape auditory recognition and learning in adults, driving it in opposite directions in different states.
Collapse
Affiliation(s)
- Rebecca M Calisi
- Department of Neurobiology, Physiology, and Behavior, The University of California , Davis , USA ; Department of Psychology, The University of California , San Diego , USA
| | | | | | | | | |
Collapse
|
24
|
Comins JA, Gentner TQ. Perceptual categories enable pattern generalization in songbirds. Cognition 2013; 128:113-8. [PMID: 23669049 DOI: 10.1016/j.cognition.2013.03.014] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2012] [Revised: 03/09/2013] [Accepted: 03/25/2013] [Indexed: 10/26/2022]
Abstract
Since Chomsky's pioneering work on syntactic structures, comparative psychologists interested in the study of language evolution have targeted pattern complexity, using formal mathematical grammars, as the key to organizing language-relevant cognitive processes across species. This focus on formal syntactic complexity, however, often disregards the close interaction in real-world signals between the structure of a pattern and its constituent elements. Whether such features of natural auditory signals shape pattern generalization is unknown. In the present paper, we train birds to recognize differently patterned strings of natural signals (song motifs). Instead of focusing on the complexity of the overtly reinforced patterns, we ask how the perceptual groupings of pattern elements influence the generalization pattern knowledge. We find that learning and perception of training patterns is agnostic to the perceptual features of underlying elements. Surprisingly, however, these same features constrain the generalization of pattern knowledge, and thus its broader use. Our results demonstrate that the restricted focus of comparative language research on formal models of syntactic complexity is, at best, insufficient to understand pattern use.
Collapse
Affiliation(s)
- Jordan A Comins
- Department of Psychology, University of California, San Diego, USA
| | | |
Collapse
|
25
|
Knudsen DP, Gentner TQ. Active recognition enhances the representation of behaviorally relevant information in single auditory forebrain neurons. J Neurophysiol 2013; 109:1690-703. [PMID: 23303858 DOI: 10.1152/jn.00461.2012] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Sensory systems are dynamic. They must process a wide range of natural signals that facilitate adaptive behaviors in a manner that depends on an organism's constantly changing goals. A full understanding of the sensory physiology that underlies adaptive natural behaviors must therefore account for the activity of sensory systems in light of these behavioral goals. Here we present a novel technique that combines in vivo electrophysiological recording from awake, freely moving songbirds with operant conditioning techniques that allow control over birds' recognition of conspecific song, a widespread natural behavior in songbirds. We show that engaging in a vocal recognition task alters the response properties of neurons in the caudal mesopallium (CM), an avian analog of mammalian auditory cortex, in European starlings. Compared with awake, passive listening, active engagement of subjects in an auditory recognition task results in neurons responding to fewer song stimuli and a decrease in the trial-to-trial variability in their driven firing rates. Mean firing rates also change during active recognition, but not uniformly. Relative to nonengaged listening, active recognition causes increases in the driven firing rates in some neurons, decreases in other neurons, and stimulus-specific changes in other neurons. These changes lead to both an increase in stimulus selectivity and an increase in the information conveyed by the neurons about the animals' behavioral task. This study demonstrates the behavioral dependence of neural responses in the avian auditory forebrain and introduces the starling as a model for real-time monitoring of task-related neural processing of complex auditory objects.
Collapse
Affiliation(s)
- Daniel P Knudsen
- Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, USA
| | | |
Collapse
|
26
|
Thompson JV, Jeanne JM, Gentner TQ. Local inhibition modulates learning-dependent song encoding in the songbird auditory cortex. J Neurophysiol 2012; 109:721-33. [PMID: 23155175 DOI: 10.1152/jn.00262.2012] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Changes in inhibition during development are well documented, but the role of inhibition in adult learning-related plasticity is not understood. In songbirds, vocal recognition learning alters the neural representation of songs across the auditory forebrain, including the caudomedial nidopallium (NCM), a region analogous to mammalian secondary auditory cortices. Here, we block local inhibition with the iontophoretic application of gabazine, while simultaneously measuring song-evoked spiking activity in NCM of European starlings trained to recognize sets of conspecific songs. We find that local inhibition differentially suppresses the responses to learned and unfamiliar songs and enhances spike-rate differences between learned categories of songs. These learning-dependent response patterns emerge, in part, through inhibitory modulation of selectivity for song components and the masking of responses to specific acoustic features without altering spectrotemporal tuning. The results describe a novel form of inhibitory modulation of the encoding of learned categories and demonstrate that inhibition plays a central role in shaping the responses of neurons to learned, natural signals.
Collapse
Affiliation(s)
- Jason V Thompson
- Neurosciences Graduate Program, University of California, San Diego, La Jolla, CA, USA
| | | | | |
Collapse
|
27
|
Abstract
One longstanding impediment to progress in understanding the neural basis of language is the development of model systems that retain language-relevant cognitive behaviors yet permit invasive cellular neuroscience methods. Recent experiments in songbirds suggest that this group may be developed into a powerful animal model, particularly for components of grammatical processing. It remains unknown, however, what a neuroscience of language perception may look like when instantiated at the cellular or network level. Here we deconstruct language perception into a minimal set of cognitive processes necessary to support grammatical processing. We then review the current state of our understanding about the neural mechanisms of these requisite cognitive processes in songbirds. We note where current knowledge is lacking, and suggest how these mechanisms may ultimately combine to support an emergent mechanism capable of processing grammatical structures of differing complexity.
Collapse
Affiliation(s)
- Justin T Kiggins
- Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, USA
| | | | | |
Collapse
|
28
|
Bregman MR, Patel AD, Gentner TQ. Stimulus-dependent flexibility in non-human auditory pitch processing. Cognition 2012; 122:51-60. [PMID: 21911217 PMCID: PMC3215778 DOI: 10.1016/j.cognition.2011.08.008] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2011] [Revised: 06/09/2011] [Accepted: 08/08/2011] [Indexed: 11/20/2022]
Abstract
Songbirds and humans share many parallels in vocal learning and auditory sequence processing. However, the two groups differ notably in their abilities to recognize acoustic sequences shifted in absolute pitch (pitch height). Whereas humans maintain accurate recognition of words or melodies over large pitch height changes, songbirds are comparatively much poorer at recognizing pitch-shifted tone sequences. This apparent disparity may reflect fundamental differences in the neural mechanisms underlying the representation of sound in songbirds. Alternatively, because non-human studies have used sine-tone stimuli almost exclusively, tolerance to pitch height changes in the context of natural signals may be underestimated. Here, we show that European starlings, a species of songbird, can maintain accurate recognition of the songs of other starlings when the pitch of those songs is shifted by as much as ±40%. We observed accurate recognition even for songs pitch-shifted well outside the range of frequencies used during training, and even though much smaller pitch shifts in conspecific songs are easily detected. With similar training using human piano melodies, recognition of the pitch-shifted melodies is very limited. These results demonstrate that non-human pitch processing is more flexible than previously thought and that the flexibility in pitch processing strategy is stimulus dependent.
Collapse
Affiliation(s)
- Micah R Bregman
- Department of Cognitive Science, UC San Diego, La Jolla, CA, United States
| | | | | |
Collapse
|
29
|
|
30
|
Abstract
Songbirds share a number of parallels with humans that make them an attractive model system for studying the behavioral and neurobiological mechanisms that underlie the learning and processing of vocal communication signals. Here we review the perceptual and cognitive mechanisms of audition in birds, and emphasize the behavioral and neural basis of song recognition. Where appropriate, we point out a number of intersections with human vocal communication behavior that suggest common mechanisms amenable to further study, and limitations of birdsong as a model for human language.
Collapse
Affiliation(s)
- Daniel P Knudsen
- University of California, San Diego, La Jolla, 92093, United States
| | | |
Collapse
|
31
|
Comins JA, Gentner TQ. Working memory for patterned sequences of auditory objects in a songbird. Cognition 2010; 117:38-53. [PMID: 20638052 DOI: 10.1016/j.cognition.2010.06.009] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2009] [Revised: 06/05/2010] [Accepted: 06/14/2010] [Indexed: 11/28/2022]
Abstract
The capacity to remember sequences is critical to many behaviors, such as navigation and communication. Adult humans readily recall the serial order of auditory items, and this ability is commonly understood to support, in part, the speech processing for language comprehension. Theories of short-term serial recall posit either use of absolute (hierarchically structured) or relative (associatively structured) position information. To date, neither of these classes of theories has been tested in a comparative auditory model. European starlings, a species of songbird, use temporally structured acoustic signals to communicate, and thus have the potential to serve as a model system for auditory working memory. Here, we explore the strategies that starlings use to detect the serial order of ecologically valid acoustic communication signals and the limits on their capacities to do so. Using a two-alternative choice operant procedure, we demonstrate that starlings can attend to the serial ordering of at least four song elements (motifs) and can use this information to classify differently ordered sequences of motifs. Removing absolute position cues from sequences while leaving relative position cues intact, causes recognition to fail. We then show that starlings can, however, recognize motif-sequences using only relative position cues, but only under rigid circumstances. The data are consistent with a strong learning bias against relative position information, and suggest that recognition of structured vocal signals in this species is inherently hierarchical.
Collapse
Affiliation(s)
- Jordan A Comins
- Department of Human Evolutionary Biology, Harvard University, Cambridge, MA 02138, USA
| | | |
Collapse
|
32
|
Thompson JV, Gentner TQ. Song recognition learning and stimulus-specific weakening of neural responses in the avian auditory forebrain. J Neurophysiol 2010; 103:1785-97. [PMID: 20107117 DOI: 10.1152/jn.00885.2009] [Citation(s) in RCA: 49] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Learning typically increases the strength of responses and the number of neurons that respond to training stimuli. Few studies have explored representational plasticity using natural stimuli, however, leaving unknown the changes that accompany learning under more realistic conditions. Here, we examine experience-dependent plasticity in European starlings, a songbird with rich acoustic communication signals tied to robust, natural recognition behaviors. We trained starlings to recognize conspecific songs and recorded the extracellular spiking activity of single neurons in the caudomedial nidopallium (NCM), a secondary auditory forebrain region analogous to mammalian auditory cortex. Training induced a stimulus-specific weakening of the neural responses (lower spike rates) to the learned songs, whereas the population continued to respond robustly to unfamiliar songs. Additional experiments rule out stimulus-specific adaptation and general biases for novel stimuli as explanations of these effects. Instead, the results indicate that associative learning leads to single neuron responses in which both irrelevant and unfamiliar stimuli elicit more robust responses than behaviorally relevant natural stimuli. Detailed analyses of these effects at a finer temporal scale point to changes in the number of motifs eliciting excitatory responses above a neuron's spontaneous discharge rate. These results show a novel form of experience-dependent plasticity in the auditory forebrain that is tied to associative learning and in which the overall strength of responses is inversely related to learned behavioral significance.
Collapse
Affiliation(s)
- Jason V Thompson
- Graduate Program in Neuroscience, UCSD Dept. of Psychology, La Jolla, CA 92093, USA
| | | |
Collapse
|
33
|
Abstract
Uncovering the roles of neural feedback in the brain is an active area of experimental research. In songbirds, the telencephalic premotor nucleus HVC receives neural feedback from both forebrain and brain stem areas. Here we present a computational model of birdsong sequencing that incorporates HVC and associated nuclei and builds on the model of sparse bursting presented in our preceding companion paper. Our model embodies the hypotheses that 1) different networks in HVC control different syllables or notes of birdsong, 2) interneurons in HVC not only participate in sparse bursting but also provide mutual inhibition between networks controlling syllables or notes, and 3) these syllable networks are sequentially excited by neural feedback via the brain stem and the afferent thalamic nucleus Uva, or a similar feedback pathway. We discuss the model's ability to unify physiological, behavioral, and lesion results and we use it to make novel predictions that can be tested experimentally. The model suggests a neural basis for sequence variations, shows that stimulation in the feedback pathway may have different effects depending on the balance of excitation and inhibition at the input to HVC from Uva, and predicts deviations from uniform expansion of syllables and gaps during HVC cooling.
Collapse
Affiliation(s)
- Leif Gibb
- Neurosciences Graduate Program, Department of Psychology, Scripps Institute of Oceanography, Center for Theoretical Biological Physics, University of California, San Diego, La Jolla, CA, USA.
| | | | | |
Collapse
|
34
|
Gibb L, Gentner TQ, Abarbanel HDI. Inhibition and recurrent excitation in a computational model of sparse bursting in song nucleus HVC. J Neurophysiol 2009; 102:1748-62. [PMID: 19515949 DOI: 10.1152/jn.00670.2007] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023] Open
Abstract
The telencephalic premotor nucleus HVC is situated at a critical point in the pattern-generating premotor circuitry of oscine songbirds. A striking feature of HVC's premotor activity is that its projection neurons burst extremely sparsely. Here we present a computational model of HVC embodying several central hypotheses: 1) sparse bursting is generated in bistable groups of recurrently connected robust nucleus of the arcopallium (RA)-projecting (HVCRA) neurons; 2) inhibitory interneurons terminate bursts in the HVCRA groups; and 3) sparse sequences of bursts are generated by the propagation of waves of bursting activity along networks of HVCRA neurons. Our model of sparse bursting places HVC in the context of central pattern generators and cortical networks using inhibition, recurrent excitation, and bistability. Importantly, the unintuitive result that inhibitory interneurons can precisely terminate the bursts of HVCRA groups while showing relatively sustained activity throughout the song is made possible by a specific constraint on their connectivity. We use the model to make novel predictions that can be tested experimentally.
Collapse
Affiliation(s)
- Leif Gibb
- Neurosciences Graduate Program, Department of Psychology, Scripps Institute of Oceanography, Center for Theoretical Biological Physics, University of California, San Diego, La Jolla, CA, USA.
| | | | | |
Collapse
|
35
|
Gentner TQ. Temporal scales of auditory objects underlying birdsong vocal recognition. J Acoust Soc Am 2008; 124:1350-9. [PMID: 18681620 PMCID: PMC2638212 DOI: 10.1121/1.2945705] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/20/2007] [Revised: 05/05/2008] [Accepted: 05/28/2008] [Indexed: 05/21/2023]
Abstract
Vocal recognition is common among songbirds, and provides an excellent model system to study the perceptual and neurobiological mechanisms for processing natural vocal communication signals. Male European starlings, a species of songbird, learn to recognize the songs of multiple conspecific males by attending to stereotyped acoustic patterns, and these learned patterns elicit selective neuronal responses in auditory forebrain neurons. The present study investigates the perceptual grouping of spectrotemporal acoustic patterns in starling song at multiple temporal scales. The results show that permutations in sequencing of submotif acoustic features have significant effects on song recognition, and that these effects are specific to songs that comprise learned motifs. The observations suggest that (1) motifs form auditory objects embedded in a hierarchy of acoustic patterns, (2) that object-based song perception emerges without explicit reinforcement, and (3) that multiple temporal scales within the acoustic pattern hierarchy convey information about the individual identity of the singer. The authors discuss the results in the context of auditory object formation and talker recognition.
Collapse
Affiliation(s)
- Timothy Q Gentner
- Department of Psychology, Neurosciences Graduate Program, University of California, San Diego, La Jolla, California 92093, USA.
| |
Collapse
|
36
|
Gentner TQ. Surprising twist on auditory representation. Focus on: "what's that sound? Auditory area CLM encodes stimulus surprise, not intensity or intensity changes". J Neurophysiol 2008; 99:2755-6. [PMID: 18544667 DOI: 10.1152/jn.90270.2008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
|
37
|
Abstract
Humans regularly produce new utterances that are understood by other members of the same language community. Linguistic theories account for this ability through the use of syntactic rules (or generative grammars) that describe the acceptable structure of utterances. The recursive, hierarchical embedding of language units (for example, words or phrases within shorter sentences) that is part of the ability to construct new utterances minimally requires a 'context-free' grammar that is more complex than the 'finite-state' grammars thought sufficient to specify the structure of all non-human communication signals. Recent hypotheses make the central claim that the capacity for syntactic recursion forms the computational core of a uniquely human language faculty. Here we show that European starlings (Sturnus vulgaris) accurately recognize acoustic patterns defined by a recursive, self-embedding, context-free grammar. They are also able to classify new patterns defined by the grammar and reliably exclude agrammatical patterns. Thus, the capacity to classify sequences from recursive, centre-embedded grammars is not uniquely human. This finding opens a new range of complex syntactic processing mechanisms to physiological investigation.
Collapse
Affiliation(s)
- Timothy Q Gentner
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, Illinois 60637, USA.
| | | | | | | |
Collapse
|
38
|
Appeltants D, Gentner TQ, Hulse SH, Balthazart J, Ball GF. The effect of auditory distractors on song discrimination in male canaries (Serinus canaria). Behav Processes 2005; 69:331-41. [PMID: 15896531 DOI: 10.1016/j.beproc.2005.01.010] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2004] [Revised: 01/20/2005] [Accepted: 01/20/2005] [Indexed: 11/30/2022]
Abstract
Male songbirds such as canaries produce complex learned vocalizations that are used in the context of mate attraction and territory defense. Successful mate attraction or territorial defense requires that a bird be able to recognize individuals based on their vocal performance and identify these songs in a noisy background. In order to learn more about how birds are able to solve this problem, we investigated, with a two-alternative choice procedure, the ability of adult male canaries to discriminate between conspecific song segments from two different birds and to maintain this discrimination when conspecific songs are superimposed with a variety of distractors. The results indicate that male canaries have the ability to discriminate, with a high level of accuracy song segments produced by two different conspecific birds. Song discrimination was partially maintained when the stimuli were masked by auditory distractors, but the accuracy of the discrimination progressively declined as a function of the number of masking distractors. The type of distractor used in the experiments (other conspecific songs or different types of artificial white noise) did not markedly affect the rate of deterioration of the song discrimination. These data indicate that adult male canaries have the perceptual abilities to discriminate and selectively attend to one ongoing sound that occurs simultaneously with one or more other sounds. The administration of a noradrenergic neurotoxin did not impair markedly the discrimination learning abilities although the number of subjects tested was too small to allow any firm conclusion. In these conditions, however, the noradrenergic lesion significantly increased the number failures to respond in the discrimination learning task suggesting a role, in canaries, of the noradrenergic system in some attentional processes underlying song learning and processing.
Collapse
Affiliation(s)
- Didier Appeltants
- Department of Cellular and Molecular Neurobiology, Research Group in Behavioral Neuroendocrinology, University of Liège, 17 Place Delcour (Bat. L1), B-4020 Liège, Belgium
| | | | | | | | | |
Collapse
|
39
|
Gentner TQ, Hulse SH, Ball GF. Functional differences in forebrain auditory regions during learned vocal recognition in songbirds. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 2004; 190:1001-10. [PMID: 15449092 DOI: 10.1007/s00359-004-0556-x] [Citation(s) in RCA: 52] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2004] [Revised: 07/30/2004] [Accepted: 07/31/2004] [Indexed: 11/27/2022]
Abstract
Converging evidence implicates the auditory forebrain regions caudal medial mesopallium (formerly cmHV) and caudal medial nidopallium in the perceptual processing of conspecific vocalizations in songbirds. Little is known however, about more specific processing within these regions especially during song-based perceptual behaviors. One hallmark of the caudal medial mesopallium and caudal medial nidopallium, areas analogous to mammalian secondary auditory cortical structures, is their robust expression of the immediate-early-gene zenk in response to conspecific songs. Using European starlings operantly trained to recognize the songs of individual conspecifics, we show that the levels and patterns of zenk protein expression in the caudal medial nidopallium and caudal medial mesopallium differ when song recognition demands are placed on the system. In the caudal medial mesopallium, expression is significantly elevated above basal levels during the recognition of familiar songs, the acquisition of novel associations for familiar songs, and the acquisition of novel song discriminations. In the caudal medial nidopallium, however, expression is significantly elevated above basal levels only during the acquisition of novel song discriminations. The results directly implicate the caudal medial nidopallium and caudal medial mesopallium in at least a portion of the auditory processes underlying vocal recognition. Moreover, the observed differences between these regions imply the functional localization (or at least the concentration) of different auditory processing mechanisms within the caudal medial nidopallium and the caudal medial mesopallium.
Collapse
Affiliation(s)
- Timothy Q Gentner
- Department of Organismal Biology and Anatomy, University of Chicago, 1027 E. 57th Street, Chicago, IL 60637, USA.
| | | | | |
Collapse
|
40
|
Abstract
The songbird auditory system is an excellent model for neuroethological studies of the mechanisms that govern the perception and cognition of natural stimuli (i.e., song), and the translation of corresponding representations into natural behaviors. One common songbird behavior is the learned recognition of individual conspecific songs. This chapter summarizes the research effort to identify the brain regions and mechanisms mediating individual song recognition in European starlings, a species of songbird. The results of laboratory behavioral studies are reviewed, which show that when adult starlings learn to recognize other individual's songs, they do so by memorizing large sets of song elements, called motifs. Recent data from single neurons in the caudal medial portion of the mesopallium are then reviewed, showing that song recognition learning leads to explicit representation of acoustic features that correspond closely to specific motifs, but only to motifs in the songs that birds have learned to recognize. This suggests that the strength and tuning of high-level auditory object representations, of the sort that presumably underlie many forms of vocal communication, are shaped by each animal's unique experience.
Collapse
Affiliation(s)
- T Q Gentner
- Department of Organismal Biology and Anatomy, University of Chicago, 1027 E. 57th Street, Chicago, IL 60637, USA.
| |
Collapse
|
41
|
Sockman KW, Gentner TQ, Ball GF. Complementary neural systems for the experience-dependent integration of mate-choice cues in European starlings. ACTA ACUST UNITED AC 2004; 62:72-81. [PMID: 15389683 DOI: 10.1002/neu.20068] [Citation(s) in RCA: 38] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Choice of a particular mate phenotype may arise out of experience with the very phenotypes under consideration. Female European starlings (Sturnus vulgaris) prefer males that sing predominantly long-bout songs over males that sing predominantly short-bout songs, and thus, song-bout length is a phenotypic parameter instrumental in releasing the female's mate choice. The preferred long-bout songs induce higher expression of the immediate early gene (IEG) ZENK in the female auditory telencephalon than short-bout songs do, but this sensitivity to song length depends on the female's recent song experience. Here, we compared the experience-dependent modulation of ZENK with that of another IEG, FOS, and report that ZENK and FOS expression in the caudomedial mesopallium and caudomedial nidopallium show different modulation properties that complement natural variation in song-bout length. As reported previously, ZENK expression was greater in response to novel long-bout than to novel short-bout songs following a 1-week experience with long-bout but not short-bout songs. In contrast, FOS expression was greater in response to novel long-bout than to novel short-bout songs following a 1-week experience with short-bout but not long-bout songs. Thus, the ZENK and FOS signaling pathways are made sensitive to variation in song length by experiences with songs at opposite ends of the starling song-variation continuum, suggesting the presence of complementary neural systems made sensitive in register with the natural axis of phenotypic variation fundamental to the female's mate choice.
Collapse
Affiliation(s)
- Keith W Sockman
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, Maryland 21218, USA.
| | | | | |
Collapse
|
42
|
Abstract
The neural representations associated with learned auditory behaviours, such as recognizing individuals based on their vocalizations, are not well described. Higher vertebrates learn to recognize complex conspecific vocalizations that comprise sequences of easily identified, naturally occurring auditory objects, which should facilitate the analysis of higher auditory pathways. Here we describe the first example of neurons selective for learned conspecific vocalizations in adult animals--in starlings that have been trained operantly to recognize conspecific songs. The neuronal population is found in a non-primary forebrain auditory region, exhibits increased responses to the set of learned songs compared with novel songs, and shows differential responses to categories of learned songs based on recognition training contingencies. Within the population, many cells respond highly selectively to a subset of specific motifs (acoustic objects) present only in the learned songs. Such neuronal selectivity may contribute to song-recognition behaviour, which in starlings is sensitive to motif identity. In this system, both top-down and bottom-up processes may modify the tuning properties of neurons during recognition learning, giving rise to plastic representations of behaviourally meaningful auditory objects.
Collapse
Affiliation(s)
- Timothy Q Gentner
- Department of Organismal Biology and Anatomy, University of Chicago, 1027 East 57th Street, Chicago, Illinois 60637, USA.
| | | |
Collapse
|
43
|
Sockman KW, Gentner TQ, Ball GF. Recent experience modulates forebrain gene-expression in response to mate-choice cues in European starlings. Proc Biol Sci 2002; 269:2479-85. [PMID: 12495492 PMCID: PMC1691185 DOI: 10.1098/rspb.2002.2180] [Citation(s) in RCA: 94] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Mate-choice decisions can be experience dependent, but we know little about how the brain processes stimuli that release such decisions. Female European starlings (Sturnus vulgaris) prefer males with long-bout songs over males with short-bout songs, and show higher expression of the immediate early gene (IEG) ZENK in the auditory forebrain when exposed to long-bout songs than when exposed to short-bout songs. We exposed female starlings to a short-day photoperiod for one of three durations and then, on an increased photophase, exposed them to one week of long-bout or short-bout song experience. We then examined their IEG response to novel long-bout versus novel short-bout songs by quantifying ZENK protein in two song-processing areas: the caudo-medial hyperstriatum ventrale and the caudo-medial neostriatum. ZENK expression in both areas increased with tenure on short-day photoperiods, suggesting that short days sensitize females to song. The ZENK response bias toward long-bout songs was greater in females with long-bout experience than in females with short-bout experience, indicating that the forebrain response bias toward a preferred trait depends on recent experience with that category of trait. This surprising level of neuroplasticity is immediately relevant to the natural history and fitness of the organism, and may underlie a mechanism for optimizing mate-choice criteria amidst locally variable distributions of secondary sexual characteristics.
Collapse
Affiliation(s)
- Keith W Sockman
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD 21218, USA.
| | | | | |
Collapse
|
44
|
Gentner TQ, Hulse SH, Duffy D, Ball GF. Response biases in auditory forebrain regions of female songbirds following exposure to sexually relevant variation in male song. J Neurobiol 2001; 46:48-58. [PMID: 11108615 DOI: 10.1002/1097-4695(200101)46:1<48::aid-neu5>3.0.co;2-3] [Citation(s) in RCA: 121] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
In many species of songbirds, individual variation between the songs of competing males is correlated with female behavioral preferences. The neural mechanisms of song based female preference in songbirds are not known. Working with female European starlings (Sturnus vulgaris), we used immunocytochemistry for ZENK protein to localize forebrain regions that respond to sexually relevant variation in conspecific male song. The number of ZENK-ir cells in ventral caudo-medial neostriatum [NCMv] was significantly higher in females exposed to longer songs than in those exposed to shorter songs, whereas variation in the total duration of song exposure yielded no significant differences in ZENK expression. ZENK expression in caudo-medial ventral hyperstriatum [cmHV] was uniformly high in all subjects, and did not vary significantly among the three groups. These results suggest that subregions of NCM in female starlings are tuned to variation in male song length, or to song features correlated therewith. Female starlings exhibit robust behavioral preferences for longer over shorter male songs (Gentner and Hulse; Anim Behav 59:443-458, 2000). Therefore, the results of this study strongly implicate NCM in at least a portion of the perceptual processes underlying the complex natural behavior of female choice.
Collapse
Affiliation(s)
- T Q Gentner
- Department of Psychology, Behavioral Neuroendocrinology Group, Johns Hopkins University, 3400 N. Charles St., Baltimore, Maryland 21218, USA.
| | | | | | | |
Collapse
|
45
|
Abstract
The ability to recognize individuals based on their vocalizations is common among many species of songbirds. Examining the psychological and neural basis of this functionally relevant behavior can provide insight into the perceptual processing of acoustically complex, real-world, communication signals. In one species of songbird, European starlings (Sturnus vulgaris), males sing long and acoustically complex songs composed of small stereotyped note clusters called motifs. Previous studies demonstrate that starlings are capable of individual vocal recognition, and suggest that vocal recognition results from the association of specific motifs with specific individuals. The present study tests this possibility by examining how variation among the motifs that comprise a song affect its discrimination and classification. Starlings were trained, using operant techniques, to associate multiple songs from a single male starling with one response, and songs from four other male starlings with another response. The level of stimulus control exerted by motif variation was then measured by having subjects classify three sets of novel song bouts in which motifs from the training songs were systematically recombined. The results demonstrate a significant, and approximately linear, relationship between song classification and the relative proportions of familiar motifs from different singers that compose a bout. The results also indicate that the motif proportion effects on song classification are primary to retroactive interference in the recall for specific motifs, and independent of any biases due to the syntactic organization of motifs within a bout. Together, the results of this study suggest that starlings organize the complex vocalizations of conspecifics by memorizing large numbers of unique song components (i.e., motifs) that are then associated with different classes. Because individual starlings tend to possess unique motif repertoires, it is likely that under natural conditions such classes will correspond to individual identity. Thus, it is likely that perceptual processing mechanisms similar to those described by the results of the present study form the basis for individual vocal recognition in starlings.
Collapse
Affiliation(s)
- T Q Gentner
- Johns Hopkins University, Department of Psychology, Baltimore, Maryland 21218, USA.
| | | |
Collapse
|
46
|
Abstract
Data from several field studies support the hypothesis that female European starlings, Sturnus vulgaris, attend to variation among the songs of conspecific males when making mate-choice decisions. However, for a variety of methodological reasons, direct evidence for female preferences based on song in starlings has been lacking. This study presents a novel technique for assaying directly female preference and choice in European starlings by using the presentation of conspecific male song as an operant reinforcer in a controlled environment. Using an apparatus in which the playback of songs from different nestboxes is under the operant control of the subject, we demonstrate how the reinforcing properties of conspecific song can be used to measure female preference and choice. The results of the study suggest three conclusions. First, female starlings prefer naturally ordered conspecific male songs over reversed songs. Second, female starlings display robust preferences for longer compared with shorter male song bouts. Behaviour in the operant apparatus varied directly with male song bout length. Third, preferences based on song bout length are sex specific. Male starlings failed to respond differentially to the same stimuli for which females showed strong preferences. These results suggest that male-male variation in song bout length is important for mate choice among starlings. In addition, we detail the use of a novel behavioural assay for measuring female preferences that can be applied to similar behaviours in other species of songbirds. Copyright 2000 The Association for the Study of Animal Behaviour.
Collapse
Affiliation(s)
- TQ Gentner
- Department of Psychology, Johns Hopkins University
| | | |
Collapse
|
47
|
Gentner TQ, Hulse SH, Bentley GE, Ball GF. Individual vocal recognition and the effect of partial lesions to HVc on discrimination, learning, and categorization of conspecific song in adult songbirds. J Neurobiol 2000; 42:117-33. [PMID: 10623906 DOI: 10.1002/(sici)1097-4695(200001)42:1<117::aid-neu11>3.0.co;2-m] [Citation(s) in RCA: 88] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Among songbirds, the capacity to associate particular songs with particular singers (i.e., vocal recognition) forms the cognitive basis for more complex communication behaviors such as female choice and territoriality. In the present study, we combine operant conditioning techniques and excitotoxic lesions to the forebrain nucleus HVc to examine the role of this region in the discrimination, associative learning, and categorization of conspecific song. We trained adult male and female European starlings, Sturnus vulgaris, to recognize simultaneously the songs of several conspecific males. Then, using a series of transfer procedures, we demonstrate that correct recognition does not generalize to song bouts containing novel motifs from familiar singers. This suggests that starlings do not make use of individually invariant source or filter characteristics for vocal recognition. We then lesioned a portion of HVc bilaterally with ibotenic acid, and exposed the birds to a series of manipulations testing the discrimination, associative learning, and categorization of conspecific song. The lesions attenuated song production among males, but retention of the basic recognition task (i.e., maintenance of the discrimination) was unaffected. However, when the response contingencies were reversed-as a test of associative learning independent of discrimination-the initial performance and subsequent learning rate were negatively correlated with the size of the HVc lesions. This suggests that HVc plays a role in the formation of associations between a song and some referent. The results of this study are discussed in light of earlier claims regarding the role of HVc in the perceptual processing of conspecific song.
Collapse
Affiliation(s)
- T Q Gentner
- Department of Psychology, Behavioral Neuroendocrinology Group, Johns Hopkins University, 3400 N. Charles Street, Baltimore, Maryland 21218, USA
| | | | | | | |
Collapse
|
48
|
Abstract
The capacity for vocal recognition of individual conspecifics is well documented in many species, but the perceptual mechanisms that underlie this ability in oscines are less well understood. Using operant conditioning, we trained three groups of European starlings on a baseline task to discriminate the songs of one male starling from those of four others. Each subject heard songs from the same five singers, but the to-be-recognized individual varied among birds. We grouped the subjects according to sex and their degree of previous exposure to the songs used as stimuli in this experiment. The first group (N=5 males) identified their own songs from those of four familiar males. The second group (N=5 males) was familiar with the song stimuli, but none of the songs was their own. The third group (N=4 females) was unfamiliar with the songs. After learning the baseline discrimination, the subjects were exposed to new natural and synthetic stimuli. The subjects maintained the ability to identify correctly an individual on the basis of novel song bouts, and showed differential responding on the basis of the sequence of song types in song bouts that were modelled using Markov chains. Based upon patterns of responding to these different stimuli, we conclude that European starlings are capable of individual vocal recognition, and that this process is mediated by mechanisms involving the memorization of individually specific song types, the sequential ordering of song types within different bouts of an individual, and perhaps by individually specific spectral (or voice) characteristics that generalize across song types. Copyright 1998 The Association for the Study of Animal Behaviour.
Collapse
Affiliation(s)
- TQ Gentner
- Department of Psychology, Johns Hopkins University
| | | |
Collapse
|
49
|
Affiliation(s)
- G F Ball
- Department of Psychology, Johns Hopkins University, Baltimore, Maryland 21218, USA
| | | |
Collapse
|
50
|
MacDougall-Shackleton SA, Hulse SH, Gentner TQ, White W. Auditory scene analysis by European starlings (Sturnus vulgaris): perceptual segregation of tone sequences. J Acoust Soc Am 1998; 103:3581-3587. [PMID: 9637040 DOI: 10.1121/1.423063] [Citation(s) in RCA: 62] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
Like humans, animals that use acoustic stimuli to perceive their world ought to be able to parse the auditory scene into functionally significant sounds. The ability to do so ought to have significant adaptive value when, for example, an animal can identify the sounds of a predator among other natural noises. In earlier work it was shown that a species of songbird, the European starling, can identify excerpts of both its own song and songs from other avian species when the songs are mixed concurrently with other natural signals. In this experiment it is demonstrated that starlings can segregate two synthetic pure-tone sequences when the sequences differ in frequency. Taken together, the experiments show that at least one nonhuman species is capable of auditory scene analysis both for natural and for non-natural acoustic stimuli. This suggests in turn that auditory scene analysis may be a general perceptual process that occurs in many species that make use of acoustic information.
Collapse
|