On reappearance and complexity in musical calling

Music is especially valued in human societies, but music-like behavior in the form of song also occurs in a variety of other animal groups including primates. The calling of our primate ancestors may well have evolved into the music of modern humans via multiple selective scenarios. But efforts to uncover these influences have been hindered by the challenge of precisely defining musical behavior in a way that could be more generally applied across species. We propose an acoustic focused reconsideration of "musicality" that could help enable independent inquiry into potential ecological pressures on the evolutionary emergence of such behavior. Using published spectrographic images (n = 832 vocalizations) from the primate vocalization literature, we developed a quantitative formulation that could be used to help recognize signatures of human-like musicality in the acoustic displays of other species. We visually scored each spectrogram along six structural features from human music-tone, interval, transposition, repetition, rhythm, and syllabic variation-and reduced this multivariate assessment into a concise measure of musical patterning, as informed by principal components analysis. The resulting acoustic reappearance diversity index (ARDI) estimates the number of different reappearing syllables within a call type. ARDI is in concordance with traditional measures of bird song complexity yet more readily identifies shorter, more subtly melodic primate vocalizations. We demonstrate the potential utility of this index by using it to corroborate several origins scenarios. When comparing ARDI scores with ecological features, our data suggest that vocalizations with diversely reappearing elements have a pronounced association with both social and environmental factors. Musical calls were moderately associated with wooded habitats and arboreal foraging, providing partial support for the acoustic adaptation hypothesis. But musical calling was most strongly associated with social monogamy, suggestive of selection for constituents of small family-sized groups by neighboring conspecifics. In sum, ARDI helps construe musical behavior along a continuum, accommodates non-human musicality, and enables gradualistic co-evolutionary paths between primate taxa-ranging from the more inhibited locational calls of archaic primates to the more exhibitional displays of modern apes.

Changes in audio-spatial working memory abilities during childhood: The role of spatial and phonological development

Working memory is a cognitive system devoted to storage and retrieval processing of information. Numerous studies on the development of working memory have investigated the processing of visuo-spatial and verbal non-spatialized information; however, little is known regarding the refinement of acoustic spatial and memory abilities across development. Here, we hypothesize that audio-spatial memory skills improve over development, due to strengthening spatial and cognitive skills such as semantic elaboration. We asked children aged 6 to 11 years old (n = 55) to pair spatialized animal calls with the corresponding animal spoken name. Spatialized sounds were emitted from an audio-haptic device, haptically explored by children with the dominant hand's index finger. Children younger than 8 anchored their exploration strategy on previously discovered sounds instead of holding this information in working memory and performed worse than older peers when asked to pair the spoken word with the corresponding animal call. In line with our hypothesis, these findings demonstrate that age-related improvements in spatial exploration and verbal coding memorization strategies affect how children learn and memorize items belonging to a complex acoustic spatial layout. Similar to vision, audio-spatial memory abilities strongly depend on cognitive development in early years of life.

Is prediction nothing more than multi-scale pattern completion of the future?

While the notion of the brain as a prediction machine has been extremely influential and productive in cognitive science, there are competing accounts of how best to model and understand the predictive capabilities of brains. One prominent framework is of a "Bayesian brain" that explicitly generates predictions and uses resultant errors to guide adaptation. We suggest that the prediction-generation component of this framework may involve little more than a pattern completion process. We first describe pattern completion in the domain of visual perception, highlighting its temporal extension, and show how this can entail a form of prediction in time. Next, we describe the forward momentum of entrained dynamical systems as a model for the emergence of predictive processing in non-predictive systems. Then, we apply this reasoning to the domain of language, where explicitly predictive models are perhaps most popular. Here, we demonstrate how a connectionist model, TRACE, exhibits hallmarks of predictive processing without any representations of predictions or errors. Finally, we present a novel neural network model, inspired by reservoir computing models, that is entirely unsupervised and memoryless, but nonetheless exhibits prediction-like behavior in its pursuit of homeostasis. These explorations demonstrate that brain-like systems can get prediction "for free," without the need to posit formal logical representations with Bayesian probabilities or an inference machine that holds them in working memory.

Phasic Off responses of auditory cortex underlie perception of sound duration

It has been proposed that sound information is separately streamed into onset and offset pathways for parallel processing. However, how offset responses contribute to auditory perception remains unclear. Here, loose-patch and whole-cell recordings in awake mouse primary auditory cortex (A1) reveal that a subset of pyramidal neurons exhibit a transient "Off" response, with its onset tightly time-locked to the sound termination and its frequency tuning similar to that of the transient "On" response. Both responses are characterized by excitation briefly followed by inhibition, with the latter mediated by parvalbumin (PV) inhibitory neurons. Optogenetically manipulating sound-evoked A1 responses at different temporal phases or artificially creating phantom sounds in A1 further reveals that the A1 phasic On and Off responses are critical for perceptual discrimination of sound duration. Our results suggest that perception of sound duration is dependent on precisely encoding its onset and offset timings by phasic On and Off responses.

Neurocognitive processing efficiency for discriminating human non-alarm rather than alarm scream calls

Across many species, scream calls signal the affective significance of events to other agents. Scream calls were often thought to be of generic alarming and fearful nature, to signal potential threats, with instantaneous, involuntary, and accurate recognition by perceivers. However, scream calls are more diverse in their affective signaling nature than being limited to fearfully alarming a threat, and thus the broader sociobiological relevance of various scream types is unclear. Here we used 4 different psychoacoustic, perceptual decision-making, and neuroimaging experiments in humans to demonstrate the existence of at least 6 psychoacoustically distinctive types of scream calls of both alarming and non-alarming nature, rather than there being only screams caused by fear or aggression. Second, based on perceptual and processing sensitivity measures for decision-making during scream recognition, we found that alarm screams (with some exceptions) were overall discriminated the worst, were responded to the slowest, and were associated with a lower perceptual sensitivity for their recognition compared with non-alarm screams. Third, the neural processing of alarm compared with non-alarm screams during an implicit processing task elicited only minimal neural signal and connectivity in perceivers, contrary to the frequent assumption of a threat processing bias of the primate neural system. These findings show that scream calls are more diverse in their signaling and communicative nature in humans than previously assumed, and, in contrast to a commonly observed threat processing bias in perceptual discriminations and neural processes, we found that especially non-alarm screams, and positive screams in particular, seem to have higher efficiency in speeded discriminations and the implicit neural processing of various scream types in humans.

Controversial stimuli: Pitting neural networks against each other as models of human cognition

Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently compare models' ability to predict human responses, we synthesize controversial stimuli: images for which different models produce distinct responses. We applied this approach to two visual recognition tasks, handwritten digits (MNIST) and objects in small natural images (CIFAR-10). For each task, we synthesized controversial stimuli to maximize the disagreement among models which employed different architectures and recognition algorithms. Human subjects viewed hundreds of these stimuli, as well as natural examples, and judged the probability of presence of each digit/object category in each image. We quantified how accurately each model predicted the human judgments. The best-performing models were a generative analysis-by-synthesis model (based on variational autoencoders) for MNIST and a hybrid discriminative-generative joint energy model for CIFAR-10. These deep neural networks (DNNs), which model the distribution of images, performed better than purely discriminative DNNs, which learn only to map images to labels. None of the candidate models fully explained the human responses. Controversial stimuli generalize the concept of adversarial examples, obviating the need to assume a ground-truth model. Unlike natural images, controversial stimuli are not constrained to the stimulus distribution models are trained on, thus providing severe out-of-distribution tests that reveal the models' inductive biases. Controversial stimuli therefore provide powerful probes of discrepancies between models and human perception.

Creating a neuroprosthesis for active tactile exploration of textures

Intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1) can produce percepts that mimic somatic sensation and, thus, has potential as an approach to sensorize prosthetic limbs. However, it is not known whether ICMS could recreate active texture exploration-the ability to infer information about object texture by using one's fingertips to scan a surface. Here, we show that ICMS of S1 can convey information about the spatial frequencies of invisible virtual gratings through a process of active tactile exploration. Two rhesus monkeys scanned pairs of visually identical screen objects with the fingertip of a hand avatar-controlled first via a joystick and later via a brain-machine interface-to find the object with denser virtual gratings. The gratings consisted of evenly spaced ridges that were signaled through individual ICMS pulses generated whenever the avatar's fingertip crossed a ridge. The monkeys learned to interpret these ICMS patterns, evoked by the interplay of their voluntary movements and the virtual textures of each object, to perform a sensory discrimination task. Discrimination accuracy followed Weber's law of just-noticeable differences (JND) across a range of grating densities; a finding that matches normal cutaneous sensation. Moreover, 1 monkey developed an active scanning strategy where avatar velocity was integrated with the ICMS pulses to interpret the texture information. We propose that this approach could equip upper-limb neuroprostheses with direct access to texture features acquired during active exploration of natural objects.

Emotionally expressed voices are retained in memory following a single exposure

Studies of voice recognition in biology suggest that long exposure may not satisfactorily represent the voice acquisition process. The current study proposes that humans can acquire a newly familiar voice from brief exposure to spontaneous speech, given a personally engaging context. Studies have shown that arousing and emotionally engaging experiences are more likely to be recorded and consolidated in memory. Yet it remains undemonstrated whether this advantage holds for voices. The present study examined the role of emotionally expressive context in the acquisition of voices following a single, 1-minute exposure by comparing recognition of voices experienced in engaging and neutral contexts at two retention intervals. Listeners were exposed to a series of emotionally nuanced and neutral videotaped narratives produced by performers, and tested on the recognition of excerpted voice samples, by indicating whether they had heard the voice before, immediately and after a one-week delay. Excerpts were voices from exposed videotaped narratives, but utilized verbal material taken from a second (nonexposed) narrative provided by the same performer. Overall, participants were consistently able to distinguish between voices that were exposed during the video session and voices that were not exposed. Voices experienced in emotional, engaging contexts were significantly better recognized than those in neutral ones both immediately and after a one-week delay. Our findings provide the first evidence that new voices can be acquired rapidly from one-time exposure and that nuanced context facilitates initially inducting new voices into a repertory of personally familiar voices in long-term memory. The results converge with neurological evidence to suggest that cerebral processes differ for familiar and unfamiliar voices.

A neural ensemble correlation code for sound category identification

Humans and other animals effortlessly identify natural sounds and categorize them into behaviorally relevant categories. Yet, the acoustic features and neural transformations that enable sound recognition and the formation of perceptual categories are largely unknown. Here, using multichannel neural recordings in the auditory midbrain of unanesthetized female rabbits, we first demonstrate that neural ensemble activity in the auditory midbrain displays highly structured correlations that vary with distinct natural sound stimuli. These stimulus-driven correlations can be used to accurately identify individual sounds using single-response trials, even when the sounds do not differ in their spectral content. Combining neural recordings and an auditory model, we then show how correlations between frequency-organized auditory channels can contribute to discrimination of not just individual sounds but sound categories. For both the model and neural data, spectral and temporal correlations achieved similar categorization performance and appear to contribute equally. Moreover, both the neural and model classifiers achieve their best task performance when they accumulate evidence over a time frame of approximately 1-2 seconds, mirroring human perceptual trends. These results together suggest that time-frequency correlations in sounds may be reflected in the correlations between auditory midbrain ensembles and that these correlations may play an important role in the identification and categorization of natural sounds.

Prepontine non-giant neurons drive flexible escape behavior in zebrafish

Many species execute ballistic escape reactions to avoid imminent danger. Despite fast reaction times, responses are often highly regulated, reflecting a trade-off between costly motor actions and perceived threat level. However, how sensory cues are integrated within premotor escape circuits remains poorly understood. Here, we show that in zebrafish, less precipitous threats elicit a delayed escape, characterized by flexible trajectories, which are driven by a cluster of 38 prepontine neurons that are completely separate from the fast escape pathway. Whereas neurons that initiate rapid escapes receive direct auditory input and drive motor neurons, input and output pathways for delayed escapes are indirect, facilitating integration of cross-modal sensory information. These results show that rapid decision-making in the escape system is enabled by parallel pathways for ballistic responses and flexible delayed actions and defines a neuronal substrate for hierarchical choice in the vertebrate nervous system.

Task-uninformative visual stimuli improve auditory spatial discrimination in humans but not the ideal observer

In order to survive and function in the world, we must understand the content of our environment. This requires us to gather and parse complex, sometimes conflicting, information. Yet, the brain is capable of translating sensory stimuli from disparate modalities into a cohesive and accurate percept with little conscious effort. Previous studies of multisensory integration have suggested that the brain’s integration of cues is well-approximated by an ideal observer implementing Bayesian causal inference. However, behavioral data from tasks that include only one stimulus in each modality fail to capture what is in nature a complex process. Here we employed an auditory spatial discrimination task in which listeners were asked to determine on which side they heard one of two concurrently presented sounds. We compared two visual conditions in which task-uninformative shapes were presented in the center of the screen, or spatially aligned with the auditory stimuli. We found that performance on the auditory task improved when the visual stimuli were spatially aligned with the auditory stimuli—even though the shapes provided no information about which side the auditory target was on. We also demonstrate that a model of a Bayesian ideal observer performing causal inference cannot explain this improvement, demonstrating that humans deviate systematically from the ideal observer model.

Listeners form average-based representations of individual voice identities

Models of voice perception propose that identities are encoded relative to an abstracted average or prototype. While there is some evidence for norm-based coding when learning to discriminate different voices, little is known about how the representation of an individual's voice identity is formed through variable exposure to that voice. In two experiments, we show evidence that participants form abstracted representations of individual voice identities based on averages, despite having never been exposed to these averages during learning. We created 3 perceptually distinct voice identities, fully controlling their within-person variability. Listeners first learned to recognise these identities based on ring-shaped distributions located around the perimeter of within-person voice spaces - crucially, these distributions were missing their centres. At test, listeners' accuracy for old/new judgements was higher for stimuli located on an untrained distribution nested around the centre of each ring-shaped distribution compared to stimuli on the trained ring-shaped distribution.

Error patterns of native and non-native listeners' perception of speech in noise

Speech perception in noise requires both bottom-up sampling of the stimulus and top-down reconstruction of the masked signal from a language model. Previous studies have provided mixed evidence about the exact role that linguistic knowledge plays in native and non-native listeners' perception of masked speech. This paper describes an analysis of whole utterance, content word, and morphosyntactic error patterns to test the prediction that non-native listeners are uniquely affected by energetic and informational masks because of limited information at multiple linguistic levels. The results reveal a consistent disadvantage for non-native listeners at all three levels in challenging listening environments.

A Gestalt inference model for auditory scene segregation

Our current understanding of how the brain segregates auditory scenes into meaningful objects is in line with a Gestaltism framework. These Gestalt principles suggest a theory of how different attributes of the soundscape are extracted then bound together into separate groups that reflect different objects or streams present in the scene. These cues are thought to reflect the underlying statistical structure of natural sounds in a similar way that statistics of natural images are closely linked to the principles that guide figure-ground segregation and object segmentation in vision. In the present study, we leverage inference in stochastic neural networks to learn emergent grouping cues directly from natural soundscapes including speech, music and sounds in nature. The model learns a hierarchy of local and global spectro-temporal attributes reminiscent of simultaneous and sequential Gestalt cues that underlie the organization of auditory scenes. These mappings operate at multiple time scales to analyze an incoming complex scene and are then fused using a Hebbian network that binds together coherent features into perceptually-segregated auditory objects. The proposed architecture successfully emulates a wide range of well established auditory scene segregation phenomena and quantifies the complimentary role of segregation and binding cues in driving auditory scene segregation.

Young children's neural processing of their mother's voice: An fMRI study

In addition to semantic content, human speech carries paralinguistic information that conveys important social cues such as a speaker's identity. For young children, their own mothers' voice is one of the most salient vocal inputs in their daily environment. Indeed, qualities of mothers' voices are shown to contribute to children's social development. Our knowledge of how the mother's voice is processed at the neural level, however, is limited. This study investigated whether the voice of a mother modulates activation in the network of regions activated by the human voice in young children differently than the voice of an unfamiliar mother. We collected fMRI data from 32 typically developing 7- and 8-year-olds as they listened to natural speech produced by their mother and another child's mother. We used emotionally-varied natural speech stimuli to approximate the range of children's day-to-day experience. We individually-defined functional ROIs in children's voice-sensitive neural network and then independently investigated the extent to which activation in these regions is modulated by speaker identity. The bilateral posterior auditory cortex, superior temporal gyrus (STG), and inferior frontal gyrus (IFG) exhibit enhanced activation in response to the voice of one's own mother versus that of an unfamiliar mother. The findings indicate that children process the voice of their own mother uniquely, and pave the way for future studies of how social information processing contributes to the trajectory of child social development.

Representations of microgeometric tactile information during object recognition

Object recognition through tactile perception involves two elements: the shape of the object (macrogeometric properties) and the material of the object (microgeometric properties). Here we sought to determine the characteristics of microgeometric tactile representations regarding object recognition through tactile perception. Participants were directed to recognize objects with different surface materials using either tactile information or visual information. With a quantitative analysis of the cognitive process regarding object recognition, Experiment 1 confirmed the same eight concepts (composed of rules defining distinct cognitive processes) commonly generated in both tactile and visual perceptions to accomplish the task, although an additional concept was generated during the visual task. Experiment 2 focused only on tactile perception. Three tactile objects with different surface materials (plastic, cloth and sandpaper) were used for the object recognition task. The participants answered a questionnaire regarding the process leading to their answers (which was designed based on the results obtained in Experiment 1) and to provide ratings on the vividness, familiarity and affective valence. We used these experimental data to investigate whether changes in material attributes (tactile information) change the characteristics of tactile representation. The observation showed that differences in tactile information resulted in differences in cognitive processes, vividness, familiarity and emotionality. These two experiments collectively indicated that microgeometric tactile information contributes to object recognition by recruiting various cognitive processes including episodic memory and emotion, similar to the case of object recognition by visual information.

Fast neural learning in dogs: A multimodal sensory fMRI study

Dogs may follow their nose, but they learn associations to many types of sensory stimuli. Are some modalities learned better than others? We used awake fMRI in 19 dogs over a series of three experiments to measure reward-related learning of visual, olfactory, and verbal stimuli. Neurobiological learning curves were generated for individual dogs by measuring activation over time within three regions of interest: the caudate nucleus, amygdala, and parietotemporal cortex. The learning curves showed that dogs formed stimulus-reward associations in as little as 22 trials. Consistent with neuroimaging studies of associative learning, the caudate showed a main effect for reward-related stimuli, but not a significant interaction with modality. However, there were significant differences in the time courses, suggesting that although multiple modalities are represented in the caudate, the rates of acquisition and habituation are modality-dependent and are potentially gated by their salience in the amygdala. Visual and olfactory modalities resulted in the fastest learning, while verbal stimuli were least effective, suggesting that verbal commands may be the least efficient way to train dogs.

Attentional responses on an auditory oddball predict false memory susceptibility

Attention and memory are highly integrated processes. Building on prior behavioral investigations, this study assesses the link between individual differences in low-level neural attentional responding and false memory susceptibility on the misinformation effect, a paradigm in which false event memories are induced via misleading post-event information. Twenty-four subjects completed the misinformation effect paradigm after which high-density (256-channel) EEG data was collected as they engaged in an auditory oddball task. Temporal-spatial decomposition was used to extract two attention-related components from the oddball data, the P3b and Classic Slow Wave. The P3b was utilized as an index of individual differences in salient target attentional responding while the slow wave was adopted as an index of variability in task-level sustained attention. Analyses of these components show a significant negative relationship between slow-wave responses to oddball non-targets and perceptual false memory endorsements, suggestive of a link between individual differences in levels of sustained attention and false memory susceptibility. These findings provide the first demonstrated link between individual differences in basic attentional responses and false memory. These results support prior behavioral work linking attention and false memory and highlight the integration between attentional processes and real-world episodic memory.

Alpha and beta band correlates of haptic perceptual grouping: Results from an orientation detection task

Behavioral and neurophysiological findings in vision suggest that perceptual grouping is not a unitary process and that different grouping principles have different processing requirements and neural correlates. The present study aims to examine whether the same occurs in the haptic modality using two grouping principles widely studied in vision, spatial proximity and texture similarity. We analyzed behavioral responses (accuracy and response times) and conducted an independent component analysis of brain oscillations in alpha and beta bands for haptic stimuli grouped by spatial proximity and texture similarity, using a speeded orientation detection task performed on a novel haptic device (MonHap). Behavioral results showed faster response times for patterns grouped by spatial proximity relative to texture similarity. Independent component clustering analysis revealed the activation of a bilateral network of sensorimotor and parietal areas while performing the task. We conclude that, as occurs in visual perception, grouping the elements of the haptic scene by means of their spatial proximity is faster than forming the same objects by means of texture similarity. In addition, haptic grouping seems to involve the activation of a network of widely distributed bilateral sensorimotor and parietal areas as reflected by the consistent event-related desynchronization found in alpha and beta bands.

Explaining individual variation in paternal brain responses to infant cries

Crying is the principal means by which newborn infants shape parental behavior to meet their needs. While this mechanism can be highly effective, infant crying can also be an aversive stimulus that leads to parental frustration and even abuse. Fathers have recently become more involved in direct caregiving activities in modern, developed nations, and fathers are more likely than mothers to physically abuse infants. In this study, we attempt to explain variation in the neural response to infant crying among human fathers, with the hope of identifying factors that are associated with a more or less sensitive response. We imaged brain function in 39 first-time fathers of newborn infants as they listened to both their own and a standardized unknown infant cry stimulus, as well as auditory control stimuli, and evaluated whether these neural responses were correlated with measured characteristics of fathers and infants that were hypothesized to modulate these responses. Fathers also provided subjective ratings of each cry stimulus on multiple dimensions. Fathers showed widespread activation to both own and unknown infant cries in neural systems involved in empathy and approach motivation. There was no significant difference in the neural response to the own vs. unknown infant cry, and many fathers were unable to distinguish between the two cries. Comparison of these results with previous studies in mothers revealed a high degree of similarity between first-time fathers and first-time mothers in the pattern of neural activation to newborn infant cries. Further comparisons suggested that younger infant age was associated with stronger paternal neural responses, perhaps due to hormonal or novelty effects. In our sample, older fathers found infant cries less aversive and had an attenuated response to infant crying in both the dorsal anterior cingulate cortex (dACC) and the anterior insula, suggesting that compared with younger fathers, older fathers may be better able to avoid the distress associated with empathic over-arousal in response to infant cries. A principal components analysis revealed that fathers with more negative emotional reactions to the unknown infant cry showed decreased activation in the thalamus and caudate nucleus, regions expected to promote positive parental behaviors, as well as increased activation in the hypothalamus and dorsal ACC, again suggesting that empathic over-arousal might result in negative emotional reactions to infant crying. In sum, our findings suggest that infant age, paternal age and paternal emotional reactions to infant crying all modulate the neural response of fathers to infant crying. By identifying neural correlates of variation in paternal subjective reactions to infant crying, these findings help lay the groundwork for evaluating the effectiveness of interventions designed to increase paternal sensitivity and compassion.

The Impact of Age, Background Noise, Semantic Ambiguity, and Hearing Loss on Recognition Memory for Spoken Sentences

PURPOSE

The goal of this study was to determine how background noise, linguistic properties of spoken sentences, and listener abilities (hearing sensitivity and verbal working memory) affect cognitive demand during auditory sentence comprehension.

METHOD

We tested 30 young adults and 30 older adults. Participants heard lists of sentences in quiet and in 8-talker babble at signal-to-noise ratios of +15 dB and +5 dB, which increased acoustic challenge but left the speech largely intelligible. Half of the sentences contained semantically ambiguous words to additionally manipulate cognitive challenge. Following each list, participants performed a visual recognition memory task in which they viewed written sentences and indicated whether they remembered hearing the sentence previously.

RESULTS

Recognition memory (indexed by d') was poorer for acoustically challenging sentences, poorer for sentences containing ambiguous words, and differentially poorer for noisy high-ambiguity sentences. Similar patterns were observed for Z-transformed response time data. There were no main effects of age, but age interacted with both acoustic clarity and semantic ambiguity such that older adults' recognition memory was poorer for acoustically degraded high-ambiguity sentences than the young adults'. Within the older adult group, exploratory correlation analyses suggested that poorer hearing ability was associated with poorer recognition memory for sentences in noise, and better verbal working memory was associated with better recognition memory for sentences in noise.

CONCLUSIONS

Our results demonstrate listeners' reliance on domain-general cognitive processes when listening to acoustically challenging speech, even when speech is highly intelligible. Acoustic challenge and semantic ambiguity both reduce the accuracy of listeners' recognition memory for spoken sentences.

SUPPLEMENTAL MATERIALS

https://doi.org/10.23641/asha.5848059.

Orthographic influence on spoken word identification: Behavioral and fMRI evidence

The current study investigated behavioral and neuroimaging evidence for orthographic influences on auditory word identification. To assess such influences, the proportion of similar sounding words (i.e. phonological neighbors) that were also spelled similarly (i.e., orthographic neighbors) was computed for each auditorily presented word as the Orthographic-to-Phonological Overlap Ratio (OPOR). Speech intelligibility was manipulated by presenting monosyllabic words in multi-talker babble at two signal-to-noise ratios: + 3 and + 10 dB SNR. Identification rates were lower for high overlap words in the challenging + 3 dB SNR condition. In addition, BOLD contrast increased with OPOR at the more difficult SNR, and decreased with OPOR under more favorable SNR conditions. Both voxel-based and region of interest analyses demonstrated robust effects of OPOR in several cingulo-opercular regions. However, contrary to prior theoretical accounts, no task-related activity was observed in posterior regions associated with phonological or orthographic processing. We suggest that, when processing is difficult, orthographic-to-phonological feature overlap increases the availability of competing responses, which then requires additional support from domain general performance systems in order to produce a single response.

Statistical Learning of Melodic Patterns Influences the Brain's Response to Wrong Notes

The theory of statistical learning has been influential in providing a framework for how humans learn to segment patterns of regularities from continuous sensory inputs, such as speech and music. This form of learning is based on statistical cues and is thought to underlie the ability to learn to segment patterns of regularities from continuous sensory inputs, such as the transition probabilities in speech and music. However, the connection between statistical learning and brain measurements is not well understood. Here we focus on ERPs in the context of tone sequences that contain statistically cohesive melodic patterns. We hypothesized that implicit learning of statistical regularities would influence what was held in auditory working memory. We predicted that a wrong note occurring within a cohesive pattern (within-pattern deviant) would lead to a significantly larger brain signal than a wrong note occurring between cohesive patterns (between-pattern deviant), even though both deviant types were equally likely to occur with respect to the global tone sequence. We discuss this prediction within a simple Markov model framework that learns the transition probability regularities within the tone sequence. Results show that signal strength was stronger when cohesive patterns were violated and demonstrate that the transitional probability of the sequence influences the memory basis for melodic patterns. Our results thus characterize how informational units are stored in auditory memory trace for deviance detection and provide new evidence about how the brain organizes sequential sound input that is useful for perception.

Music viewed by its entropy content: A novel window for comparative analysis

Polyphonic music files were analyzed using the set of symbols that produced the Minimal Entropy Description, which we call the Fundamental Scale. This allowed us to create a novel space to represent music pieces by developing: (a) a method to adjust a textual description from its original scale of observation to an arbitrarily selected scale, (b) a method to model the structure of any textual description based on the shape of the symbol frequency profiles, and (c) the concept of higher order entropy as the entropy associated with the deviations of a frequency-ranked symbol profile from a perfect Zipfian profile. We call this diversity index the ‘2nd Order Entropy’. Applying these methods to a variety of musical pieces showed how the space of ‘symbolic specific diversity-entropy’ and that of ‘2nd order entropy’ captures characteristics that are unique to each music type, style, composer and genre. Some clustering of these properties around each musical category is shown. These methods allow us to visualize a historic trajectory of academic music across this space, from medieval to contemporary academic music. We show that the description of musical structures using entropy, symbol frequency profiles and specific symbolic diversity allows us to characterize traditional and popular expressions of music. These classification techniques promise to be useful in other disciplines for pattern recognition and machine learning.

Auditory object perception: A neurobiological model and prospective review

Interaction with the world is a multisensory experience, but most of what is known about the neural correlates of perception comes from studying vision. Auditory inputs enter cortex with its own set of unique qualities, and leads to use in oral communication, speech, music, and the understanding of emotional and intentional states of others, all of which are central to the human experience. To better understand how the auditory system develops, recovers after injury, and how it may have transitioned in its functions over the course of hominin evolution, advances are needed in models of how the human brain is organized to process real-world natural sounds and "auditory objects". This review presents a simple fundamental neurobiological model of hearing perception at a category level that incorporates principles of bottom-up signal processing together with top-down constraints of grounded cognition theories of knowledge representation. Though mostly derived from human neuroimaging literature, this theoretical framework highlights rudimentary principles of real-world sound processing that may apply to most if not all mammalian species with hearing and acoustic communication abilities. The model encompasses three basic categories of sound-source: (1) action sounds (non-vocalizations) produced by 'living things', with human (conspecific) and non-human animal sources representing two subcategories; (2) action sounds produced by 'non-living things', including environmental sources and human-made machinery; and (3) vocalizations ('living things'), with human versus non-human animals as two subcategories therein. The model is presented in the context of cognitive architectures relating to multisensory, sensory-motor, and spoken language organizations. The models' predictive values are further discussed in the context of anthropological theories of oral communication evolution and the neurodevelopment of spoken language proto-networks in infants/toddlers. These phylogenetic and ontogenetic frameworks both entail cortical network maturations that are proposed to at least in part be organized around a number of universal acoustic-semantic signal attributes of natural sounds, which are addressed herein.

Noise correlations in the human brain and their impact on pattern classification

Multivariate decoding methods, such as multivoxel pattern analysis (MVPA), are highly effective at extracting information from brain imaging data. Yet, the precise nature of the information that MVPA draws upon remains controversial. Most current theories emphasize the enhanced sensitivity imparted by aggregating across voxels that have mixed and weak selectivity. However, beyond the selectivity of individual voxels, neural variability is correlated across voxels, and such noise correlations may contribute importantly to accurate decoding. Indeed, a recent computational theory proposed that noise correlations enhance multivariate decoding from heterogeneous neural populations. Here we extend this theory from the scale of neurons to functional magnetic resonance imaging (fMRI) and show that noise correlations between heterogeneous populations of voxels (i.e., voxels selective for different stimulus variables) contribute to the success of MVPA. Specifically, decoding performance is enhanced when voxels with high vs. low noise correlations (measured during rest or in the background of the task) are selected during classifier training. Conversely, voxels that are strongly selective for one class in a GLM or that receive high classification weights in MVPA tend to exhibit high noise correlations with voxels selective for the other class being discriminated against. Furthermore, we use simulations to show that this is a general property of fMRI data and that selectivity and noise correlations can have distinguishable influences on decoding. Taken together, our findings demonstrate that if there is signal in the data, the resulting above-chance classification accuracy is modulated by the magnitude of noise correlations.

A central challenge in cognitive neuroscience is decoding mental representations from patterns of brain activity. With functional magnetic resonance imaging (fMRI), multivariate decoding methods like multivoxel pattern analysis (MVPA) have produced numerous discoveries about the brain. However, what information these methods draw upon remains the subject of debate. Typically, each voxel is thought to contribute information through its selectivity (i.e., how differently it responds to the classes being decoded), with improved sensitivity reflecting the aggregation of selectivity across voxels. We show that this interpretation downplays an important factor: MVPA is also highly attuned to noise correlations between voxels with opposite selectivity. Across several analyses of an fMRI dataset, we demonstrate a positive relationship between the magnitude of noise correlations and multivariate decoding performance. Indeed, voxels more selective for one class, or heavily weighted in MVPA, tend to be more strongly correlated with voxels selective for the opposite class. Furthermore, using a model to simulate different levels of selectivity and noise correlations, we find that the benefit of noise correlations for decoding is a general property of fMRI data. These findings help elucidate the computational underpinnings of multivariate decoding in cognitive neuroscience and provide insight into the nature of neural representations.

Phonological Working Memory for Words and Nonwords in Cerebral Cortex

PURPOSE

The primary purpose of this study was to identify the brain bases of phonological working memory (the short-term maintenance of speech sounds) using behavioral tasks analogous to clinically sensitive assessments of nonword repetition. The secondary purpose of the study was to identify how individual differences in brain activation were related to participants' nonword repetition abilities.

METHOD

We used functional magnetic resonance imaging to measure neurophysiological response during a nonword discrimination task derived from standard clinical assessments of phonological working memory. Healthy adult control participants (N = 16) discriminated pairs of real words or nonwords under varying phonological working memory load, which we manipulated by parametrically varying the number of syllables in target (non)words. Participants' cognitive and phonological abilities were also measured using standardized assessments.

RESULTS

Neurophysiological responses in bilateral superior temporal gyrus, inferior frontal gyrus, and supplementary motor area increased with greater phonological working memory load. Activation in left superior temporal gyrus during nonword discrimination correlated with participants' performance on standard clinical nonword repetition tests.

CONCLUSION

These results suggest that phonological working memory is related to the function of cortical structures that canonically underlie speech perception and production.

Electrophysiology of Perception and Processing of Phonological Information as Indices of Toddlers' Language Performance

PURPOSE

The toddler years are a critical period for language development and growth. We investigated how event-related potentials (ERPs) to repeated and novel nonwords are associated with clinical assessments of language in young children. In addition, nonword repetition (NWR) was used to measure phonological working memory to determine the unique and collective contribution of ERP measures of phonemic discrimination and NWR as predictors of language ability.

METHOD

Forty children between the ages of 24-48 months participated in an ERP experiment to determine phonemic discrimination to repeated and novel nonwords in an old/new design. Participants also completed a NWR task to explore the contribution of phonological working memory in predicting language.

RESULTS

ERP analyses revealed that faster responses to novel stimuli correlated with higher language performance on clinical assessments of language. Regression analyses revealed that an earlier component was associated with lower level phonemic sensitivity, and a later component was indexing phonological working memory skills similar to NWR.

CONCLUSION

Our findings suggest that passive ERP responses indexing phonological discrimination and phonological working memory are strongly related to behavioral measures of language.

Discrete call types referring to predation risk enhance the efficiency of the meerkat sentinel system

Sentinel behaviour, a form of coordinated vigilance, occurs in a limited range of species, mostly in cooperative breeders. In some species sentinels confirm their presence vocally by giving a single sentinel call type, whereby the rate and subtle acoustic changes provide graded information on the variation of perceived predation risk. In contrast, meerkat (Suricata suricatta) sentinels produce six different sentinel call types. Here we show that manipulation of perception of danger has different effects on the likelihood of emitting these different call types, and that these call types affect foraging individuals differently. Increasing the perceived predation risk by playing back alarm calls decreased the production rate of the common short note calls and increased the production rate of the rare long calls. Playbacks of short note calls increased foraging behaviour and decreased vigilance in the rest of the group, whereas the opposite was observed when playing long calls. This suggests that the common call types act as all-clear signals, while the rare call types have a warning function. Therefore, meerkats increase the efficiency of their sentinel system by producing several discrete call types that represent changes in predation risk and lead to adjustments of the group's vigilance behaviour.

Building better biomarkers: brain models in translational neuroimaging

Despite its great promise, neuroimaging has yet to substantially impact clinical practice and public health. However, a developing synergy between emerging analysis techniques and data-sharing initiatives has the potential to transform the role of neuroimaging in clinical applications. We review the state of translational neuroimaging and outline an approach to developing brain signatures that can be shared, tested in multiple contexts and applied in clinical settings. The approach rests on three pillars: (i) the use of multivariate pattern-recognition techniques to develop brain signatures for clinical outcomes and relevant mental processes; (ii) assessment and optimization of their diagnostic value; and (iii) a program of broad exploration followed by increasingly rigorous assessment of generalizability across samples, research contexts and populations. Increasingly sophisticated models based on these principles will help to overcome some of the obstacles on the road from basic neuroscience to better health and will ultimately serve both basic and applied goals.

Painted Goby Larvae under High-CO2 Fail to Recognize Reef Sounds

Atmospheric CO₂ levels have been increasing at an unprecedented rate due to anthropogenic activity. Consequently, ocean pCO₂ is increasing and pH decreasing, affecting marine life, including fish. For many coastal marine fishes, selection of the adult habitat occurs at the end of the pelagic larval phase. Fish larvae use a range of sensory cues, including sound, for locating settlement habitat. This study tested the effect of elevated CO₂ on the ability of settlement-stage temperate fish to use auditory cues from adult coastal reef habitats. Wild late larval stages of painted goby (Pomatoschistus pictus) were exposed to control pCO₂ (532 μatm, pH 8.06) and high pCO₂ (1503 μatm, pH 7.66) conditions, likely to occur in nearshore regions subjected to upwelling events by the end of the century, and tested in an auditory choice chamber for their preference or avoidance to nighttime reef recordings. Fish reared in control pCO₂ conditions discriminated reef soundscapes and were attracted by reef recordings. This behaviour changed in fish reared in the high CO₂ conditions, with settlement-stage larvae strongly avoiding reef recordings. This study provides evidence that ocean acidification might affect the auditory responses of larval stages of temperate reef fish species, with potentially significant impacts on their survival.

EEG signatures accompanying auditory figure-ground segregation

In everyday acoustic scenes, figure-ground segregation typically requires one to group together sound elements over both time and frequency. Electroencephalogram was recorded while listeners detected repeating tonal complexes composed of a random set of pure tones within stimuli consisting of randomly varying tonal elements. The repeating pattern was perceived as a figure over the randomly changing background. It was found that detection performance improved both as the number of pure tones making up each repeated complex (figure coherence) increased, and as the number of repeated complexes (duration) increased - i.e., detection was easier when either the spectral or temporal structure of the figure was enhanced. Figure detection was accompanied by the elicitation of the object related negativity (ORN) and the P400 event-related potentials (ERPs), which have been previously shown to be evoked by the presence of two concurrent sounds. Both ERP components had generators within and outside of auditory cortex. The amplitudes of the ORN and the P400 increased with both figure coherence and figure duration. However, only the P400 amplitude correlated with detection performance. These results suggest that 1) the ORN and P400 reflect processes involved in detecting the emergence of a new auditory object in the presence of other concurrent auditory objects; 2) the ORN corresponds to the likelihood of the presence of two or more concurrent sound objects, whereas the P400 reflects the perceptual recognition of the presence of multiple auditory objects and/or preparation for reporting the detection of a target object.

Abnormal Complex Auditory Pattern Analysis in Schizophrenia Reflected in an Absent Missing Stimulus Mismatch Negativity

The simple mismatch negativity (MMN) to tones deviating physically (in pitch, loudness, duration, etc.) from repeated standard tones is robustly reduced in schizophrenia. Although generally interpreted to reflect memory or cognitive processes, simple MMN likely contains some activity from non-adapted sensory cells, clouding what process is affected in schizophrenia. Research in healthy participants has demonstrated that MMN can be elicited by deviations from abstract auditory patterns and complex rules that do not cause sensory adaptation. Whether persons with schizophrenia show abnormalities in the complex MMN is unknown. Fourteen schizophrenia participants and 16 matched healthy underwent EEG recording while listening to 400 groups of 6 tones 330 ms apart, separated by 800 ms. Occasional deviant groups were missing the 4th or 6th tone (50 groups each). Healthy participants generated a robust response to a missing but expected tone. The schizophrenia group was significantly impaired in activating the missing stimulus MMN, generating no significant activity at all. Schizophrenia affects the ability of "primitive sensory intelligence" and pre-attentive perceptual mechanisms to form implicit groups in the auditory environment. Importantly, this deficit must relate to abnormalities in abstract complex pattern analysis rather than sensory problems in the disorder. The results indicate a deficit in parsing of the complex auditory scene which likely impacts negatively on successful social navigation in schizophrenia. Knowledge of the location and circuit architecture underlying the true novelty-related MMN and its pathophysiology in schizophrenia will help target future interventions.

Unsupervised Learning in an Ensemble of Spiking Neural Networks Mediated by ITDP

We propose a biologically plausible architecture for unsupervised ensemble learning in a population of spiking neural network classifiers. A mixture of experts type organisation is shown to be effective, with the individual classifier outputs combined via a gating network whose operation is driven by input timing dependent plasticity (ITDP). The ITDP gating mechanism is based on recent experimental findings. An abstract, analytically tractable model of the ITDP driven ensemble architecture is derived from a logical model based on the probabilities of neural firing events. A detailed analysis of this model provides insights that allow it to be extended into a full, biologically plausible, computational implementation of the architecture which is demonstrated on a visual classification task. The extended model makes use of a style of spiking network, first introduced as a model of cortical microcircuits, that is capable of Bayesian inference, effectively performing expectation maximization. The unsupervised ensemble learning mechanism, based around such spiking expectation maximization (SEM) networks whose combined outputs are mediated by ITDP, is shown to perform the visual classification task well and to generalize to unseen data. The combined ensemble performance is significantly better than that of the individual classifiers, validating the ensemble architecture and learning mechanisms. The properties of the full model are analysed in the light of extensive experiments with the classification task, including an investigation into the influence of different input feature selection schemes and a comparison with a hierarchical STDP based ensemble architecture.

Fetal Development: Voice Processing in Normotensive and Hypertensive Pregnancies

Recent observation of maternal voice recognition provides evidence of rudimentary memory and learning in healthy term fetuses. However, such higher order auditory processing has not been examined in the presence of maternal hypertension, which is associated with reduced and/or impaired uteroplacental blood flow. In this study, voice processing was examined in 40 fetuses (gestational ages of 33 to 41 weeks) of hypertensive and normotensive women. Fetuses received 2 min of no sound, 2 min of a tape-recorded story read by their mothers or by a female stranger, and 2 min of no sound while fetal heart rate was recorded. Results demonstrated that fetuses in the normotensive group had heart rate accelerations during the playing of their mother's voice, whereas the response occurred in the hypertensive group following maternal voice offset. Across all fetuses, a greater fetal heart rate change was observed when the amniotic fluid index was above compared to below the median (i.e., 150 mm), indicating that amniotic fluid volume may be an independent moderator of fetal auditory sensitivity. It was concluded that differential fetal responding to the mother's voice in pregnancies complicated by maternal hypertension may reflect functional elevation of sensorineural threshold or a delay in auditory system maturation, signifying functional differences during fetal life or subtle differences in the development of the central nervous system.

[The Effect of Cognitive Load on the Development of Muscle Fatigue]

The aim of our study was to examine the effect of cognitive load on the development of muscle fatigue. Sub- jects performed 250 maximal compressions of a handgrip dynamometer in response to the target acoustic stimulus. Stimuli were presented in three paradigms with different levels of cognitive load: (1) the reaction task with one type of signals (2) the acoustic stimuli recognition task with the 50% target and 50% non-target stimuli, where voluntary attention was involved, (3) the recognition task with the 17% target and 83% non tar- get stimuli, that led to the activation of voluntary and involuntary attention systems. As fatigue indicators amplitude of compression force in response to the target stimulus, the maximum voluntary contraction and rating of perceived exertion were used. Our results showed the reduced level of fatigue in experiment with the activation of voluntary and involuntary attention systems. Moreover, the activation of involuntary attention, accompanied by mismatch negativity in the EEG, enhanced the effect of voluntary attention.

Head-Mounted Sensory Augmentation Device: Designing a Tactile Language

Sensory augmentation operates by synthesizing new information then displaying it through an existing sensory channel and can be used to help people with impaired sensing or to assist in tasks where sensory information is limited or sparse, for example, when navigating in a low visibility environment. This paper presents the design of a 2nd generation head-mounted vibrotactile interface as a sensory augmentation prototype designed to present navigation commands that are intuitive, informative, and minimize information overload. We describe an experiment in a structured environment in which the user navigates along a virtual wall whilst the position and orientation of the user's head is tracked in real time by a motion capture system. Navigation commands in the form of vibrotactile feedback are presented according to the user's distance from the virtual wall and their head orientation. We test the four possible combinations of two command presentation modes (continuous, discrete) and two command types (recurring, single). We evaluated the effectiveness of this 'tactile language' according to the users' walking speed and the smoothness of their trajectory parallel to the virtual wall. Results showed that recurring continuous commands allowed users to navigate with lowest route deviation and highest walking speed. In addition, subjects preferred recurring continuous commands over other commands.

Monkey׳s short-term auditory memory nearly abolished by combined removal of the rostral superior temporal gyrus and rhinal cortices

While monkeys easily acquire the rules for performing visual and tactile delayed matching-to-sample, a method for testing recognition memory, they have extraordinary difficulty acquiring a similar rule in audition. Another striking difference between the modalities is that whereas bilateral ablation of the rhinal cortex (RhC) leads to profound impairment in visual and tactile recognition, the same lesion has no detectable effect on auditory recognition memory (Fritz et al., 2005). In our previous study, a mild impairment in auditory memory was obtained following bilateral ablation of the entire medial temporal lobe (MTL), including the RhC, and an equally mild effect was observed after bilateral ablation of the auditory cortical areas in the rostral superior temporal gyrus (rSTG). In order to test the hypothesis that each of these mild impairments was due to partial disconnection of acoustic input to a common target (e.g., the ventromedial prefrontal cortex), in the current study we examined the effects of a more complete auditory disconnection of this common target by combining the removals of both the rSTG and the MTL. We found that the combined lesion led to forgetting thresholds (performance at 75% accuracy) that fell precipitously from the normal retention duration of ~30 to 40s to a duration of ~1 to 2s, thus nearly abolishing auditory recognition memory, and leaving behind only a residual echoic memory. This article is part of a Special Issue entitled SI: Auditory working memory.

Neural Mechanisms Behind Identification of Leptokurtic Noise and Adaptive Behavioral Response

Large-scale human interaction through, for example, financial markets causes ceaseless random changes in outcome variability, producing frequent and salient outliers that render the outcome distribution more peaked than the Gaussian distribution, and with longer tails. Here, we study how humans cope with this evolutionary novel leptokurtic noise, focusing on the neurobiological mechanisms that allow the brain, 1) to recognize the outliers as noise and 2) to regulate the control necessary for adaptive response. We used functional magnetic resonance imaging, while participants tracked a target whose movements were affected by leptokurtic noise. After initial overreaction and insufficient subsequent correction, participants improved performance significantly. Yet, persistently long reaction times pointed to continued need for vigilance and control. We ran a contrasting treatment where outliers reflected permanent moves of the target, as in traditional mean-shift paradigms. Importantly, outliers were equally frequent and salient. There, control was superior and reaction time was faster. We present a novel reinforcement learning model that fits observed choices better than the Bayes-optimal model. Only anterior insula discriminated between the 2 types of outliers. In both treatments, outliers initially activated an extensive bottom-up attention and belief network, followed by sustained engagement of the fronto-parietal control network.

Corruption of the dentate gyrus by "dominant" granule cells: Implications for dentate gyrus function in health and disease

The dentate gyrus (DG) and area CA3 of the hippocampus are highly organized lamellar structures which have been implicated in specific cognitive functions such as pattern separation and pattern completion. Here we describe how the anatomical organization and physiology of the DG and CA3 are consistent with structures that perform pattern separation and completion. We then raise a new idea related to the complex circuitry of the DG and CA3 where CA3 pyramidal cell 'backprojections' play a potentially important role in the sparse firing of granule cells (GCs), considered important in pattern separation. We also propose that GC axons, the mossy fibers, already known for their highly specialized structure, have a dynamic function that imparts variance--'mossy fiber variance'--which is important to pattern separation and completion. Computational modeling is used to show that when a subset of GCs become 'dominant,' one consequence is loss of variance in the activity of mossy fiber axons and a reduction in pattern separation and completion in the model. Empirical data are then provided using an example of 'dominant' GCs--subsets of GCs that develop abnormally and have increased excitability. Notably, these abnormal GCs have been identified in animal models of disease where DG-dependent behaviors are impaired. Together these data provide insight into pattern separation and completion, and suggest that behavioral impairment could arise from dominance of a subset of GCs in the DG-CA3 network.

Evidence for a Caregiving Instinct: Rapid Differentiation of Infant from Adult Vocalizations Using Magnetoencephalography

Crying is the most salient vocal signal of distress. The cries of a newborn infant alert adult listeners and often elicit caregiving behavior. For the parent, rapid responding to an infant in distress is an adaptive behavior, functioning to ensure offspring survival. The ability to react rapidly requires quick recognition and evaluation of stimuli followed by a co-ordinated motor response. Previous neuroimaging research has demonstrated early specialized activity in response to infant faces. Using magnetoencephalography, we found similarly early (100-200 ms) differences in neural responses to infant and adult cry vocalizations in auditory, emotional, and motor cortical brain regions. We propose that this early differential activity may help to rapidly identify infant cries and engage affective and motor neural circuitry to promote adaptive behavioral responding, before conscious awareness. These differences were observed in adults who were not parents, perhaps indicative of a universal brain-based "caregiving instinct."

Recognition of Frequency Modulated Whistle-Like Sounds by a Bottlenose Dolphin (Tursiops truncatus) and Humans with Transformations in Amplitude, Duration and Frequency

Bottlenose dolphins (Tursiops truncatus) use the frequency contour of whistles produced by conspecifics for individual recognition. Here we tested a bottlenose dolphin's (Tursiops truncatus) ability to recognize frequency modulated whistle-like sounds using a three alternative matching-to-sample paradigm. The dolphin was first trained to select a specific object (object A) in response to a specific sound (sound A) for a total of three object-sound associations. The sounds were then transformed by amplitude, duration, or frequency transposition while still preserving the frequency contour of each sound. For comparison purposes, 30 human participants completed an identical task with the same sounds, objects, and training procedure. The dolphin's ability to correctly match objects to sounds was robust to changes in amplitude with only a minor decrement in performance for short durations. The dolphin failed to recognize sounds that were frequency transposed by plus or minus ½ octaves. Human participants demonstrated robust recognition with all acoustic transformations. The results indicate that this dolphin's acoustic recognition of whistle-like sounds was constrained by absolute pitch. Unlike human speech, which varies considerably in average frequency, signature whistles are relatively stable in frequency, which may have selected for a whistle recognition system invariant to frequency transposition.

The representational-hierarchical view of pattern separation: Not just hippocampus, not just space, not just memory?

Pattern separation (PS) has been defined as a process of reducing overlap between similar input patterns to minimize interference amongst stored representations. The present article describes this putative PS process from the "representational-hierarchical" perspective (R-H), which uses a hierarchical continuum instead of a cognitive modular processing framework to describe the organization of the ventral visual perirhinal-hippocampal processing stream. Instead of trying to map psychological constructs onto anatomical modules in the brain, the R-H model suggests that the function of brain regions depends upon what representations they contain. We begin by discussing a main principle of the R-H framework, the resolution of "ambiguity" of lower level representations via the formation of unique conjunctive representations in higher level areas, and how this process is remarkably similar to definitions of PS. Work from several species and experimental approaches suggest that this principle of resolution of ambiguity via conjunctive representations has considerable explanatory power, leads to wide possibilities for experimentation, and also supports some perhaps surprising conclusions.

Natural Vocalizations in the Mammalian Inferior Colliculus are Broadly Encoded by a Small Number of Independent Multi-Units

How complex natural sounds are represented by the main converging center of the auditory midbrain, the central inferior colliculus, is an open question. We applied neural discrimination to determine the variation of detailed encoding of individual vocalizations across the best frequency gradient of the central inferior colliculus. The analysis was based on collective responses from several neurons. These multi-unit spike trains were recorded from guinea pigs exposed to a spectrotemporally rich set of eleven species-specific vocalizations. Spike trains of disparate units from the same recording were combined in order to investigate whether groups of multi-unit clusters represent the whole set of vocalizations more reliably than only one unit, and whether temporal response correlations between them facilitate an unambiguous neural representation of the vocalizations. We found a spatial distribution of the capability to accurately encode groups of vocalizations across the best frequency gradient. Different vocalizations are optimally discriminated at different locations of the best frequency gradient. Furthermore, groups of a few multi-unit clusters yield improved discrimination over only one multi-unit cluster between all tested vocalizations. However, temporal response correlations between units do not yield better discrimination. Our study is based on a large set of units of simultaneously recorded responses from several guinea pigs and electrode insertion positions. Our findings suggest a broadly distributed code for behaviorally relevant vocalizations in the mammalian inferior colliculus. Responses from a few non-interacting units are sufficient to faithfully represent the whole set of studied vocalizations with diverse spectrotemporal properties.

An Energy-Efficient and Scalable Deep Learning/Inference Processor With Tetra-Parallel MIMD Architecture for Big Data Applications

Deep Learning algorithm is widely used for various pattern recognition applications such as text recognition, object recognition and action recognition because of its best-in-class recognition accuracy compared to hand-crafted algorithm and shallow learning based algorithms. Long learning time caused by its complex structure, however, limits its usage only in high-cost servers or many-core GPU platforms so far. On the other hand, the demand on customized pattern recognition within personal devices will grow gradually as more deep learning applications will be developed. This paper presents a SoC implementation to enable deep learning applications to run with low cost platforms such as mobile or portable devices. Different from conventional works which have adopted massively-parallel architecture, this work adopts task-flexible architecture and exploits multiple parallelism to cover complex functions of convolutional deep belief network which is one of popular deep learning/inference algorithms. In this paper, we implement the most energy-efficient deep learning and inference processor for wearable system. The implemented 2.5 mm × 4.0 mm deep learning/inference processor is fabricated using 65 nm 8-metal CMOS technology for a battery-powered platform with real-time deep inference and deep learning operation. It consumes 185 mW average power, and 213.1 mW peak power at 200 MHz operating frequency and 1.2 V supply voltage. It achieves 411.3 GOPS peak performance and 1.93 TOPS/W energy efficiency, which is 2.07× higher than the state-of-the-art.

Benefits for Voice Learning Caused by Concurrent Faces Develop over Time

Recognition of personally familiar voices benefits from the concurrent presentation of the corresponding speakers’ faces. This effect of audiovisual integration is most pronounced for voices combined with dynamic articulating faces. However, it is unclear if learning unfamiliar voices also benefits from audiovisual face-voice integration or, alternatively, is hampered by attentional capture of faces, i.e., “face-overshadowing”. In six study-test cycles we compared the recognition of newly-learned voices following unimodal voice learning vs. bimodal face-voice learning with either static (Exp. 1) or dynamic articulating faces (Exp. 2). Voice recognition accuracies significantly increased for bimodal learning across study-test cycles while remaining stable for unimodal learning, as reflected in numerical costs of bimodal relative to unimodal voice learning in the first two study-test cycles and benefits in the last two cycles. This was independent of whether faces were static images (Exp. 1) or dynamic videos (Exp. 2). In both experiments, slower reaction times to voices previously studied with faces compared to voices only may result from visual search for faces during memory retrieval. A general decrease of reaction times across study-test cycles suggests facilitated recognition with more speaker repetitions. Overall, our data suggest two simultaneous and opposing mechanisms during bimodal face-voice learning: while attentional capture of faces may initially impede voice learning, audiovisual integration may facilitate it thereafter.

Performance and strategy comparisons of human listeners and logistic regression in discriminating underwater targets

To improve the design of underwater target recognition systems based on auditory perception, this study compared human listeners with automatic classifiers. Performances measures and strategies in three discrimination experiments, including discriminations between man-made and natural targets, between ships and submarines, and among three types of ships, were used. In the experiments, the subjects were asked to assign a score to each sound based on how confident they were about the category to which it belonged, and logistic regression, which represents linear discriminative models, also completed three similar tasks by utilizing many auditory features. The results indicated that the performances of logistic regression improved as the ratio between inter- and intra-class differences became larger, whereas the performances of the human subjects were limited by their unfamiliarity with the targets. Logistic regression performed better than the human subjects in all tasks but the discrimination between man-made and natural targets, and the strategies employed by excellent human subjects were similar to that of logistic regression. Logistic regression and several human subjects demonstrated similar performances when discriminating man-made and natural targets, but in this case, their strategies were not similar. An appropriate fusion of their strategies led to further improvement in recognition accuracy.

A Digital Liquid State Machine With Biologically Inspired Learning and Its Application to Speech Recognition

This paper presents a bioinspired digital liquid-state machine (LSM) for low-power very-large-scale-integration (VLSI)-based machine learning applications. To the best of the authors' knowledge, this is the first work that employs a bioinspired spike-based learning algorithm for the LSM. With the proposed online learning, the LSM extracts information from input patterns on the fly without needing intermediate data storage as required in offline learning methods such as ridge regression. The proposed learning rule is local such that each synaptic weight update is based only upon the firing activities of the corresponding presynaptic and postsynaptic neurons without incurring global communications across the neural network. Compared with the backpropagation-based learning, the locality of computation in the proposed approach lends itself to efficient parallel VLSI implementation. We use subsets of the TI46 speech corpus to benchmark the bioinspired digital LSM. To reduce the complexity of the spiking neural network model without performance degradation for speech recognition, we study the impacts of synaptic models on the fading memory of the reservoir and hence the network performance. Moreover, we examine the tradeoffs between synaptic weight resolution, reservoir size, and recognition performance and present techniques to further reduce the overhead of hardware implementation. Our simulation results show that in terms of isolated word recognition evaluated using the TI46 speech corpus, the proposed digital LSM rivals the state-of-the-art hidden Markov-model-based recognizer Sphinx-4 and outperforms all other reported recognizers including the ones that are based upon the LSM or neural networks.

On cross-language consonant identification in second language noise

Speech perception in everyday conditions is highly affected by the presence of noise of a different nature. The presence of overlapping speakers is considered an especially challenging scenario, as it introduces both energetic and informational masking. The efficacy of the masking also depends on the familiarity with the language of both the target and masking stimuli. This work analyses consonant identification by non-native English speakers in N-talker natural babble noise and babble-modulated noise, by varying the number of talkers in the babble. In particular, only English consonants that are also present in all the native languages of the subjects are used. As the subjects are familiar with the consonants used, this study can be considered a step towards a deeper analysis on perception of first language speech in the presence of second language maskers.

[ACTIVE AND PASSIVE DISCRIMINATION OF MOVING SOUNDS: EVENT-RELATED RESPONSES OF HUMAN BRAIN]

The current study investigates auditory event-related potentials (ERPs) and mismatch negativity (MMN) during active and passive discrimination of stationary and moving sound stimuli presented according to the oddball paradigm. Standard stimuli represented stationary midline sounds. Deviant stimuli simulated sound source location shifts (to the left/right from head midline) produced by linear or stepwise changes of interaural time delay (ITD). The event-related responses were evaluated by peak amplitudes of N1 waves and mean amplitudes of MMN, P3a, P3b and reorienting negativity (RON) components. The N1 amplitude was larger in active than in passive conditions, and was unaffected by spatial dynamic changes of the deviant stimuli. The deviant motion pattern (smooth or stepwise) affected only MMN and RON obtained in passive listening conditions. Abrupt deviant displacement elicited larger MMN and RON components than smooth motion. Drawing listeners' attention to the deviant stimuli resulted in suppression of MMN/RON sensitivity to auditory motion pattern.