Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Categorical face perception in fish: How a fish brain warps reality to dissociate "same" from "different"

Categorical perception (CP) is the phenomenon by which a smoothly varying stimulus property undergoes a nonlinear transformation during processing in the brain. Consequently, the stimuli are perceived as belonging to distinct categories separated by a sharp boundary. Originally thought to be largely innate, the discovery of CP in tasks such as novel image discrimination has piqued the interest of cognitive scientists because it provides compelling evidence that learning can shape a category's perceptual boundaries. CP has been particularly closely studied in human face perception. In nonprimates, there is evidence for CP for sound and color discrimination, but not for image or face discrimination. Here, we investigate the potential for learned CP in a lower vertebrate, the damselfish Pomacentrus amboinensis. Specifically, we tested whether the ability of these fish to discriminate complex facial patterns tracked categorical rather than metric differences in the stimuli. We first trained the fish to discriminate sets of two facial patterns. Next, we morphed between these patterns and determined the just noticeable difference (JND) between a morph and original image. Finally, we tested for CP by analyzing the discrimination ability of the fish for pairs of JND stimuli along the spectrum of morphs between two original images. Discrimination performance was significant for the image pair straddling the boundary between categories, and chance for equivalent stimulus pairs on either side, thus producing the classic "category boundary" effect. Our results reveal how perception can be influenced in a top-down manner even in the absence of a visual cortex.

Vocal tract constancy in birds and humans

Humans perceive speech as being relatively stable despite acoustic variation caused by vocal tract (VT) differences between speakers. Humans use perceptual 'vocal tract normalisation' (VTN) and other processes to achieve this stability. Similarity in vocal apparatus/acoustics between birds and humans means that birds might also experience VT variation. This has the potential to impede bird communication. No known studies have explicitly examined this, but a number of studies show perceptual stability or 'perceptual constancy' in birds similar to that seen in humans when dealing with VT variation. This review explores similarities between birds and humans and concludes that birds show sufficient evidence of perceptual constancy to warrant further research in this area. Future work should 1) quantify the multiple sources of variation in bird vocalisations, including, but not limited to VT variations, 2) determine whether vocalisations are perniciously disrupted by any of these and 3) investigate how birds reduce variation to maintain perceptual constancy and perceptual efficiency.

Rapidly learned song-discrimination without behavioral reinforcement in adult male zebra finches (Taeniopygia guttata)

Zebra finches communicate via several distinct vocalizations, of which song is the most studied. Behavioral observations indicate that adults are able to discriminate among the songs of different conspecific individuals. In the wild, zebra finches live in structured but mobile colonies, and encounter new individuals on a frequent basis. Thus it seems plausible that adult finches might have the capacity to recognize and remember new songs they encounter on a single day, but this has never been directly tested. Here we devised a simple observational assay to determine whether adult male zebra finches show recognition of a song they have heard repeatedly from taped playbacks, over a single three hour period the day before. We quantified the rate of production of six discrete behaviors (short calls, contact calls, singing, short hops, long hops, and beak swipes) made by adult male zebra finches as they listened to the playbacks. At the onset of song playback, all birds suspended these behaviors and sat silently-occasionally moving their heads. Then, after a measurable period ("response latency"), the birds resumed these activities. We observed that the response latency was long (approximately 10 min) when birds were hearing a particular song for the first time. The response latency was much shorter (approximately 1-2 min) when the birds had heard the same song the day before. Thus, functional song memories must result from as little as 3 h of passive song-exposure. These results suggest that ongoing song learning may play a natural role in the daily life of adult zebra finches, and provide a behavioral reference point for studies of molecular and physiological plasticity in the adult auditory system.

Influence of restraint and acute isolation on the selectivity of the adult zebra finch zenk gene response to acoustic stimuli

Zebra finches respond to certain auditory stimuli with the activation of the immediate early gene zenk. It has been shown that the amount of sound-mediated zenk gene expression varies in the zebra finch caudomedial neostriatum (NCM), apparently correlated with stimulus type (conspecific>heterospecific>noise>tones) and familiarity. Here we tested the impact of two additional factors-song-specific acoustical properties and testing conditions-on the specificity of the sound-mediated zenk response, as assessed by in situ hybridization. A variant of a normal conspecific song was first produced by randomizing the spectral content while retaining the amplitude envelope ('song-enveloped noise'). This stimulus and related controls were presented to birds which were either free in cages or restrained in a stereotaxic instrument, after isolation either overnight or for only 1 h prior to testing. We confirmed prior results that unrestrained birds show a greater zenk response to normal conspecific song than to other acoustic stimuli. However, under restraint, birds showed little or no selectivity for conspecific song compared to matched stimuli lacking a song organization. Thus the specificity of the zenk response to song is not determined simply by the acoustic structure and familiarity of the stimulus. We conclude that the intrinsic selectivity of sensory responses measured in the CNS may be influenced by factors associated with attention, arousal or vigilance, and may be significantly altered by experimental conditions that involve physical restraint.

Auditory sensitivity in the great tit: perception of signals in the presence and absence of noise

Absolute and masked auditory thresholds (critical masking ratios) were determined behaviourally in the great tit, Parus major, using a GO/NOGO-procedure. Absolute sensitivity was measured between 0.25 and 10 kHz. In the absence of noise, great tits were most sensitive to frequencies between 2 and 4 kHz. In background noise, however, the sensitivity was only a function of the noise level and was independent of frequency. Critical masking ratios determined for signals between 0.25 and 8 kHz were almost constant (median values varied between 23.8 and 25.9 dB) irrespective of signal frequency. Therefore, in contrast to the majority of bird species, great tits have unusually low critical masking ratios at high frequencies. This means that great tits can use high-frequency vocalizations to communicate efficiently in noisy (i.e. natural) environments. Copyright 1998 The Association for the Study of Animal Behaviour.