Object expectations alter information use during visual recognition

Computational reconstruction of mental representations using human behavior

Revealing how the mind represents information is a longstanding goal of cognitive science. However, there is currently no framework for reconstructing the broad range of mental representations that humans possess. Here, we ask participants to indicate what they perceive in images made of random visual features in a deep neural network. We then infer associations between the semantic features of their responses and the visual features of the images. This allows us to reconstruct the mental representations of multiple visual concepts, both those supplied by participants and other concepts extrapolated from the same semantic space. We validate these reconstructions in separate participants and further generalize our approach to predict behavior for new stimuli and in a new task. Finally, we reconstruct the mental representations of individual observers and of a neural network. This framework enables a large-scale investigation of conceptual representations.

The processing of spatial frequencies through time in visual word recognition

This study examined the temporal profile of spatial frequency processing in a word reading task in 16 normal adult readers. They had to report the word presented in a 200 ms display using a four-alternative forced-choice task (4AFC). The stimuli were made of an additive combination of the signal (i.e. the target word) and of a visual white noise patch wherein the signal-to-noise ratio varied randomly across stimulus duration. Four spatial frequency conditions were defined for the signal component of the stimulus (bandpass Butterworth filters with center frequencies of 1.2, 2.4, 4.8 and 9.6 cycles per degree). In contrast to the coarse-to-fine theory of visual recognition, the results show that the highest spatial frequency range dominates early processing, with a shift toward lower spatial frequencies at later points during stimulus exposure. This pattern interacted in a complex way with the temporal frequency content of signal-to-noise oscillations. The outcome of individual data patterns classification by a machine learning algorithm according to the corresponding spatial frequency band further shows that the most salient spatial frequency signature is obtained when the time dimension within data patterns is recoded into its Fourier transform.

Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.