Language can boost otherwise unseen objects into visual awareness

Testing the involvement of low-level visual representations during spoken word processing with non-Western students and meditators practicing Sudarshan Kriya Yoga

Previous studies, using the Continuous Flash Suppression (CFS) paradigm, observed that (Western) university students are better able to detect otherwise invisible pictures of objects when they are presented with the corresponding spoken word shortly before the picture appears. Here we attempted to replicate this effect with non-Western university students in Goa (India). A second aim was to explore the performance of (non-Western) meditators practicing Sudarshan Kriya Yoga in Goa in the same task. Some previous literature suggests that meditators may excel in some tasks that tap visual attention, for example by exercising better endogenous and exogenous control of visual awareness than non-meditators. The present study replicated the finding that congruent spoken cue words lead to significantly higher detection sensitivity than incongruent cue words in non-Western university students. Our exploratory meditator group also showed this detection effect but both frequentist and Bayesian analyses suggest that the practice of meditation did not modulate it. Overall, our results provide further support for the notion that spoken words can activate low-level category-specific visual features that boost the basic capacity to detect the presence of a visual stimulus that has those features. Further research is required to conclusively test whether meditation can modulate visual detection abilities in CFS and similar tasks.

A revised diffusion model for conflict tasks

The recently developed diffusion model for conflict tasks (DMC) Ulrich et al. (Cognitive Psychology, 78, 148-174, 2015) provides a good account of data from all standard conflict tasks (e.g., Stroop, Simon, and flanker tasks) within a common evidence accumulation framework. A central feature of DMC's processing dynamics is that there is an initial phase of rapid accumulation of distractor evidence that is then selectively withdrawn from the decision mechanism as processing continues. We argue that this assumption is potentially troubling because it could be viewed as implying qualitative changes in the representation of distractor information over the time course of processing. These changes suggest more than simple inhibition or suppression of distractor information, as they involve evidence produced by distractor processing "changing sign" over time. In this article, we (a) develop a revised DMC (RDMC) whose dynamics operate strictly within the limits of inhibition/suppression (i.e., evidence strength can change monotonically, but cannot change sign); (b) demonstrate that RDMC can predict the full range of delta plots observed in the literature (i.e., both positive-going and negative-going); and (c) show that the model provides excellent fits to Simon and flanker data used to benchmark the original DMC at both the individual and group level. Our model provides a novel account of processing differences across Simon and flanker tasks. Specifically, that they differ in how distractor information is processed on congruent trials, rather than incongruent trials: congruent trials in the Simon task show relatively slow attention shifting away from distractor information (i.e., location) while complete and rapid attention shifting occurs in the flanker task. Our new model highlights the importance of considering dynamic interactions between top-down goals and bottom-up stimulus effects in conflict processing.

Using agreement probability to study differences in types of concepts and conceptualizers

Agreement probability p(a) is a homogeneity measure of lists of properties produced by participants in a Property Listing Task (PLT) for a concept. Agreement probability's mathematical properties allow a rich analysis of property-based descriptions. To illustrate, we use p(a) to delve into the differences between concrete and abstract concepts in sighted and blind populations. Results show that concrete concepts are more homogeneous within sighted and blind groups than abstract ones (i.e., exhibit a higher p(a) than abstract ones) and that concrete concepts in the blind group are less homogeneous than in the sighted sample. This supports the idea that listed properties for concrete concepts should be more similar across subjects due to the influence of visual/perceptual information on the learning process. In contrast, abstract concepts are learned based mainly on social and linguistic information, which exhibit more variability among people, thus, making the listed properties more dissimilar across subjects. Relative to abstract concepts, the difference in p(a) between sighted and blind is not statistically significant. Though this is a null result, and should be considered with care, it is expected because abstract concepts should be learned by paying attention to the same social and linguistic input in both, blind and sighted, and thus, there is no reason to expect that the respective lists of properties should differ. Finally, we used p(a) to classify concrete and abstract concepts with a good level of certainty. All these analyses suggest that p(a) can be fruitfully used to study data obtained in a PLT.

The interactive effect of external rewards and self-determined choice on memory

Both external motivational incentives (e.g., monetary reward) and internal motivational incentives (e.g., self-determined choice) have been found to promote memory, but much less is known about how these two types of incentives interact with each other to affect memory. The current study (N = 108) examined how performance-dependent monetary rewards affected the role of self-determined choice in memory performance, also known as the choice effect. Using a modified and better controlled version of the choice paradigm and manipulating levels of reward, we demonstrated an interactive effect between monetary reward and self-determined choice on 1-day delayed memory performance. Specifically, the choice effect on memory decreased when we introduced the performance-dependent external rewards. These results are discussed in terms of understanding how external and internal motivators interact to impact learning and memory.

The association of targets modulates the search efficiency in multitarget searches

Previous studies have found that distractors can affect visual search efficiency when associated with the target in a single-target search. However, multitarget searches are frequently necessary in daily life. In the present study, we examined how the association of targets in a multitarget search affected performance when searching for two targets simultaneously (Experiment 1). In addition, we explored whether the association affected switch cost (Experiment 2) and preparation cost (Experiment 3). Participants were required to learn associations between different colors or shapes and then performed feature search and conjunction search tasks. For all experiments, the results of search efficiency showed that for conjunction search, the search efficiency under the associated condition was significantly higher than that under the neutral condition. Similarly, the response times in the associated condition were significantly faster than those in the neutral condition under the conjunction search condition in Experiments 1 and 2. However, in Experiment 3, the response times in the associated condition were longer than those in the neutral condition. These results indicate that the association between targets can improve the efficiency of multitarget searches. Furthermore, associations can reduce the time spent searching for individual targets and the switch cost; however, the preparation cost increases.

Carving joints into nature: reengineering scientific concepts in light of concept-laden evidence

A new wave of proposals suggests that scientists must reassess scientific concepts in light of accumulated evidence. However, reengineering scientific concepts in light of data is challenging because scientific concepts affect the evidence itself in multiple ways. Among other possible influences, concepts (i) prime scientists to overemphasize within-concept similarities and between-concept differences; (ii) lead scientists to measure conceptually relevant dimensions more accurately; (iii) serve as units of scientific experimentation, communication, and theory-building; and (iv) affect the phenomena themselves. When looking for improved ways to carve nature at its joints, scholars must take the concept-laden nature of evidence into account to avoid entering a vicious circle of concept-evidence mutual substantiation.

Conceptual Associations Generate Sensory Predictions

A crucial ability of the human brain is to learn and exploit probabilistic associations between stimuli to facilitate perception and behavior by predicting future events. Although studies have shown how perceptual relationships are used to predict sensory inputs, relational knowledge is often between concepts rather than percepts (e.g., we learned to associate cats with dogs, rather than specific images of cats and dogs). Here, we asked if and how sensory responses to visual input may be modulated by predictions derived from conceptual associations. To this end we exposed participants of both sexes to arbitrary word-word pairs (e.g., car-dog) repeatedly, creating an expectation of the second word, conditional on the occurrence of the first. In a subsequent session, we exposed participants to novel word-picture pairs, while measuring fMRI BOLD responses. All word-picture pairs were equally likely, but half of the pairs conformed to the previously formed conceptual (word-word) associations, whereas the other half violated this association. Results showed suppressed sensory responses throughout the ventral visual stream, including early visual cortex, to pictures that corresponded to the previously expected words compared with unexpected words. This suggests that the learned conceptual associations were used to generate sensory predictions that modulated processing of the picture stimuli. Moreover, these modulations were tuning specific, selectively suppressing neural populations tuned toward the expected input. Combined, our results suggest that recently acquired conceptual priors are generalized across domains and used by the sensory brain to generate category-specific predictions, facilitating processing of expected visual input.SIGNIFICANCE STATEMENT Perceptual predictions play a crucial role in facilitating perception and the integration of sensory information. However, little is known about whether and how the brain uses more abstract, conceptual priors to form sensory predictions. In our preregistered study, we show that priors derived from recently acquired arbitrary conceptual associations result in category-specific predictions that modulate perceptual processing throughout the ventral visual hierarchy, including early visual cortex. These results suggest that the predictive brain uses prior knowledge across various domains to modulate perception, thereby extending our understanding of the extensive role predictions play in perception.

Auditory scene context facilitates visual recognition of objects in consistent visual scenes

Visual object recognition is facilitated by contextually consistent scenes in which the object is embedded. Scene gist representations extracted from the scenery backgrounds yield this scene consistency effect. Here we examined whether the scene consistency effect is specific to the visual domain or if it is crossmodal. Through four experiments, the accuracy of the naming of briefly presented visual objects was assessed. In each trial, a 4-s sound clip was presented and a visual scene containing the target object was briefly shown at the end of the sound clip. In a consistent sound condition, an environmental sound associated with the scene in which the target object typically appears was presented (e.g., forest noise for a bear target object). In an inconsistent sound condition, a sound clip contextually inconsistent with the target object was presented (e.g., city noise for a bear). In a control sound condition, a nonsensical sound (sawtooth wave) was presented. When target objects were embedded in contextually consistent visual scenes (Experiment 1: a bear in a forest background), consistent sounds increased object-naming accuracy. In contrast, sound conditions did not show a significant effect when target objects were embedded in contextually inconsistent visual scenes (Experiment 2: a bear in a pedestrian crossing background) or in a blank background (Experiments 3 and 4). These results suggested that auditory scene context has weak or no direct influence on visual object recognition. It seems likely that consistent auditory scenes indirectly facilitate visual object recognition by promoting visual scene processing.

Tool use acquisition induces a multifunctional interference effect during object processing: evidence from the sensorimotor mu rhythm

A fundamental characteristic of human development is acquiring and accumulating tool use knowledge through observation and sensorimotor experience. Recent studies showed that, in children and adults, different action possibilities to grasp-to-move and grasp-to-use objects generate a conflict that extinguishes neural motor resonance phenomena during visual object processing. In this study, a training protocol coupled with EEG recordings was administered in virtual reality to healthy adults to evaluate whether a similar conflict occurs between novel tool use knowledge. Participants perceived and manipulated two novel 3D tools trained beforehand with either single or double-usage. A weaker reduction of mu-band (10-13 Hz) power, accompanied by a reduced inter-trial phase coherence, was recorded during the perception of the tool associated with the double-usage. These effects started within the first 200 ms of visual object processing and were predominantly recorded over the left motor system. Furthermore, interacting with the double usage tool delayed grasp-to-reach movements. The results highlight a multifunctional interference effect, such as tool use acquisition reduces the neural motor resonance phenomenon and inhibits the activation of the motor system during subsequent object recognition. These results imply that learned tool use information guides sensorimotor processes of objects.

Guns Are Not Faster to Enter Awareness After Seeing a Black Face: Absence of Race-Priming in a Gun/Tool Task During Continuous Flash Suppression

In the Weapon Identification Task (WIT), Black faces prime the identification of guns compared with tools. We measured race-induced changes in visual awareness of guns and tools using continuous flash suppression (CFS). Eighty-four participants, primed with Black or Asian faces, indicated the location of a gun or tool target that was temporarily rendered invisible through CFS, which provides a sensitive measure of effects on early visual processing. The same participants also completed a standard (non-CFS) WIT. We replicated the standard race-priming effect in the WIT. In the CFS task, Black and Asian primes did not affect the time guns and tools needed to enter awareness. Thus, race priming does not alter early visual processing but does change the identification of guns and tools. This confirms that race-priming originates from later post-perceptual memory- or response-related processing.

A label isn't just a label: Brief training leads to label-dependent visuo-cortical processing in adults

The current study examines the extent to which hearing individual-level names (e.g., Jimmy) and category-level labels (e.g., Hitchel) paired with novel objects impacts neural responses across a brief (6 min) learning period. Event-related Potentials (ERPs) were recorded while adult participants (n = 44) viewed and heard exemplars of two different species of named novel objects. ERPs were examined for each labeling condition and compared across the first and second half of the learning trials (∼3 min/half). Mean amplitude decreased for the P1 and increased for the N170 from the first to the second half of trials. The decrease in P1 was right lateralized. In addition, the P1 amplitude recorded over right occipitotemporal regions was greater than left occipitotemporal areas, but only for objects paired with individual-level labels. Category-level labels did not show regional P1 differences. The N250 component was greatest over the right occipitotemporal region and was enhanced for objects labeled with individual-level relative to category-level names during the second half of trials. Overall, these findings highlight the unfolding of label-dependent visual processing across a short training period in adults. The results suggest that linguistic labels have an important, top-down impact, on visual processing and that label specificity shapes visuo-cortical responses within a 6-min learning period.

Long-term memory representations for audio-visual scenes

In this study, we investigated the nature of long-term memory representations for naturalistic audio-visual scenes. Whereas previous research has shown that audio-visual scenes are recognized more accurately than their unimodal counterparts, it remains unclear whether this benefit stems from audio-visually integrated long-term memory representations or a summation of independent retrieval cues. We tested two predictions for audio-visually integrated memory representations. First, we used a modeling approach to test whether recognition performance for audio-visual scenes is more accurate than would be expected from independent retrieval cues. This analysis shows that audio-visual integration is not necessary to explain the benefit of audio-visual scenes relative to purely auditory or purely visual scenes. Second, we report a series of experiments investigating the occurrence of study-test congruency effects for unimodal and audio-visual scenes. Most importantly, visually encoded information was immune to additional auditory information presented during testing, whereas auditory encoded information was susceptible to additional visual information presented during testing. This renders a true integration of visual and auditory information in long-term memory representations unlikely. In sum, our results instead provide evidence for visual dominance in long-term memory. Whereas associative auditory information is capable of enhancing memory performance, the long-term memory representations appear to be primarily visual.

What You See Is What You Hear: Sounds Alter the Contents of Visual Perception

Visual object recognition is not performed in isolation but depends on prior knowledge and context. Here, we found that auditory context plays a critical role in visual object perception. Using a psychophysical task in which naturalistic sounds were paired with noisy visual inputs, we demonstrated across two experiments (young adults; ns = 18-40 in Experiments 1 and 2, respectively) that the representations of ambiguous visual objects were shifted toward the visual features of an object that were related to the incidental sound. In a series of control experiments, we found that these effects were not driven by decision or response biases (ns = 40-85) nor were they due to top-down expectations (n = 40). Instead, these effects were driven by the continuous integration of audiovisual inputs during perception itself. Together, our results demonstrate that the perceptual experience of visual objects is directly shaped by naturalistic auditory context, which provides independent and diagnostic information about the visual world.

The action-sentence compatibility effect (ACE): Meta-analysis of a benchmark finding for embodiment

The embodied account of language comprehension has been one of the most influential theoretical developments in the recent decades addressing the question how humans comprehend and represent language. To examine its assumptions, many studies have made use of behavioral paradigms involving basic compatibility effects. The action-sentence compatibility effect (ACE) is one of the most influential of these compatibility effects and is the most widely cited evidence for the embodied account of language comprehension. However, recently there have been difficulties in extending or even in reliably replicating the ACE. The conflicting findings concerning the ACE and its extensions lead to the discussion of whether the ACE is indeed a reliable effect. In a first step we conducted a meta-analysis using a random-effects model. This analysis revealed a small but significant effect size of the ACE. Furthermore, the task-parameter Delay occurred as a factor of interest in whether the ACE appears with positive or negative effect direction. A second meta-analytic approach (Fisher's method) supports these findings. Additionally, an analysis of publication bias suggests that there is bias in the ACE literature. In post-hoc analyses of the recent multi-lab investigation of the ACE (Morey et al., 2021), evidence for individual differences in the ACE was found. However, further analyses indicate that these differences are likely due to item-specific variability and the specific way in which items were assigned to conditions in the counterbalancing lists.

Diversity in dermatology: International medical graduates and their role in the diversity of a specialty

A diverse medical workforce improves patient care. Dermatology is the second-least diverse medical specialty in the United States, and many recent publications have discussed the different reasons and possible solutions to improve this disparity. A quarter of physicians in the United States are international medical graduates, which directly affects the cultural diversity in health care. Dermatology has the lowest percentage of international medical graduates in its active physician workforce. Among other measures, the inclusion of more international medical graduates in residency programs can help improve the diversity in this specialty and alleviate any disparities in dermatological care delivery in underserved communities.

Localizing, describing, interpreting: effects of different audio text structures on attributing meaning to digital pictures

Based on previous research on multimedia learning and text comprehension, an eye-tracking study was conducted to examine the influence of audio text coherence on visual attention and memory in a multimedia learning situation with a focus on picture comprehension. Audio text coherence was manipulated by the type of LDI structure, that is, whether localization, description, and interpretation followed in immediate succession for each pictorial detail or whether localizations and description of details were separated from their interpretation. Results show that with a LDI integrated structure compared to a LDI separated structure the referred-to picture elements were fixated longer during interpretation parts, and linkages between descriptions and interpretations were better recalled and recognized. The effects on recall and recognition of linkages were fully mediated by fixation times. This pattern of results can be explained by an interplay between audio text coherence and dual coding processes. It points out the importance of local coherence and the provision of localization information in audio explanations as well as visual attention to allow for dual coding processes that can be used to better attribute meaning to picture details. Practical implications for the design of educational videos, audio texts on websites, and audio guides are discussed.

A voice without a mouth no more: The neurobiology of language and consciousness

Most research on the neurobiology of language ignores consciousness and vice versa. Here, language, with an emphasis on inner speech, is hypothesised to generate and sustain self-awareness, i.e., higher-order consciousness. Converging evidence supporting this hypothesis is reviewed. To account for these findings, a 'HOLISTIC' model of neurobiology of language, inner speech, and consciousness is proposed. It involves a 'core' set of inner speech production regions that initiate the experience of feeling and hearing words. These take on affective qualities, deriving from activation of associated sensory, motor, and emotional representations, involving a largely unconscious dynamic 'periphery', distributed throughout the whole brain. Responding to those words forms the basis for sustained network activity, involving 'default mode' activation and prefrontal and thalamic/brainstem selection of contextually relevant responses. Evidence for the model is reviewed, supporting neuroimaging meta-analyses conducted, and comparisons with other theories of consciousness made. The HOLISTIC model constitutes a more parsimonious and complete account of the 'neural correlates of consciousness' that has implications for a mechanistic account of mental health and wellbeing.

Cross-modal facilitation of auditory discrimination in a frog

Stimulation in one sensory modality can affect perception in a separate modality, resulting in diverse effects including illusions in humans. This can also result in cross-modal facilitation, a process where sensory performance in one modality is improved by stimulation in another modality. For instance, a simple sound can improve performance in a visual task in both humans and cats. However, the range of contexts and underlying mechanisms that evoke such facilitation effects remain poorly understood. Here, we demonstrated cross-modal stimulation in wild-caught túngara frogs, a species with well-studied acoustic preferences in females. We first identified that a combined visual and seismic cue (vocal sac movement and water ripple) was behaviourally relevant for females choosing between two courtship calls in a phonotaxis assay. We then found that this combined cross-modal stimulus rescued a species-typical acoustic preference in the presence of background noise that otherwise abolished the preference. These results highlight how cross-modal stimulation can prime attention in receivers to improve performance during decision-making. With this, we provide the foundation for future work uncovering the processes and conditions that promote cross-modal facilitation effects.

Semantic Expectation Effects on Object Detection: Using Figure Assignment to Elucidate Mechanisms

Recent evidence suggesting that object detection is improved following valid rather than invalid labels implies that semantics influence object detection. It is not clear, however, whether the results index object detection or feature detection. Further, because control conditions were absent and labels and objects were repeated multiple times, the mechanisms are unknown. We assessed object detection via figure assignment, whereby objects are segmented from backgrounds. Masked bipartite displays depicting a portion of a mono-oriented object (a familiar configuration) on one side of a central border were shown once only for 90 or 100 ms. Familiar configuration is a figural prior. Accurate detection was indexed by reports of an object on the familiar configuration side of the border. Compared to control experiments without labels, valid labels improved accuracy and reduced response times (RTs) more for upright than inverted objects (Studies 1 and 2). Invalid labels denoting different superordinate-level objects (DSC; Study 1) or same superordinate-level objects (SSC; Study 2) reduced accuracy for upright displays only. Orientation dependency indicates that effects are mediated by activated object representations rather than features which are invariant over orientation. Following invalid SSC labels (Study 2), accurate detection RTs were longer than control for both orientations, implicating conflict between semantic representations that had to be resolved before object detection. These results demonstrate that object detection is not just affected by semantics, it entails semantics.

Generalized Approach/Avoidance Responses to Degraded Affective Stimuli: An Informational Account

Two studies tested whether affective stimuli presented auditorily spontaneously trigger approach/avoidance reactions toward neutral visual stimuli. Contrary to hypotheses, Experiment 1 revealed that when the target was present, participants responded faster after positive (vs. negative) stimuli, and faster to the absence of the target following negative (vs. positive) stimuli, whatever the response modality (i.e., approach/avoidance). Instructions were to approach/avoid stimuli depending on whether a target was presented or not presented. We proposed that affective stimuli were used in this study as information about the presence/absence of the target. In Experiment 2, we replicated the results of Experiment 1 when participants responded to the presence/absence of the target, whereas an approach/avoidance compatibility effect was observed when each response modality was associated with a target. These results indicate that affective stimuli influence approach/avoidance across perceptual modalities and suggest that the link between affective stimuli and behavioral tendencies could be mediated by informational value of affect.

Concurrent semantic priming and lexical interference for close semantic relations in blocked-cyclic picture naming: Electrophysiological signatures

In the present study, we employed event-related brain potentials to investigate the effects of semantic similarity on different planning stages during language production. We manipulated semantic similarity by controlling feature overlap within taxonomical hierarchies. In a blocked-cyclic naming task, participants named pictures in repeated cycles, blocked in semantically close, distant, or unrelated conditions. Only closely related items, but not distantly related items, induced semantic blocking effects. In the first presentation cycle, naming was facilitated, and amplitude modulations in the N1 component around 140-180 ms post-stimulus onset predicted this behavioral facilitation. In contrast, in later cycles, naming was delayed, and a negative-going posterior amplitude modulation around 250-350 ms post-stimulus onset predicted this interference. These findings indicate easier object recognition or identification underlying initial facilitation and increased difficulties during lexical selection. The N1 modulation was reduced but persisted in later cycles in which interference dominated, and the posterior negativity was also present in cycle 1 in which facilitation dominated, demonstrating concurrent effects of conceptual priming and lexical interference in all naming cycles. Our assumptions about the functional role these two opposing forces play in producing semantic context effects are further supported by the finding that the joint modulation of these two ERPs on naming latency exclusively emerged when naming closely related, but not unrelated items. The current findings demonstrate that close relations, but not distant taxonomic relations, induce stronger semantic blocking effects, and that temporally overlapping electrophysiological signatures reflect a trade-off between facilitatory priming and interfering lexical competition.

From non-conscious processing to conscious events: a minimalist approach

The minimalist approach that we develop here is a framework that allows to appreciate how non-conscious processing and conscious contents shape human cognition, broadly defined. It is composed of three simple principles. First, cognitive processes are inherently non-conscious, while their inputs and (interim) outputs may be consciously experienced. Second, non-conscious processes and elements of the cognitive architecture prioritize information for conscious experiences. Third, conscious events are composed of series of conscious contents and non-conscious processes, with increased duration leading to more opportunity for processing. The narrowness of conscious experiences is conceptualized here as a solution to the problem of channeling the plethora of non-conscious processes into action and communication processes that are largely serial. The framework highlights the importance of prioritization for consciousness, and we provide an illustrative review of three main factors that shape prioritization-stimulus strength, motivational relevance and mental accessibility. We further discuss when and how this framework (i) is compatible with previous theories, (ii) enables new understandings of established findings and models, and (iii) generates new predictions and understandings.

Linguistic labels cue biological motion perception and misperception

Linguistic labels exert a particularly strong top-down influence on perception. The potency of this influence has been ascribed to their ability to evoke category-diagnostic features of concepts. In doing this, they facilitate the formation of a perceptual template concordant with those features, effectively biasing perceptual activation towards the labelled category. In this study, we employ a cueing paradigm with moving, point-light stimuli across three experiments, in order to examine how the number of biological motion features (form and kinematics) encoded in lexical cues modulates the efficacy of lexical top-down influence on perception. We find that the magnitude of lexical influence on biological motion perception rises as a function of the number of biological motion-relevant features carried by both cue and target. When lexical cues encode multiple biological motion features, this influence is robust enough to mislead participants into reporting erroneous percepts, even when a masking level yielding high performance is used.

Unpredictive linguistic verbal cues accelerate congruent visual targets into awareness in a breaking continuous flash suppression paradigm

One of the most influential ideas within the domain of cognition is that of embodied cognition, in which the experienced world is the result of an interplay between an organism's physiology, sensorimotor system, and its environment. An aspect of this idea is that linguistic information activates sensory representations automatically. For example, hearing the word 'red' would automatically activate sensory representations of this color. But does linguistic information prioritize access to awareness of congruent visual information? Here, we show that linguistic verbal cues accelerate matching visual targets into awareness by using a breaking continuous flash suppression paradigm. In a speeded reaction time task, observers heard spoken color labels (e.g., red) followed by colored targets that were either congruent (red), incongruent (green), or neutral (a neutral noncolor word) with respect to the labels. Importantly, and in contrast to previous studies investigating a similar question, the incidence of congruent trials was not higher than that of incongruent trials. Our results show that RTs were selectively shortened for congruent verbal-visual pairings, and that this shortening occurred over a wide range of cue-target intervals. We suggest that linguistic verbal information preactivates sensory representations, so that hearing the word 'red' preactivates (visual) sensory information internally.

Auditory and Semantic Cues Facilitate Decoding of Visual Object Category in MEG

Sounds (e.g., barking) help us to visually identify objects (e.g., a dog) that are distant or ambiguous. While neuroimaging studies have revealed neuroanatomical sites of audiovisual interactions, little is known about the time course by which sounds facilitate visual object processing. Here we used magnetoencephalography to reveal the time course of the facilitatory influence of natural sounds (e.g., barking) on visual object processing and compared this to the facilitatory influence of spoken words (e.g., "dog"). Participants viewed images of blurred objects preceded by a task-irrelevant natural sound, a spoken word, or uninformative noise. A classifier was trained to discriminate multivariate sensor patterns evoked by animate and inanimate intact objects with no sounds, presented in a separate experiment, and tested on sensor patterns evoked by the blurred objects in the 3 auditory conditions. Results revealed that both sounds and words, relative to uninformative noise, significantly facilitated visual object category decoding between 300-500 ms after visual onset. We found no evidence for earlier facilitation by sounds than by words. These findings provide evidence for a semantic route of facilitation by both natural sounds and spoken words, whereby the auditory input first activates semantic object representations, which then modulate the visual processing of objects.

Dissociating conscious and unconscious influences on visual detection effects

The scope of unconscious processing is highly debated, with recent studies showing that even high-level functions such as perceptual integration and category-based attention occur unconsciously. For example, upright faces that are suppressed from awareness through interocular suppression break into awareness more quickly than inverted faces. Similarly, verbal object cues boost otherwise invisible objects into awareness. Here, we replicate these findings, but find that they reflect a general difference in detectability not specific to interocular suppression. To dissociate conscious and unconscious influences on visual detection effects, we use an additional discrimination task to rule out conscious processes as a cause for these differences. Results from this detection-discrimination dissociation paradigm reveal that, while face orientation is processed unconsciously, category-based attention requires awareness. These findings provide insights into the function of conscious perception and offer an experimental approach for mapping out the scope and limits of unconscious processing.

Synesthesia does not help to recover perceptual dominance following flash suppression

Grapheme-colour synesthesia occurs when letters or numbers elicit an abnormal colour sensation (e.g., printed black letters are perceived as coloured). This phenomenon is typically reported following explicit presentation of graphemes. Very few studies have investigated colour sensations in synesthesia in the absence of visual awareness. We took advantage of the dichoptic flash suppression paradigm to temporarily render a stimulus presented to one eye invisible. Synesthetic alphanumeric and non-synesthetic stimuli were presented to 21 participants (11 synesthetes) in achromatic and chromatic experimental conditions. The test stimulus was first displayed to one eye and then masked by a sudden presentation of visual noise in the other eye (flash suppression). The time for an image to be re-perceived following the onset of the suppressive noise was calculated. Trials where there was no flash suppression performed but instead mimicked the perceptual suppression of the flash were also tested. Results showed that target detection by synesthetes was significantly better than by controls in the absence of flash suppression. No difference was found between the groups in the flash suppression condition. Our findings suggest that synesthesia is associated with enhanced perception for overt recognition, but does not provide an advantage in recovering from a perceptual suppression. Further studies are needed to investigate synesthesia in relation to visual awareness.

Symbolic categorization of novel multisensory stimuli in the human brain

When primates (both human and non-human) learn to categorize simple visual or acoustic stimuli by means of non-verbal matching tasks, two types of changes occur in their brain: early sensory cortices increase the precision with which they encode sensory information, and parietal and lateral prefrontal cortices develop a categorical response to the stimuli. Contrary to non-human animals, however, our species mostly constructs categories using linguistic labels. Moreover, we naturally tend to define categories by means of multiple sensory features of the stimuli. Here we trained adult subjects to parse a novel audiovisual stimulus space into 4 orthogonal categories, by associating each category to a specific symbol. We then used multi-voxel pattern analysis (MVPA) to show that during a cross-format category repetition detection task three neural representational changes were detectable. First, visual and acoustic cortices increased both precision and selectivity to their preferred sensory feature, displaying increased sensory segregation. Second, a frontoparietal network developed a multisensory object-specific response. Third, the right hippocampus and, at least to some extent, the left angular gyrus, developed a shared representational code common to symbols and objects. In particular, the right hippocampus displayed the highest level of abstraction and generalization from a format to the other, and also predicted symbolic categorization performance outside the scanner. Taken together, these results indicate that when humans categorize multisensory objects by means of language the set of changes occurring in the brain only partially overlaps with that described by classical models of non-verbal unisensory categorization in primates.

Building semantic memory from embodied and distributional language experience

Humans seamlessly make sense of a rapidly changing environment, using a seemingly limitless knowledgebase to recognize and adapt to most situations we encounter. This knowledgebase is called semantic memory. Embodied cognition theories suggest that we represent this knowledge through simulation: understanding the meaning of coffee entails reinstantiating the neural states involved in touching, smelling, seeing, and drinking coffee. Distributional semantic theories suggest that we are sensitive to statistical regularities in natural language, and that a cognitive mechanism picks up on these regularities and transforms them into usable semantic representations reflecting the contextual usage of language. These appear to present contrasting views on semantic memory, but do they? Recent years have seen a push toward combining these approaches under a common framework. These hybrid approaches augment our understanding of semantic memory in important ways, but current versions remain unsatisfactory in part because they treat sensory-perceptual and distributional-linguistic data as interacting but distinct types of data that must be combined. We synthesize several approaches which, taken together, suggest that linguistic and embodied experience should instead be considered as inseparably entangled: just as sensory and perceptual systems are reactivated to understand meaning, so are experience-based representations endemic to linguistic processing; further, sensory-perceptual experience is susceptible to the same distributional principles as language experience. This conclusion produces a characterization of semantic memory that accounts for the interdependencies between linguistic and embodied data that arise across multiple timescales, giving rise to concept representations that reflect our shared and unique experiences. This article is categorized under: Psychology > Language Neuroscience > Cognition Linguistics > Language in Mind and Brain.

Eye movements and the label feedback effect: Speaking modulates visual search via template integrity

The label-feedback hypothesis (Lupyan, 2012) proposes that language modulates low- and high-level visual processing, such as priming visual object perception. Lupyan and Swingley (2012) found that repeating target names facilitates visual search, resulting in shorter response times (RTs) and higher accuracy. In the present investigation, we conceptually replicated and extended their study, using additional control conditions and recording eye movements during search. Our goal was to evaluate whether self-directed speech influences target locating (i.e. attentional guidance) or object perception (i.e., distractor rejection and target appreciation). In three experiments, during object search, people spoke target names, nonwords, irrelevant (absent) object names, or irrelevant (present) object names (all within-participants). Experiments 1 and 2 examined search RTs and accuracy: Speaking target names improved performance, without differences among the remaining conditions. Experiment 3 incorporated eye-tracking: Gaze fixation patterns suggested that language does not affect attentional guidance, but instead affects both distractor rejection and target appreciation. When search trials were conditionalized according to distractor fixations, language effects became more orderly: Search was fastest while people spoke target names, followed in linear order by the nonword, distractor-absent, and distractor-present conditions. We suggest that language affects template maintenance during search, allowing fluent differentiation of targets and distractors. Materials, data, and analyses can be retrieved here: https://osf.io/z9ex2/.

Effect of stimulus dimension on perception and cognition

Stimuli characteristics ha a decisive role in our perception and cognition. In the present study, we aimed to evaluate the effects of dimension of stimuli, two-dimensional (2D) versus three-dimensional (3D), on perception and working memory. In the first experiment, using the method of eye tracking, a higher blink rate, pupil size, and the number of saccade for three compared to 2D stimuli revealed a higher perceptual demand of 3D stimuli. In the second experiment, visual search task shows a higher response time for 3D stimuli and an equal performance with 2- and 3D stimuli in spatial working memory task. In the third experiment, four working memory tasks with high and low cognitive and perceptual load revealed 3D stimuli are memorized better in the both low and high load of working memory. We can conclude that 3D stimulus, compared 2D, imposes a higher load on perceptual system, but it is memorized better. It could be concluded that the phenomenon of filtering should occur in the early perceptual system for preventing overload.

Speaking for seeing: Sentence structure guides visual event apprehension

Human experience and communication are centred on events, and event apprehension is a rapid process that draws on the visual perception and immediate categorization of event roles ("who does what to whom"). We demonstrate a role for syntactic structure in visual information uptake for event apprehension. An event structure foregrounding either the agent or patient was activated during speaking, transiently modulating the apprehension of subsequently viewed unrelated events. Speakers of Dutch described pictures with actives and passives (agent and patient foregrounding, respectively). First fixations on pictures of unrelated events that were briefly presented (for 300 ms) next were influenced by the active or passive structure of the previously produced sentence. Going beyond the study of how single words cue object perception, we show that sentence structure guides the viewpoint taken during rapid event apprehension.

A Future of Words: Language and the Challenge of Abstract Concepts

The paper outlines one of the most important challenges that embodied and grounded theories need to face, i.e., that to explain how abstract concepts (abstractness) are acquired, represented, and used. I illustrate the view according to which abstract concepts are grounded not only in sensorimotor experiences, like concrete concepts, but also and to a greater extent in linguistic, social, and inner experiences. Specifically, I discuss the role played by metacognition, inner speech, social metacognition, and interoception. I also present evidence showing that the weight of linguistic, social, and inner experiences varies depending on the considered sub-kind of abstract concepts (e.g., mental states and spiritual concepts, numbers, emotions, social concepts). I argue that the challenge to explain abstract concepts representation implies the recognition of: a. the role of language, intended as inner and social tool, in shaping our mind; b. the importance of differences across languages; c. the existence of different kinds of abstract concepts; d. the necessity to adopt new paradigms, able to capture the use of abstract concepts in context and interactive situations. This challenge should be addressed with an integrated approach that bridges developmental, anthropological, and neuroscientific studies. This approach extends embodied and grounded views incorporating insights from distributional statistics views of meaning, from pragmatics and semiotics.

Towards Understanding the Task Dependency of Embodied Language Processing: The Influence of Colour During Language-Vision Interactions

A main challenge for theories of embodied cognition is to understand the task dependency of embodied language processing. One possibility is that perceptual representations (e.g., typical colour of objects mentioned in spoken sentences) are not activated routinely but the influence of perceptual representation emerges only when context strongly supports their involvement in language. To explore this question, we tested the effects of colour representations during language processing in three visual-world eye-tracking experiments. On critical trials, participants listened to sentence-embedded words associated with a prototypical colour (e.g., ‘…spinach…’) while they inspected a visual display with four printed words (Experiment 1), coloured or greyscale line drawings (Experiment 2) and a ‘blank screen’ after a preview of coloured or greyscale line drawings (Experiment 3). Visual context always presented a word/object (e.g., frog) associated with the same prototypical colour (e.g. green) as the spoken target word and three distractors. When hearing spinach participants did not prefer the written word frog compared to other distractor words (Experiment 1). In Experiment 2, colour competitors attracted more overt attention compared to average distractors, but only for the coloured condition and not for greyscale trials. Finally, when the display was removed at the onset of the sentence, and in contrast to the previous blank-screen experiments with semantic competitors, there was no evidence of colour competition in the eye-tracking record (Experiment 3). These results fit best with the notion that the main role of perceptual representations in language processing is to contextualize language in the immediate environment.

Effects of Language on Visual Perception

Does language change what we perceive? Does speaking different languages cause us to perceive things differently? We review the behavioral and electrophysiological evidence for the influence of language on perception, with an emphasis on the visual modality. Effects of language on perception can be observed both in higher-level processes such as recognition and in lower-level processes such as discrimination and detection. A consistent finding is that language causes us to perceive in a more categorical way. Rather than being fringe or exotic, as they are sometimes portrayed, we discuss how effects of language on perception naturally arise from the interactive and predictive nature of perception.

Language is the missing link in action-perception coupling: an EEG study

The paper reports an electrophysiological (EEG) study investigating how language is involved in perception–action relations in musically trained and untrained participants. Using an original backward priming paradigm, participants were exposed to muted point-light videos of violinists performing piano or forte nuances followed by a congruent vs. incongruent word. After the video presentation, participants were asked to decide whether the musician was playing a piano or forte musical nuance. EEG results showed a greater P200 event-related potential for trained participants at the occipital site, and a greater N400 effect for untrained participants at the central site. Musically untrained participants were more accurate when the word was semantically congruent with the gesture than when it was incongruent. Overall, language seems to influence the performance of untrained participants, for which perception–action couplings are less automatized.

(Not so) Great Expectations: Listening to Foreign-Accented Speech Reduces the Brain's Anticipatory Processes

This study examines the effect of foreign-accented speech on the predictive ability of our brain. Listeners actively anticipate upcoming linguistic information in the speech signal so as to facilitate and reduce processing load. However, it is unclear whether or not listeners also do this when they are exposed to speech from non-native speakers. In the present study, we exposed native Dutch listeners to sentences produced by native and non-native speakers while measuring their brain activity using electroencephalography. We found that listeners’ brain activity differed depending on whether they listened to native or non-native speech. However, participants’ overall performance as measured by word recall rate was unaffected. We discussed the results in relation to previous findings as well as the automaticity of anticipation.

Other Species' Alarm Calls Evoke a Predator-Specific Search Image in Birds

Many animals produce vocal alarm signals when they detect a predator, and heterospecific species sharing predators often eavesdrop on and respond to these calls [1]. Despite the widespread occurrence of interspecific eavesdropping in animals, its underlying cognitive process remains to be elucidated. If alarm calls, like human referential words, denote a specific predator type (e.g., "snake!"), then receivers may retrieve a mental image of the predator when hearing these calls [2-4]. Here, using a recently developed experimental paradigm [5], I test whether heterospecific alarm calls evoke a predator-specific visual search image in wild birds. During playback of snake-specific alarm calls produced by Japanese tits (Parus minor), coal tits (Periparus ater) approach a wooden stick being moved in a snake-like manner. However, coal tits do not approach the same stick when hearing other call types or if the stick's movement is dissimilar to that of a snake. Thus, Japanese tit snake alarms cause coal tits to specifically enhance visual attention to snakelike objects. These results provide experimental evidence for the evocation of visual search images by heterospecific alarm calls, highlighting the importance of integrating cross-modal information in interspecific eavesdropping.

Emotion Words' Effect on Visual Awareness and Attention of Emotional Faces

To explore whether the meaning of a word changes visual processing of emotional faces (i.e., visual awareness and visual attention), we performed two complementary studies. In Experiment 1, we presented participants with emotion and control words and then tracked their visual awareness for two competing emotional faces using a binocular rivalry paradigm. Participants experienced the emotional face congruent with the emotion word for longer than a word-incongruent emotional face, as would be expected if the word was biasing awareness toward the (unseen) face. In Experiment 2, we similarly presented participants with emotion and control words prior to presenting emotional faces using a divided visual field paradigm. Emotion words were congruent with either the emotional face in the right or left visual field. After the presentation of faces, participants saw a dot in either the left or right visual field. Participants were slower to identify the location of the dot when it appeared in the same visual field as the emotional face congruent with the emotion word. The effect was limited to the left hemisphere (RVF), as would be expected for linguistic integration of the word with the face. Since the task was not linguistic, but rather a simple dot-probe task, participants were slower in their responses under these conditions because they likely had to disengage from the additional linguistic processing caused by the word-face integration. These findings indicate that emotion words bias visual awareness for congruent emotional faces, as well as shift attention toward congruent emotional faces.

Finding categories through words: More nameable features improve category learning

What are the cognitive consequences of having a name for something? Having a word for a feature makes it easier to communicate about a set of exemplars belonging to the same category (e.g., "the red things"). But might it also make it easier to learn the category itself? Here, we provide evidence that the ease of learning category distinctions based on simple visual features is predicted from the ease of naming those features. Across seven experiments, participants learned categories composed of colors or shapes that were either easy or more difficult to name in English. Holding the category structure constant, when the underlying features of the category were easy to name, participants were faster and more accurate in learning the novel category. These results suggest that compact verbal labels may facilitate hypothesis formation during learning: it is easier to pose the hypothesis "it is about redness" than "it is about that pinkish-purplish color". Our results have consequences for understanding how developmental and cross-linguistic differences in a language's vocabulary affect category learning and conceptual development.

The syntax-semantics interface in animal vocal communication

Syntax (rules for combining words or elements) and semantics (meaning of expressions) are two pivotal features of human language, and interaction between them allows us to generate a limitless number of meaningful expressions. While both features were traditionally thought to be unique to human language, research over the past four decades has revealed intriguing parallels in animal communication systems. Many birds and mammals produce specific calls with distinct meanings, and some species combine multiple meaningful calls into syntactically ordered sequences. However, it remains largely unclear whether, like phrases or sentences in human language, the meaning of these call sequences depends on both the meanings of the component calls and their syntactic order. Here, leveraging recently demonstrated examples of meaningful call combinations, we introduce a framework for exploring the interaction between syntax and semantics (i.e. the syntax-semantic interface) in animal vocal sequences. We outline methods to test the cognitive mechanisms underlying the production and perception of animal vocal sequences and suggest potential evolutionary scenarios for syntactic communication. We hope that this review will stimulate phenomenological studies on animal vocal sequences as well as experimental studies on the cognitive processes, which promise to provide further insights into the evolution of language. This article is part of the theme issue 'What can animal communication teach us about human language?'

The cognitive penetrability of perception: A blocked debate and a tentative solution

Despite the extensive body of psychological findings suggesting that cognition influences perception, the debate between defenders and detractors of the cognitive penetrability of perception persists. While detractors demand more strictness in psychological experiments, proponents consider that empirical studies show that cognitive penetrability occurs. These considerations have led some theorists to propose that the debate has reached a dead end. The issue about where perception ends and cognition begins is, I argue, one of the reasons why the debate is cornered. Another reason is the inability of psychological studies to present uncontroversial interpretations of the results obtained. To dive into other kinds of empirical sources is, therefore, required to clarify the debate. In this paper, I explain where the debate is blocked, and suggest that neuroscientific evidence together with the predictive coding account, might decant the discussion on the side of the penetrability thesis.

Sound-Symbolism Effects in the Absence of Awareness: A Replication Study

People have been shown to link particular sounds with particular shapes. For instance, the round-sounding nonword bouba tends to be associated with curved shapes, whereas the sharp-sounding nonword kiki is deemed to be related to angular shapes. People's tendency to associate sounds and shapes has been observed across different languages. In the present study, we reexamined the claim by Hung, Styles, and Hsieh (2017) that such sound-shape mappings can occur before an individual becomes aware of the visual stimuli. More precisely, we replicated their first experiment, in which congruent and incongruent stimuli (e.g., bouba presented in a round shape or an angular shape, respectively) were rendered invisible through continuous flash suppression. The results showed that congruent combinations, on average, broke suppression faster than incongruent combinations, thus providing converging evidence for Hung and colleagues' assertions. Collectively, these findings now provide a solid basis from which to explore the boundary conditions of the effect.