Neighing, barking, and drumming horses-object related sounds help and hinder picture naming

Neighing dogs: Semantic context effects of environmental sounds in spoken word production - a replication and extension

Semantic context effects are well established using both words and pictures as stimuli. One such effect, semantic interference, is observed in naming latencies when a categorically related distractor word or picture is presented together with a target picture (e.g., dog-LION). Recently, this effect has also been shown to occur when an environmental sound (e.g., a dog barking) is presented as an auditory distractor during picture naming and when a distractor picture is presented with a target sound for naming. The purpose of the current study was twofold: (1) to replicate the semantic interference effect in the picture-sound interference (PSI) paradigm and (2) determine whether a semantic interference effect is also observable when distractor words are presented with environmental sounds as target auditory objects for naming, using a novel sound-word interference (SWI) paradigm. We replicated the semantic interference effect in Experiment 1 with environmental sound distractors. Experiment 2 demonstrated significant semantic interference during an SWI paradigm for the first time. We discuss the implications of these results for our understanding of the origin and locus of the semantic interference effect according to current theories of lexical selection.

Long-term memory representations for audio-visual scenes

In this study, we investigated the nature of long-term memory representations for naturalistic audio-visual scenes. Whereas previous research has shown that audio-visual scenes are recognized more accurately than their unimodal counterparts, it remains unclear whether this benefit stems from audio-visually integrated long-term memory representations or a summation of independent retrieval cues. We tested two predictions for audio-visually integrated memory representations. First, we used a modeling approach to test whether recognition performance for audio-visual scenes is more accurate than would be expected from independent retrieval cues. This analysis shows that audio-visual integration is not necessary to explain the benefit of audio-visual scenes relative to purely auditory or purely visual scenes. Second, we report a series of experiments investigating the occurrence of study-test congruency effects for unimodal and audio-visual scenes. Most importantly, visually encoded information was immune to additional auditory information presented during testing, whereas auditory encoded information was susceptible to additional visual information presented during testing. This renders a true integration of visual and auditory information in long-term memory representations unlikely. In sum, our results instead provide evidence for visual dominance in long-term memory. Whereas associative auditory information is capable of enhancing memory performance, the long-term memory representations appear to be primarily visual.

Auditory enhancement of visual searches for event scenes

Increasing research has revealed that uninformative spatial sounds facilitate the early processing of visual stimuli. This study examined the crossmodal interactions of semantically congruent stimuli by assessing whether the presentation of event-related characteristic sounds facilitated or interfered with the visual search for corresponding event scenes in pictures. The search array consisted of four images: one target and three non-target pictures. Auditory stimuli were presented to participants in synchronization with picture onset using three types of sounds: a sound congruent with a target, a sound congruent with a distractor, or a control sound. The control sound varied across six experiments, alternating between a sound unrelated to the search stimuli, white noise, and no sound. Participants were required to swiftly localize a target position while ignoring the sound presentation. Visual localization resulted in rapid responses when a sound that was semantically related to the target was played. Furthermore, when a sound was semantically related to a distractor picture, the response times were longer. When the distractor-congruent sound was used, participants incorrectly localized the distractor position more often than at the chance level. These findings were replicated when the experiments ruled out the possibility that participants would learn picture-sound pairs during the visual tasks (i.e., the possibility of brief training during the experiments). Overall, event-related crossmodal interactions occur based on semantic representations, and audiovisual associations may develop as a result of long-term experiences rather than brief training in a laboratory.

Word Identification With Temporally Interleaved Competing Sounds by Younger and Older Adult Listeners

OBJECTIVES

The purpose of this experiment was to contribute to our understanding of the nature of age-related changes in competing speech perception using a temporally interleaved task.

DESIGN

Younger and older adults (n = 16/group) participated in this study. The target was a five-word sentence. The masker was one of the following: another five-word sentence; five brief samples of modulated noise; or five brief samples of environmental sounds. The stimuli were presented in a temporally interleaved manner, where the target and masker alternated in time, always beginning with the target. Word order was manipulated in the target (and in the masker during trials with interleaved words) to compare performance when the five words in each stream did versus did not create a syntactically correct sentence. Talker voice consistency also was examined by contrasting performance when each word in the target was spoken by the same talker or by different talkers; a similar manipulation was used for the masker when it consisted of words. Participants were instructed to repeat back the target words and ignore the intervening words or sounds. Participants also completed a subset of tests from the NIH Cognitive Toolbox.

RESULTS

Performance on this interleaved task was significantly associated with listener age and with a metric of cognitive flexibility, but it was not related to the degree of high-frequency hearing loss. Younger adults' performance on this task was better than that of older adults, especially for words located toward the end of the sentence. Both groups of participants were able to take advantage of correct word order in the target, and both were negatively affected, to a modest extent, when the masker words were in correct syntactic order. The two groups did not differ in how phonetic similarity between target and masker words influenced performance, and interleaved environmental sounds or noise had only a minimal effect for all listeners. The most robust difference between listener groups was found for the use of voice consistency: older adults, as compared with younger adults, were less able to take advantage of a consistent target talker within a trial.

CONCLUSIONS

Younger adults outperformed older adults when masker words were interleaved with target words. Results suggest that this difference was unlikely to be related to energetic masking and/or peripheral hearing loss. Rather, age-related changes in cognitive flexibility and problems encoding voice information appeared to underlie group differences. These results support the contention that, in real-life competing speech situations that produce both energetic and informational masking, older adults' problems are due to both peripheral and nonperipheral changes.

Effects of transcranial magnetic stimulation over the left posterior superior temporal gyrus on picture-word interference

Word-production theories argue that during language production, a concept activates multiple lexical candidates in left temporal cortex, and the intended word is selected from this set. Evidence for theories on spoken-word production comes, for example, from the picture-word interference task, where participants name pictures superimposed by congruent (e.g., picture: rabbit, distractor "rabbit"), categorically related (e.g., distractor "sheep"), or unrelated (e.g., distractor "fork") words. Typically, whereas congruent distractors facilitate naming, related distractors slow down picture naming relative to unrelated distractors, resulting in semantic interference. However, the neural correlates of semantic interference are debated. Previous neuroimaging studies have shown that the left mid-to-posterior STG (pSTG) is involved in the interference associated with semantically related distractors. To probe the functional relevance of this area, we targeted the left pSTG with focal repetitive transcranial magnetic stimulation (rTMS) while subjects performed a picture-word interference task. Unexpectedly, pSTG stimulation did not affect the semantic interference effect but selectively increased the congruency effect (i.e., faster naming with congruent distractors). The facilitatory TMS effect selectively occurred in the more difficult list with an overall lower name agreement. Our study adds new evidence to the causal role of the left pSTG in the interaction between picture and distractor representations or processing streams, only partly supporting previous neuroimaging studies. Moreover, the observed unexpected condition-specific facilitatory rTMS effect argues for an interaction of the task- or stimulus-induced brain state with the modulatory TMS effect. These issues should be systematically addressed in future rTMS studies on language production.

The Embodied Penman: Effector-Specific Motor-Language Integration During Handwriting

Several studies have illuminated how processing manual action verbs (MaVs) affects the programming or execution of concurrent hand movements. Here, to circumvent key confounds in extant designs, we conducted the first assessment of motor-language integration during handwriting-a task in which linguistic and motoric processes are co-substantiated. Participants copied MaVs, non-manual action verbs, and non-action verbs as we collected measures of motor programming and motor execution. Programming latencies were similar across conditions, but execution was faster for MaVs than for the other categories, regardless of whether word meanings were accessed implicitly or explicitly. In line with the Hand-Action-Network Dynamic Language Embodiment (HANDLE) model, such findings suggest that effector-congruent verbs can prime manual movements even during highly automatized tasks in which motoric and verbal processes are naturally intertwined. Our paradigm opens new avenues for fine-grained explorations of embodied language processes.

Activating Semantic Knowledge During Spoken Words and Environmental Sounds: Evidence From the Visual World Paradigm

Two visual world experiments investigated the activation of semantically related concepts during the processing of environmental sounds and spoken words. Participants heard environmental sounds such as barking or spoken words such as "puppy" while viewing visual arrays with objects such as a bone (semantically related competitor) and candle (unrelated distractor). In Experiment 1, a puppy (target) was also included in the visual array; in Experiment 2, it was not. During both types of auditory stimuli, competitors were fixated significantly more than distractors, supporting the coactivation of semantically related concepts in both cases; comparisons of the two types of auditory stimuli also revealed significantly larger effects with environmental sounds than spoken words. We discuss implications of these results for theories of semantic knowledge.

Semantic interference in the picture-word interference task: Is there a pre-lexical, conceptual contribution to the effect?

Picture naming takes longer in the presence of a semantic-categorically related distractor word compared to an unrelated distractor word. This semantic interference effect in the picture-word interference (PWI) task is an empirical cornerstone in speech production research and of central importance in theory development and evaluation. Prominent models locate the effect at an abstract lexical level, yet only few studies have tested for a possible pre-lexical, conceptual contribution. Moreover, those studies that did are not conclusive. We re-explored the locus of semantic interference by contrasting two task versions that were implemented in as parallel a fashion as possible, but differed with respect to the processing stages involved: naming pictures (requiring conceptual processing and lexical processing) and deciding on their natural size (requiring conceptual processing only). We predicted semantic interference in naming, replicating the standard effect. If part of the effect is localized at the conceptual level, we predicted interference in size decision, too. We found semantic effects in both tasks but with different polarity - interference in naming and facilitation in size decision. This pattern supports the view that semantic interference in PWI has its locus at the lexical level and its origin at the conceptual level.

Localizing semantic interference from distractor sounds in picture naming: A dual-task study

In this study we explored the locus of semantic interference in a novel picture-sound interference task in which participants name pictures while ignoring environmental distractor sounds. In a previous study using this task (Mädebach, Wöhner, Kieseler, & Jescheniak, in Journal of Experimental Psychology: Human Perception and Performance, 43, 1629-1646, 2017), we showed that semantically related distractor sounds (e.g., BARKING_dog) interfere with a picture-naming response (e.g., "horse") more strongly than unrelated distractor sounds do (e.g., DRUMMING_drum). In the experiment reported here, we employed the psychological refractory period (PRP) approach to explore the locus of this effect. We combined a geometric form classification task (square vs. circle; Task 1) with the picture-sound interference task (Task 2). The stimulus onset asynchrony (SOA) between the tasks was systematically varied (0 vs. 500 ms). There were three central findings. First, the semantic interference effect from distractor sounds was replicated. Second, picture naming (in Task 2) was slower with the short than with the long task SOA. Third, both effects were additive-that is, the semantic interference effects were of similar magnitude at both task SOAs. This suggests that the interference arises during response selection or later stages, not during early perceptual processing. This finding corroborates the theory that semantic interference from distractor sounds reflects a competitive selection mechanism in word production.