Eye movements during the production of nouns and pronouns

Unpacking the multilingualism continuum: An investigation of language variety co-activation in simultaneous interpreters

This study examines the phonological co-activation of a task-irrelevant language variety in mono- and bivarietal speakers of German with and without simultaneous interpreting (SI) experience during German comprehension and production. Assuming that language varieties in bivarietal speakers are co-activated analogously to the co-activation observed in bilinguals, the hypothesis was tested in the Visual World paradigm. Bivarietalism and SI experience were expected to affect co-activation, as bivarietalism requires communication-context based language-variety selection, while SI hinges on concurrent comprehension and production in two languages; task type was not expected to affect co-activation as previous evidence suggests the phenomenon occurs during comprehension and production. Sixty-four native speakers of German participated in an eye-tracking study and completed a comprehension and a production task. Half of the participants were trained interpreters and half of each sub-group were also speakers of Swiss German (i.e., bivarietal speakers). For comprehension, a growth-curve analysis of fixation proportions on phonological competitors revealed cross-variety co-activation, corroborating the hypothesis that co-activation in bivarietals' minds bears similar traits to language co-activation in multilingual minds. Conversely, co-activation differences were not attributable to SI experience, but rather to differences in language-variety use. Contrary to expectations, no evidence for phonological competition was found for either same- nor cross-variety competitors in either production task (interpreting- and word-naming variety). While phonological co-activation during production cannot be excluded based on our data, exploring the effects of additional demands involved in a production task hinging on a language-transfer component (oral translation from English to Standard German) merit further exploration in the light of a more nuanced understanding of the complexity of the SI task.

Representations underlying pronoun choice in Italian and English

Research has shown that speakers use fewer pronouns when the referential candidates are more similar and hence compete more strongly. Here we examined the locus of such an effect, investigating (1) whether pronoun use is affected by the referents’ competition at a non-linguistic level only (non-linguistic competition account) or whether it is also affected by competition arising from the antecedents’ similarities (linguistic competition account) and (2) the extent to which this depends on the type of pronoun. Speakers used Italian null pronouns and English pronouns less often (relative to full nouns) when the referential candidates compete more strongly situationally, while the antecedents’ semantic, grammatical or phonological similarity did not affect the rates of either pronouns, providing support for the non-linguistic competition account. However, unlike English pronouns, Italian null pronouns were unaffected by gender congruence between human referents, running counter to the gender effect for the use of non-gendered overt pronouns reported earlier. Hence, while both null and overt pronouns are sensitive to non-linguistic competition, what similarity affects non-linguistic competition partly depends on the type of pronouns.

Seeing for speaking: Semantic and lexical information provided by briefly presented, naturalistic action scenes

At the interface between scene perception and speech production, we investigated how rapidly action scenes can activate semantic and lexical information. Experiment 1 examined how complex action-scene primes, presented for 150 ms, 100 ms, or 50 ms and subsequently masked, influenced the speed with which immediately following action-picture targets are named. Prime and target actions were either identical, showed the same action with different actors and environments, or were unrelated. Relative to unrelated primes, identical and same-action primes facilitated naming the target action, even when presented for 50 ms. In Experiment 2, neutral primes assessed the direction of effects. Identical and same-action scenes induced facilitation but unrelated actions induced interference. In Experiment 3, written verbs were used as targets for naming, preceded by action primes. When target verbs denoted the prime action, clear facilitation was obtained. In contrast, interference was observed when target verbs were phonologically similar, but otherwise unrelated, to the names of prime actions. This is clear evidence for word-form activation by masked action scenes. Masked action pictures thus provide conceptual information that is detailed enough to facilitate apprehension and naming of immediately following scenes. Masked actions even activate their word-form information-as is evident when targets are words. We thus show how language production can be primed with briefly flashed masked action scenes, in answer to long-standing questions in scene processing.

Looking at a contrast object before speaking boosts referential informativeness, but is not essential

Variation in referential form has traditionally been accounted for by theoretical frameworks focusing on linguistic and discourse features. Despite the explosion of interest in eye tracking methods in psycholinguistics, the role of visual scanning behaviour in informative reference production is yet to be comprehensively investigated. Here we examine the relationship between speakers' fixations to relevant referents and the form of the referring expressions they produce. Overall, speakers were fully informative across simple and (to a lesser extent) more complex displays, providing appropriately modified referring expressions to enable their addressee to locate the target object. Analysis of contrast fixations revealed that looking at a contrast object boosts but is not essential for full informativeness. Contrast fixations which take place immediately before speaking provide the greatest boost. Informative referring expressions were also associated with later speech onsets than underinformative ones. Based on the finding that fixations during speech planning facilitate but do not fully predict informative referring, direct visual scanning is ruled out as a prerequisite for informativeness. Instead, pragmatic expectations of informativeness may play a more important role. Results are consistent with a goal-based link between eye movements and language processing, here applied for the first time to production processes.

The effects of utterance timing and stimulation of left prefrontal cortex on the production of referential expressions

We examined the relationship between the timing of utterance initiation and the choice of referring expressions, e.g., pronouns (it), zeros (…and went down), or descriptive NPs (the pink pentagon). We examined language production in healthy adults, and used anodal transcranial direct current stimulation (tDCS) to test the involvement of the left prefrontal cortex (PFC) in the timing of utterance production and the selection of reference forms in a discourse context. Twenty-two subjects (11 anodal, 11sham) described fast-paced actions, e.g. The gray oval flashes, then it moves right 2 blocks. We only examined trials in contexts that supported pronoun/zero use. For sham participants, pronouns/zeros increased on trials with longer latencies to initiate the target utterance, and trials where the previous trial was short. We argue that both of these conditions enabled greater message pre-planning and greater discourse connectedness: The strongest predictor of pronoun/zero usage was the presence of a connector word like and or then, which was also tended to occur on trials with longer latencies. For the anodal participants, the latency effect disappeared. PFC stimulation appeared to enable participants to produce utterances with greater discourse connectedness, even while planning incrementally.

Explicit and Emergent Mechanisms of Information Status

It is well established that language production and comprehension are influenced by information status, for example, whether information is given, new, topical, or predictable, and many scholars suggest that an important component of information status is keeping track of what information is in common ground (i.e., what is shared), and what is not. Information status affects both speakers' choices (e.g., word order, pronoun use, prosodic prominence) and how listeners interpret the speaker's meaning (e.g., Chafe, 1994; Prince, 1981). Although there is a wealth of scholarly work on information status (for a review, see Arnold, Kaiser, Kahn, & Kim, 2013), there is no consensus on the mechanisms by which it is used, and in fact relatively little discussion of the underlying representations and psycholinguistic mechanisms. Moreover, a major challenge to understanding information status is that its effects are notoriously variable. This study considers existing proposals about information status, focusing on two questions: (a) how is it represented; and (b) by what mechanisms is it used? I propose that it is important to consider whether representations and mechanisms can be classified as either explicit or emergent. Based on a review of existing evidence, I argue that information status representations are most likely emergent, but the mechanisms by which they are used are both explicit and emergent. This review provides one of the first considerations of information status processing across multiple domains.

The Eyes Are Right When the Mouth Is Wrong

When describing visual scenes, speakers typically gaze at objects while preparing their names. In a study of the relation between eye movements and speech, a corpus of self-corrected speech errors was analyzed. If errors result from rushed word preparation, insufficient visual information, or failure to check prepared names against objects, speakers should spend less time gazing at referents before uttering errors than before uttering correct names. Counter to predictions, gazes to referents before errors (e.g., gazes to an axe before saying “ham-” [hammer]) highly resembled gazes to referents before correct names (e.g., gazes to an axe before saying “axe”). However, speakers gazed at referents for more time after initiating erroneous compared with correct names, apparently while they prepared corrections. Assuming that gaze nonetheless reflects word preparation, errors were not associated with insufficient preparation. Nor were errors systematically associated with decreased inspection of objects. Like gesture, gaze may accurately reflect a speaker's intentions even when the accompanying speech does not.

Effects of Psychological Attention on Pronoun Comprehension

Pronoun comprehension is facilitated for referents that are focused in the discourse context. Discourse focus has been described as a function of attention, especially shared attention, but few studies have explicitly tested this idea. Two experiments used an exogenous capture cue paradigm to demonstrate that listeners' visual attention at the onset of a story influences their preferences during pronoun resolution later in the story. In both experiments trial-initial attention modulated listeners' transitory biases while considering referents for the pronoun, whether it was in response to the capture cue or not. These biases even had a small influence on listeners' final interpretation of the pronoun. These results provide independently-motivated evidence that the listener's attention influences the on-line processes of pronoun comprehension. Trial-initial attentional shifts were made on the basis of non-shared, private information, demonstrating that attentional effects on pronoun comprehension are not restricted to shared attention among interlocutors.

Parallel object activation and attentional gating of information: evidence from eye movements in the multiple object naming paradigm

Do we access information from any object we can see, or do we access information only from objects that we intend to name? In 3 experiments using a modified multiple object naming paradigm, subjects were required to name several objects in succession when previews appeared briefly and simultaneously in the same location as the target as well as at another location. In Experiment 1, preview benefit-faster processing of the target when the preview was related (a mirror image of the target) compared to unrelated (semantically and phonologically)-was found for the preview in the target location but not a location that was never to be named. In Experiment 2, preview benefit was found if a related preview appeared in either the target location or the third-to-be-named location. Experiment 3 showed the difference between results from the first 2 experiments was not due to the number of objects on the screen. These data suggest that attention serves to gate visual input about objects based on the intention to name them and that information from one intended-to-be-named object can facilitate processing of an object in another location.

Reference production in young speakers with and without autism: effects of discourse status and processing constraints

We examine the referential choices (pronouns/zeros vs. names/descriptions) made during a narrative by high-functioning children and adolescents with autism and a well-matched typically developing control group. The process of choosing appropriate referring expressions has been proposed to depend on two areas of cognitive functioning: (a) judging the attention and knowledge of one's interlocutor, and (b) the use of memory and attention mechanisms to represent the discourse situation. We predicted possible group differences, since autism is often associated with deficits in (a) mentalizing and (b) memory and attention, as well as a more general tendency to have difficulty with the pragmatic aspects of language use. Results revealed that some of the participants with autism were significantly less likely to produce pronouns or zeros in some discourse contexts. However, the difference was only one of degree. Overall, all participants in our analysis exhibited fine-grained sensitivity to the discourse context. Furthermore, referential choices for all participants were modulated by factors related to the cognitive effort of language production.

Speech planning during multiple-object naming: effects of ageing

Two experiments were conducted with younger and older speakers. In Experiment 1, participants named single objects that were intact or visually degraded, while hearing distractor words that were phonologically related or unrelated to the object name. In both younger and older participants naming latencies were shorter for intact than for degraded objects and shorter when related than when unrelated distractors were presented. In Experiment 2, the single objects were replaced by object triplets, with the distractors being phonologically related to the first object's name. Naming latencies and gaze durations for the first object showed degradation and relatedness effects that were similar to those in single-object naming. Older participants were slower than younger participants when naming single objects and slower and less fluent on the second but not the first object when naming object triplets. The results of these experiments indicate that both younger and older speakers plan object names sequentially, but that older speakers use this planning strategy less efficiently.

When you name the pizza you look at the coin and the bread: eye movements reveal semantic activation during word production

Two eyetracking experiments tested for activation of category coordinate and perceptually related concepts when speakers prepare the name of an object. Speakers saw four visual objects in a 2 x 2 array and identified and named a target picture on the basis of either category (e.g., "What is the name of the musical instrument?") or visual-form (e.g., "What is the name of the circular object?") instructions. There were more fixations on visual-form competitors and category coordinate competitors than on unrelated objects during name preparation, but the increased overt attention did not affect naming latencies. The data demonstrate that eye movements are a sensitive measure of the overlap between the conceptual (including visual-form) information that is accessed in preparation for word production and the conceptual knowledge associated with visual objects. Furthermore, these results suggest that semantic activation of competitor concepts does not necessarily affect lexical selection, contrary to the predictions of lexical-selection-by-competition accounts (e.g., Levelt, Roelofs, & Meyer, 1999).

Observing the what and when of language production for different age groups by monitoring speakers' eye movements

Research on adult age differences in language production has traditionally focused on either the production of single words or the properties of language samples. Older adults are more prone to word retrieval failures than are younger adults (e.g., ). Older adults also tend to produce fewer ideas per utterance and fewer left-branching syntactic structures (e.g., ). The use of eye movement monitoring in the study of language production allows researchers to examine word production processes in the context of multiword utterances, bridging the gap between behavior in word production studies and spontaneous speech samples. This paper outlines one view of how speakers plan and produce utterances, summarizes the literature on age-related changes in production, presents an overview of the published research on speakers' gaze during picture description, and recaps a study using eye movement monitoring to explore age-related changes in language production.

Speakers gaze at objects while preparing intentionally inaccurate labels for them

When describing scenes, speakers gaze at objects while preparing their names (Z. M. Griffin & K. Bock, 2000). In this study, the authors investigated whether gazes to referents occurred in the absence of a correspondence between visual features and word meaning. Speakers gazed significantly longer at objects before intentionally labeling them inaccurately with the names of similar things (e.g., calling a horse a dog) than when labeling them accurately. This held for grammatical subjects and objects as well as agents and patients. Moreover, the time spent gazing at a referent before labeling it with a novel word or accurate name was similar and decreased as speakers gained experience using the novel word. These results suggest that visual attention in speaking may be directed toward referents in the absence of any association between their visual forms and the words used to talk about them.

Spoken word production: a theory of lexical access

A core operation in speech production is the preparation of words from a semantic base. The theory of lexical access reviewed in this article covers a sequence of processing stages beginning with the speaker's focusing on a target concept and ending with the initiation of articulation. The initial stages of preparation are concerned with lexical selection, which is zooming in on the appropriate lexical item in the mental lexicon. The following stages concern form encoding, i.e., retrieving a word's morphemic phonological codes, syllabifying the word, and accessing the corresponding articulatory gestures. The theory is based on chronometric measurements of spoken word production, obtained, for instance, in picture-naming tasks. The theory is largely computationally implemented. It provides a handle on the analysis of multiword utterance production as well as a guide to the analysis and design of neuroimaging studies of spoken utterance production.