Orthographic directionality and thematic role illustration in English and Arabic

Literacy overrides effects of animacy: A picture-naming study with pre-literate German children and adult speakers of German and Arabic

Animacy plays a key role for human cognition, which is also reflected in the way humans process language. However, while experiments on sentence processing show reliable effects of animacy on word order and grammatical function assignment, effects of animacy on conjoined noun phrases (e.g., fish and shoe vs. shoe and fish) have yielded inconsistent results. In the present study, we tested the possibility that effects of animacy are outranked by reading and writing habits. We examined adult speakers of German (left-to-right script) and speakers of Arabic (right-to-left script), as well as German preschool children who do not yet know how to read and write. Participants were tested in a picture naming task that presented an animate and an inanimate entity next to one another. On half of the trials, the animate entity was located on the left and, on the other half, it was located on the right side of the screen. We found that adult German and Arabic speakers differed in their order of naming. Whereas German speakers were much more likely to mention the animate entity first when it was presented on the left than on the right, a reverse tendency was observed for speakers of Arabic. Thus, in literate adults, the ordering of conjoined noun phrases was influenced by reading and writing habits rather than by the animacy status of an entity. By contrast, pre-literate children preferred to start their utterances with the animate entity regardless of position, suggesting that effects of animacy in adults have been overwritten by effects of literacy.

Electrophysiological hallmarks for event relations and event roles in working memory

The ability to maintain events (i.e., interactions between/among objects) in working memory is crucial for our everyday cognition, yet the format of this representation is poorly understood. The current ERP study was designed to answer two questions: How is maintaining events (e.g., the tiger hit the lion) neurally different from maintaining item coordinations (e.g., the tiger and the lion)? That is, how is the event relation (present in events but not coordinations) represented? And how is the agent, or initiator of the event encoded differently from the patient, or receiver of the event during maintenance? We used a novel picture-sentence match-across-delay approach in which the working memory representation was "pinged" during the delay, replicated across two ERP experiments with Chinese and English materials. We found that maintenance of events elicited a long-lasting late sustained difference in posterior-occipital electrodes relative to non-events. This effect resembled the negative slow wave reported in previous studies of working memory, suggesting that the maintenance of events in working memory may impose a higher cost compared to coordinations. Although we did not observe significant ERP differences associated with pinging the agent vs. the patient during the delay, we did find that the ping appeared to dampen the ongoing sustained difference, suggesting a shift from sustained activity to activity silent mechanisms. These results suggest a new method by which ERPs can be used to elucidate the format of neural representation for events in working memory.

The Alignment of Agent-First Preferences with Visual Event Representations: Contrasting German and Arabic

How does non-linguistic, visual experience affect language production? A series of experiments addressed this question by examining linguistic and visual preferences for agent positions in transitive action scenarios. In Experiment 1, 30 native German speakers described event scenes where agents were positioned either to the right or to the left of patients. Produced utterances had longer speech onset times for scenes with right- rather than left-positioned agents, suggesting that the visual organization of events can affect sentence production. In Experiment 2 another cohort of 36 native German participants indicated their aesthetic preference for left- or right-positioned agents in mirrored scenes and displayed a preference for scenes with left-positioned agents. In Experiment 3, 37 Arabic native participants performed the same non-verbal task showing the reverse preference. Our findings demonstrate that non-linguistic visual preferences seem to affect sentence production, which in turn may rely on the writing system of a specific language.

Rightward directional bias in art produced by cultures without a written language

In this study, we coded art painted on rocks located in southern Africa, which was painted with a mixture of ochre, blood, and clay by the San, a Neolithic culture with no written language. These images depict a mixture of humans and animals in a variety of contexts, including (but not limited to) hunts and dances. We calculated a laterality index for the collected available art from each region, finding that although there was variability across regions in the direction of the laterality scores, most regions contained a majority of figures facing rightward. This is in stark contrast with reports of artists drawing leftward facing animals and human profiles (an effect that is influenced by native language writing system direction, gender, and handedness), but interestingly our sample also contained regions with strong leftward biases. Our results are, however, in accord with studies that report people preferring images that depict left-to-right motion, as well as the left-to-right bias in depicting transitive actions, an effect that seems to result from greater right hemispheric activation in scene processing and interpretation. Thus, this study shows that in the absence of a writing system, right-lateralized neural architecture may guide the hands of artists.

Depicting Visual Motion in Still Images: Forward Leaning and a Left to Right Bias for Lateral Movement

What artistic conventions are used to convey the motion of animate and inanimate items in still images, such as drawings and photographs? One graphic convention involves depicting items leaning forward into their movement, with greater leaning conveying greater speed. Though this convention could derive from the natural leaning forward of people and animals as they run, it is also applied to depictions of inanimate objects (eg cars and trains). It is proposed that it is this convention that allows the italicization of text to convey notions of motion and speed. Evidence for this is obtained from three sources: the use of italicization on book covers (in book titles); judgments of typeface connotations; and performance measures during the semantic classification of words appearing in italicized and non-italicized fonts. Inspection of the availability of italic fonts in Hebrew indicates an additional artistic convention for conveying motion, based on a fundamental bias, yet to be confirmed, for people to expect to see, or prefer to see, lateral movement (real or implied) in a left to right direction, rather than a right to left direction. Evidence for such a bias is found in photographs of a range of animate and inanimate items archived on Google Images. Whereas a rightward bias is found for photographs of animate and inanimate items in motion (the more so, the faster the motion being conveyed), either no bias or a leftward bias is found for the same items in static pose. Possible origins of a fundamental left to right bias for visual motion, and future lines of research able to evaluate them, are identified.

Spatial biases in understanding descriptions of static scenes: the role of reading and writing direction

Prior studies on reasoning tasks have shown lateral spatial biases on mental model construction, which converge with known spatial biases in the mental representation of number, time, and events. The latter have been shown to be related to habitual reading and writing direction. The present study bridges and extends both research strands by looking at the processes of mental model construction in language comprehension and examining how they are influenced by reading and writing direction. Sentences like "the table is between the lamp and the TV" were auditorily presented to groups of mono- and bidirectional readers in languages with left-to-right or right-to-left scripts, and participants were asked to draw the described scene. There was a clear preference for deploying the lateral objects in the direction marked by the script of the input language and some hints of a much smaller effect of the degree of practice with the script. These lateral biases occurred in the context of universal strategies for working memory management.

Literacy shapes thought: the case of event representation in different cultures

There has been a lively debate whether conceptual representations of actions or scenes follow a left-to-right spatial transient when participants depict such events or scenes. It was even suggested that conceptualizing the agent on the left side represents a universal. We review the current literature with an emphasis on event representation and on cross-cultural studies. While there is quite some evidence for spatial bias for representations of events and scenes in diverse cultures, their extent and direction depend on task demands, one‘s native language, and importantly, on reading and writing direction. Whether transients arise only in subject-verb-object languages, due to their linear sentential position of event participants, is still an open issue. We investigated a group of illiterate speakers of Yucatec Maya, a language with a predominant verb-object-subject structure. They were compared to illiterate native speakers of Spanish. Neither group displayed a spatial transient. Given the current literature, we argue that learning to read and write has a strong impact on representations of actions and scenes. Thus, while it is still under debate whether language shapes thought, there is firm evidence that literacy does.

Script Directionality Affects Depiction of Depth in Representational Drawings

This research examined the influence of directional reading/writing habits on the representation of depth in a scene. Participants with English vs. Arabic language backgrounds were asked to represent an imagined scene containing two houses, a “near house” and a “far house.” Nearly all participants drew the near house larger than the far house and drew the near house before drawing the far house. However, significant group differences in spatial strategies and movement biases were noted. Whereas the majority of native English readers drew the near house on the left side of the page and the far house to the right of it, native Arabic readers showed a slight right bias in placement of the near house and tended to place the far house to the left of the near house. This effect of script direction characterized right-handed and left-handed users of each group. Taken together, the findings support a cultural account of asymmetries in representational drawing reflecting biases arising from prolonged experience in reading and writing in a particular direction.

Spatial Representation of Action Phrases Among Bidirectional Readers

Perceptual bias in simple visuospatial tasks, such as line bisection seen among healthy dextrals, has often been attributed to the hemispheric activation hypothesis. The often reported leftward perceptual bias was explained by an activation of the right hemisphere during visuospatial tasks. However, imposed scanning direction and stimuli saliency have also been used to explain these spatial asymmetries. One example of scanning direction is the well-trained one resulting from reading direction. Here, we present studies that target the role of reading direction on nonverbal tasks: line bisection, esthetic preference, and straight-ahead pointing by comparing left-to-right and right-to-left readers. The findings are discussed regarding the interaction between cultural factors, such as reading habits, and biological factors, such as cerebral lateralization.

Is the future the right time?

Spanish and English speakers tend to conceptualize time as running from left to right along a mental line. Previous research suggests that this representational strategy arises from the participants' exposure to a left-to-right writing system. However, direct evidence supporting this assertion suffers from several limitations and relies only on the visual modality. This study subjected to a direct test the reading hypothesis using an auditory task. Participants from two groups (Spanish and Hebrew) differing in the directionality of their orthographic system had to discriminate temporal reference (past or future) of verbs and adverbs (referring to either past or future) auditorily presented to either the left or right ear by pressing a left or a right key. Spanish participants were faster responding to past words with the left hand and to future words with the right hand, whereas Hebrew participants showed the opposite pattern. Our results demonstrate that the left-right mapping of time is not restricted to the visual modality and that the direction of reading accounts for the preferred directionality of the mental time line. These results are discussed in the context of a possible mechanism underlying the effects of reading direction on highly abstract conceptual representations.

Disembodying cognition

The idea that concepts are embodied by our motor and sensory systems is popular in current theorizing about cognition. Embodied cognition accounts come in different versions and are often contrasted with a purely symbolic amodal view of cognition. Simulation, or the hypothesis that concepts simulate the sensory and motor experience of real world encounters with instances of those concepts, has been prominent in psychology and cognitive neuroscience. Here, with a focus on spatial thought and language, I review some of the evidence cited in support of simulation versions of embodied cognition accounts. While these data are extremely interesting and many of the experiments are elegant, knowing how to best interpret the results is often far from clear. I point out that a quick acceptance of embodied accounts runs the danger of ignoring alternate hypotheses and not scrutinizing neuroscience data critically. I also review recent work from my lab that raises questions about the nature of sensory motor grounding in spatial thought and language. In my view, the question of whether or not cognition is grounded is more fruitfully replaced by questions about gradations in this grounding. A focus on disembodying cognition, or on graded grounding, opens the way to think about how humans abstract. Within neuroscience, I propose that three functional anatomic axes help frame questions about the graded nature of grounded cognition. First, are questions of laterality differences. Do association cortices in both hemispheres instantiate the same kind of sensory or motor information? Second, are questions about ventral dorsal axes. Do neuronal ensembles along this axis shift from conceptual representations of objects to the relationships between objects? Third, are questions about gradients centripetally from sensory and motor cortices towards and within perisylvian cortices. How does sensory and perceptual information become more language-like and then get transformed into language proper?

In hindsight, life flows from left to right

Three experiments investigated the mental representation of meaningful event sequences. Experiment 1 used extended (5 min long) naturalistic scenes excerpted from commercial movies. Experiments 2 and 3 presented everyday activities by means of sequences of six photographs. All experiments found both left-right and distance effects in an order decision task, suggesting that when contemplated in hindsight, experienced events unfold along a left-to-right analogical mental line. Present results are discussed in the context of the mental representation of other kinds of ordinal sequences, and other left-right effects reported in non-ordinal domains.

A case of impaired verbalization but preserved gesticulation of motion events

In most cultures, most of the time, when people talk they gesture. We took advantage of a rare opportunity to explore the relation between the verbalization and gesticulation of motion events by studying Marcel, an English speaker with a unilateral left-hemisphere lesion affecting frontal, parietal, and temporal sectors of the perisylvian cortex. Marcel has intact semantic knowledge of the three major classes of words that are commonly used in English descriptions of motion events - specifically, concrete nouns, action verbs, and spatial prepositions - as well as intact syntactic knowledge of how these word classes are typically combined in the intransitive motion construction (e.g., The ball rolled down the hill). However, his ability to retrieve the lexical-phonological structures of these words is severely impaired. Despite this profound anomia, he is still remarkably skilled at producing iconic manual depictions of motion events, as demonstrated in two experiments involving spontaneous gestures and one experiment involving elicited gestures. Moreover, the structural characteristics of Marcel's gestures are clearly sensitive to the idiosyncratic meanings of English verbs and prepositions, and they may also be sensitive to the way motion events are syntactically packaged in the intransitive motion construction. These findings improve our understanding of how some brain-damaged individuals with severe aphasia but without manual apraxia can successfully employ gesture to augment the semantic content of their speech.