The Listener as Addressee in Face-to-Face Dialogue

Digital connection, real bonding: Brief online chats boost interpersonal closeness regardless of the conversational topic

This study explores how the quality of brief dyadic written exchanges (lasting under 5 min) on a virtual platform and the nature of the conversational topic (abstract or concrete), influences physical, interpersonal, and psychological closeness between interlocutors. In the first experiment, participants engaged in written conversations on either an abstract or concrete topic under two conditions: (i) an interactive condition, where participants exchanged messages with another person, and (ii) a non-interactive condition, where participants wrote independently on the same topic, aware that another person was simultaneously doing the same. Results indicated that participants in the interactive condition reported feeling significantly closer to their interlocutor than those in the non-interactive condition. In addition, greater perceived pleasantness, intimacy, and the importance of the other person's contribution to the conversation were associated with increased feelings of closeness. However, inconclusive evidence was obtained regarding the interaction of the other person's contribution with the abstractness of the conversational topic during the written exchanges in fostering feelings of closeness. The second experiment focused only on the interactive condition, where we examined interpersonal dynamics across different subcategories of abstract (e.g., philosophical/spiritual, emotional, social, physical/spatio-temporal) and concrete topics (e.g., tools, animals, food). The results of the first experiment were replicated, reinforcing the idea that the quality of the virtual exchange-rather than the topic itself-plays a crucial role in fostering feelings of closeness between individuals.

Zooming in and out of semantics: proximal-distal construal levels and prominence hierarchies

We argue that the "Prominence Hierarchy" within linguistics can be subsumed under the "Construal Level Theory" within psychology and that a wide spectrum of grammatical phenomena, ranging from case assignment to number, definiteness, verbal agreement, voice, direct/inverse morphology, and syntactic word-order respond to Prominence Hierarchies (PH), or semantic scales. In fact, the field of prominence hierarchies, as expressed through the languages of the world, continues to be riddled with riddles. We identify a set of conundrums: (A) vantage point and animacy, (B) individuation and narrow reference phenomena, (C) fronting mechanisms, (D) abstraction, and (E) cultural variance and flexibility. We here propose an account for the existence of these hierarchies and their pervasive effects on grammar by relying on psychological Construal Level Theory (CLT). We suggest that both PH and CLT structure the external world according to proximity or distance from the "Me, Here and Now" (MHN) perspective. In language, MHN has the effect of structuring grammars; in cognition, it structures our lives, our preferences, and choices.

Head motion synchrony in unidirectional and bidirectional verbal communication

Interpersonal communication includes verbal and nonverbal communication. Verbal communication comprises one-way (e.g., a speech or lecture) and interactive verbal communication (e.g., daily conversations or meetings), which we frequently encounter. Nonverbal communication has considerable influence on interpersonal communication, and body motion synchrony is known to be an important factor for successful communication and social interaction. However, most research on body motion synchrony has been elucidated by either the setting of one-way verbal transmission or the verbal interaction setting, and it remains unclear whether verbal directionality and interactivity affect body motion synchrony. One-way and two-way (interactive) verbal communication is implicated in designed or undesigned leader-follower relationships, and also in the complexity and diversity of interpersonal interactions, where two-way verbal communication is more complex and diverse than in the one-way condition. In this study, we tested head motion synchrony between the one-way verbal communication condition (in which the roles of the speaker and listener are fixed) and the two-way verbal communication condition (where the speaker and listener can freely engage in a conversation). Therefore, although no statistically significant difference in synchrony activity (relative frequency) was found, a statistically significant difference was observed in synchrony direction (temporal lead-lag structure as mimicry) and intensity. Specifically, the synchrony direction in two-way verbal communication was close to zero, but this in one-way verbal communication was synchronized with the listener's movement predominantly delayed. Furthermore, synchrony intensity, in terms of the degree of variation in the phase difference distribution, was significantly higher in the one-way verbal communication than in the two-way condition, with bigger time-shifts being observed in the latter. This result suggests that verbal interaction does not affect the overall frequency of head motion synchrony but does affect the temporal lead-lag structure and coherence.

The CANDOR corpus: Insights from a large multimodal dataset of naturalistic conversation

People spend a substantial portion of their lives engaged in conversation, and yet, our scientific understanding of conversation is still in its infancy. Here, we introduce a large, novel, and multimodal corpus of 1656 conversations recorded in spoken English. This 7+ million word, 850-hour corpus totals more than 1 terabyte of audio, video, and transcripts, with moment-to-moment measures of vocal, facial, and semantic expression, together with an extensive survey of speakers' postconversation reflections. By taking advantage of the considerable scope of the corpus, we explore many examples of how this large-scale public dataset may catalyze future research, particularly across disciplinary boundaries, as scholars from a variety of fields appear increasingly interested in the study of conversation.

Shared reality, memory goal satisfaction, and psychological well-being during conversational remembering

Conversational remembering, or sharing autobiographical memories with others, occurs frequently in everyday communication. The current project examined how the experience of shared reality with a conversation partner when describing autobiographical memories to them can operate to enhance the self, social, and directive uses of a recalled memory and explored the role of shared reality experienced as a result of conversational remembering in psychological well-being. In this project, conversational remembering was examined using experimental (Study 1) and daily diary (Study 2) methodologies. Results indicated that experiencing a shared reality during conversational remembering of an autobiographical memory enhanced self, social, and directive memory goal fulfilment and was positively associated with greater psychological well-being. The current investigation highlights important benefits of sharing our life stories with others, especially those with whom we develop a sense of shared reality.

The puzzle of ideography

An ideography is a general-purpose code made of pictures that do not encode language, which can be used autonomously - not just as a mnemonic prop - to encode information on a broad range of topics. Why are viable ideographies so hard to find? I contend that self-sufficient graphic codes need to be narrowly specialized. Writing systems are only an apparent exception: At their core, they are notations of a spoken language. Even if they also encode nonlinguistic information, they are useless to someone who lacks linguistic competence in the encoded language or a related one. The versatility of writing is thus vicarious: Writing borrows it from spoken language. Why is it so difficult to build a fully generalist graphic code? The most widespread answer points to a learnability problem. We possess specialized cognitive resources for learning spoken language, but lack them for graphic codes. I argue in favor of a different account: What is difficult about graphic codes is not so much learning or teaching them as getting every user to learn and teach the same code. This standardization problem does not affect spoken or signed languages as much. Those are based on cheap and transient signals, allowing for easy online repairing of miscommunication, and require face-to-face interactions where the advantages of common ground are maximized. Graphic codes lack these advantages, which makes them smaller in size and more specialized.

Facial Emotion Expressions in Human–Robot Interaction: A Survey

Facial expressions are an ideal means of communicating one’s emotions or intentions to others. This overview will focus on human facial expression recognition as well as robotic facial expression generation. In the case of human facial expression recognition, both facial expression recognition on predefined datasets as well as in real-time will be covered. For robotic facial expression generation, hand-coded and automated methods i.e., facial expressions of a robot are generated by moving the features (eyes, mouth) of the robot by hand-coding or automatically using machine learning techniques, will also be covered. There are already plenty of studies that achieve high accuracy for emotion expression recognition on predefined datasets, but the accuracy for facial expression recognition in real-time is comparatively lower. In the case of expression generation in robots, while most of the robots are capable of making basic facial expressions, there are not many studies that enable robots to do so automatically. In this overview, state-of-the-art research in facial emotion expressions during human–robot interaction has been discussed leading to several possible directions for future research.

Connection Heals Wounds: Feeling Listened to Reduces Speakers' Loneliness Following a Social Rejection Disclosure

Memories of rejection contribute to feeling lonely. However, high-quality listening that conveys well-meaning attention and understanding when speakers discuss social rejection may help them to reconnect. Speakers may experience less loneliness because they feel close and connected (relatedness) to the listener and because listening supports self-congruent expression (autonomy). Five experiments (total N = 1,643) manipulated listening during visualized (Studies 1, 4, 5) and actual (Studies 2, 3) conversations. We used different methods (video vignettes; in-person; computer-mediated; recall; written scenarios) to compare high-quality with regular (all studies) and poor (Study 1) listening. Findings across studies showed that high-quality listening reduced speakers' state loneliness after they shared past experiences of social rejection. Parallel mediation analyses indicated that both feeling related to the listener and autonomy satisfaction (particularly its self-congruence component; Study 5) mediated the effect of listening on loneliness. These results provide novel insights into the hitherto unexplored effect of listening on state loneliness.

Feeling Heard: Experiences of Listening (or Not) at Work

Listening has been identified as a key workplace skill, important for ensuring high-quality communication, building relationships, and motivating employees. However, recent research has increasingly suggested that speaker perceptions of good listening do not necessarily align with researcher or listener conceptions of good listening. While many of the benefits of workplace listening rely on employees feeling heard, little is known about what constitutes this subjective perception. To better understand what leaves employees feeling heard or unheard, we conducted 41 interviews with bank employees, who collectively provided 81 stories about listening interactions they had experienced at work. Whereas, prior research has typically characterized listening as something that is perceived through responsive behaviors within conversation, our findings suggest conversational behaviors alone are often insufficient to distinguish between stories of feeling heard vs. feeling unheard. Instead, our interviewees felt heard or unheard only when listeners met their subjective needs and expectations. Sometimes their needs and expectations could be fulfilled through conversation alone, and other times action was required. Notably, what would be categorized objectively as good listening during an initial conversation could be later counteracted by a failure to follow-through in ways expected by the speaker. In concert, these findings contribute to both theory and practice by clarifying how listening behaviors take on meaning from the speakers' perspective and the circumstances under which action is integral to feeling heard. Moreover, they point toward the various ways listeners can engage to help speakers feel heard in critical conversations.

Towards an Engagement-Aware Attentive Artificial Listener for Multi-Party Interactions

Listening to one another is essential to human-human interaction. In fact, we humans spend a substantial part of our day listening to other people, in private as well as in work settings. Attentive listening serves the function to gather information for oneself, but at the same time, it also signals to the speaker that he/she is being heard. To deduce whether our interlocutor is listening to us, we are relying on reading his/her nonverbal cues, very much like how we also use non-verbal cues to signal our attention. Such signaling becomes more complex when we move from dyadic to multi-party interactions. Understanding how humans use nonverbal cues in a multi-party listening context not only increases our understanding of human-human communication but also aids the development of successful human-robot interactions. This paper aims to bring together previous analyses of listener behavior analyses in human-human multi-party interaction and provide novel insights into gaze patterns between the listeners in particular. We are investigating whether the gaze patterns and feedback behavior, as observed in the human-human dialogue, are also beneficial for the perception of a robot in multi-party human-robot interaction. To answer this question, we are implementing an attentive listening system that generates multi-modal listening behavior based on our human-human analysis. We are comparing our system to a baseline system that does not differentiate between different listener types in its behavior generation. We are evaluating it in terms of the participant’s perception of the robot, his behavior as well as the perception of third-party observers.

Comparing Frequency of Listener Responses Between Adolescents with and Without ASD During Conversation

In conversation, the listener plays an active role in conversation success, specifically by providing listener feedback which signals comprehension and interest. Previous work has shown that frequency of feedback positively correlates with conversation success. Because individuals with ASD are known to struggle with various conversational skills, e.g., turn-taking and commenting, this study examines their use of listener feedback by comparing the frequency of feedback produced by 20 adolescents with ASD and 23 neurotypical (NT) adolescents. We coded verbal and nonverbal listener feedback during the time when participants were listening in a semi-structured interview with a research assistant. Results show that ASD participants produced significantly fewer instances of listener feedback than NT adolescents, which likely contributes to difficulties with social interactions.

Predictions of Miscommunication in Verbal Communication During Collaborative Joint Action

Purpose The purpose of the current study was to examine the lexical and pragmatic factors that may contribute to turn-by-turn failures in communication (i.e., miscommunication) that arise regularly in interactive communication. Method Using a corpus from a collaborative dyadic building task, we investigated what differentiated successful from unsuccessful communication and potential factors associated with the choice to provide greater lexical information to a conversation partner. Results We found that more successful dyads' language tended to be associated with greater lexical density, lower ambiguity, and fewer questions. We also found participants were more lexically dense when accepting and integrating a partner's information (i.e., grounding) but were less lexically dense when responding to a question. Finally, an exploratory analysis suggested that dyads tended to spend more lexical effort when responding to an inquiry and used assent language accurately-that is, only when communication was successful. Conclusion Together, the results suggest that miscommunication both emerges and benefits from ambiguous and lexically dense utterances.

Gender, experimenter gender and medium of report influence the content of autobiographical memory report

In this study, we examined the role of context in autobiographical memory narratives, specifically as it pertains to gender among emerging adults. Male and female participants reported stressful events in their lives in the presence of an experimenter, and were randomly assigned either to report events verbally or type them, and to report in the presence of a male or female experimenter. Narratives were coded for factual and interpretive content. Results revealed that men verbally reporting to women reported longer narratives than all other groups. Women's narrative length did not vary by medium of report or conversational partner, but women used proportionally fewer internal state phrases when verbally reporting to men than when reporting to women. Women also used proportionally fewer evaluative statements in verbal reports than in typed narratives. Of these important interactions among context, gender, and experimenter gender, some findings, such as men's longer narratives and women's reduced internal states, were counter to expectations. These findings highlight the importance of methodological influences in autobiographical memory studies, in regard to both the context generated by experimental methods, and how gender differences are understood.

Perception of 'Back-Channeling' Nonverbal Feedback in Musical Duo Improvisation

In witnessing face-to-face conversation, observers perceive authentic communication according to the social contingency of nonverbal feedback cues (‘back-channeling’) by non-speaking interactors. The current study investigated the generality of this function by focusing on nonverbal communication in musical improvisation. A perceptual experiment was conducted to test whether observers can reliably identify genuine versus fake (mismatched) duos from musicians’ nonverbal cues, and how this judgement is affected by observers’ musical background and rhythm perception skill. Twenty-four musicians were recruited to perform duo improvisations, which included solo episodes, in two styles: standard jazz (where rhythm is based on a regular pulse) or free improvisation (where rhythm is non-pulsed). The improvisations were recorded using a motion capture system to generate 16 ten-second point-light displays (with audio) of the soloist and the silent non-soloing musician (‘back-channeler’). Sixteen further displays were created by splicing soloists with back-channelers from different duos. Participants (N = 60) with various musical backgrounds were asked to rate the point-light displays as either real or fake. Results indicated that participants were sensitive to the real/fake distinction in the free improvisation condition independently of musical experience. Individual differences in rhythm perception skill did not account for performance in the free condition, but were positively correlated with accuracy in the standard jazz condition. These findings suggest that the perception of back-channeling in free improvisation is not dependent on music-specific skills but is a general ability. The findings invite further study of the links between interpersonal dynamics in conversation and musical interaction.

Timing in turn-taking and its implications for processing models of language

The core niche for language use is in verbal interaction, involving the rapid exchange of turns at talking. This paper reviews the extensive literature about this system, adding new statistical analyses of behavioral data where they have been missing, demonstrating that turn-taking has the systematic properties originally noted by Sacks et al. (1974; hereafter SSJ). This system poses some significant puzzles for current theories of language processing: the gaps between turns are short (of the order of 200 ms), but the latencies involved in language production are much longer (over 600 ms). This seems to imply that participants in conversation must predict (or 'project' as SSJ have it) the end of the current speaker's turn in order to prepare their response in advance. This in turn implies some overlap between production and comprehension despite their use of common processing resources. Collecting together what is known behaviorally and experimentally about the system, the space for systematic explanations of language processing for conversation can be significantly narrowed, and we sketch some first model of the mental processes involved for the participant preparing to speak next.

Unaddressed participants' gaze in multi-person interaction: optimizing recipiency

One of the most intriguing aspects of human communication is its turn-taking system. It requires the ability to process on-going turns at talk while planning the next, and to launch this next turn without considerable overlap or delay. Recent research has investigated the eye movements of observers of dialogs to gain insight into how we process turns at talk. More specifically, this research has focused on the extent to which we are able to anticipate the end of current and the beginning of next turns. At the same time, there has been a call for shifting experimental paradigms exploring social-cognitive processes away from passive observation toward on-line processing. Here, we present research that responds to this call by situating state-of-the-art technology for tracking interlocutors’ eye movements within spontaneous, face-to-face conversation. Each conversation involved three native speakers of English. The analysis focused on question–response sequences involving just two of those participants, thus rendering the third momentarily unaddressed. Temporal analyses of the unaddressed participants’ gaze shifts from current to next speaker revealed that unaddressed participants are able to anticipate next turns, and moreover, that they often shift their gaze toward the next speaker before the current turn ends. However, an analysis of the complex structure of turns at talk revealed that the planning of these gaze shifts virtually coincides with the points at which the turns first become recognizable as possibly complete. We argue that the timing of these eye movements is governed by an organizational principle whereby unaddressed participants shift their gaze at a point that appears interactionally most optimal: It provides unaddressed participants with access to much of the visual, bodily behavior that accompanies both the current speaker’s and the next speaker’s turn, and it allows them to display recipiency with regard to both speakers’ turns.

Is nonverbal communication disrupted in interactions involving patients with schizophrenia?

BACKGROUND

Nonverbal communication is a critical feature of successful social interaction and interpersonal rapport. Social exclusion is a feature of schizophrenia. This experimental study investigated if the undisclosed presence of a patient with schizophrenia in interaction changes nonverbal communication (ie, speaker gesture and listener nodding).

METHOD

3D motion-capture techniques recorded 20 patient (1 patient, 2 healthy participants) and 20 control (3 healthy participants) interactions. Participants rated their experience of rapport with each interacting partner. Patients' symptoms, social cognition, and executive functioning were assessed. Four hypotheses were tested: (1) Compared to controls, patients display less speaking gestures and listener nods. (2) Patients' increased symptom severity and poorer social cognition are associated with patients' reduced gesture and nods. (3) Patients' partners compensate for patients' reduced nonverbal behavior by gesturing more when speaking and nodding more when listening. (4) Patients' reduced nonverbal behavior, increased symptom severity, and poorer social cognition are associated with others experiencing poorer rapport with the patient.

RESULTS

Patients gestured less when speaking. Patients with more negative symptoms nodded less as listeners, while their partners appeared to compensate by gesturing more as speakers. Patients with more negative symptoms also gestured more when speaking, which, alongside increased negative symptoms and poorer social cognition, was associated with others experiencing poorer patient rapport.

CONCLUSIONS

Patients' symptoms are associated with the nonverbal behavior of patients and their partners. Patients' increased negative symptoms and gesture use are associated with poorer interpersonal rapport. This study provides specific evidence about how negative symptoms impact patients' social interactions.