Perceptual prioritization of self-associated voices

Not mine, it's yours: Liberal decision threshold underpins self-other memory prioritization

The Self-reference effect (SRE) is a well-established phenomenon of enhanced memory for self-associated stimuli. Previous research assumed the SRE is underpinned by an attentional boost enhancing early visual processing, as speculated in the Self-attention Network (SAN1). If so, self-relevance should counteract impaired memory for visually degraded stimuli (e.g., those presented at low contrast). In two experiments we investigated if the attentional enhancement attributed to the SAN occurs at the encoding (Experiment 1) or retrieval (Experiment 2) stage of recognition memory. Using a computerised version of the shopping task, participants sorted miscellaneous shopping objects into self- and other-owned bags (encoding) and then completed a surprise recognition source memory test, indicating if the object was self-, other-owned or not recognised (retrieval). Notably, in Experiment 1, half the objects appeared under low-contrast conditions during the object sorting task (encoding). Conversely, in Experiment 2 the low-contrast manipulation occurred in the subsequent surprise recognition source memory test. Contrary to previous findings, neither manipulation produced the SRE at the behavioural level, with equally accurate source memory for self- and other-owned objects. A Hierarchical Linear Ballistic Accumulator model probed the decisional mechanisms underlying memory performance. This model revealed a caution decision threshold for self- compared with other-owned objects potentially accounts for self-relevance memory advantages. This suggests decision-level factors contribute to the SRE and potentially play a larger role than early attentional factors.

Mine for life: Charting ownership effects in memory from adolescence to old age

This study investigates the self-reference effect (SRE) with an ownership memory task across several age groups, providing the first age exploration of implicit ownership memory biases from adolescence to older adulthood (N = 159). Using a well-established ownership task, participants were required to sort images of grocery items as belonging to themselves or to a fictitious unnamed Other. After sorting and a brief distractor task, participants completed a surprise one-step source memory test. Overall, there was a robust SRE, with greater source memory accuracy for self-owned items. The SRE attenuated with age, such that the magnitude of difference between self and other memory diminished into older adulthood. Importantly, these findings were not due to a deterioration of memory for self-owned items, but rather an increase in memory performance for other-owned items. Linear mixed effects analyses showed self-biases in reaction times, such that self-owned items were identified more rapidly compared with other owned items. Again, age interacted with this effect showing that the responses of older adults were slowed, especially for other-owned items. Several theoretical implications were drawn from these findings, but we suggest that older adults may not experience ownership-related biases to the same degree as younger adults. Consequently, SREs through the lens of mere ownership may attenuate with age.

Listen to yourself! Prioritization of self-associated and own voice cues

Self-cues such as one's own name or face attract attention, reflecting a bias for stimuli connected to self to be prioritized in cognition. Recent evidence suggests that even external voices can elicit this self-prioritization effect; in a voice-label matching task, external voices assigned to the Self-identity label 'you' elicited faster responses than those assigned to 'friend' or 'stranger' (Payne et al., Br. J. Psychology, 112, 585-610). However, it is not clear whether external voices assigned to Self are prioritized over participants' own voices. We explore this issue in two experiments. In Exp 1 (N = 35), a voice-label matching task comprising three external voices confirmed that reaction time and accuracy are improved when an external voice cue is assigned to Self rather than Friend or Stranger. In Exp 2 (N = 90), one of the voice cues was replaced with a recording of the participant's own voice. Reaction time and accuracy showed a consistent advantage for the participant's own-voice, even when it was assigned to the 'friend' or 'stranger' identity. These findings show that external voices can elicit self-prioritization effects if associated with Self, but they are not prioritized above individuals' own voices. This has implications for external voice production technology, suggesting own-voice imitation may be beneficial.

A multiverse assessment of the reliability of the self-matching task as a measurement of the self-prioritization effect

The self-matching task (SMT) is widely used to investigate the cognitive mechanisms underlying the self-prioritization effect (SPE), wherein performance is enhanced for self-associated stimuli compared to other-associated ones. Although the SMT robustly elicits the SPE, there is a lack of data quantifying the reliability of this paradigm. This is problematic, given the prevalence of the reliability paradox in cognitive tasks: many well-established cognitive tasks demonstrate relatively low reliability when used to evaluate individual differences, despite exhibiting replicable effects at the group level. To fill this gap, this preregistered study investigated the reliability of SPE derived from the SMT using a multiverse approach, combining all possible indicators and baselines reported in the literature. We first examined the robustness of 24 SPE measures across 42 datasets (N = 2250) using a meta-analytical approach. We then calculated the split-half reliability (r) and intraclass correlation coefficient (ICC2) for each SPE measure. Our findings revealed a robust group-level SPE across datasets. However, when evaluating individual differences, SPE indices derived from reaction time (RT) and efficiency exhibited relatively higher, compared to other SPE indices, but still unsatisfied split-half reliability (approximately 0.5). The reliability across multiple time points, as assessed by ICC2, RT, and efficiency, demonstrated moderate levels of test-retest reliability (close to 0.5). These findings revealed the presence of a reliability paradox in the context of SMT-based SPE assessment. We discussed the implications of how to enhance individual-level reliability using this paradigm for future study design.

Embodiment in the enfacement illusion is mediated by self-other overlap

The enfacement illusion is a facial version of the rubber hand illusion, in which participants experience tactile stimulation of their own faces synchronously with the observation of the same stimulation applied to another's face. In previous studies, participants have reported experiencing an illusory embodiment of the other's face following synchronous compared to asynchronous stimulation. In a series of three experiments, we addressed the following three questions: (i) how does similarity between the self and the other, operationalized here as being of the same or different gender to the other, impact the experience of embodiment in the enfacement illusion; (ii) does the experience of embodiment result from alterations to the self-concept; and (iii) is susceptibility to the experience of embodiment associated with interoceptive processing, i.e. perception of the internal state of the body? Results indicate that embodiment is facilitated by the similarity between the self and the other and is mediated by the incorporation of the other into the self-concept, but sensitivity to one's own internal states does not impact upon embodiment within the enfacement illusion. This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Self-ownership, not self-production, modulates bias and agency over a synthesised voice

Voices are fundamentally social stimuli, and their importance to the self may be underpinned by how far they can be used to express the self and achieve communicative goals. This paper examines how self-bias and agency over a synthesised voice is altered when that voice is used to represent the self in social interaction. To enable participants to use a new voice, a novel two-player game was created, in which participants communicated online using a text-to-speech (TTS) synthesised voice. We then measured self-bias and sense of agency attributed to this synthesised voice, comparing participants who had used their new voice to interact with another person (n = 44) to a control group of participants (n = 44) who had been only briefly exposed to the voices. We predicted that the new, synthesised self-voice would be more perceptually prioritised after it had been self-produced. Further, that participants' sense of agency over the voice would be increased, if they had experienced self-producing the voice, relative to those who only owned it. Contrary to the hypothesis, the results indicated that both experimental participants and control participants similarly prioritised the new synthesised voice and experienced a similar degree of agency over it, relative to voices owned by others. Critically then, being able to produce the new voice in a social interaction did not modulate bias towards it nor participant's sense of agency over it. These results suggest that merely having ownership over a new voice may be sufficient to generate a perceptual bias and a sense of agency over it.

You have interrupted me again!: making voice assistants more dementia-friendly with incremental clarification

In spontaneous conversation, speakers seldom have a full plan of what they are going to say in advance: they need to conceptualise and plan incrementally as they articulate each word in turn. This often leads to long pauses mid-utterance. Listeners either wait out the pause, offer a possible completion, or respond with an incremental clarification request (iCR), intended to recover the rest of the truncated turn. The ability to generate iCRs in response to pauses is therefore important in building natural and robust everyday voice assistants (EVA) such as Amazon Alexa. This becomes crucial with people with dementia (PwDs) as a target user group since they are known to pause longer and more frequently, with current state-of-the-art EVAs interrupting them prematurely, leading to frustration and breakdown of the interaction. In this article, we first use two existing corpora of truncated utterances to establish the generation of clarification requests as an effective strategy for recovering from interruptions. We then proceed to report on, analyse, and release SLUICE-CR: a new corpus of 3,000 crowdsourced, human-produced iCRs, the first of its kind. We use this corpus to probe the incremental processing capability of a number of state-of-the-art large language models (LLMs) by evaluating (1) the quality of the model's generated iCRs in response to incomplete questions and (2) the ability of the said LLMs to respond correctly after the users response to the generated iCR. For (1), our experiments show that the ability to generate contextually appropriate iCRs only emerges at larger LLM sizes and only when prompted with example iCRs from our corpus. For (2), our results are in line with (1), that is, that larger LLMs interpret incremental clarificational exchanges more effectively. Overall, our results indicate that autoregressive language models (LMs) are, in principle, able to both understand and generate language incrementally and that LLMs can be configured to handle speech phenomena more commonly produced by PwDs, mitigating frustration with today's EVAs by improving their accessibility.

Attention and emotion shape self-voice prioritization in speech processing

Both self-voice and emotional speech are salient signals that are prioritized in perception. Surprisingly, self-voice perception has been investigated to a lesser extent than the self-face. Therefore, it remains to be clarified whether self-voice prioritization is boosted by emotion, and whether self-relevance and emotion interact differently when attention is focused on who is speaking vs. what is being said. Thirty participants listened to 210 prerecorded words spoken in one's own or an unfamiliar voice and differing in emotional valence in two tasks, manipulating the attention focus on either speaker identity or speech emotion. Event-related potentials (ERP) of the electroencephalogram (EEG) informed on the temporal dynamics of self-relevance, emotion, and attention effects. Words spoken in one's own voice elicited a larger N1 and Late Positive Potential (LPP), but smaller N400. Identity and emotion interactively modulated the P2 (self-positivity bias) and LPP (self-negativity bias). Attention to speaker identity modulated more strongly ERP responses within 600 ms post-word onset (N1, P2, N400), whereas attention to speech emotion altered the late component (LPP). However, attention did not modulate the interaction of self-relevance and emotion. These findings suggest that the self-voice is prioritized for neural processing at early sensory stages, and that both emotion and attention shape self-voice prioritization in speech processing. They also confirm involuntary processing of salient signals (self-relevance and emotion) even in situations in which attention is deliberately directed away from those cues. These findings have important implications for a better understanding of symptoms thought to arise from aberrant self-voice monitoring such as auditory verbal hallucinations.

Is self always prioritised? Attenuating the ownership self-reference effect in memory

The current study demonstrates the abolishment of the Ownership Self Reference Effect (OSRE) when elaborate details of a distant other-referent are provided. In a 2 (High versus Low information) × 2 (Self versus Other) experimental design, we tested the capacity for the SRE to be modulated with social saliency. Using a well-established ownership paradigm (Collard et al., 2020; Cunningham et al., 2008; Sparks et al., 2016), when the other was made socially salient (i.e. details and characteristics about the other were provided to the participant prior to encoding), no SRE emerged, such that self-owned and other-owned items were recalled with comparable accuracy. In contrast, when the other was not salient (i.e., no details about them were provided), participants accurately recalled a higher proportion of self-owned items, demonstrating a typical SRE in source memory. The degree of self- or other- referencing was not related to measured variables of closeness, similarity or shared traits with the other. Although the SRE is an established and robust effect, the findings of the current study illustrate critical circumstances in which the self is no longer prioritised above the other. In line with our predictions, we suggest that the self has automatic attributed social salience (e.g. through ownership) and that enhancing social salience by elaborating details of the other, prioritisation can expand to encapsulate an other beyond the self and influence incidental memory.

Self-prioritization with unisensory and multisensory stimuli in a matching task

A shape-label matching task is commonly used to examine the self-advantage in motor reaction-time responses (the Self-Prioritization Effect; SPE). In the present study, auditory labels were introduced, and, for the first time, responses to unisensory auditory, unisensory visual, and multisensory object-label stimuli were compared across block-type (i.e., trials blocked by sensory modality type, and intermixed trials of unisensory and multisensory stimuli). Auditory stimulus intensity was presented at either 50 dB (Group 1) or 70 dB (Group 2). The participants in Group 2 also completed a multisensory detection task, making simple speeded motor responses to the shape and sound stimuli and their multisensory combinations. In the matching task, the SPE was diminished in intermixed trials, and in responses to the unisensory auditory stimuli as compared with the multisensory (visual shape+auditory label) stimuli. In contrast, the SPE did not differ in responses to the unisensory visual and multisensory (auditory object+visual label) stimuli. The matching task was associated with multisensory ‘costs’ rather than gains, but response times to self- versus stranger-associated stimuli were differentially affected by the type of multisensory stimulus (auditory object+visual label or visual shape+auditory label). The SPE was thus modulated both by block-type and the combination of object and label stimulus modalities. There was no SPE in the detection task. Taken together, these findings suggest that the SPE with unisensory and multisensory stimuli is modulated by both stimulus- and task-related parameters within the matching task. The SPE does not transfer to a significant motor speed gain when the self-associations are not task-relevant.

Instrumental learning in social interactions: Trait learning from faces and voices

Recent research suggests that reinforcement learning may underlie trait formation in social interactions with faces. The current study investigated whether the same learning mechanisms could be engaged for trait learning from voices. On each trial of a training phase, participants (N = 192) chose from pairs of human or slot machine targets that varied in the (1) reward value and (2) generosity of their payouts. Targets were either auditory (voices or tones; Experiment 1) or visual (faces or icons; Experiment 2) and were presented sequentially before payout feedback. A test phase measured participant choice behaviour, and a post-test recorded their target preference ratings. For auditory targets, we found a significant effect of reward only on target choices, but saw higher preference ratings for more generous humans and slot machines. For visual targets, findings from previous studies were replicated: participants learned about both generosity and reward, but generosity was prioritised in the human condition. These findings provide one of the first demonstrations of reinforcement learning of reward with auditory stimuli in a social learning task, but suggest that the use of auditory targets does alter learning in this paradigm. Conversely, reinforcement learning of reward and trait information with visual stimuli remains intact even when sequential presentation introduces a delay in feedback.