Right-lateralized unconscious, but not conscious, processing of affective environmental sounds

Listening to two speakers: Capacity and tradeoffs in neural speech tracking during Selective and Distributed Attention

Speech comprehension is severely compromised when several people talk at once, due to limited perceptual and cognitive resources. In such circumstances, top-down attention mechanisms can actively prioritize processing of task-relevant speech. However, behavioral and neural evidence suggest that this selection is not exclusive, and the system may have sufficient capacity to process additional speech input as well. Here we used a data-driven approach to contrast two opposing hypotheses regarding the system's capacity to co-represent competing speech: Can the brain represent two speakers equally or is the system fundamentally limited, resulting in tradeoffs between them? Neural activity was measured using magnetoencephalography (MEG) as human participants heard concurrent speech narratives and engaged in two tasks: Selective Attention, where only one speaker was task-relevant and Distributed Attention, where both speakers were equally relevant. Analysis of neural speech-tracking revealed that both tasks engaged a similar network of brain regions involved in auditory processing, attentional control and speech processing. Interestingly, during both Selective and Distributed Attention the neural representation of competing speech showed a bias towards one speaker. This is in line with proposed 'bottlenecks' for co-representation of concurrent speech and suggests that good performance on distributed attention tasks may be achieved by toggling attention between speakers over time.

An ecological investigation of the capacity to follow simultaneous speech and preferential detection of ones’ own name

Many situations require focusing attention on one speaker, while monitoring the environment for potentially important information. Some have proposed that dividing attention among 2 speakers involves behavioral trade-offs, due to limited cognitive resources. However the severity of these trade-offs, particularly under ecologically-valid circumstances, is not well understood. We investigated the capacity to process simultaneous speech using a dual-task paradigm simulating task-demands and stimuli encountered in real-life. Participants listened to conversational narratives (Narrative Stream) and monitored a stream of announcements (Barista Stream), to detect when their order was called. We measured participants’ performance, neural activity, and skin conductance as they engaged in this dual-task. Participants achieved extremely high dual-task accuracy, with no apparent behavioral trade-offs. Moreover, robust neural and physiological responses were observed for target-stimuli in the Barista Stream, alongside significant neural speech-tracking of the Narrative Stream. These results suggest that humans have substantial capacity to process simultaneous speech and do not suffer from insufficient processing resources, at least for this highly ecological task-combination and level of perceptual load. Results also confirmed the ecological validity of the advantage for detecting ones’ own name at the behavioral, neural, and physiological level, highlighting the contribution of personal relevance when processing simultaneous speech.

Linguistic processing of task-irrelevant speech at a cocktail party

Paying attention to one speaker in a noisy place can be extremely difficult, because to-be-attended and task-irrelevant speech compete for processing resources. We tested whether this competition is restricted to acoustic-phonetic interference or if it extends to competition for linguistic processing as well. Neural activity was recorded using Magnetoencephalography as human participants were instructed to attend to natural speech presented to one ear, and task-irrelevant stimuli were presented to the other. Task-irrelevant stimuli consisted either of random sequences of syllables, or syllables structured to form coherent sentences, using hierarchical frequency-tagging. We find that the phrasal structure of structured task-irrelevant stimuli was represented in the neural response in left inferior frontal and posterior parietal regions, indicating that selective attention does not fully eliminate linguistic processing of task-irrelevant speech. Additionally, neural tracking of to-be-attended speech in left inferior frontal regions was enhanced when competing with structured task-irrelevant stimuli, suggesting inherent competition between them for linguistic processing.

The Interpersonal Neurobiology of Intersubjectivity

In 1975, Colwyn Trevarthen first presented his groundbreaking explorations into the early origins of human intersubjectivity. His influential model dictates that, during intimate and playful spontaneous face-to-face protoconversations, the emotions of both the 2-3-month-old infant and mother are nonverbally communicated, perceived, mutually regulated, and intersubjectively shared. This primordial basic interpersonal interaction is expressed in synchronized rhythmic-turn-taking transactions that promote the intercoordination and awareness of positive brain states in both. In this work, I offer an interpersonal neurobiological model of Trevarthen's intersubjective protoconversations as rapid, reciprocal, bidirectional visual-facial, auditory-prosodic, and tactile-gestural right brain-to-right brain implicit nonverbal communications between the psychobiologically attuned mother and the developing infant. These co-constructed positive emotional interactions facilitate the experience-dependent maturation of the infant's right brain, which is in an early critical period of growth. I then address the central role of interpersonal synchrony in intersubjectivity, expressed in a mutual alignment or coupling between the minds and bodies of the mother and infant in face-to-face protoconversations, as well as how these right brain-to-right brain emotional transmissions generate bioenergetic positively charged interbrain synchrony within the dyad. Following this, I offer recent brain laterality research on the essential functions of the right temporoparietal junction, a central node of the social brain, in face-to-face nonverbal communications. In the next section, I describe the ongoing development of the protoconversation over the 1st year and beyond, and the co-creation of a fundamental energy-dependent, growth-promoting social emotional matrix that facilitates the emergence of the highly adaptive human functions of mutual play and mutual love. In the final section, I discuss the clinical applications of this interpersonal neurobiological model of intersubjectivity, which has a long history in the psychotherapy literature. Toward that end, I offer very recent paradigm-shifting hyperscanning research that simultaneously measures both the patient and therapist during a psychotherapeutic interaction. Using the Trevarthen's two-person intersubjective model, this research demonstrates changes in both brains of the therapeutic dyad and the critical role of nonverbal communications in an emotionally-focused psychotherapy session. These studies specifically document interbrain synchronization between the right temporoparietal junction of the patient and the right temporoparietal junction of the clinician, a right brain-to-right brain nonverbal communication system in the co-constructed therapeutic alliance. Lastly, I discuss the relationship between the affect communicating functions of the intersubjective motivational system and the affect regulating functions of the attachment motivational system.

Detecting (Un)seen Change: The Neural Underpinnings of (Un)conscious Prediction Errors

Detecting changes in the environment is fundamental for our survival. According to predictive coding theory, detecting these irregularities relies both on incoming sensory information and our top-down prior expectations (or internal generative models) about the world. Prediction errors (PEs), detectable in event-related potentials (ERPs), occur when there is a mismatch between the sensory input and our internal model (i.e., a surprise event). Many changes occurring in our environment are irrelevant for survival and may remain unseen. Such changes, even if subtle, can nevertheless be detected by the brain without emerging into consciousness. What remains unclear is how these changes are processed in the brain at the network level. Here, we used a visual oddball paradigm in which participants engaged in a central letter task during electroencephalographic (EEG) recordings while presented with task-irrelevant high- or low-coherence background, random-dot motion. Critically, once in a while, the direction of the dots changed. After the EEG session, we confirmed that changes in motion direction at high- and low-coherence were visible and invisible, respectively, using psychophysical measurements. ERP analyses revealed that changes in motion direction elicited PE regardless of the visibility, but with distinct spatiotemporal patterns. To understand these responses, we applied dynamic causal modeling (DCM) to the EEG data. Bayesian Model Averaging showed visible PE relied on a release from adaptation (repetition suppression) within bilateral MT+, whereas invisible PE relied on adaptation at bilateral V1 (and left MT+). Furthermore, while feedforward upregulation was present for invisible PE, the visible change PE also included downregulation of feedback between right MT+ to V1. Our findings reveal a complex interplay of modulation in the generative network models underlying visible and invisible motion changes.

Evidence Integration in Natural Acoustic Textures during Active and Passive Listening

Many natural sounds can be well described on a statistical level, for example, wind, rain, or applause. Even though the spectro-temporal profile of these acoustic textures is highly dynamic, changes in their statistics are indicative of relevant changes in the environment. Here, we investigated the neural representation of change detection in natural textures in humans, and specifically addressed whether active task engagement is required for the neural representation of this change in statistics. Subjects listened to natural textures whose spectro-temporal statistics were modified at variable times by a variable amount. Subjects were instructed to either report the detection of changes (active) or to passively listen to the stimuli. A subset of passive subjects had performed the active task before (passive-aware vs passive-naive). Psychophysically, longer exposure to pre-change statistics was correlated with faster reaction times and better discrimination performance. EEG recordings revealed that the build-up rate and size of parieto-occipital (PO) potentials reflected change size and change time. Reduced effects were observed in the passive conditions. While P2 responses were comparable across conditions, slope and height of PO potentials scaled with task involvement. Neural source localization identified a parietal source as the main contributor of change-specific potentials, in addition to more limited contributions from auditory and frontal sources. In summary, the detection of statistical changes in natural acoustic textures is predominantly reflected in parietal locations both on the skull and source level. The scaling in magnitude across different levels of task involvement suggests a context-dependent degree of evidence integration.

Emotion Elicitation: A Comparison of Pictures and Films

Pictures and film clips are widely used and accepted stimuli to elicit emotions. Based on theoretical arguments it is often assumed that the emotional effects of films exceed those of pictures, but to date this assumption has not been investigated directly. The aim of the present study was to compare pictures and films in terms of their capacity to induce emotions verified by means of explicit measures. Stimuli were (a) single pictures presented for 6 s, (b) a set of three consecutive pictures with emotionally congruent contents presented for 2 s each, (c) short film clips with a duration of 6 s. A total of 144 participants rated their emotion and arousal states following stimulus presentation. Repeated-measures ANOVAs revealed that the film clips and 3-picture version were as effective as the classical 1-picture method to elicit positive emotions, however, modulation toward positive valence was little. Modulation toward negative valence was more effective in general. Film clips were less effective than pictorial stimuli in producing the corresponding emotion states (all p < 0.001) and were less arousing (all p ≤ 0.02). Possible reasons for these unexpected results are discussed.