Remote photoplethysmography (rPPG) in the wild: Remote heart rate imaging via online webcams

Remote photoplethysmography (rPPG) is a low-cost technique to measure physiological parameters such as heart rate by analyzing videos of a person. There has been growing attention to this technique due to the increased possibilities and demand for running psychological experiments on online platforms. Technological advancements in commercially available cameras and video processing algorithms have led to significant progress in this field. However, despite these advancements, past research indicates that suboptimal video recording conditions can severely compromise the accuracy of rPPG. In this study, we aimed to develop an open-source rPPG methodology and test its performance on videos collected via an online platform, without control of the hardware of the participants and the contextual variables, such as illumination, distance, and motion. Across two experiments, we compared the results of the rPPG extraction methodology to a validated dataset used for rPPG testing. Furthermore, we then collected 231 online video recordings and compared the results of the rPPG extraction to finger pulse oximeter data acquired with a validated mobile heart rate application. Results indicated that the rPPG algorithm was highly accurate, showing a significant degree of convergence with both datasets thus providing an improved tool for recording and analyzing heart rate in online experiments.

With hand on heart: A cardiac Rubber Hand Illusion

Body illusions such as the Rubber Hand Illusion (RHI) have highlighted how multisensory integration underpins the sense of one's own body. Much of this research has focused on senses arising from outside the body (e.g. vision and touch), but sensations from within the body may also play a role. In a pre-registered study, participants completed a cardiac variation of the RHI, where taps to the finger occurred in or out of time with the heartbeat. We replicated the RHI effect, showing that synchronous but not asynchronous taps to the real and rubber hand increased sensations of embodiment over the rubber hand and caused a shift in the perceived hand location. However, there were no significant influences of cardiac timing on embodiment, nor did it interact with visuo-tactile synchrony. An exploratory analysis found a three-way interaction between synchrony, cardiac timing and interoceptive accuracy as measured by a heartbeat counting task, such that greater interoceptive accuracy was associated with lower embodiment ratings in the systole condition compared to diastole, but only during synchronous stimulation. Although our novel methodology successfully replicated the RHI, our findings suggest that the cooccurence of vision and touch with cardiac signals may make little contribution to the sense of one's body.

Beyond peripersonal boundaries: insights from crossmodal interactions

We experience our self as a body located in space. However, how information about self-location is integrated into multisensory processes underlying the representation of the peripersonal space (PPS), is still unclear. Prior studies showed that the presence of visual information related to oneself modulates the multisensory processes underlying PPS. Here, we used the crossmodal congruency effect (CCE) to test whether this top-down modulation depends on the spatial location of the body-related visual information. Participants responded to tactile events on their bodies while trying to ignore a visual distractor presented on the mirror reflection of their body (Self) either in the peripersonal space (Near) or in the extrapersonal space (Far). We found larger CCE when visual events were presented on the mirror reflection in the peripersonal space, as compared to the extrapersonal space. These results suggest that top-down modulation of the multisensory bodily self is only possible within the PPS.

Social interoception: Perceiving events during cardiac afferent activity makes people more suggestible to other people's influence

Our judgements are often influenced by other people's views and opinions. Interoception also influences decision making, but little is known about its role in social influence and particularly, the extent to which other people may influence our decisions. Across two experiments, using different forms of social influence, participants judged the trustworthiness of faces presented either during the systolic phase of the cardiac cycle, when baroreceptors convey information from the heart to the brain, or during diastolic phase, when baroreceptors are quiescent. We quantified the extent to which participants changed their minds (as an index of social influence) following the social feedback, in order to compare two competing hypotheses. According to the Arousal-Confidence Hypothesis, cardiac signals create a context of heightened bodily arousal that increases confidence in perceptual judgements. People should, therefore, be less subject to social influence during systole. By contrast, according to the Uncertainty-Conformity Hypothesis, cardiac signals increase neural noise and sensory attenuation, such that people should display greater effects of social influence during systole, as they then underweight private interoceptive signals in favour of the external social information. Across two studies that used different kind of social interactions, we found that participants changed their minds more when faces were presented at systole. Our results, therefore, support the Uncertainly-Conformity hypothesis and highlight how cardiac afferent signals contribute to shape our social decision-making in different types of social interactions.

Temporal dynamics of the Rubber Hand Illusion

It is widely accepted that the representation of the body is not fixed and immutable, but rather flexible and constantly updated based on a continuous stream of multisensory information. This mechanism can be very useful to adapt to several situations, but it would not be adaptive if the body representation was too malleable or if it wasn't capable of restoring its integrity after a transient modification. Here we used the Rubber Hand Illusion (RHI) to investigate how quickly the body representation can be modified. Previous studies have investigated the timing of the onset and offset of the illusion, however, they did not assess a fine temporal resolution. Here, we used a potentiometer to record a moment-by-moment rating of the feeling of owning the RH for two minutes during the visuo-tactile stimulation and two minutes following the stimulation. Our results suggest that the feeling of Ownership is already established during the first 19 s of stimulation then it continues to grow, but at a much slower pace. The feeling of Ownership disappears within 66 s from the end of the stimulation. This work sheds new light on the temporal dynamics of the RHI and the malleability of the body self-consciousness.

Seeing Through Each Other's Hearts: Inferring Others' Heart Rate as a Function of Own Heart Rate Perception and Perceived Social Intelligence

Successful social interactions require a good understanding of the emotional states of other people. This information is often not directly communicated but must be inferred. As all emotional experiences are also imbedded in the visceral or interoceptive state of the body (i.e., accelerating heart rate during arousal), successfully inferring the interoceptive states of others may open a window into their emotional state. But how well can people do that? Here, we replicate recent results showing that people can discriminate between the cardiac states (i.e., the resting heartrate) of other people by simply looking at them. We further tested whether the ability to infer the interoceptive states of others depends on one's own interoceptive abilities. We measured people's performance in a cardioception task and their self-reported interoceptive accuracy. Whilst neither was directly associated to their ability to infer the heartrate of another person, we found a significant interaction. Specifically, overestimating one's own interoceptive capacities was associated with a worse performance at inferring the heartrate of others. In contrast, underestimating one's own interoceptive capacities did not have such influence. This pattern suggests that deficient beliefs about own interoceptive capacities can have detrimental effects on inferring the interoceptive states of other people.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s42761-022-00151-4.

Event-related transcutaneous vagus nerve stimulation modulates behaviour and pupillary responses during an auditory oddball task

Transcutaneous auricular vagus nerve stimulation (taVNS) is a neuromodulatory technique that is thought to activate the Locus Coeruleus-Noradrenaline (LC-NA) system. Standard taVNS protocols consist of the administration of intermittent or continuous stimulation over long periods. However, there is currently a limited understanding of the temporal dynamics of taVNS modulation of cognitive processes, as well as its mechanisms of action. We argue that novel stimulation approaches, informed by established theories of the LC-NA system, are needed to further our understanding of the neurocognitive underpinnings of taVNS. In this pre-registered study, we tested whether an "event-related" taVNS protocol can modulate the LC-NA system. In a within-subject design (single session) we delivered brief trains of taVNS (3 s) during an auditory oddball paradigm. The taVNS was time-locked to the target stimuli presentation and randomly interleaved with sham stimulation. Response times (RT) and stimuli-driven pupillary diameter (PD) were used as indices of LC-NA activity. Results revealed that active taVNS increased RT to targets, as compared to sham trials. Notably, in line with current theories of LC-NA functioning, taVNS modulation of target-related pupil dilation depended on pre-stimulation PD, an index of baseline LC-NA activity. In particular, active (vs. sham) taVNS was associated with smaller pupil dilation in trials where the baseline PD was small. These results demonstrate, for the first time, the effectiveness of brief event-related taVNS in the modulation of cognitive processes and highlight the importance of using pupil size as an index of tonic and phasic LC-NA activity.

Do you hear what I see? An audio-visual paradigm to assess emotional egocentricity bias

We often use our own emotions to understand other people's emotions. However, emotional egocentric biases (EEB), namely the tendency to use one's own emotional state when relating to others' emotions, may hinder this process, especially when emotions are incongruent. We capitalised on the classic EEB task to develop a new version that is easier to implement and control. Unlike the original EEB task that relies on a combination of private (e.g. touch) and public (e.g. vision) sensory information, our EEB task (AV-EEB) used audio-visual stimuli to evoke congruent/incongruent emotions in participants. Auditory and visual signals are both public, in that they can be shared among individuals, and make the task easier to implement and control. We provide lab-based and online validations of the AV-EEB, and demonstrate a positive relationship between EEB and social negative potency. This new, easily implemented version of the EEB task can accelerate the investigation of egocentricity biases in several research areas.

Seeing myself through my heart: Cortical processing of a single heartbeat speeds up self-face recognition

Recent research has highlighted the contribution of interoceptive signals to different aspects of bodily self-consciousness (BSC) by means of the cardio-visual stimulation - i.e. perceiving a pulsing stimulus in synchrony with one's own heart. Here, for the first time, we investigate the effects of individual heartbeats on a critical feature of BSC, namely the recognition of one's own face. Across two studies, we explored the cardiac-timing effects on a classic self-face recognition task. In Study 1, participants saw morphed faces that contained different percentages of the self-face and that of another unfamiliar individual. Study 2 used a similar design, albeit participants saw morphed faces of the self-face and that of a familiar other to provide a better control of self-familiarity. Results from both studies consistently revealed that the cortical processing of cardiac afferent signals conveyed by the firing of arterial baroreceptors affects the speed, but not the accuracy, of self-face recognition, when a single picture is presented during cardiac systole, as compared to diastole. This effect is stronger and more stable for stimuli with more self-cues than other-cues and for 'ambiguous' stimuli - i.e. at the individual point of subjective equality. Results from Study 2 also revealed that cardiac effects on the speed of self-face recognition cannot be explained simply on the basis of the imbalanced familiarity between the self's and other's faces used. The present findings highlight the interoceptive contributions to self-recognition and may be expand our understanding of pathological disturbances of self-experience.

Multisensory integration, body representation and hyperactivity of the immune system

Multisensory stimuli are integrated over a delimited window of temporal asynchronies. This window is highly variable across individuals, but the origins of this variability are still not clear. We hypothesized that immune system functioning could partially account for this variability. In two experiments, we investigated the relationship between key aspects of multisensory integration in allergic participants and healthy controls. First, we tested the temporal constraint of multisensory integration, as measured by the temporal binding window. Second, we tested multisensory body representation, as indexed by the Rubber Hand Illusion (RHI). Results showed that allergic participants have a narrower temporal binding window and are less susceptible to the RHI than healthy controls. Overall, we provide evidence linking multisensory integration processes and the activity of the immune system. The present findings are discussed within the context of the effect of immune molecules on the brain mechanisms enabling multisensory integration and multisensory body representation.

Multisensory integration induces body ownership of a handtool, but not any handtool

Bodily boundaries are computed by integrating multisensory bodily signals and can be experimentally manipulated using bodily illusions. Research on tool use demonstrates that tools alter body representations motorically to account for changes in a user's action repertoire. The present experiment sought to unify perceptual and motoric accounts of tool embodiment using a modified Rubber Hand Illusion (RHI) that also addressed the skill and practice aspects of the tool use literature. In Experiment 1, synchronous multisensory stimulation induced perceptual embodiment of a tool, chopsticks. The embodiment of chopsticks was stronger for more skilled participants, and if the illusion was preceded by tool use. In Experiment 2, the illusion was not elicited with a different type of tool, a teacup, showing that not all objects can be incorporated. This experiment helps to clarify the role of perceptual and motoric embodiment and suggests future avenues for research into tools embodiment using this method.

Multisensory body representation in autoimmune diseases

Body representation has been linked to the processing and integration of multisensory signals. An outstanding example of the pivotal role played by multisensory mechanisms in body representation is the Rubber Hand Illusion (RHI). In this paradigm, multisensory stimulation induces a sense of ownership over a fake limb. Previous work has shown high interindividual differences in the susceptibility to the RHI. The origin of this variability remains largely unknown. Given the tight and bidirectional communication between the brain and the immune system, we predicted that the origin of this variability could be traced, in part, to the immune system’s functioning, which is altered by several clinical conditions, including Coeliac Disease (CD). Consistent with this prediction, we found that the Rubber Hand Illusion is stronger in CD patients as compared to healthy controls. We propose a biochemical mechanism accounting for the dependency of multisensory body representation upon the Immune system. Our finding has direct implications for a range of neurological, psychiatric and immunological conditions where alterations of multisensory integration, body representation and dysfunction of the immune system co-exist.