Combining electroencephalographic activity and instantaneous heart rate for assessing brain-heart dynamics during visual emotional elicitation in healthy subjects

Time-varying information measures: an adaptive estimation of information storage with application to brain-heart interactions

Network Physiology is a rapidly growing field of study that aims to understand how physiological systems interact to maintain health. Within the information theory framework the information storage (IS) allows to measure the regularity and predictability of a dynamic process under stationarity assumption. However, this assumption does not allow to track over time the transient pathways occurring in the dynamical activity of a physiological system. To address this limitation, we propose a time-varying approach based on the recursive least squares algorithm (RLS) for estimating IS at each time instant, in non-stationary conditions. We tested this approach in simulated time-varying dynamics and in the analysis of electroencephalographic (EEG) signals recorded from healthy volunteers and timed with the heartbeat to investigate brain-heart interactions. In simulations, we show that the proposed approach allows to track both abrupt and slow changes in the information stored in a physiological system. These changes are reflected in its evolution and variability over time. The analysis of brain-heart interactions reveals marked differences across the cardiac cycle phases of the variability of the time-varying IS. On the other hand, the average IS values exhibit a weak modulation over parieto-occiptal areas of the scalp. Our study highlights the importance of developing more advanced methods for measuring IS that account for non-stationarity in physiological systems. The proposed time-varying approach based on RLS represents a useful tool for identifying spatio-temporal dynamics within the neurocardiac system and can contribute to the understanding of brain-heart interactions.

Dynamic fluctuations in ascending heart-to-brain communication under mental stress

Dynamical information exchange between central and autonomic nervous systems, as referred to functional brain-heart interplay, occurs during emotional and physical arousal. It is well documented that physical and mental stress lead to sympathetic activation. Nevertheless, the role of autonomic inputs in nervous system-wise communication under mental stress is yet unknown. In this study, we estimated the causal and bidirectional neural modulations between electroencephalogram (EEG) oscillations and peripheral sympathetic and parasympathetic activities using a recently proposed computational framework for a functional brain-heart interplay assessment, namely the sympathovagal synthetic data generation model. Mental stress was elicited in 37 healthy volunteers by increasing their cognitive demands throughout three tasks associated with increased stress levels. Stress elicitation induced an increased variability in sympathovagal markers, as well as increased variability in the directional brain-heart interplay. The observed heart-to-brain interplay was primarily from sympathetic activity targeting a wide range of EEG oscillations, whereas variability in the efferent direction seemed mainly related to EEG oscillations in the γ band. These findings extend current knowledge on stress physiology, which mainly referred to top-down neural dynamics. Our results suggest that mental stress may not cause an increase in sympathetic activity exclusively as it initiates a dynamic fluctuation within brain-body networks including bidirectional interactions at a brain-heart level. We conclude that directional brain-heart interplay measurements may provide suitable biomarkers for a quantitative stress assessment and bodily feedback may modulate the perceived stress caused by increased cognitive demand.

Atypical Response to Affective Touch in Children with Autism: Multi-Parametric Exploration of the Autonomic System

This study aimed at evaluating the autonomic response to pleasant affective touch in children with Autism Spectrum Disorders (ASD) and age-matched typically developing (TD) peers, thanks to multiple autonomic nervous system (ANS) parameters and by contrasting CT (C-tactile fibers) high- vs. low-density territory stimulations. We measured pupil diameter, skin conductance, and heart rate during gentle stroking of two skin territories (CT high- and low-density, respectively, forearm and palm of the hand) in thirty 6-12-year-old TD children and twenty ASD children. TD children showed an increase in pupil diameter and skin conductance associated with a heart rate deceleration in response to tactile stimulations at the two locations. Only the pupil was influenced by the stimulated location, with a later dilation peak following CT low-density territory stimulation. Globally, ASD children exhibited reduced autonomic responses, as well as different ANS baseline values compared to TD children. These atypical ANS responses to pleasant touch in ASD children were not specific to CT-fiber stimulation. Overall, these results point towards both basal autonomic dysregulation and lower tactile autonomic evoked responses in ASD, possibly reflecting lower arousal and related to social disengagement.

The Role of the Emotional Sequence in the Communication of the Territorial Cheeses: A Neuromarketing Approach

Over the past few years, many studies have shown how territoriality can be considered a driver for purchasing agri-food products. Products with certification of origin are perceived as more sustainable, safer and of better quality. At the same time, producers of traditional products often belong to small entities that struggle to compete with large multinational food corporations, having less budget to allocate to product promotion. In this study, we propose a neuromarketing approach, showing how the use of these techniques can help in choosing the most effective commercial in terms of likeability and ability to activate mnemonic processes. Two commercials were filmed for the purpose of this study. They differed from each other in terms of emotional sequence. The first aimed primarily at eliciting positive emotions derived from the product description. The second aimed to generate negative emotions during the early stages, highlighting the negative consequences of humans' loss of contact with nature and tradition and then eliciting positive emotions by presenting cheese production using traditional techniques as a solution to the problem. Based on the literature on the emotional sequences in social advertising, we hypothesised that the second commercial would generate an overall better emotional reaction and activate mnemonic processes to a greater extent. Our results partially support the research hypotheses, providing useful insights both to marketers and for future research on the topic.

Heart Rate Variability Analysis for Seizure Detection in Neonatal Intensive Care Units

In Neonatal Intensive Care Units (NICUs), the early detection of neonatal seizures is of utmost importance for a timely clinical intervention. Over the years, several neonatal seizure detection systems were proposed to detect neonatal seizures automatically and speed up seizure diagnosis, most based on the EEG signal analysis. Recently, research has focused on other possible seizure markers, such as electrocardiography (ECG). This work proposes an ECG-based NSD system to investigate the usefulness of heart rate variability (HRV) analysis to detect neonatal seizures in the NICUs. HRV analysis is performed considering time-domain, frequency-domain, entropy and multiscale entropy features. The performance is evaluated on a dataset of ECG signals from 51 full-term babies, 29 seizure-free. The proposed system gives results comparable to those reported in the literature: Area Under the Receiver Operating Characteristic Curve = 62%, Sensitivity = 47%, Specificity = 67%. Moreover, the system’s performance is evaluated in a real clinical environment, inevitably affected by several artefacts. To the best of our knowledge, our study proposes for the first time a multi-feature ECG-based NSD system that also offers a comparative analysis between babies suffering from seizures and seizure-free ones.

Functional assessment of bidirectional cortical and peripheral neural control on heartbeat dynamics: a brain-heart study on thermal stress

The study of functional brain-heart interplay (BHI) from non-invasive recordings has gained much interest in recent years. Previous endeavors aimed at understanding how the two dynamical systems exchange information, providing novel holistic biomarkers and important insights on essential cognitive aspects and neural system functioning. However, the interplay between cardiac sympathovagal and cortical oscillations still has much room for further investigation. In this study, we introduce a new computational framework for a functional BHI assessment, namely the Sympatho-Vagal Synthetic Data Generation Model, combining cortical (electroencephalography, EEG) and peripheral (cardiac sympathovagal) neural dynamics. The causal, bidirectional neural control on heartbeat dynamics was quantified on data gathered from 26 human volunteers undergoing a cold-pressor test. Results show that thermal stress induces heart-to-brain functional interplay sustained by EEG oscillations in the delta and gamma bands, primarily originating from sympathetic activity, whereas brain-to-heart interplay originates over central brain regions through sympathovagal control. The proposed methodology provides a viable computational tool for the functional assessment of the causal interplay between cortical and cardiac neural control.

Functional brain-heart interplay extends to the multifractal domain

The study of functional brain-heart interplay has provided meaningful insights in cardiology and neuroscience. Regarding biosignal processing, this interplay involves predominantly neural and heartbeat linear dynamics expressed via time and frequency domain-related features. However, the dynamics of central and autonomous nervous systems show nonlinear and multifractal behaviours, and the extent to which this behaviour influences brain-heart interactions is currently unknown. Here, we report a novel signal processing framework aimed at quantifying nonlinear functional brain-heart interplay in the non-Gaussian and multifractal domains that combines electroencephalography (EEG) and heart rate variability series. This framework relies on a maximal information coefficient analysis between nonlinear multiscale features derived from EEG spectra and from an inhomogeneous point-process model for heartbeat dynamics. Experimental results were gathered from 24 healthy volunteers during a resting state and a cold pressor test, revealing that synchronous changes between brain and heartbeat multifractal spectra occur at higher EEG frequency bands and through nonlinear/complex cardiovascular control. We conclude that significant bodily, sympathovagal changes such as those elicited by cold-pressure stimuli affect the functional brain-heart interplay beyond second-order statistics, thus extending it to multifractal dynamics. These results provide a platform to define novel nervous-system-targeted biomarkers. This article is part of the theme issue 'Advanced computation in cardiovascular physiology: new challenges and opportunities'.

HRV analysis: a non-invasive approach to discriminate between newborns with and without seizures. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021;2021:52-55. [PMID: 34891237 DOI: 10.1109/embc46164.2021.9629741]

Early neonatal seizures detection is one of the most challenging issues in Neonatal Intensive Care Units. Several EEG-based Neonatal Seizure Detectors were proposed to support the clinical staff. However, less invasive and more easily interpretable methods than EEG are still missing. In this work, we investigated if Heart Rate Variability analysis and related measures as input features of supervised classifiers could be a valid support for discriminating between newborns with seizures and seizure-free ones. The proposed methods were validated on 52 subjects (33 with seizures and 19 seizure-free) of a public dataset collected at the Helsinki University Hospital. Encouraging results are achieved using a Linear Support Vector Machine, obtaining about 87% Area Under ROC Curve. This suggests that Heart Rate Variability analysis might be a non-invasive pre-screening tool to identify newborns with seizures.Clinical Relevance- Heart Rate Variability analysis for detecting newborns with seizures in NICUs could speed up the diagnosis process and appropriate treatments for a better neurodevelopmental outcome of the infant.

Disruption of the Cortical-Vagal Communication Network in Parkinson's Disease

Parkinson's disease (PD) is a neuropathy characterized by motor disorders, but it has also been associated with the presence of autonomic alterations as a result of degradation of the dopaminergic system. Studying the relation between Band Power time series (BPts) and Heart Rate Variability (HRV), has been proposed as a tool to explore the bidirectional communication pathways between cortex and autonomic control. This work presents a primer analysis on study brain ↔ heart interaction on a databse of PD patients under two conditions: without and after levadopa (L-dopa) intake. Additionally a healthy control population was also analyzed, and used as comparison level between both conditions. Results show PD affects pathways by reducing the number of connections, specially association of beta and power and the second faster component of HRV seems to be more sensitive to L-dopa administration.

The role of electroencephalography electrical reference in the assessment of functional brain-heart interplay: From methodology to user guidelines

BACKGROUND

The choice of EEG reference has been widely studied. However, the choice of the most appropriate re-referencing for EEG data is still debated. Moreover, the role of EEG reference in the estimation of functional Brain-Heart Interplay (BHI), together with different multivariate modelling strategies, has not been investigated yet.

METHODS

This study identifies the best methodology combining a proper EEG electrical reference and signal processing methods for an effective functional BHI assessment. The effects of the EEG reference among common average, mastoids average, Laplacian reference, Cz reference, and the reference electrode standardization technique (REST) were explored throughout different BHI methods including synthetic data generation (SDG) model, heartbeat-evoked potentials, heartbeat-evoked oscillations, and maximal information coefficient.

RESULTS

The SDG model exhibited high robustness between EEG references, whereas the maximal information coefficient method exhibited a high sensitivity. The common average and REST references for EEG showed a good consistency in the between-method comparisons. Laplacian, and Cz references significantly bias a BHI measurement.

COMPARISON WITH EXISTING METHODS

The use of EEG reference based on a common average outperforms on the use of other references for consistency in estimating directed functional BHI. We do not recommend the use of EEG references based on analytical derivations as the experimental conditions may not meet the requirements of their optimal estimation, particularly in clinical settings.

CONCLUSION

The use of a common average for EEG electrical reference is concluded to be the most appropriate choice for a quantitative, functional BHI assessment.

Bidirectional intrinsic modulation of EEG band power time series and spectral components of heart rate variability

Some hypotheses relate oscillations of EEG band power with autonomic processes derived from homeostatic control modulated by structures like the Central Autonomic Network and the Autonomic Nervous System. This research project studies the causal relationships between fluctuations of an autonomic process marker like the Heart Rate Variability (HRV) and the proposed EEG band power time series (BPts). To verify the existence of directional causal relationships, using Granger Causality (GC) test between HRV and BPts. Analyses were performed using two databases, of 9 and 14 subjects respectively. Experiments consisted of spontaneous breathing and a controlled breathing task (CBT). GC was tested over Intrinsinc Mode Functions of HRV derived from Empirical Mode Decomposition and BPts computed over α, β and γ bands. Positive GC tests were observed through each experimental task, channels, IMFs, EEG band, and direction. The largest number of positive GC relationships were found from BPts to HRV when testing, higher EEG band and IMF with lower spectral content. Opposite direction achieves lower total counts, but more related with IMFs of higher spectral content. Its presence also suggests that some homeostatic condition alters the BPts course given its increment under the CBT. It is important to notice that in both cases γ band achieves larger values for almost all of the studied conditions. Suggesting that such band has an important influence over HRV, but alterations on breathing condition also produce changes on BPts evolution, suggesting that the closed loop for homeostatic control alters neural dynamics at cortical level.

Quantifying multidimensional control mechanisms of cardiovascular dynamics during multiple concurrent stressors

Heartbeat regulation is achieved through different routes originating from central autonomic network sources, as well as peripheral control mechanisms. While previous studies successfully characterized cardiovascular regulatory mechanisms during a single stressor, to the best of our knowledge, a combination of multiple concurrent elicitations leading to the activation of different autonomic regulatory routes has not been investigated yet. Therefore, in this study, we propose a novel modeling framework for the quantification of heartbeat regulatory mechanisms driven by different neural routes. The framework is evaluated using two heartbeat datasets gathered from healthy subjects undergoing physical and mental stressors, as well as their concurrent administration. Experimental results indicate that more than 70% of the heartbeat regulatory dynamics is driven by the physical stressor when combining physical and cognitive/emotional stressors. The proposed framework provides quantitative insights and novel perspectives for neural activity on cardiac control dynamics, likely highlighting new biomarkers in the psychophysiology and physiopathology fields. A Matlab implementation of the proposed tool is available online.

Toward brain-heart computer interfaces: a study on the classification of upper limb movements using multisystem directional estimates

OBJECTIVE

Brain-computer interfaces (BCI) exploit computational features from brain signals to perform a given task. Despite recent neurophysiology and clinical findings indicating the crucial role of functional interplay between brain and cardiovascular dynamics in locomotion, heartbeat information remains to be included in common BCI systems. In this study, we exploit the multidimensional features of directional and functional interplay between electroencephalographic and heartbeat spectra to classify upper limb movements into three classes.

APPROACH

We gathered data from 26 healthy volunteers that performed 90 movements; the data were processed using a recently proposed framework for brain-heart interplay (BHI) assessment based on synthetic physiological data generation. Extracted BHI features were employed to classify, through sequential forward selection scheme and k-nearest neighbors algorithm, among resting state and three classes of movements according to the kind of interaction with objects.

MAIN RESULTS

The results demonstrated that the proposed brain-heart computer interface (BHCI) system could distinguish between rest and movement classes automatically with an average 90% of accuracy.

SIGNIFICANCE

Further, this study provides neurophysiology insights indicating the crucial role of functional interplay originating at the cortical level onto the heart in the upper limb neural control. The inclusion of functional BHI insights might substantially improve the neuroscientific knowledge about motor control, and this may lead to advanced BHCI systems performances.

Multivariate Correlation Measures Reveal Structure and Strength of Brain-Body Physiological Networks at Rest and During Mental Stress

In this work, we extend to the multivariate case the classical correlation analysis used in the field of network physiology to probe dynamic interactions between organ systems in the human body. To this end, we define different correlation-based measures of the multivariate interaction (MI) within and between the brain and body subnetworks of the human physiological network, represented, respectively, by the time series of δ, θ, α, and β electroencephalographic (EEG) wave amplitudes, and of heart rate, respiration amplitude, and pulse arrival time (PAT) variability (η, ρ, π). MI is computed: (i) considering all variables in the two subnetworks to evaluate overall brain-body interactions; (ii) focusing on a single target variable and dissecting its global interaction with all other variables into contributions arising from the same subnetwork and from the other subnetwork; and (iii) considering two variables conditioned to all the others to infer the network topology. The framework is applied to the time series measured from the EEG, electrocardiographic (ECG), respiration, and blood volume pulse (BVP) signals recorded synchronously via wearable sensors in a group of healthy subjects monitored at rest and during mental arithmetic and sustained attention tasks. We find that the human physiological network is highly connected, with predominance of the links internal of each subnetwork (mainly η-ρ and δ-θ, θ-α, α-β), but also statistically significant interactions between the two subnetworks (mainly η-β and η-δ). MI values are often spatially heterogeneous across the scalp and are modulated by the physiological state, as indicated by the decrease of cardiorespiratory interactions during sustained attention and by the increase of brain-heart interactions and of brain-brain interactions at the frontal scalp regions during mental arithmetic. These findings illustrate the complex and multi-faceted structure of interactions manifested within and between different physiological systems and subsystems across different levels of mental stress.

Association between Cardiac Autonomic Control and Postural Control in Patients with Parkinson's Disease

Parkinson’s disease (PD) is a neurodegenerative disorder that affects postural and cardiac autonomic control. However, since it is unknown whether these changes are associated, the objective of this study was to determine whether such a relationship exists. Twenty-three patients with PD participated. The RR intervals were recorded in different positions and heart rate variability (HRV) was analyzed. Postural sway was analyzed based on the center of pressure. No significant differences on HRV indices were induced by postural change. A correlation was found between these indices and postural control, high frequency (HF), and anterior-posterior (AP) root mean square (RMS-AP) (r = 0.422, p = 0.045), low frequency (LF)/HF, and AP mean velocity (r = 0.478, p = 0.021). A correlation was found between HRV induced by postural change and postural control, Δ LF/HF and RMS-AP (r = 0.448, p = 0.032), Δ LF/HF and ellipse area (r = 0.505, p = 0.014), Δ LF/HF and AP mean velocity (r = −0.531; p = 0.009), and Δ LF and AP mean velocity (r = −0.424, p = 0.044). There is an association between the autonomic and postural systems, such that PD patients with blunted cardiac autonomic function in both the supine and orthostatic positions have worse postural control.

A framework to quantify controlled directed interactions in network physiology applied to cognitive function assessment

The complex nature of physiological systems where multiple organs interact to form a network is complicated by direct and indirect interactions, with varying strength and direction of influence. This study proposes a novel framework which quantifies directional and pairwise couplings, while controlling for the effect of indirect interactions. Simulation results confirm the superiority of this framework in uncovering directional primary links compared to previous published methods. In a practical application of cognitive attention and alertness tasks, the method was used to assess controlled directed interactions between the cardiac, respiratory and brain activities (prefrontal cortex). It revealed increased interactions during the alertness task between brain wave activity on the left side of the brain with heart rate and respiration compared to resting phases. During the attention task, an increased number of right brain wave interactions involving respiration was also observed compared to rest, in addition to left brain wave activity with heart rate. The proposed framework potentially assesses directional interactions in complex network physiology and may detect cognitive dysfunctions associated with altered network physiology.

Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing

Emotions play a critical role in our daily lives, so the understanding and recognition of emotional responses is crucial for human research. Affective computing research has mostly used non-immersive two-dimensional (2D) images or videos to elicit emotional states. However, immersive virtual reality, which allows researchers to simulate environments in controlled laboratory conditions with high levels of sense of presence and interactivity, is becoming more popular in emotion research. Moreover, its synergy with implicit measurements and machine-learning techniques has the potential to impact transversely in many research areas, opening new opportunities for the scientific community. This paper presents a systematic review of the emotion recognition research undertaken with physiological and behavioural measures using head-mounted displays as elicitation devices. The results highlight the evolution of the field, give a clear perspective using aggregated analysis, reveal the current open issues and provide guidelines for future research.

Affective Cortical Asymmetry at the Early Developmental Emergence of Emotional Expression

Emotions have an important survival function. Vast amounts of research have demonstrated how affect-related changes in physiology promote survival by effecting short-term and long-term changes in adaptive behavior. However, if emotions truly serve such an inherent function, they should be pervasive across species and be established early in life. Here, using electroencephalographic (EEG) brain activity we sought to characterize core neurophysiological features underlying affective function at the emergence of emotional expression [i.e., at the developmental age when human infants start to show reliable stimulus-elicited emotional states (4–6 months)]. Using an approach that eschews traditional EEG frequency band delineations (like theta, alpha), we demonstrate that negative emotional states induce a strong right hemispheric increase in the prominence of the resonant frequency (∼5–6 Hz) in the infant frontal EEG. Increased rightward asymmetry was strongly correlated with increased heart rate responses to emotionally negative states compared with neutral states. We conclude that functional frontal asymmetry is a key component of emotional processing and suggest that the rightward asymmetry in prominence of the resonant frequency during negative emotional states might reflect functional asymmetry in the central representation of anatomically driven asymmetry in the autonomic nervous system. Our findings indicate that the specific mode hallmarking emotional processing in the frontal cortex is established in parallel with the emergence of stable emotional states very early during development, despite the well known protracted maturation of frontal cortex.

Methodological Considerations on EEG Electrical Reference: A Functional Brain-Heart Interplay Study

The growing interest in the study of functional brain-heart interplay (BHI) has motivated the development of novel methodological frameworks for its quantification. While a combination of electroencephalography (EEG) and heartbeat-derived series has been widely used, the role of EEG preprocessing on a BHI quantification is yet unknown. To this extent, here we investigate on four different EEG electrical referencing techniques associated with BHI quantifications over 4-minute resting-state in 15 healthy subjects. BHI methods include the synthetic data generation model, heartbeat-evoked potentials, heartbeat-evoked oscillations, and maximal information coefficient (MIC). EEG signals were offline referenced under the Cz channel, common average, mastoids average, and Laplacian method, and statistical comparisons were performed to assess similarities between references and between BHI techniques. Results show a topographical agreement between BHI estimation methods depending on the specific EEG reference. Major differences between BHI methods occur with the Laplacian reference, while major differences between EEG references are with the MIC analysis. We conclude that the choice of EEG electrical reference may significantly affect a functional BHI quantification.

Information Transfer in Linear Multivariate Processes Assessed through Penalized Regression Techniques: Validation and Application to Physiological Networks

The framework of information dynamics allows the dissection of the information processed in a network of multiple interacting dynamical systems into meaningful elements of computation that quantify the information generated in a target system, stored in it, transferred to it from one or more source systems, and modified in a synergistic or redundant way. The concepts of information transfer and modification have been recently formulated in the context of linear parametric modeling of vector stochastic processes, linking them to the notion of Granger causality and providing efficient tools for their computation based on the state-space (SS) representation of vector autoregressive (VAR) models. Despite their high computational reliability these tools still suffer from estimation problems which emerge, in the case of low ratio between data points available and the number of time series, when VAR identification is performed via the standard ordinary least squares (OLS). In this work we propose to replace the OLS with penalized regression performed through the Least Absolute Shrinkage and Selection Operator (LASSO), prior to computation of the measures of information transfer and information modification. First, simulating networks of several coupled Gaussian systems with complex interactions, we show that the LASSO regression allows, also in conditions of data paucity, to accurately reconstruct both the underlying network topology and the expected patterns of information transfer. Then we apply the proposed VAR-SS-LASSO approach to a challenging application context, i.e., the study of the physiological network of brain and peripheral interactions probed in humans under different conditions of rest and mental stress. Our results, which document the possibility to extract physiologically plausible patterns of interaction between the cardiovascular, respiratory and brain wave amplitudes, open the way to the use of our new analysis tools to explore the emerging field of Network Physiology in several practical applications.

Assessing the Quality of Heart Rate Variability Estimated from Wrist and Finger PPG: A Novel Approach Based on Cross-Mapping Method

The non-invasiveness of photoplethysmographic (PPG) acquisition systems, together with their cost-effectiveness and easiness of connection with IoT technologies, is opening up to the possibility of their widespread use. For this reason, the study of the reliability of PPG and pulse rate variability (PRV) signal quality has become of great scientific, technological, and commercial interest. In this field, sensor location has been demonstrated to play a crucial role. The goal of this study was to investigate PPG and PRV signal quality acquired from two body locations: finger and wrist. We simultaneously acquired the PPG and electrocardiographic (ECG) signals from sixteen healthy subjects (aged 28.5 ± 3.5, seven females) who followed an experimental protocol of affective stimulation through visual stimuli. Statistical tests demonstrated that PPG signals acquired from the wrist and the finger presented different signal quality indexes (kurtosis and Shannon entropy), with higher values for the wrist-PPG. Then we propose to apply the cross-mapping (CM) approach as a new method to quantify the PRV signal quality. We found that the performance achieved using the two sites was significantly different in all the experimental sessions (p < 0.01), and the PRV dynamics acquired from the finger were the most similar to heart rate variability (HRV) dynamics.

Assessing Autonomic Function from Electrodermal Activity and Heart Rate Variability During Cold-Pressor Test and Emotional Challenge

Standard functional assessment of autonomic nervous system (ANS) activity on cardiovascular control relies on spectral analysis of heart rate variability (HRV) series. However, difficulties in obtaining a reliable measure of sympathetic activity from HRV spectra limits the exploitation of sympatho-vagal metrics. On the other hand, measures of electrodermal activity (EDA) have been demonstrated to provide a reliable quantifier of sympathetic dynamics. In this study we propose novel indices of phasic autonomic regulation mechanisms by combining HRV and EDA correlates and thoroughly investigating their time-varying dynamics. HRV and EDA series were gathered from 26 healthy subjects during a cold-pressor test and emotional stimuli. Instantaneous linear and nonlinear (bispectral) estimates of vagal dynamics were obtained from HRV through inhomogeneous point-process models, and combined with a sensitive maker of sympathetic tone from EDA spectral power. A wavelet decomposition analysis was applied to estimate phasic components of the proposed sympatho-vagal indices. Results show significant statistical differences for the proposed indices between the cold-pressor elicitation and previous resting state. Furthermore, an accuracy of 73.08% was achieved for the automatic emotional valence recognition. The proposed nonlinear processing of phasic ANS markers brings novel insights on autonomic functioning that can be exploited in the field of affective computing and psychophysiology.

Real vs. immersive-virtual emotional experience: Analysis of psycho-physiological patterns in a free exploration of an art museum

Virtual reality is a powerful tool in human behaviour research. However, few studies compare its capacity to evoke the same emotional responses as in real scenarios. This study investigates psycho-physiological patterns evoked during the free exploration of an art museum and the museum virtualized through a 3D immersive virtual environment (IVE). An exploratory study involving 60 participants was performed, recording electroencephalographic and electrocardiographic signals using wearable devices. The real vs. virtual psychological comparison was performed using self-assessment emotional response tests, whereas the physiological comparison was performed through Support Vector Machine algorithms, endowed with an effective feature selection procedure for a set of state-of-the-art metrics quantifying cardiovascular and brain linear and nonlinear dynamics. We included an initial calibration phase, using standardized 2D and 360° emotional stimuli, to increase the accuracy of the model. The self-assessments of the physical and virtual museum support the use of IVEs in emotion research. The 2-class (high/low) system accuracy was 71.52% and 77.08% along the arousal and valence dimension, respectively, in the physical museum, and 75.00% and 71.08% in the virtual museum. The previously presented 360° stimuli contributed to increasing the accuracy in the virtual museum. Also, the real vs. virtual classifier accuracy was 95.27%, using only EEG mean phase coherency features, which demonstrates the high involvement of brain synchronization in emotional virtual reality processes. These findings provide an important contribution at a methodological level and to scientific knowledge, which will effectively guide future emotion elicitation and recognition systems using virtual reality.

Functional Linear and Nonlinear Brain–Heart Interplay during Emotional Video Elicitation: A Maximum Information Coefficient Study

Brain and heart continuously interact through anatomical and biochemical connections. Although several brain regions are known to be involved in the autonomic control, the functional brain–heart interplay (BHI) during emotional processing is not fully characterized yet. To this aim, we investigate BHI during emotional elicitation in healthy subjects. The functional linear and nonlinear couplings are quantified using the maximum information coefficient calculated between time-varying electroencephalography (EEG) power spectra within the canonical bands (δ,θ,α,β and γ), and time-varying low-frequency and high-frequency powers from heartbeat dynamics. Experimental data were gathered from 30 healthy volunteers whose emotions were elicited through pleasant and unpleasant high-arousing videos. Results demonstrate that functional BHI increases during videos with respect to a resting state through EEG oscillations not including the γ band (>30 Hz). Functional linear coupling seems associated with a high-arousing positive elicitation, with preferred EEG oscillations in the θ band ([4,8) Hz) especially over the left-temporal and parietal cortices. Differential functional nonlinear coupling between emotional valence seems to mainly occur through EEG oscillations in the δ,θ,α bands and sympathovagal dynamics, as well as through δ,α,β oscillations and parasympathetic activity mainly over the right hemisphere. Functional BHI through δ and α oscillations over the prefrontal region seems primarily nonlinear. This study provides novel insights on synchronous heartbeat and cortical dynamics during emotional video elicitation, also suggesting that a nonlinear analysis is needed to fully characterize functional BHI.

Lateralization of directional brain-heart information transfer during visual emotional elicitation

Previous studies have characterized the physiological interactions between central nervous system (brain) and peripheral cardiovascular system (heart) during affective elicitation in healthy subjects; however, questions related to the directionality of this functional interplay have been gaining less attention from the scientific community. Here, we explore brain-heart interactions during visual emotional elicitation in healthy subjects using measures of Granger causality (GC), a widely used descriptor of causal influences between two dynamical systems. The proposed approach inferences causality between instantaneous cardiovagal dynamics estimated from inhomogeneous point-process models of the heartbeat and high-density electroencephalogram (EEG) dynamics in 22 healthy subjects who underwent pleasant/unpleasant affective elicitation by watching pictures from the International Affective Picture System database. Particularly, we calculated the GC indexes between the EEG spectrogram in the canonical θ-, α-, β-, and γ-bands and both the instantaneous mean heart rate and its continuous parasympathetic modulations (i.e., the instantaneous HF power). Thus we looked for significant statistical differences among GC values estimated during the resting state, neutral elicitation, and pleasant/unpleasant arousing elicitation. As compared with resting state, coupling strength increases significantly in the left hemisphere during positive stimuli and in the right hemisphere during negative stimuli. Our results further reveal a correlation between emotional valence and lateralization of the dynamical information transfer going from brain-to-heart, mainly localized in the prefrontal, somatosensory, and posterior cortexes, and of the information transfer from heart-to-brain, mainly reflected into the fronto-parietal cortex oscillations in the γ-band (30 −45 Hz).

Emotional expression in psychiatric conditions: New technology for clinicians

AIM

Emotional expressions are one of the most widely studied topics in neuroscience, from both clinical and non-clinical perspectives. Atypical emotional expressions are seen in various psychiatric conditions, including schizophrenia, depression, and autism spectrum conditions. Understanding the basics of emotional expressions and recognition can be crucial for diagnostic and therapeutic procedures. Emotions can be expressed in the face, gesture, posture, voice, and behavior and affect physiological parameters, such as the heart rate or body temperature. With modern technology, clinicians can use a variety of tools ranging from sophisticated laboratory equipment to smartphones and web cameras. The aim of this paper is to review the currently used tools using modern technology and discuss their usefulness as well as possible future directions in emotional expression research and treatment strategies.

METHODS

The authors conducted a literature review in the PubMed, EBSCO, and SCOPUS databases, using the following key words: 'emotions,' 'emotional expression,' 'affective computing,' and 'autism.' The most relevant and up-to-date publications were identified and discussed. Search results were supplemented by the authors' own research in the field of emotional expression.

RESULTS

We present a critical review of the currently available technical diagnostic and therapeutic methods. The most important studies are summarized in a table.

CONCLUSION

Most of the currently available methods have not been adequately validated in clinical settings. They may be a great help in everyday practice; however, they need further testing. Future directions in this field include more virtual-reality-based and interactive interventions, as well as development and improvement of humanoid robots.

Brain Dynamics Induced by Pleasant/Unpleasant Tactile Stimuli Conveyed by Different Fabrics

In this study, we investigated brain dynamics from electroencephalographic (EEG) signals during affective tactile stimulation conveyed by the dynamical contact with different fabrics. Thirty-three healthy subjects (16 females) were enrolled to interact with a haptic device able to mimic caress-like stimuli conveyed by strips of different fabrics moved back and forth at different velocities. Specifically, two velocity levels (i.e., 9.4 and 65 mm/sec) and two kinds of fabric (i.e., burlap and silk) were selected to deliver pleasant and unpleasant affective elicitations, according to subjects' self-assessment. EEG power spectra and functional connectivity were then calculated and analyzed. Experimental results, reported in terms of p-value topographic maps, demonstrated that caresses administered through unpleasant fabrics increased brain activity in the θ (4-8 Hz), α (8-14 Hz), and β (14-30 Hz) bands, whereas the use of pleasant fabrics enhanced functional connections in specific areas (e.g., frontal, occipital, and temporal cortices) depending on the oscillations frequency and caressing velocity. Furthermore, we adopted K-NN algorithms to automatically recognize the pleasantness of the haptic stimulation at a single-subject level using EEG power spectra, achieving a recognition accuracy up to 74.24%. Finally, we showed how brain oscillation power in the α and β bands over contralateral frontal- and central-cortex were the most informative features characterizing the pleasantness of a tactile stimulus on the forearm.

A Multiclass Arousal Recognition using HRV Nonlinear Analysis and Affective Images

This paper reports on a multiclass arousal recognition system based on autonomic nervous system linear and nonlinear dynamics during affective visual elicitation. We propose a new hybrid method based on Lagged Poincaré Plot (LPP) and symbolic analysis, hereinafter called LPP_symb. This tool uses symbolic analysis to evaluate the irregularity of the trends of Lagged Poincaré Plot (LPP) quantifiers over the lags, and is here applied to investigate complex Heart Rate Variability (HRV) changes during emotion stimuli. In the experimental protocol 22 healthy subjects were elicited through a passive visualization of affective images gathered from the international affective picture system. LPP_symb and standard HRV analysis (defined in time and frequency domains) were applied to HRV series of one minute length. Then, an ad-hoc pattern recognition algorithm based on quadratic discriminant classifier was implemented and validated through a leave-onesubject-out procedure. The best performance of the proposed classification algorithm for recognizing the four classes of arousal was obtained using nine features comprising heartbeat complex dynamics, achieving an accuracy of 71.59%.

EEG Hyperconnectivity Study on Saxophone Quartet Playing in Ensemble

A professional quartet of saxophonists playing in ensemble provides a perfect scenario to study the eventual occurrence of synchronous oscillatory brain activity across subjects. Here, we applied hyperscanning methodologies for simultaneously recordings of electroencephalographic (EEG) signals from four professional saxophonists while they observ an audiovideo recording of their own previous musical performance. An ad-hoc musical composition was written for the study. At debriefing, the subjects were asked to answer two questionnaires to assess their empathy trait and the musical leadership. In order to estimate the hyperconnectivity of each musician we proposed a measure which combines phase synchronization index of brain oscillations and graph theory framework. The inter-connectivity level of each musician was statistically compared. Statistical results revealed a significant lower hyperconnectivity in the left Brodmann area 44 for the Soprano with respect to the other three members. Recent theories attributed this brain region (Broca's area) to music generation, empathy processes and communication. We hypothesize a relationship between brain-to-brain connectivity level and the musical role within the quartet.

A new Modelling Framework to Study Time-Varying Directional Brain-Heart Interactions: Preliminary Evaluations and Perspectives

We propose a novel modelling framework to study non-stationary, directional brain-heart interplay in a time varying fashion. Considering electroencephalographic (EEG) signals and Heart Rate Variability (HRV) series as inputs, a new multivariate formulation is derived from proper coupling functions linking cortical electrical activity and heartbeat dynamics generation models. These neural-autonomic coupling rules are formalised according to the current knowledge on the central autonomic network and fully parametrised in adaptive coefficients quantifying the information outflow from-brain-to- heart as well as from-heart-to-brain. Such coefficients can be effectively estimated by solving the model inverse problem, and profitably exploited for a novel assessment of brain-heart interactions. Here we show preliminary experimental results gathered from 27 healthy volunteers undergoing significant sympatho-vagal perturbations through cold-pressor test and discuss prospective uses of this novel methodological frame- work. Specifically, we highlight how the directional brain-heart coupling significantly increases during prolonged baroreflex elicitation with specific time delays and throughout specific brain areas, especially including fronto-parietal regions and lateralisation mechanisms in the temporal cortices.

Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors

Affective Computing has emerged as an important field of study that aims to develop systems that can automatically recognize emotions. Up to the present, elicitation has been carried out with non-immersive stimuli. This study, on the other hand, aims to develop an emotion recognition system for affective states evoked through Immersive Virtual Environments. Four alternative virtual rooms were designed to elicit four possible arousal-valence combinations, as described in each quadrant of the Circumplex Model of Affects. An experiment involving the recording of the electroencephalography (EEG) and electrocardiography (ECG) of sixty participants was carried out. A set of features was extracted from these signals using various state-of-the-art metrics that quantify brain and cardiovascular linear and nonlinear dynamics, which were input into a Support Vector Machine classifier to predict the subject’s arousal and valence perception. The model’s accuracy was 75.00% along the arousal dimension and 71.21% along the valence dimension. Our findings validate the use of Immersive Virtual Environments to elicit and automatically recognize different emotional states from neural and cardiac dynamics; this development could have novel applications in fields as diverse as Architecture, Health, Education and Videogames.

Resting-state brain correlates of instantaneous autonomic outflow

A prominent pathway of brain-heart interaction is represented by autonomic nervous system (ANS) heartbeat modulation. While within-brain resting state networks have been the object of intense functional Magnetic Resonance Imaging (fMRI) research, technological and methodological limitations have hampered research on the central correlates of cardiovascular control dynamics. Here we combine the high temporal and spatial resolution as well as data volume afforded by the Human Connectome Project with a probabilistic model of heartbeat dynamics to characterize central correlates of sympathetic and parasympathetic ANS activity at rest. We demonstrate an involvement of a number of brain regions such as the Insular cortex, Frontal Gyrus, Lateral Occipital Cortex, Paracingulate and Cingulate Gyrus and Precuneous Cortex, as well as subcortical structures (Thalamus, Putamen, Pallidum, Brain-Stem, Hippocampus, Amygdala, and Right Caudate) in the modulation of ANS-mediated cardiovascular control, possibly indicating a broader definition of the central autonomic network (CAN). Our findings provide a basis for an informed neurobiological interpretation of the numerous studies which employ HRV-related measures as standalone biomarkers in health and disease.

Analysis of generic coupling between EEG activity and PETCO2 in free breathing and breath-hold tasks using Maximal Information Coefficient (MIC)

Brain activations related to the control of breathing are not completely known. The respiratory system is a non-linear system. However, the relationship between neural and respiratory dynamics is usually estimated through linear correlation measures, completely neglecting possible underlying nonlinear interactions. This study evaluate the linear and nonlinear coupling between electroencephalographic (EEG) signal and variations in carbon dioxide (CO₂) signal related to different breathing task. During a free breathing and a voluntary breath hold tasks, the coupling between EEG power in nine different brain regions in delta (1–3 Hz) and alpha (8–13 Hz) bands and end-tidal CO₂ (P_ET CO₂) was evaluated. Specifically, the generic associations (i.e. linear and nonlinear correlations) and a “pure” nonlinear correlations were evaluated using the maximum information coefficient (MIC) and MIC-ρ² between the two signals, respectively (where ρ² represents the Pearson’s correlation coefficient). Our results show that in delta band, MIC indexes discriminate the two tasks in several regions, while in alpha band the same behaviour is observed for MIC-ρ², suggesting a generic coupling between delta EEG power and P_ETCO₂ and a pure nonlinear interaction between alpha EEG power and P_ETCO₂. Moreover, higher indexes values were found for breath hold task respect to free breathing.

Heart rate variability analysis during muscle fatigue due to prolonged isometric contraction

Fatigue can be defined as the muscular condition occurring before the inability to perform a task. It can be assessed through the evaluation of the median and mean frequency of the spectrum of the surface electromyography series. Previous studies investigated the relationship between heartbeat dynamics and muscular activity. However, exploitation of such cardiovascular measures to automatically identify muscle fatigue during fatiguing exercises is still missing. To this extent, HRV signals were gathered from 32 subjects during an isometric contraction task, and features defined in the time, frequency and nonlinear domains were investigated. We used surface electromyography to label the occurrence of muscle fatigue. Statistically significant differences were observed by comparing features related to fatigued subjects with the non-fatigued ones. Moreover, a pattern recognition system capable to achieve an average accuracy of 78.24% was implemented. These results confirmed the hypothesis that a relationship between heartbeat dynamics and muscle fatigue might exist.

A novel Heart Rate Variability analysis using Lagged Poincaré plot: A study on hedonic visual elicitation

This paper reports on a novel method for the analysis of Heart Rate Variability (HRV) through Lagged Poincaré Plot (LPP) theory. Specifically a hybrid method, LPP_symb, including LPP quantifiers and related symbolic dynamics was proposed. LPP has been applied to investigate the autonomic response to pleasant and unpleasant pictures extracted from the International Affective Picture System (IAPS). IAPS pictures are standardized in terms of level of arousal, i.e. the intensity of the evoked emotion, and valence, i.e. the level of pleasantness/unpleasantness, according to the Circumplex model of Affects (CMA). Twenty-two healthy subjects were enrolled in the experiment, which comprised four sessions with increasing arousal level. Within each session valence increased from positive to negative. An ad-hoc pattern recognition algorithm using a Leave-One-Subject-Out (LOSO) procedure based on a Quadratic Discriminant Classifier (QDC) was implemented. Our pattern recognition system was able to classify pleasant and unpleasant sessions with an accuracy of 71.59%. Therefore, we can suggest the use of the LPP_symb for emotion recognition.

Neurocardiology: Cardiovascular Changes and Specific Brain Region Infarcts

There are complex and dynamic reflex control networks between the heart and the brain, including cardiac and intrathoracic ganglia, spinal cord, brainstem, and central nucleus. Recent literature based on animal model and clinical trials indicates a close link between cardiac function and nervous systems. It is noteworthy that the autonomic nervous-based therapeutics has shown great potential in the management of atrial fibrillation, ventricular arrhythmia, and myocardial remodeling. However, the potential mechanisms of postoperative brain injury and cardiovascular changes, particularly heart rate variability and the presence of arrhythmias, are not understood. In this chapter, we will describe mechanisms of brain damage undergoing cardiac surgery and focus on the interaction between cardiovascular changes and damage to specific brain regions.

Causal brain-heart information transfer during visual emotional elicitation in healthy subjects: Preliminary evaluations and future perspectives

Complex heartbeat dynamics is known to reflect subject's emotional state, thanks to numerous links to brain cortical and subcortical regions. Likewise, specific brain regions are deeply involved in vagally-mediated emotional processing and regulation. Nevertheless, although the brain-heart interplay has been studied during visual emotion elicitation, directional interactions have not been investigated so far. To fill this gap, in this study we investigate brain-heart dynamics during emotional elicitation in healthy subjects through measures of Granger causality (GC) between the two physiological systems. Data were gathered from 22 healthy volunteers who underwent pleasant/ unpleasant affective elicitation using pictures from the International Affective Picture System. Neutral emotional stimuli were elicited as well. High density electroencephalogram (EEG) signals were processed to obtain time-varying maps of cortical activation, whereas the associated instantaneous cardiovascular dynamics was estimated through inhomogeneous point-process models. Concerning the information transfer brain-to-heart, GE highlighted significant valence-dependent lateralization with respect to resting states. Furthermore, as a proof of concept, the study of heart-to-brain dynamics considering EEG oscillations in the γ band (30-45 Hz) highlighted differential information transfer between neutral and positive elicitations directed to the prefrontal cortex.

Disentanglement of sympathetic and parasympathetic activity by instantaneous analysis of human heartbeat dynamics

Spectral analysis of heart rate variability (HRV) is one of the most effective techniques for the assessment of the influence of the autonomic nervous system (ANS) on the heartbeat. Despite its widespread use, it has been demonstrated that HRV subdivision in the low frequency (LF) and high frequency (HF) bands does not accurately reflect separate sympathetic and parasympathetic influences, respectively, mainly due to overlap of the two branches in the low frequencies. Here we propose two novel indices, namely the instantaneous sympathetic autonomic index (SAI) and parasympathetic autonomic index (PAI), that are able to separately assess the time-varying ANS synergic functions. The application of the paradigm is presented here by associating proper combinations of orthonormal Laguerre functions defined within the heartbeat point-process continuous model. Preliminary results from ten subjects recorded during a tilt-table protocol show that the proposed methodology, differently than the traditional spectral parameters, is able to separately track the independent changes associated with the two ANS branches.

Exploratory analysis of nonlinear coupling between EEG global field power and end-tidal carbon dioxide in free breathing and breath-hold tasks

Brain activations underlying control of breathing are not completely known. Furthermore, the coupling between neural and respiratory dynamics is usually estimated through linear correlation measures, thus totally disregarding possible underlying nonlinear interactions. To overcome these limitations, in this preliminary study we propose a nonlinear coupling analysis of simultaneous recordings of electroencephalographic (EEG) and respiratory signals at rest and after variation of carbon dioxide (CO2) level. Specifically, a CO2 increase was induced by a voluntary breath hold task. EEG global field power (GFP) in different frequency bands and end-tidal CO2 (PETCO2) were estimated in both conditions. The maximum information coefficient (MIC) and MIC-ρ2 (where ρ represents the Pearson's correlation coefficient) between the two signals were calculated to identify generic associations (i.e. linear and nonlinear correlations) and nonlinear correlations, respectively. With respect to a free breathing state, our results suggest that a breath hold state is characterized by an increased coupling between respiration activity and specific EEG oscillations, mainly involving linear and nonlinear interactions in the delta band (1-4 Hz), and prevalent nonlinear interactions in the alpha band (8-13 Hz).

Gender-specific automatic valence recognition of affective olfactory stimulation through the analysis of the electrodermal activity

This study reports on the development of a gender-specific classification system able to discern between two valence levels of smell, through information gathered from electrodermal activity (EDA) dynamics. Specifically, two odorants were administered to 32 healthy volunteers (16 males) while monitoring EDA. CvxEDA model was used to process the EDA signal and extract features from both tonic and phasic components. The feature set was used as input to a K-NN classifier implementing a leave-one-subject-out procedure. Results show strong differences in the accuracy of valence recognition between men (62.5%) and women (78%). We can conclude that affective olfactory stimulation significantly affect EDA dynamics with a highly specific gender dependency.

Valence-dependent changes in visual arousing elicitation: an exploratory study in EEG gamma oscillations

Emotion regulation involves several brain areas such as prefrontal cortex, amygdala, and insular cortex. However, considering different levels of arousing elicitations, how such a brain dynamics is affected by emotional pleasant/unpleasant (valence) elicitation is not fully understood. To this aim, we propose an Electroencephalographic (EEG)-based preliminary study in which 22 healthy subjects were elicited through affective pictures gathered from the International Affective Picture System. Considering 4 arousing levels, each of which including two valence levels (pleasant and unpleasant), we investigated EEG power spectra and functional connectivity. Focusing on gamma oscillations (> 32 Hz), because of their known sensitivity to valence changes, results revealed no significant changes between pleasant/unpleasant elicitation when lower and higher arousing level occurred. Conversely, valence changes in the intermediate arousing sessions were associated with changes in the prefrontal and occipital regions. Additionally, different arousing levels of pleasant elicitations affected short-range connectivity over the right hemisphere, whereas different arousing levels of unpleasant elicitations affected medium-range connectivity over the left hemisphere.