A Novel Filter for Tracking Real-World Cognitive Stress using Multi-Time-Scale Point Process Observations

Unveiling productivity: The interplay of cognitive arousal and expressive typing in remote work

Cognitive Arousal, frequently elicited by environmental stressors that exceed personal coping resources, manifests in measurable physiological markers, notably in galvanic skin responses. This effect is prominent in cognitive tasks such as composition, where fluctuations in these biomarkers correlate with individual expressiveness. It is crucial to understand the nexus between cognitive arousal and expressiveness. However, there has not been a concrete study that investigates this inter-relation concurrently. Addressing this, we introduce an innovative methodology for simultaneous monitoring of these elements. Our strategy employs Bayesian analysis in a multi-state filtering format to dissect psychomotor performance (captured through typing speed), galvanic skin response or skin conductance (SC), and heart rate variability (HRV). This integrative analysis facilitates the quantification of expressive behavior and arousal states. At the core, we deploy a state-space model connecting one latent psychological arousal condition to neural activities impacting sweating (inferred through SC responses) and another latent state to expressive behavior during typing. These states are concurrently evaluated with model parameters using an expectation-maximization algorithms approach. Assessments using both computer-simulated data and experimental data substantiate the validity of our approach. Outcomes display distinguishable latent state patterns in expressive typing and arousal across different computer software used in office management, offering profound implications for Human-Computer Interaction (HCI) and productivity analysis. This research marks a significant advancement in decoding human productivity dynamics, with extensive repercussions for optimizing performance in telecommuting scenarios.

Regulation of brain cognitive states through auditory, gustatory, and olfactory stimulation with wearable monitoring

Inspired by advances in wearable technologies, we design and perform human-subject experiments. We aim to investigate the effects of applying safe actuation (i.e., auditory, gustatory, and olfactory) for the purpose of regulating cognitive arousal and enhancing the performance states. In two proposed experiments, subjects are asked to perform a working memory experiment called n-back tasks. Next, we incorporate listening to different types of music, drinking coffee, and smelling perfume as safe actuators. We employ signal processing methods to seamlessly infer participants' brain cognitive states. The results demonstrate the effectiveness of the proposed safe actuation in regulating the arousal state and enhancing performance levels. Employing only wearable devices for human monitoring and using safe actuation intervention are the key components of the proposed experiments. Our dataset fills the existing gap of the lack of publicly available datasets for the self-management of internal brain states using wearable devices and safe everyday actuators. This dataset enables further machine learning and system identification investigations to facilitate future smart work environments. This would lead us to the ultimate idea of developing practical automated personalized closed-loop architectures for managing internal brain states and enhancing the quality of life.

A Point-Process Approach for Tracking Valence using a Respiration Belt

Emotional valence is difficult to be inferred since it is related to several psychological factors and is affected by inter- and intra-subject variability. Changes in emotional valence have been found to cause a physiological response in respiration signals. In this study, we propose a state-space model and decode the valence by analyzing a person's respiration pattern. Particularly, we generate a binary point process based on features that are indicative of changes in respiration pattern as a result of an emotional valence response. High valence is typically associated with faster and deeper breathing. As a result, (i)depth of breath, (ii)rate of respiration, and (iii) breathing cycle time are indicators of high valence and used to generate the binary point process representing underlying neural stimuli associated with changes in valence. We utilize an expectation-maximization (EM) framework to decode a hidden valence state and the associated valence index. This predicted valence state is compared to self-reported valence ratings to optimize the parameters and determine the accuracy of the model. The accuracy of the model in predicting high and low valence events is found to be 77% and 73%, respectively. Our study can be applied towards the long term analysis of valence. Additionally, it has applications in a closed-loop system procedures and wearable design paradigm to track and regulate the emotional valence.

Physiological characterization of electrodermal activity enables scalable near real-time autonomic nervous system activation inference

Electrodermal activities (EDA) are any electrical phxenomena observed on the skin. Skin conductance (SC), a measure of EDA, shows fluctuations due to autonomic nervous system (ANS) activation induced sweat secretion. Since it can capture psychophysiological information, there is a significant rise in the research work for tracking mental and physiological health with EDA. However, the current state-of-the-art lacks a physiologically motivated approach for real-time inference of ANS activation from EDA. Therefore, firstly, we propose a comprehensive model for the SC dynamics. The proposed model is a 3D state-space representation of the direct secretion of sweat via pore opening and diffusion followed by corresponding evaporation and reabsorption. As the input to the model, we consider a sparse signal representing the ANS activation that causes the sweat glands to produce sweat. Secondly, we derive a scalable fixed-interval smoother-based sparse recovery approach utilizing the proposed comprehensive model to infer the ANS activation enabling edge computation. We incorporate a generalized-cross-validation to tune the sparsity level. Finally, we propose an Expectation-Maximization based deconvolution approach for learning the model parameters during the ANS activation inference. For evaluation, we utilize a dataset with 26 participants, and the results show that our comprehensive state-space model can successfully describe the SC variations with high scalability, showing the feasibility of real-time applications. Results validate that our physiology-motivated state-space model can comprehensively explain the EDA and outperforms all previous approaches. Our findings introduce a whole new perspective and have a broader impact on the standard practices of EDA analysis.

A State-space Investigation of Impact of Music on Cognitive Performance during a Working Memory Experiment

Stress has effects on productivity and performance. Poor stress management may lead to reduced productivity and performance. Non-invasive actuators such as music have the potential to effectively regulate stress. In this study, using a state-space approach, we obtain a performance state to investigate the performance during a working memory task while playing two different types of music in the background. In our experiments, participants performed a working memory task while listening to calming and vexing music of their choice. We utilize the binary correct/incorrect response and the continuous reaction time of the response from the participants to quantify the performance. The state-space quantification reveals that vexing music has a statistically significant positive impact on the obtained performance state. This indicates the feasibility of designing non-invasive closed-loop systems to regulate stress for maximizing performance and productivity.

Decoding a Music-Modulated Cognitive Arousal State using Electrodermal Activity and Functional Near-infrared Spectroscopy Measurements

Biofeedback systems sense different physiological activities and help with gaining self-awareness. Understanding music's impact on the arousal state is of great importance for biofeedback stress management systems. In this study, we investigate a cognitive-stress-related arousal state modulated by different types of music. During our experiments, each subject was presented with neurological stimuli that elicit a cognitive-stress-related arousal response in a working memory experiment. Moreover, this cognitive-stress-related arousal was modulated by calming and vexing music played in the background. Electrodermal activity and functional near-infrared spectroscopy (fNIRS) measurements both contain information related to cognitive arousal and were collected in our study. By considering various fNIRS features, we selected three features based on variance, root mean square, and local fNIRS peaks as the most informative fNIRS observations in terms of cognitive arousal. The rate of neural impulse occurrence underlying EDA was taken as a binary observation. To retain a low computational complexity for our decoder and select the best fNIRS-based observations, two features were chosen as fNIRS-based observations at a time. A decoder based on one binary and two continuous observations was utilized to estimate the hidden cognitive-stress-related arousal state. This was done by using a Bayesian filtering approach within an expectation-maximization framework. Our results indicate that the decoded cognitive arousal modulated by vexing music was higher than calming music. Among the three fNIRS observations selected, a combination of observations based on root mean square and local fNIRS peaks resulted in the best decoded states for our experimental settings. This study serves as a proof of concept for utilizing fNIRS and EDA measurements to develop a low-dimensional decoder for tracking cognitive-stress-related arousal levels.

Decoding a Neurofeedback-Modulated Cognitive Arousal State to Investigate Performance Regulation by the Yerkes-Dodson Law

Enhancing the productivity of humans by regulating arousal during cognitive tasks is a challenging topic in psychology that has a great potential to transform workplaces for increased productivity and educational systems for enhanced performance. In this study, we assess the feasibility of using the Yerkes-Dodson law from psychology to improve performance during a working memory experiment. We employ a Bayesian filtering approach to track cognitive arousal and performance. In particular, by utilizing skin conductance signal recorded during a working memory experiment in the presence of music, we decode a cognitive arousal state. This is done by considering the rate of neural impulse occurrences and their amplitudes as observations for the arousal model. Similarly, we decode a performance state using the number of correct and incorrect responses, and the reaction time as binary and continuous behavioral observations, respectively. We estimate the arousal and performance states within an expectation-maximization framework. Thereafter, we design an arousal-performance model on the basis of the Yerkes-Dodson law and estimate the model parameters via regression analysis. In this experiment musical neurofeedback was used to modulate cognitive arousal. Our investigations indicate that music can be used as a mode of actuation to influence arousal and enhance the cognitive performance during working memory tasks. Our findings can have a significant impact on designing future smart workplaces and online educational systems.

A Mixed Filtering Approach for Real-Time Seizure State Tracking Using Multi-Channel Electroencephalography Data

Real-time continuous tracking of seizure state is necessary to develop feedback neuromodulation therapy that can prevent or terminate a seizure early. Due to its high temporal resolution, high scalp coverage, and non-invasive applicability, electroencephalography (EEG) is a good candidate for seizure tracking. In this research, we make multiple seizure state estimations using a mixed-filter and multiple channels found over the entire sensor space; then by applying a Kalman filter, we produce a single seizure state estimation made up of these individual estimations. Using a modified wrapper feature selection, we determine two optimal features of mixed data type, one continuous and one binary analyzing all available channels. These features are used in a state-space framework to model the continuous hidden seizure state. Expectation maximization is performed offline on the training and validation data sets to estimate unknown parameters. The seizure state estimation process is performed for multiple channels, and the seizure state estimation is derived using a square-root Kalman filter. A second expectation maximization step is utilized to estimate the unknown square-root Kalman filter parameters. This method is tested in a real-time applicable way for seizure state estimation. Applying this approach, we obtain a single seizure state estimation with quantitative information about the likelihood of a seizure occurring, which we call seizure probability. Our results on the experimental data (CHB-MIT EEG database) validate the proposed estimation method and we achieve an average accuracy, sensitivity, and specificity of 92.7%, 92.8%, and 93.4%, respectively. The potential applications of this seizure estimation model are for closed-loop neuromodulation and long-term quantitative analysis of seizure treatment efficacy.

A mixed filter algorithm for sympathetic arousal tracking from skin conductance and heart rate measurements in Pavlovian fear conditioning

Pathological fear and anxiety disorders can have debilitating impacts on individual patients and society. The neural circuitry underlying fear learning and extinction has been known to play a crucial role in the development and maintenance of anxiety disorders. Pavlovian conditioning, where a subject learns an association between a biologically-relevant stimulus and a neutral cue, has been instrumental in guiding the development of therapies for treating anxiety disorders. To date, a number of physiological signal responses such as skin conductance, heart rate, electroencephalography and cerebral blood flow have been analyzed in Pavlovian fear conditioning experiments. However, physiological markers are often examined separately to gain insight into the neural processes underlying fear acquisition. We propose a method to track a single brain-related sympathetic arousal state from physiological signal features during fear conditioning. We develop a state-space formulation that probabilistically relates features from skin conductance and heart rate to the unobserved sympathetic arousal state. We use an expectation-maximization framework for state estimation and model parameter recovery. State estimation is performed via Bayesian filtering. We evaluate our model on simulated and experimental data acquired in a trace fear conditioning experiment. Results on simulated data show the ability of our proposed method to estimate an unobserved arousal state and recover model parameters. Results on experimental data are consistent with skin conductance measurements and provide good fits to heartbeats modeled as a binary point process. The ability to track arousal from skin conductance and heart rate within a state-space model is an important precursor to the development of wearable monitors that could aid in patient care. Anxiety and trauma-related disorders are often accompanied by a heightened sympathetic tone and the methods described herein could find clinical applications in remote monitoring for therapeutic purposes.

Skin Conductance as a Viable Alternative for Closing the Deep Brain Stimulation Loop in Neuropsychiatric Disorders

Markers from local field potentials, neurochemicals, skin conductance, and hormone concentrations have been proposed as a means of closing the loop in Deep Brain Stimulation (DBS) therapy for treating neuropsychiatric and movement disorders. Developing a closed-loop DBS controller based on peripheral signals would require: (i) the recovery of a biomarker from the source neural stimuli underlying the peripheral signal variations; (ii) the estimation of an unobserved brain or central nervous system related state variable from the biomarker. The state variable is application-specific. It is emotion-related in the case of depression or post-traumatic stress disorder, and movement-related for Parkinson's or essential tremor. We present a method for closing the DBS loop in neuropsychiatric disorders based on the estimation of sympathetic arousal from skin conductance measurements. We deconvolve skin conductance via an optimization formulation utilizing sparse recovery and obtain neural impulses from sympathetic nerve fibers stimulating the sweat glands. We perform this deconvolution via a two-step coordinate descent procedure that recovers the sparse neural stimuli and estimates physiological system parameters simultaneously. We next relate an unobserved sympathetic arousal state to the probability that these neural impulses occur and use Bayesian filtering within an Expectation-Maximization framework for estimation. We evaluate our method on a publicly available data-set examining the effect of different types of stress on peripheral signal changes including body temperature, skin conductance and heart rate. A high degree of arousal is estimated during cognitive tasks, as are much lower levels during relaxation. The results demonstrate the ability to decode psychological arousal from neural activity underlying skin conductance signal variations. The complete pipeline from recovering neural stimuli to decoding an emotion-related brain state using skin conductance presents a promising methodology for the ultimate realization of a closed-loop DBS controller. Closed-loop DBS treatment would additionally help reduce unnecessary power consumption and improve therapeutic gains.