Signal processing of functional NIRS data acquired during overt speaking

Support vector machine prediction of individual Autism Diagnostic Observation Schedule (ADOS) scores based on neural responses during live eye-to-eye contact

Social difficulties during interactions with others are central to autism spectrum disorder (ASD). Understanding the links between these social difficulties and their underlying neural processes is a primary aim focused on improved diagnosis and treatment. In keeping with this goal, we have developed a multivariate classification method based on neural data acquired by functional near infrared spectroscopy, fNIRS, during live eye-to-eye contact with adults who were either typically developed (TD) or individuals with ASD. The ASD diagnosis was based on the gold-standard Autism Diagnostic Observation Schedule (ADOS) which also provides an index of symptom severity. Using a nested cross-validation method, a support vector machine (SVM) was trained to discriminate between ASD and TD groups based on the neural responses during eye-to-eye contact. ADOS scores were not applied in the classification training. To test the hypothesis that SVM identifies neural activity patterns related to one of the neural mechanisms underlying the behavioral symptoms of ASD, we determined the correlation coefficient between the SVM scores and the individual ADOS scores. Consistent with the hypothesis, the correlation between observed and predicted ADOS scores was 0.72 (p < 0.002). Findings suggest that multivariate classification methods combined with the live interaction paradigm of eye-to-eye contact provide a promising approach to link neural processes and social difficulties in individuals with ASD.

Effects of age on the interactions of attentional and emotional processes: a prefrontal fNIRS study

The aging of attentional and emotional functions has been extensively studied but relatively independently. Therefore, the relationships between aging and the interactions of attentional and emotional processes remain elusive. This study aimed to determine how age affected the interactions between attentional and emotional processes during adulthood. One-hundred forty adults aged 18-79 performed the emotional variant of the Attention Network Test, which probed alerting, orienting, and executive control in the presence and absence of threatening faces. During this task, contexts with varying levels of task preparatory processes were created to modulate the effect of threatening faces on attention, and functional near-infrared spectroscopy (fNIRS) was used to examine the neural underpinnings of the behavioural effects. The behavioural results showed that aging was associated with a significant decline in alerting efficiency, and there was a statistical trend for age-related deficits in executive control. Despite these age differences, age did not significantly moderate the interactions among attentional networks or between attention and emotion. Additionally, the fNIRS results showed that decreased frontal cortex functioning might underlie the age-related decline in executive control. Therefore, while aging has varying effects on different attentional networks, the interactions of attentional and emotional processes remain relatively unaffected by age.

Effects of age, gender, and education on task performance and prefrontal cortex processing during emotional and non-emotional verbal fluency tests

The emotional semantic fluency test (SFT) is an emerging verbal fluency test that requires controlled access to emotional lexical information. Currently, how demographic variables influence neurocognitive processing during this test remains elusive. The present study compared the effects of age, gender, and education on task performance and prefrontal cortex (PFC) processing during the non-emotional and emotional SFTs. One-hundred and thirty-three Cantonese-speaking adults aged 18-79 performed the non-emotional and emotional SFTs while their PFC activation was measured using functional near-infrared spectroscopy. Results showed that more education predicted better non-emotional SFT performance, whereas younger age, being female, and more education predicted better emotional SFT performance. Only age significantly affected PFC activation during the SFTs, and the effect was comparable between the two SFTs. Thus, compared with its non-emotional analog, the emotional SFT is influenced by overlapping yet distinct demographic variables. There is a similar age-related reorganization of PFC function across SFT performances.

Neural mechanisms for emotional contagion and spontaneous mimicry of live facial expressions

Viewing a live facial expression typically elicits a similar expression by the observer (facial mimicry) that is associated with a concordant emotional experience (emotional contagion). The model of embodied emotion proposes that emotional contagion and facial mimicry are functionally linked although the neural underpinnings are not known. To address this knowledge gap, we employed a live two-person paradigm (n = 20 dyads) using functional near-infrared spectroscopy during live emotive face-processing while also measuring eye-tracking, facial classifications and ratings of emotion. One dyadic partner, 'Movie Watcher', was instructed to emote natural facial expressions while viewing evocative short movie clips. The other dyadic partner, 'Face Watcher', viewed the Movie Watcher's face. Task and rest blocks were implemented by timed epochs of clear and opaque glass that separated partners. Dyadic roles were alternated during the experiment. Mean cross-partner correlations of facial expressions (r = 0.36 ± 0.11 s.e.m.) and mean cross-partner affect ratings (r = 0.67 ± 0.04) were consistent with facial mimicry and emotional contagion, respectively. Neural correlates of emotional contagion based on covariates of partner affect ratings included angular and supramarginal gyri, whereas neural correlates of the live facial action units included motor cortex and ventral face-processing areas. Findings suggest distinct neural components for facial mimicry and emotional contagion. This article is part of a discussion meeting issue 'Face2face: advancing the science of social interaction'.

Performance comparison of systemic activity correction in functional near-infrared spectroscopy for methods with and without short distance channels

Significance: Functional near-infrared spectroscopy (fNIRS) is a promising tool for neurofeedback (NFB) or brain-computer interfaces (BCIs). However, fNIRS signals are typically highly contaminated by systemic activity (SA) artifacts, and, if not properly corrected, NFB or BCIs run the risk of being based on noise instead of brain activity. This risk can likely be reduced by correcting for SA, in particular when short-distance channels (SDCs) are available. Literature comparing correction methods with and without SDCs is still sparse, specifically comparisons considering single trials are lacking. Aim: This study aimed at comparing the performance of SA correction methods with and without SDCs. Approach: Semisimulated and real motor task data of healthy older adults were used. Correction methods without SDCs included a simple and a more advanced spatial filter. Correction methods with SDCs included a regression approach considering only the closest SDC and two GLM-based methods, one including all eight SDCs and one using only two a priori selected SDCs as regressors. All methods were compared with data uncorrected for SA and correction performance was assessed with quality measures quantifying signal improvement and spatial specificity at single trial level. Results: All correction methods were found to improve signal quality and enhance spatial specificity as compared with the uncorrected data. Methods with SDCs usually outperformed methods without SDCs. Correction methods without SDCs tended to overcorrect the data. However, the exact pattern of results and the degree of differences observable between correction methods varied between semisimulated and real data, and also between quality measures. Conclusions: Overall, results confirmed that both Δ [ HbO ] and Δ [ HbR ] are affected by SA and that correction methods with SDCs outperform methods without SDCs. Nonetheless, improvements in signal quality can also be achieved without SDCs and should therefore be given priority over not correcting for SA.

Emotional modulation of cortical activity during gum chewing: A functional near-infrared spectroscopy study

Distinct brain regions are known to be associated with various emotional states. Cortical activity may be modulated by emotional states that are triggered by flavors during food intake. We examined cortical activity during chewing with different flavors and assessed the emotional modulation of cortical activity using multichannel near-infrared spectroscopy. Thirty-six right-handed volunteers participated in this crossover trial. The participants experienced positive and negative emotions from chewing flavorful (palatable) or less flavorful (unpalatable) gums, respectively for 5 min. Participants rated the taste, odor, and deliciousness of each gum using a visual analog scale. Bilateral hemodynamic responses in the frontal and parietal lobes, bilateral masseter muscle activation, and heart rate were measured during gum chewing. Changes in all measured data during gum chewing were also evaluated. The ratings of the tastes and odors of each gum significantly differed among the participants (P &lt; 0.001). Hemodynamic response changes were significantly elevated in the bilateral primary sensorimotor cortex during gum-chewing, in comparison to resting. The difference in hemodynamic responses between palatable and unpalatable gum conditions was detected in the left frontopolar/dorsolateral prefrontal cortex. Muscle activation and heart rate were not significantly different between different gum types. Our findings indicate that differential processing in the left prefrontal cortex might be responsible for the emotional states caused by palatable and unpalatable foods.

Neural correlates of eye contact and social function in autism spectrum disorder

Reluctance to make eye contact during natural interactions is a central diagnostic criterion for autism spectrum disorder (ASD). However, the underlying neural correlates for eye contacts in ASD are unknown, and diagnostic biomarkers are active areas of investigation. Here, neuroimaging, eye-tracking, and pupillometry data were acquired simultaneously using two-person functional near-infrared spectroscopy (fNIRS) during live "in-person" eye-to-eye contact and eye-gaze at a video face for typically-developed (TD) and participants with ASD to identify the neural correlates of live eye-to-eye contact in both groups. Comparisons between ASD and TD showed decreased right dorsal-parietal activity and increased right ventral temporal-parietal activity for ASD during live eye-to-eye contact (p≤0.05, FDR-corrected) and reduced cross-brain coherence consistent with atypical neural systems for live eye contact. Hypoactivity of right dorsal-parietal regions during eye contact in ASD was further associated with gold standard measures of social performance by the correlation of neural responses and individual measures of: ADOS-2, Autism Diagnostic Observation Schedule, 2nd Edition (r = -0.76, -0.92 and -0.77); and SRS-2, Social Responsiveness Scale, Second Edition (r = -0.58). The findings indicate that as categorized social ability decreases, neural responses to real eye-contact in the right dorsal parietal region also decrease consistent with a neural correlate for social characteristics in ASD.

Young female participants show blunted placebo effects associated with blunted responses to a cue predicting a safe stimulus in the right dorsolateral prefrontal cortex

Discrimination of cues predicting non-nociceptive/nociceptive stimuli is essential for predicting whether a non-painful or painful stimulus will be administered and for eliciting placebo/nocebo (pain reduction/pain enhancement) effects. Dysfunction of the neural system involved in placebo effects has been implicated in the pathology of chronic pain, while female sex is one of the important risk factors for development of chronic pain in young adults. The dorsolateral prefrontal cortex (dl-PFC) is suggested to be involved in placebo effects and is sensitive to sex and age. In this study, to examine the neural mechanisms by which sex and age alter placebo and nocebo effects, we analyzed cerebral hemodynamic activities in the dl-PFC in different sex and age groups during a differential conditioning task. During the training session, two different sounds were followed by low- and high-intensity electrical shocks. In the following recording session, electrical shocks, the intensity of which was mismatched to the sounds, were occasionally administered to elicit placebo and nocebo effects. In young female participants, both placebo effects and hemodynamic responses to the conditioned sounds in the right dl-PFC were significantly lower than those in elderly female participants, while there were no age differences in male participants. The hemodynamic responses to the sound paired with the safe stimulus in the right dl-PFC were significantly correlated with placebo effects, except in the young female group. These results suggest that blunted placebo effects in the young female participants are ascribed to blunted responses to the sound associated with the safe stimulus in the right dl-PFC, and that sex- and age-related factors may alter the responsiveness of the right dl-PFC to associative cues predicting a safe stimulus.

Parallel factor analysis for multidimensional decomposition of functional near-infrared spectroscopy data

SIGNIFICANCE

Current techniques for data analysis in functional near-infrared spectroscopy (fNIRS), such as artifact correction, do not allow to integrate the information originating from both wavelengths, considering only temporal and spatial dimensions of the signal's structure. Parallel factor analysis (PARAFAC) has previously been validated as a multidimensional decomposition technique in other neuroimaging fields.

AIM

We aimed to introduce and validate the use of PARAFAC for the analysis of fNIRS data, which is inherently multidimensional (time, space, and wavelength).

APPROACH

We used data acquired in 17 healthy adults during a verbal fluency task to compare the efficacy of PARAFAC for motion artifact correction to traditional two-dimensional decomposition techniques, i.e., target principal (tPCA) and independent component analysis (ICA). Correction performance was further evaluated under controlled conditions with simulated artifacts and hemodynamic response functions.

RESULTS

PARAFAC achieved significantly higher improvement in data quality as compared to tPCA and ICA. Correction in several simulated signals further validated its use and promoted it as a robust method independent of the artifact's characteristics.

CONCLUSIONS

This study describes the first implementation of PARAFAC in fNIRS and provides validation for its use to correct artifacts. PARAFAC is a promising data-driven alternative for multidimensional data analyses in fNIRS and this study paves the way for further applications.

Prefrontal cortical activation associated with prospective memory while walking around a real-world street environment

Rostral PFC (area 10) activation is common during prospective memory (PM) tasks. But it is not clear what mental processes these activations index. Three candidate explanations from cognitive neuroscience theory are: (i) monitoring of the environment; (ii) spontaneous intention retrieval; (iii) a combination of the two. These explanations make different predictions about the temporal and spatial patterns of activation that would be seen in rostral PFC in naturalistic settings. Accordingly, we plotted functional events in PFC using portable fNIRS while people were carrying out a PM task outside the lab and responding to cues when they were encountered, to decide between these explanations. Nineteen people were asked to walk around a street in London, U.K. and perform various tasks while also remembering to respond to prospective memory (PM) cues when they detected them. The prospective memory cues could be either social (involving greeting a person) or non-social (interacting with a parking meter) in nature. There were also a number of contrast conditions which allowed us to determine activation specifically related to the prospective memory components of the tasks. We found that maintaining both social and non-social intentions was associated with widespread activation within medial and right hemisphere rostral prefrontal cortex (BA 10), in agreement with numerous previous lab-based fMRI studies of prospective memory. In addition, increased activation was found within lateral prefrontal cortex (BA 45 and 46) when people were maintaining a social intention compared to a non-social one. The data were then subjected to a GLM-based method for automatic identification of functional events (AIDE), and the position of the participants at the time of the activation events were located on a map of the physical space. The results showed that the spatial and temporal distribution of these events was not random, but aggregated around areas in which the participants appeared to retrieve their future intentions (i.e., where they saw intentional cues), as well as where they executed them. Functional events were detected most frequently in BA 10 during the PM conditions compared to other regions and tasks. Mobile fNIRS can be used to measure higher cognitive functions of the prefrontal cortex in "real world" situations outside the laboratory in freely ambulant individuals. The addition of a "brain-first" approach to the data permits the experimenter to determine not only when haemodynamic changes occur, but also where the participant was when it happened. This can be extremely valuable when trying to link brain and cognition.

Activation in Right Dorsolateral Prefrontal Cortex Underlies Stuttering Anticipation

People who stutter learn to anticipate many of their overt stuttering events. Despite the critical role of anticipation, particularly how responses to anticipation shape stuttering behaviors, the neural bases associated with anticipation are unknown. We used a novel approach to identify anticipated and unanticipated words, which were produced by 22 adult stutterers in a delayed-response task while hemodynamic activity was measured using functional near infrared spectroscopy (fNIRS). Twenty-two control participants were included such that each individualized set of anticipated and unanticipated words was produced by one stutterer and one control participant. We conducted an analysis on the right dorsolateral prefrontal cortex (R-DLPFC) based on converging lines of evidence from the stuttering and cognitive control literatures. We also assessed connectivity between the R-DLPFC and right supramarginal gyrus (R-SMG), two key nodes of the frontoparietal network (FPN), to assess the role of cognitive control, and particularly error-likelihood monitoring, in stuttering anticipation. All analyses focused on the five-second anticipation phase preceding the go signal to produce speech. The results indicate that anticipated words are associated with elevated activation in the R-DLPFC, and that compared to non-stutterers, stutterers exhibit greater activity in the R-DLPFC, irrespective of anticipation. Further, anticipated words are associated with reduced connectivity between the R-DLPFC and R-SMG. These findings highlight the potential roles of the R-DLPFC and the greater FPN as a neural substrate of stuttering anticipation. The results also support previous accounts of error-likelihood monitoring and action-stopping in stuttering anticipation. Overall, this work offers numerous directions for future research with clinical implications for targeted neuromodulation.

An Approach to Neuroimaging Interpersonal Interactions in Mental Health Interventions

BACKGROUND

Conventional paradigms in clinical neuroscience tend to be constrained in terms of ecological validity, raising several challenges to studying the mechanisms mediating treatments and outcomes in clinical settings. Addressing these issues requires real-world neuroimaging techniques that are capable of continuously collecting data during free-flowing interpersonal interactions and that allow for experimental designs that are representative of the clinical situations in which they occur.

METHODS

In this work, we developed a paradigm that fractionates the major components of human-to-human verbal interactions occurring in clinical situations and used functional near-infrared spectroscopy to assess the brain systems underlying clinician-client discourse (N = 30).

RESULTS

Cross-brain neural coupling between people was significantly greater during clinical interactions compared with everyday life verbal communication, particularly between the prefrontal cortex (e.g., inferior frontal gyrus) and inferior parietal lobule (e.g., supramarginal gyrus). The clinical tasks revealed extensive increases in activity across the prefrontal cortex, especially in the rostral prefrontal cortex (area 10), during periods in which participants were required to silently reason about the dysfunctional cognitions of the other person.

CONCLUSIONS

This work demonstrates a novel experimental approach to investigating the neural underpinnings of interpersonal interactions that typically occur in clinical settings, and its findings support the idea that particular prefrontal systems might be critical to cultivating mental health.

Evaluation of fNIRS signal components elicited by cognitive and hypercapnic stimuli

Functional near infrared spectroscopy (fNIRS) measurements are confounded by signal components originating from multiple physiological causes, whose activities may vary temporally and spatially (across tissue layers, and regions of the cortex). Furthermore, the stimuli can induce evoked effects, which may lead to over or underestimation of the actual effect of interest. Here, we conducted a temporal, spectral, and spatial analysis of fNIRS signals collected during cognitive and hypercapnic stimuli to characterize effects of functional versus systemic responses. We utilized wavelet analysis to discriminate physiological causes and employed long and short source-detector separation (SDS) channels to differentiate tissue layers. Multi-channel measures were analyzed further to distinguish hemispheric differences. The results highlight cardiac, respiratory, myogenic, and very low frequency (VLF) activities within fNIRS signals. Regardless of stimuli, activity within the VLF band had the largest contribution to the overall signal. The systemic activities dominated the measurements from the short SDS channels during cognitive stimulus, but not hypercapnic stimulus. Importantly, results indicate that characteristics of fNIRS signals vary with type of the stimuli administered as cognitive stimulus elicited variable responses between hemispheres in VLF band and task-evoked temporal effect in VLF, myogenic and respiratory bands, while hypercapnic stimulus induced a global response across both hemispheres.

Investigating Language and Domain-General Processing in Neurotypicals and Individuals With Aphasia - A Functional Near-Infrared Spectroscopy Pilot Study

Brain reorganization patterns associated with language recovery after stroke have long been debated. Studying mechanisms of spontaneous and treatment-induced language recovery in post-stroke aphasia requires a network-based approach given the potential for recruitment of perilesional left hemisphere language regions, homologous right hemisphere language regions, and/or spared bilateral domain-general regions. Recent hardware, software, and methodological advances in functional near-infrared spectroscopy (fNIRS) make it well-suited to examine this question. fNIRS is cost-effective with minimal contraindications, making it a robust option to monitor treatment-related brain activation changes over time. Establishing clear activation patterns in neurotypical adults during language and domain-general cognitive processes via fNIRS is an important first step. Some fNIRS studies have investigated key language processes in healthy adults, yet findings are challenging to interpret in the context of methodological limitations. This pilot study used fNIRS to capture brain activation during language and domain-general processing in neurotypicals and individuals with aphasia. These findings will serve as a reference when interpreting treatment-related changes in brain activation patterns in post-stroke aphasia in the future. Twenty-four young healthy controls, seventeen older healthy controls, and six individuals with left hemisphere stroke-induced aphasia completed two language tasks (i.e., semantic feature, picture naming) and one domain-general cognitive task (i.e., arithmetic) twice during fNIRS. The probe covered bilateral frontal, parietal, and temporal lobes and included short-separation detectors for scalp signal nuisance regression. Younger and older healthy controls activated core language regions during semantic feature processing (e.g., left inferior frontal gyrus pars opercularis) and lexical retrieval (e.g., left inferior frontal gyrus pars triangularis) and domain-general regions (e.g., bilateral middle frontal gyri) during hard versus easy arithmetic as expected. Consistent with theories of post-stroke language recovery, individuals with aphasia activated areas outside the traditional networks: left superior frontal gyrus and left supramarginal gyrus during semantic feature judgment; left superior frontal gyrus and right precentral gyrus during picture naming; and left inferior frontal gyrus pars opercularis during arithmetic processing. The preliminary findings in the stroke group highlight the utility of using fNIRS to study language and domain-general processing in aphasia.

Bidirectional Connectivity Between Broca's Area and Wernicke's Area During Interactive Verbal Communication

Aim: This investigation aims to advance the understanding of neural dynamics that underlies live and natural interactions during spoken dialogue between two individuals. Introduction: The underlying hypothesis is that functional connectivity between canonical speech areas in the human brain will be modulated by social interaction. Methods: Granger causality was applied to compare directional connectivity across Broca's and Wernicke's areas during verbal conditions consisting of interactive and noninteractive communication. Thirty-three pairs of healthy adult participants alternately talked and listened to each other while performing an object naming and description task that was either interactive or not during hyperscanning with functional near-infrared spectroscopy (fNIRS). In the noninteractive condition, the speaker named and described a picture-object without reference to the partner's description. In the interactive condition, the speaker performed the same task but included an interactive response about the preceding comments of the partner. Causality measures of hemodynamic responses from Broca's and Wernicke's areas were compared between real, surrogate, and shuffled trials within dyads. Results: The interactive communication was characterized by bidirectional connectivity between Wernicke's and Broca's areas of the listener's brain. Whereas this connectivity was unidirectional in the speaker's brain. In the case of the noninteractive condition, both speaker's and listener's brains showed unidirectional top-down (Broca's area to Wernicke's area) connectivity. Conclusion: Together, directional connectivity as determined by Granger analysis reveals bidirectional flow of neuronal information during dynamic two-person verbal interaction for processes that are active during listening (reception) and not during talking (production). Findings are consistent with prior contrast findings (general linear model) showing neural modulation of the receptive language system associated with Wernicke's area during a two-person live interaction. Impact statement The neural dynamics that underlies real-life social interactions is an emergent topic of interest. Dynamically coupled cross-brain neural mechanisms between interacting partners during verbal dialogue have been shown within Wernicke's area. However, it is not known how within-brain long-range neural mechanisms operate during these live social functions. Using Granger causality analysis, we show bidirectional neural activity between Broca's and Wernicke's areas during interactive dialogue compared with a noninteractive control task showing only unidirectional activity. Findings are consistent with an Interactive Brain Model where long-range neural mechanisms process interactive processes associated with rapid and spontaneous spoken social cues.

Speech planning and execution in children who stutter: Preliminary findings from a fNIRS investigation

Few studies have investigated the neural mechanisms underlying speech production in children who stutter (CWS), despite the critical importance of understanding these mechanisms closer to the time of stuttering onset. The relative contributions of speech planning and execution in CWS therefore are also unknown. Using functional near-infrared spectroscopy, the current study investigated neural mechanisms of planning and execution in a small sample of 9-12 year-old CWS and controls (N = 12) by implementing two tasks that manipulated speech planning and execution loads. Planning was associated with atypical activation in bilateral inferior frontal gyrus and right supramarginal gyrus. Execution was associated with atypical activation in bilateral precentral gyrus and inferior frontal gyrus, as well as right supramarginal gyrus and superior temporal gyrus. The CWS exhibited some activation patterns that were similar to the adults who stutter (AWS) as reported in our previous study: atypical planning in frontal areas including left inferior frontal gyrus and atypical execution in fronto-temporo-parietal regions including left precentral gyrus, and right inferior frontal, superior temporal, and supramarginal gyri. However, differences also emerged. Whereas CWS and AWS both appear to exhibit atypical activation in right inferior and supramarginal gyri during execution, only CWS appear to exhibit this same pattern during planning. In addition, the CWS appear to exhibit atypical activation in left inferior frontal and right precentral gyri related to execution, whereas AWS do not. These preliminary results are discussed in the context of possible impairments in sensorimotor integration and inhibitory control for CWS.

Guiding functional near-infrared spectroscopy optode-layout design using individual (f)MRI data: effects on signal strength

Significance: Designing optode layouts is an essential step for functional near-infrared spectroscopy (fNIRS) experiments as the quality of the measured signal and the sensitivity to cortical regions-of-interest depend on how optodes are arranged on the scalp. This becomes particularly relevant for fNIRS-based brain-computer interfaces (BCIs), where developing robust systems with few optodes is crucial for clinical applications. Aim: Available resources often dictate the approach researchers use for optode-layout design. We investigated whether guiding optode layout design using different amounts of subject-specific magnetic resonance imaging (MRI) data affects the fNIRS signal quality and sensitivity to brain activation when healthy participants perform mental-imagery tasks typically used in fNIRS-BCI experiments. Approach: We compared four approaches that incrementally incorporated subject-specific MRI information while participants performed mental-calculation, mental-rotation, and inner-speech tasks. The literature-based approach (LIT) used a literature review to guide the optode layout design. The probabilistic approach (PROB) employed individual anatomical data and probabilistic maps of functional MRI (fMRI)-activation from an independent dataset. The individual fMRI (iFMRI) approach used individual anatomical and fMRI data, and the fourth approach used individual anatomical, functional, and vascular information of the same subject (fVASC). Results: The four approaches resulted in different optode layouts and the more informed approaches outperformed the minimally informed approach (LIT) in terms of signal quality and sensitivity. Further, PROB, iFMRI, and fVASC approaches resulted in a similar outcome. Conclusions: We conclude that additional individual MRI data lead to a better outcome, but that not all the modalities tested here are required to achieve a robust setup. Finally, we give preliminary advice to efficiently using resources for developing robust optode layouts for BCI and neurofeedback applications.

Examination of the Prefrontal Cortex Hemodynamic Responses to the Fist-Edge-Palm Task in Naïve Subjects Using Functional Near-Infrared Spectroscopy

The Fist-Edge-Palm (FEP) task, a manual hand task, has been used to detect frontal dysfunctions in clinical situations: its performance failures are observed in various prefrontal cortex (PFC)-related disorders, including schizophrenia. However, previous imaging studies reported that the performance of the FEP task activated motor-related areas, but not the PFC. Here, we aimed to investigate the relationships between the performance of the FEP task and PFC functions. Hemodynamic activity in the PFC, including the dorsolateral PFC (area 46) and frontal pole (area 10), was recorded. Healthy young subjects performed the FEP task as well as a palm tapping (PT) task (control task) three times. The subjects also completed a Wisconsin Card Sorting Test (WCST) and Schizotypal Personality Scale (STA) questionnaire. We found that hemodynamic activity (Oxy-Hb) in the PFC increased in the first trial of the FEP task but decreased considerably in the second and third trials compared to the PT task. The number of performance errors in the FEP task also decreased in the second and third trials. Error reduction (i.e., learning) in the FEP task between the first and second trials was negatively correlated with schizotypal trait and the number of perseveration errors in the WCST. Furthermore, changes in the PFC hemodynamic activity between the first and second trials were positively correlated with error reduction in the FEP task between the first and second trials, and negatively correlated with the number of perseveration errors in the WCST. These results suggest that learning in the FEP task requires PFC activation, which is negatively associated with perseveration errors in the WCST. The results further suggest that the FEP task, in conjunction with near-infrared spectroscopy, may be useful as a diagnostic method for various disorders with PFC dysfunction.

Facial and neural mechanisms during interactive disclosure of biographical information

Pairs of participants mutually communicated (or not) biographical information to each other. By combining simultaneous eye-tracking, face-tracking and functional near-infrared spectroscopy, we examined how this mutual sharing of information modulates social signalling and brain activity. When biographical information was disclosed, participants directed more eye gaze to the face of the partner and presented more facial displays. We also found that spontaneous production and observation of facial displays was associated with activity in the left SMG and right dlPFC/IFG, respectively. Moreover, mutual information-sharing increased activity in bilateral TPJ and left dlPFC, as well as cross-brain synchrony between right TPJ and left dlPFC. This suggests that a complex long-range mechanism is recruited during information-sharing. These multimodal findings support the second-person neuroscience hypothesis, which postulates that communicative interactions activate additional neurocognitive mechanisms to those engaged in non-interactive situations. They further advance our understanding of which neurocognitive mechanisms underlie communicative interactions.

Comparison of Human Social Brain Activity During Eye-Contact With Another Human and a Humanoid Robot

Robot design to simulate interpersonal social interaction is an active area of research with applications in therapy and companionship. Neural responses to eye-to-eye contact in humans have recently been employed to determine the neural systems that are active during social interactions. Whether eye-contact with a social robot engages the same neural system remains to be seen. Here, we employ a similar approach to compare human-human and human-robot social interactions. We assume that if human-human and human-robot eye-contact elicit similar neural activity in the human, then the perceptual and cognitive processing is also the same for human and robot. That is, the robot is processed similar to the human. However, if neural effects are different, then perceptual and cognitive processing is assumed to be different. In this study neural activity was compared for human-to-human and human-to-robot conditions using near infrared spectroscopy for neural imaging, and a robot (Maki) with eyes that blink and move right and left. Eye-contact was confirmed by eye-tracking for both conditions. Increased neural activity was observed in human social systems including the right temporal parietal junction and the dorsolateral prefrontal cortex during human-human eye contact but not human-robot eye-contact. This suggests that the type of human-robot eye-contact used here is not sufficient to engage the right temporoparietal junction in the human. This study establishes a foundation for future research into human-robot eye-contact to determine how elements of robot design and behavior impact human social processing within this type of interaction and may offer a method for capturing difficult to quantify components of human-robot interaction, such as social engagement.

Interpersonal Agreement and Disagreement During Face-to-Face Dialogue: An fNIRS Investigation

Although the neural systems that underlie spoken language are well-known, how they adapt to evolving social cues during natural conversations remains an unanswered question. In this work we investigate the neural correlates of face-to-face conversations between two individuals using functional near infrared spectroscopy (fNIRS) and acoustical analyses of concurrent audio recordings. Nineteen pairs of healthy adults engaged in live discussions on two controversial topics where their opinions were either in agreement or disagreement. Participants were matched according to their a priori opinions on these topics as assessed by questionnaire. Acoustic measures of the recorded speech including the fundamental frequency range, median fundamental frequency, syllable rate, and acoustic energy were elevated during disagreement relative to agreement. Consistent with both the a priori opinion ratings and the acoustic findings, neural activity associated with long-range functional networks, rather than the canonical language areas, was also differentiated by the two conditions. Specifically, the frontoparietal system including bilateral dorsolateral prefrontal cortex, left supramarginal gyrus, angular gyrus, and superior temporal gyrus showed increased activity while talking during disagreement. In contrast, talking during agreement was characterized by increased activity in a social and attention network including right supramarginal gyrus, bilateral frontal eye-fields, and left frontopolar regions. Further, these social and visual attention networks were more synchronous across brains during agreement than disagreement. Rather than localized modulation of the canonical language system, these findings are most consistent with a model of distributed and adaptive language-related processes including cross-brain neural coupling that serves dynamic verbal exchanges.

Comparison of short-channel separation and spatial domain filtering for removal of non-neural components in functional near-infrared spectroscopy signals

Significance: With the increasing popularity of functional near-infrared spectroscopy (fNIRS), the need to determine localization of the source and nature of the signals has grown. Aim: We compare strategies for removal of non-neural signals for a finger-thumb tapping task, which shows responses in contralateral motor cortex and a visual checkerboard viewing task that produces activity within the occipital lobe. Approach: We compare temporal regression strategies using short-channel separation to a spatial principal component (PC) filter that removes global signals present in all channels. For short-channel temporal regression, we compare non-neural signal removal using first and combined first and second PCs from a broad distribution of short channels to limited distribution on the forehead. Results: Temporal regression of non-neural information from broadly distributed short channels did not differ from forehead-only distribution. Spatial PC filtering provides results similar to short-channel separation using the temporal domain. Utilizing both first and second PCs from short channels removes additional non-neural information. Conclusions: We conclude that short-channel information in the temporal domain and spatial domain regression filtering methods remove similar non-neural components represented in scalp hemodynamics from fNIRS signals and that either technique is sufficient to remove non-neural components.

Comparing fNIRS signal qualities between approaches with and without short channels

Functional near-infrared spectroscopy (fNIRS) is a non-invasive technique used to measure changes in oxygenated (HbO) and deoxygenated (HbR) hemoglobin, related to neuronal activity. fNIRS signals are contaminated by the systemic responses in the extracerebral tissue (superficial layer) of the head, as fNIRS uses a back-reflection measurement. Using shorter channels that are only sensitive to responses in the extracerebral tissue but not in the deeper layers where target neuronal activity occurs has been a ‘gold standard’ to reduce the systemic responses in the fNIRS data from adults. When shorter channels are not available or feasible for implementation, an alternative, i.e., anti-correlation (Anti-Corr) method has been adopted. To date, there has not been a study that directly assesses the outcomes from the two approaches. In this study, we compared the Anti-Corr method with the ‘gold standard’ in reducing systemic responses to improve fNIRS neural signal qualities. We used eight short channels (8-mm) in a group of adults, and conducted a principal component analysis (PCA) to extract two components that contributed the most to responses in the 8 short channels, which were assumed to contain the global components in the extracerebral tissue. We then used a general linear model (GLM), with and without including event-related regressors, to regress out the 2 principal components from regular fNIRS channels (30 mm), i.e., two GLM-PCA methods. Our results found that, the two GLM-PCA methods showed similar performance, both GLM-PCA methods and the Anti-Corr method improved fNIRS signal qualities, and the two GLM-PCA methods had better performance than the Anti-Corr method.

Neural processes for live pro-social dialogue between dyads with socioeconomic disparity

An emerging theoretical framework suggests that neural functions associated with stereotyping and prejudice are associated with frontal lobe networks. Using a novel neuroimaging technique, functional near-infrared spectroscopy (fNIRS), during a face-to-face live communication paradigm, we explore an extension of this model to include live dynamic interactions. Neural activations were compared for dyads of similar and dissimilar socioeconomic backgrounds. The socioeconomic status of each participant was based on education and income levels. Both groups of dyads engaged in pro-social dialectic discourse during acquisition of hemodynamic signals. Post-scan questionnaires confirmed increased anxiety and effort for high-disparity dyads. Consistent with the frontal lobe hypothesis, left dorsolateral pre-frontal cortex (DLPFC), frontopolar area and pars triangularis were more active during speech dialogue in high than in low-disparity groups. Further, frontal lobe signals were more synchronous across brains for high- than low-disparity dyads. Convergence of these behavioral, neuroimaging and neural coupling findings associate left frontal lobe processes with natural pro-social dialogue under 'out-group' conditions and advance both theoretical and technical approaches for further investigation.

Joint Attention During Live Person-to-Person Contact Activates rTPJ, Including a Sub-Component Associated With Spontaneous Eye-to-Eye Contact

Eye-to-eye contact is a spontaneous behavior between interacting partners that occurs naturally during social interactions. However, individuals differ with respect to eye gaze behaviors such as frequency of eye-to-eye contacts, and these variations may reflect underlying differences in social behavior in the population. While the use of eye signaling to indicate a shared object of attention in joint attention tasks has been well-studied, the effects of the natural variation in establishing eye contact during joint attention have not been isolated. Here, we investigate this question using a novel two-person joint attention task. Participants were not instructed regarding the use of eye contacts; thus all mutual eye contact events between interacting partners that occurred during the joint attention task were spontaneous and varied with respect to frequency. We predicted that joint attention systems would be modulated by differences in the social behavior across participant pairs, which could be measured by the frequency of eye contact behavior. We used functional near-infrared spectroscopy (fNIRS) hyperscanning and eye-tracking to measure the neural signals associated with joint attention in interacting dyads and to record the number of eye contact events between them. Participants engaged in a social joint attention task in which real partners used eye gaze to direct each other's attention to specific targets. Findings were compared to a non-social joint attention task in which an LED cue directed both partners' attention to the same target. The social joint attention condition showed greater activity in right temporoparietal junction than the non-social condition, replicating prior joint attention results. Eye-contact frequency modulated the joint attention activity, revealing bilateral activity in social and high level visual areas associated with partners who made more eye contact. Additionally, when the number of mutual eye contact events was used to classify each pair as either "high eye contact" or "low eye contact" dyads, cross-brain coherence analysis revealed greater coherence between high eye contact dyads than low eye contact dyads in these same areas. Together, findings suggest that variation in social behavior as measured by eye contact modulates activity in a subunit of the network associated with joint attention.

Cerebral Hemodynamic Responses to the Sensory Conflict Between Visual and Rotary Vestibular Stimuli: An Analysis With a Multichannel Near-Infrared Spectroscopy (NIRS) System

Sensory conflict among visual, vestibular, and somatosensory information induces vertiginous sensation and postural instability. To elucidate the cognitive mechanisms of the integration between the visual and vestibular cues in humans, we analyzed the cortical hemodynamic responses during sensory conflict between visual and horizontal rotatory vestibular stimulation using a multichannel near-infrared spectroscopy (NIRS) system. The subjects sat on a rotatory chair that was accelerated at 3°/s² for 20 s to the right or left, kept rotating at 60°/s for 80 s, and then decelerated at 3°/s² for 20 s. The subjects were instructed to watch white stripes projected on a screen surrounding the chair during the acceleration and deceleration periods. The white stripes moved in two ways; in the "congruent" condition, the stripes moved in the opposite direction of chair rotation at 3°/s² (i.e., natural visual stimulation), whereas in the "incongruent" condition, the stripes moved in the same direction of chair rotation at 3°/s² (i.e., conflicted visual stimulation). The cortical hemodynamic activity was recorded from the bilateral temporoparietal regions. Statistical analyses using NIRS-SPM software indicated that hemodynamic activity increased in the bilateral temporoparietal junctions (TPJs) and human MT+ complex, including the medial temporal (MT) area and medial superior temporal (MST) area in the incongruent condition. Furthermore, the subjective strength of the vertiginous sensation was negatively correlated with hemodynamic activity in the dorsal part of the supramarginal gyrus (SMG) in and around the intraparietal sulcus (IPS). These results suggest that sensory conflict between the visual and vestibular stimuli promotes cortical cognitive processes in the cortical network consisting of the TPJ, the medial temporal gyrus (MTG), and IPS, which might contribute to self-motion perception to maintain a sense of balance or equilibrioception during sensory conflict.

Real-Time Eye-to-Eye Contact Is Associated With Cross-Brain Neural Coupling in Angular Gyrus

Direct eye contact between two individuals is a salient social behavior known to initiate and promote interpersonal interaction. However, the neural processes that underlie these live interactive behaviors and eye-to-eye contact are not well understood. The Dynamic Neural Coupling Hypothesis presents a general theoretical framework proposing that shared interactive behaviors are represented by cross-brain signal coherence. Using functional near-infrared spectroscopy (fNIRS) adapted for hyper scanning, we tested this hypothesis specifically for neural mechanisms associated with eye-to-eye gaze between human participants compared to similar direct eye-gaze at a dynamic video of a face and predicted that the coherence of neural signals between the two participants during reciprocal eye-to-eye contact would be greater than coherence observed during direct eye-gaze at a dynamic video for those signals originating in social and face processing systems. Consistent with this prediction cross-brain coherence was increased for signals within the angular gyrus (AG) during eye-to-eye contact relative to direct eye-gaze at a dynamic face video (p < 0.01). Further, activity in the right temporal-parietal junction (TPJ) was increased in the real eye-to-eye condition (p < 0.05, FDR corrected). Together, these findings advance a functional and mechanistic understanding of the AG and cross-brain neural coupling associated with real-time eye-to-eye contact.

Optimization of wavelet coherence analysis as a measure of neural synchrony during hyperscanning using functional near-infrared spectroscopy

Significance: The expanding field of human social interaction is enabled by functional near-infrared spectroscopy (fNIRS) that acquires hemodynamic signals during live two-person interactions. These advances call for development of methods to quantify interactive processes. Aim: Wavelet coherence analysis has been applied to cross-brain neural coupling. However, fNIRS-specific computations have not been explored. This investigation determines the effects of global mean removal, wavelet equation, and choice of oxyhemoglobin versus deoxyhemoglobin signals. Approach: We compare signals with a known coherence with acquired signals to determine optimal computational approaches. The known coherence was calculated using three visual stimulation sequences of a contrast-reversing checkerboard convolved with the canonical hemodynamic response function. This standard was compared with acquired human fNIRS responses within visual cortex using the same sequences. Results: Observed coherence was consistent with known coherence with highest correlations within the wavelength range between 10 and 20 s. Removal of the global mean improved the correlation irrespective of the specific equation for wavelet coherence, and the oxyhemoglobin signal was associated with a marginal correlation advantage. Conclusions: These findings provide both methodological and computational guidance that enhances the validity and interpretability of wavelet coherence analysis for fNIRS signals acquired during live social interactions.

Prefrontal activation related to spontaneous creativity with rock music improvisation: A functional near-infrared spectroscopy study

Understanding how the brain modulates improvisation has been the focus of numerous studies in recent years. Models have suggested regulation of activity between default mode and executive control networks play a role in improvisational execution. Several studies comparing formulaic to improvised sequences support this framework and document increases in activity in medial frontal lobe with decreased activity in the dorsolateral prefrontal cortex (DLPFC). These patterns can be influenced through training and neural responses may differ between in beginner and expert musicians. Our goal was to test the generalizability of this framework and determine similarity in neural activity in the prefrontal cortex during improvisation. Twenty guitarists performed improvised and formulaic sequences in a blues rock format while brain activity was recorded using functional near-infrared spectroscopy. Results indicate similar modulation in DLPFC as seen previously. Specific decreases of activity from left DLPFC in the end compared to beginning or middle of improvised sequences were also found. Despite the range of skills of participants, we also found significant correlation between subjective feelings of improvisational performance and modulation in left DLPFC. Processing of subjective feelings regardless of skill may contribute to neural modulation and may be a factor in understanding neural activity during improvisation.

Co-localization of theta-band activity and hemodynamic responses during face perception: simultaneous electroencephalography and functional near-infrared spectroscopy recordings

Face-specific neural processes in the human brain have been localized to multiple anatomical structures and associated with diverse and dynamic social functions. The question of how various face-related systems and functions may be bound together remains an active area of investigation. We hypothesize that face processing may be associated with specific frequency band oscillations that serve to integrate distributed face processing systems. Using a multimodal imaging approach, including electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS), simultaneous signals were acquired during face and object picture viewing. As expected for face processing, hemodynamic activity in the right occipital face area (OFA) increased during face viewing compared to object viewing, and in a subset of participants, the expected N170 EEG response was observed for faces. Based on recently reported associations between the theta band and visual processing, we hypothesized that increased hemodynamic activity in a face processing area would also be associated with greater theta-band activity originating in the same area. Consistent with our hypothesis, theta-band oscillations were also localized to the right OFA for faces, whereas alpha- and beta-band oscillations were not. Together, these findings suggest that theta-band oscillations originating in the OFA may be part of the distributed face-specific processing mechanism.

Signal Processing in fNIRS: A Case for the Removal of Systemic Activity for Single Trial Data

Researchers using functional near infrared spectroscopy (fNIRS) are increasingly aware of the problem that conventional filtering methods do not eliminate systemic noise at frequencies overlapping with the task frequency. This is a problem when signals are averaged for analysis, even more so when single trial data are used as in online neurofeedback or BCI applications where insufficiently preprocessed data means feeding back noise instead of brain activity or when looking for brain-behavior relationships on a trial-by-trial basis. For removing this task-related noise statistical approaches have been proposed. Yet as evidence is lacking on how these approaches perform on independent data, choosing one approach over another can be difficult. Here signal quality at the single trial level was considered together with statistical effects to inform this choice. Compared were conventional band-pass filtering and wavelet minimum description length detrending and the combination of both with a more elaborate, published preprocessing approach for a motor execution—motor imagery data set. Temporal consistency between Δ[HbO] and Δ[HbR] and two measures of the spatial specificity of signals that are proposed here served as measures of data quality. Both improved strongly for the combinationed preprocessing approaches. Statistical effects showed a strong tendency toward getting smaller for the combined approaches. This underlines the importance to adequately deal with noise in fNIRS recordings and demonstrates how the quality of statistical correction approaches can be estimated.

Communication of emotion via drumming: dual-brain imaging with functional near-infrared spectroscopy

Nonverbal communication of emotion is essential to human interaction and relevant to many clinical applications, yet it is an understudied topic in social neuroscience. Drumming is an ancient nonverbal communication modality for expression of emotion that has not been previously investigated in this context. We investigate the neural response to live, natural communication of emotion via drumming using a novel dual-brain neuroimaging paradigm. Hemodynamic signals were acquired using whole-head functional near-infrared spectroscopy (fNIRS). Dyads of 36 subjects participated in two conditions, drumming and talking, alternating between 'sending' (drumming or talking to partner) and 'receiving' (listening to partner) in response to emotionally salient images from the International Affective Picture System. Increased frequency and amplitude of drum strikes was behaviorally correlated with higher arousal and lower valence measures and neurally correlated with temporoparietal junction (TPJ) activation in the listener. Contrast comparisons of drumming greater than talking also revealed neural activity in right TPJ. Together, findings suggest that emotional content communicated by drumming engages right TPJ mechanisms in an emotionally and behaviorally sensitive fashion. Drumming may provide novel, effective clinical approaches for treating social-emotional psychopathology.

A cross-brain neural mechanism for human-to-human verbal communication

Neural mechanisms that mediate dynamic social interactions remain understudied despite their evolutionary significance. The interactive brain hypothesis proposes that interactive social cues are processed by dedicated brain substrates and provides a general theoretical framework for investigating the underlying neural mechanisms of social interaction. We test the specific case of this hypothesis proposing that canonical language areas are upregulated and dynamically coupled across brains during social interactions based on talking and listening. Functional near-infrared spectroscopy (fNIRS) was employed to acquire simultaneous deoxyhemoglobin (deOxyHb) signals of the brain on partners who alternated between speaking and listening while doing an Object Naming & Description task with and without interaction in a natural setting. Comparison of interactive and non-interactive conditions confirmed an increase in neural activity associated with Wernicke's area including the superior temporal gyrus (STG) during interaction (P = 0.04). However, the hypothesis was not supported for Broca's area. Cross-brain coherence determined by wavelet analyses of signals originating from the STG and the subcentral area was greater during interaction than non-interaction (P < 0.01). In support of the interactive brain hypothesis these findings suggest a dynamically coupled cross-brain neural mechanism dedicated to pathways that share interpersonal information.

Evoked Hemodynamic Response Estimation to Auditory Stimulus Using Recursive Least Squares Adaptive Filtering with Multidistance Measurement of Near-Infrared Spectroscopy

The performance of functional near-infrared spectroscopy (fNIRS) is sometimes degraded by the interference caused by the physical or the systemic physiological activities. Several interferences presented during fNIRS recordings are mainly induced by cardiac pulse, breathing, and spontaneous physiological low-frequency oscillations. In previous work, we introduced a multidistance measurement to reduce physiological interference based on recursive least squares (RLS) adaptive filtering. Monte Carlo simulations have been implemented to evaluate the performance of RLS adaptive filtering. However, its suitability and performance on human data still remain to be evaluated. Here, we address the issue of how to detect evoked hemodynamic response to auditory stimulus using RLS adaptive filtering method. A multidistance probe based on continuous wave fNIRS is devised to achieve the fNIRS measurement and further study the brain functional activation. This study verifies our previous findings that RLS adaptive filtering is an effective method to suppress global interference and also provides a practical way for real-time detecting brain activity based on multidistance measurement.