Improving the rigor of psychophysiology research

Unconditioned response to a naturally aversive stimulus is associated with sensitized defensive responding and self-reported fearful traits in a PTSD sample

Unconditioned responding (UCR) to a naturally aversive stimulus is associated with defensive responding to a conditioned threat cue (CS+) and a conditioned safety cue (CS-) in trauma-exposed individuals during fear acquisition. However, the relationships of UCR with defensive responses during extinction training, posttraumatic stress disorder (PTSD) symptom severity, and fearful traits in trauma-exposed individuals are not known. In a sample of 100 trauma-exposed adults with a continuum of PTSD severity, we recorded startle responses and skin conductance responses (SCR) during fear acquisition and extinction training using a 140 psi, 250-ms air blast to the larynx as the unconditioned stimulus. We explored dimensional associations of two different measures of UCR (unconditioned startle and unconditioned SCR) with conditioned defensive responding to CS+ and CS-, conditioned fear (CS+ minus CS-), PTSD symptom severity, and a measure of fearful traits (composite of fear survey schedule, anxiety sensitivity index, and Connor-Davidson resilience scale). Unconditioned startle was positively associated with startle potentiation to the threat cue and the safety cue across both learning phases (CS+ Acquisition, CS- Acquisition, CS+ Extinction Training, CS- Extinction Training) and with fearful traits. Unconditioned SCR was positively associated with SCR to the CS+ and CS- and SCR difference score during Acquisition. Neither type of UCR was associated with PTSD symptom severity. Our findings suggest that UCR, particularly unconditioned startle to a naturally aversive stimulus, may inform research on biomarkers and treatment targets for symptoms of pervasive and persistent fear in trauma-exposed individuals.

Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of a study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.

Increasing Value and Reducing Waste of Research on Neurofeedback Effects in Post-traumatic Stress Disorder: A State-of-the-Art-Review

Post-Traumatic Stress Disorder (PTSD) is often considered challenging to treat due to factors that contribute to its complexity. In the last decade, more attention has been paid to non-pharmacological or non-psychological therapies for PTSD, including neurofeedback (NFB). NFB is a promising non-invasive technique targeting specific brainwave patterns associated with psychiatric symptomatology. By learning to regulate brain activity in a closed-loop paradigm, individuals can improve their functionality while reducing symptom severity. However, owing to its lax regulation and heterogeneous legal status across different countries, the degree to which it has scientific support as a psychiatric treatment remains controversial. In this state-of-the-art review, we searched PubMed, Cochrane Central, Web of Science, Scopus, and MEDLINE and identified meta-analyses and systematic reviews exploring the efficacy of NFB for PTSD. We included seven systematic reviews, out of which three included meta-analyses (32 studies and 669 participants) that targeted NFB as an intervention while addressing a single condition-PTSD. We used the MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 and the criteria described by Cristea and Naudet (Behav Res Therapy 123:103479, 2019, https://doi.org/10.1016/j.brat.2019.103479 ) to identify sources of research waste and increasing value in biomedical research. The seven assessed reviews had an overall extremely poor quality score (5 critically low, one low, one moderate, and none high) and multiple sources of waste while opening opportunities for increasing value in the NFB literature. Our research shows that it remains unclear whether NFB training is significantly beneficial in treating PTSD. The quality of the investigated literature is low and maintains a persistent uncertainty over numerous points, which are highly important for deciding whether an intervention has clinical efficacy. Just as importantly, none of the reviews we appraised explored the statistical power, referred to open data of the included studies, or adjusted their pooled effect sizes for publication bias and risk of bias. Based on the obtained results, we identified some recurrent sources of waste (such as a lack of research decisions based on sound questions or using an appropriate methodology in a fully transparent, unbiased, and useable manner) and proposed some directions for increasing value (homogeneity and consensus) in designing and reporting research on NFB interventions in PTSD.

Past, Present, and Future of Human Chemical Communication Research

Although chemical signaling is an essential mode of communication in most vertebrates, it has long been viewed as having negligible effects in humans. However, a growing body of evidence shows that the sense of smell affects human behavior in social contexts ranging from affiliation and parenting to disease avoidance and social threat. This article aims to (a) introduce research on human chemical communication in the historical context of the behavioral sciences; (b) provide a balanced overview of recent advances that describe individual differences in the emission of semiochemicals and the neural mechanisms underpinning their perception, that together demonstrate communicative function; and (c) propose directions for future research toward unraveling the molecular principles involved and understanding the variability in the generation, transmission, and reception of chemical signals in increasingly ecologically valid conditions. Achieving these goals will enable us to address some important societal challenges but are within reach only with the aid of genuinely interdisciplinary approaches.

Predicting Fear Extinction in Posttraumatic Stress Disorder

Fear extinction is the basis of exposure therapies for posttraumatic stress disorder (PTSD), but half of patients do not improve. Predicting fear extinction in individuals with PTSD may inform personalized exposure therapy development. The participants were 125 trauma-exposed adults (96 female) with a range of PTSD symptoms. Electromyography, electrocardiogram, and skin conductance were recorded at baseline, during dark-enhanced startle, and during fear conditioning and extinction. Using a cross-validated, hold-out sample prediction approach, three penalized regressions and conventional ordinary least squares were trained to predict fear-potentiated startle during extinction using 50 predictor variables (5 clinical, 24 self-reported, and 21 physiological). The predictors, selected by penalized regression algorithms, were included in multivariable regression analyses, while univariate regressions assessed individual predictors. All the penalized regressions outperformed OLS in prediction accuracy and generalizability, as indexed by the lower mean squared error in the training and holdout subsamples. During early extinction, the consistent predictors across all the modeling approaches included dark-enhanced startle, the depersonalization and derealization subscale of the dissociative experiences scale, and the PTSD hyperarousal symptom score. These findings offer novel insights into the modeling approaches and patient characteristics that may reliably predict fear extinction in PTSD. Penalized regression shows promise for identifying symptom-related variables to enhance the predictive modeling accuracy in clinical research.

Facilitated extinction but impaired extinction recall by eye movement manipulation in humans - Indications for action mechanisms and the applicability of eye movement desensitization

Eye movement desensitization and reprocessing (EMDR) therapy utilizes the manipulation of eye movements to reduce affective distress during fear-exposure. Animal research recently suggested a potential neural mechanism underlying these effects, by which increased activity of the superior colliculus (SC), mediating visual attention, increases the inhibition of the basolateral amygdala (BLA), mediating defensive plasticity. We tested such mechanism in forty healthy humans using a multiple-day single-cue fear conditioning and extinction paradigm. The activity of the SC during extinction was experimentally manipulated by eye movements, as half of the participants executed saccadic eye movements (n = 20; major SC involvement), while the other half executed smooth eye pursuits (n = 20; minor SC involvement). Amygdala-mediated fear-potentiated startle responses and fear bradycardia, as well as threat expectancy was analyzed. Saccadic eye movements facilitated the extinction of fear bradycardia and fear-potentiated startle responses. Higher saccadic accuracy and range correlated with reduced fear-potentiated startle. However, during extinction recall, fear-potentiated startle and fear bradycardia resurged and partly reached levels obtained after fear acquisition. Threat expectancy was not affected by different eye movements and was not elevated during extinction recall. Within limitations, results support an inhibitory SC-BLA pathway in humans by which eye movements may reduce low-level defensive responding, but not threat expectancy. Yet, manipulating eye movements during extinction learning seems to impair extinction recall for behavioral and physiological defensive response indices. Thus, increasing SC activity might enhance initial efficacy of exposure treatment, but additional strategies seem necessary for sustained fear attenuation.

Effects of selective attention on the C1 ERP component: A systematic review and meta-analysis

The C1 event-related potential (ERP) captures the earliest stage of feedforward processing in the primary visual cortex (V1). An ongoing debate is whether top-down selective attention can modulate the C1. One side of the debate pointed out that null findings appear to outnumber positive findings; thus, selective attention does not seem to influence the C1. However, this suggestion is not based on a valid approach to summarizing evidence across studies. Therefore, we conducted a systematic review and meta-analysis investigating the effects of selective attention on the C1, involving 47 experiments and 794 subjects in total. Despite heterogeneity across studies, results suggested that attention has a moderate effect on the C1 (Cohen's d z $$ {d}_z $$  = 0.33, p < .0001); that is, C1 amplitude is larger for visual stimuli that are attended than unattended. These results suggest that C1 is affected by top-down selective attention.

Internal consistency and test-retest reliability of the P3 event-related potential (ERP) elicited by alcoholic and non-alcoholic beverage pictures

Addiction researchers are interested in the ability of neural signals, like the P3 component of the ERP, to index individual differences in liability factors like motivational reactivity to alcohol/drug cues. The reliability of these measures directly impacts their ability to index individual differences, yet little attention has been paid to their psychometric properties. The present study fills this gap by examining within-session internal consistency reliability (ICR) and between-session test-retest reliability (TRR) of the P3 amplitude elicited by images of alcoholic beverages (Alcohol Cue P3) and non-alcoholic drinks (NADrink Cue P3) as well as the difference between them, which isolates alcohol cue-specific reactivity in the P3 (ACR-P3). Analyses drew on data from a large sample of alcohol-experienced emerging adults (session 1 N = 211, 55% female, aged 18-20 yr; session 2 N = 98, 66% female, aged 19-21 yr). Evaluated against domain-general thresholds, ICR was excellent (M ± SD; r= 0.902 ± 0.030) and TRR was fair (r = 0.706 ± 0.020) for Alcohol Cue P3 and NADrink Cue P3, whereas for ACR-P3, ICR and TRR were poor (r = 0.370 ± 0.071; r = 0.201 ± 0.042). These findings indicate that individual differences in the P3 elicited by cues for ingested liquid rewards are highly reliable and substantially stable over 8-10 months. Individual differences in alcohol cue-specific P3 reactivity were less reliable and less stable. The conditions under which alcohol/drug cue-specific reactivity in neural signals is adequately reliable and stable remain to be discovered.

EPOS: EEG Processing Open-Source Scripts

Background

Since the replication crisis, standardization has become even more important in psychological science and neuroscience. As a result, many methods are being reconsidered, and researchers’ degrees of freedom in these methods are being discussed as a potential source of inconsistencies across studies.

New Method

With the aim of addressing these subjectivity issues, we have been working on a tutorial-like EEG (pre-)processing pipeline to achieve an automated method based on the semi-automated analysis proposed by Delorme and Makeig.

Results

Two scripts are presented and explained step-by-step to perform basic, informed ERP and frequency-domain analyses, including data export to statistical programs and visual representations of the data. The open-source software EEGlab in MATLAB is used as the data handling platform, but scripts based on code provided by Mike Cohen (2014) are also included.

Comparison with existing methods

This accompanying tutorial-like article explains and shows how the processing of our automated pipeline affects the data and addresses, especially beginners in EEG-analysis, as other (pre)-processing chains are mostly targeting rather informed users in specialized areas or only parts of a complete procedure. In this context, we compared our pipeline with a selection of existing approaches.

Conclusion

The need for standardization and replication is evident, yet it is equally important to control the plausibility of the suggested solution by data exploration. Here, we provide the community with a tool to enhance the understanding and capability of EEG-analysis. We aim to contribute to comprehensive and reliable analyses for neuro-scientific research.

Brain event-related potentials predict individual differences in inhibitory control

Stop-signal reaction time (SSRT), the time needed to cancel an already-initiated motor response, quantifies individual differences in inhibitory control. Electrophysiological correlates of SSRT have primarily focused on late event-related potential (ERP) components over midline scalp regions from successfully inhibited stop trials. SSRT is robustly associated with the P300, there is mixed evidence for N200 involvement, and there is little information on the role of early ERP components. Here, machine learning was first used to interrogate ERPs during both successful and failed stop trials from 64 scalp electrodes at 4 ms resolution (n = 148). The most predictive model included data from both successful and failed stop trials, with a cross-validated Pearson's r of 0.32 between measured and predicted SSRT, significantly higher than null models. From successful stop trials, spatio-temporal features overlapping the N200 in right frontal areas and the P300 in frontocentral areas predicted SSRT, as did early ERP activity (<200 ms). As a demonstration of the reproducibility of these findings, the application of this model to a separate dataset of 97 participants was also significant (r = 0.29). These results show that ERPs during failed stops are relevant to SSRT, and that both early and late ERP activity contribute to individual differences in SSRT. Notably, the right lateralized N200, which predicted SSRT here, is not often observed in neurotypical adults. Both the ascending slope and peak of the P300 component predicted SSRT. These results were replicable, both within the training sample and when applied to ERPs from a separate dataset.

Making ERP research more transparent: Guidelines for preregistration

A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.

A systematic review of physiological responses to odours with a focus on current methods used in event-related study designs

OBJECTIVES

In odour research, there is still a lack of knowledge regarding the detailed understanding of the determinants and the magnitude of an odour's impact on human psychophysiology. Therefore, the present review aims to summarize current evidence on psychophysiological responses to olfactory events, to highlight diversity in research methods, and to provide recommendations for further research.

MATERIAL AND METHODS

Predefined search items were used for literature research in two databases, focussing on recent investigations of cardiac and electrodermal responses to short (<10 s) olfactory stimulations, combined with self-reports on odour experience, in a healthy population. The selected 27 publications were evaluated with regard to their methods and their findings on psychophysiological correlates of odour stimulation, following a conceptual scheme proposing mediating and moderating factors of physiological responses to odour stimuli.

RESULTS

The cardiac and electrodermal activity generally followed a discriminative pattern depending on the perceived pleasantness of an odour. Moreover, the trigeminal aspect of an odour stimulus became evident in electrodermal activity in several studies. Finally, for many of the here addressed potentially mediating and moderating variables, initial findings were obtained in some studies but these await corroboration by future research. With regard to the applied methodology, the reviewed studies were highly diverse, in terms of odour application, study design, and analysis of the time series data.

CONCLUSIONS

Future research is needed to advance our understanding of, and theoretical concepts beyond, psychophysiological responses to olfactory events, and to achieve experimentally validated methodological guidelines for psychophysiological measurements in olfaction research.

An Introduction and Illustration of Bayesian Modeling in Couple and Family Therapy Research

Bayesian modeling is becoming increasing popular as a method for data analyses in the social sciences and can move couple, marriage, and family therapy (C/MFT) research forward. Bayesian modeling helps researchers better understand the uncertainty of findings and incorporate previous research into analyses. Other benefits of Bayesian modeling are the straightforward interpretation of findings, high-quality inferences even with small samples (in combination with an informative prior), and the ability to work with complex data structures (observations nested in relationships and time points) which are common in C/MFT research. This article introduces the benefits of Bayesian modeling and provides an example of an Actor-Partner Interdependence Model using R. Information on how to conduct the same analyses using Stata and MPlus is provided in the Supplemental Information.

Methodological choices in event-related potential (ERP) research and their impact on internal consistency reliability and individual differences: An examination of the error-related negativity (ERN) and anxiety

Researchers in clinical psychophysiology make several methodological decisions during the analysis of event-related potentials (ERPs). In the current study, we review these choices from the perspective of individual differences. We focus on baseline period and reference scheme (i.e., average, mastoid, current source density), as well as choices regarding where (i.e., single electrode site vs. pooling of sites), when (i.e., area, area around peak), and how (i.e., subtraction- or regression-based difference scores) to quantify ERPs. To illustrate the impact of these analytic pathways on internal consistency reliability and individual differences, we focus on the error-related negativity (ERN) and anxiety-and present data from 2 samples: 1st, in adults with diagnosed generalized anxiety disorder (GAD); 2nd, in relation to continuous self-reported symptoms of GAD in a large community sample of female adolescents. Results generally indicated similar internal consistency and between-subjects effect sizes across all evaluated methods. Nonetheless, some patterns of variation emerged, such as that, across both data sets, difference-based ERN measures, especially with mastoid reference, yielded more robust associations with GAD diagnosis and symptoms, despite somewhat lower internal consistency. The current analyses suggest that the association between ERN and anxiety is robust across a range of commonly used methodological choices. The present study is an example of how systematic analyses of analytic strategies on measures of internal consistency and between-subjects variability could help inform individual-differences ERP research. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Improving the Rigor and Replicability of Applied Psychophysiology Research: Sample Size, Standardization, Transparency, and Preregistration

Scientific research across a number of areas, including applied psychophysiology, biofeedback, and neurofeedback, is facing considerable scrutiny for poor replication rates, high numbers of false-positive findings, and insufficient scientific rigor. There are many factors underlying this replication crisis in scientific research; yet incentives for more rigorous research practices at the institutional and editorial levels lag behind the need for improvement. The author provides examples of replication and rigor difficulties in scientific research with an eye toward psychophysiological research, including researcher flexibility in data analysis, “p-hacking,” insufficient sample sizes, and lack of availability and implementation of rigorous methodological and publication guidelines. Subsequently, the author highlights examples and opportunities for improvement, including decreasing researcher flexibility, reporting sample size information, increasing sample sizes through collaboration, improving reporting standards/following established guidelines for reporting psychophysiological data, and increasing adoption of preregistration and registered reports. The author concludes that the Association for Applied Psychophysiology and Biofeedback (AAPB) can improve clinical practice and perception of public and scientific credibility by implementing rigorous and transparent research practices with a focus on replicability and clear methodological and reporting techniques and standards.

Are Difficult-To-Study Populations too Difficult to Study in a Reliable Way?

Abstract. Replication studies, pre-registration, and increases in statistical power will likely improve the reliability of scientific evidence. However, these measures face critical limitations in populations that are inherently difficult to study. Members of difficult-to-study populations (e.g., patients, children, non-human animals) are less accessible to researchers, which typically results in small-sample studies that are infeasible to replicate. Nevertheless, meta-analyses on clinical neuropsychological data suggest that difficult-to-study populations can be studied in a reliable way. These analyses often produce unbiased effect-size estimates despite aggregating across severely underpowered original studies. This finding can be attributed to a neuropsychological research culture involving the non-selective reporting of results from standardized and validated test procedures. Consensus guidelines, test manuals, and psychometric evidence constrain the methodological choices made by neuropsychologists, who regularly report the results from neuropsychological test batteries irrespective of their statistical significance or novelty. Comparable shifts toward more standardization and validation, complete result reports, and between-lab collaborations can allow for a meaningful and reliable study of psychological phenomena in other difficult-to-study populations.

Fear Extinction Retention: Is It What We Think It Is?

There has been an explosion of research on fear extinction in humans in the past 2 decades. This has not only generated major insights, but also brought a new goal into focus: how to maintain extinction memory over time (i.e., extinction retention). We argue that there are still important conceptual and procedural challenges in human fear extinction research that hamper advancement in the field. We use extinction retention and the extinction retention index to exemplarily illustrate these challenges. Our systematic literature search identified 16 different operationalizations of the extinction retention index. Correlation coefficients among these different operationalizations as well as among measures of fear/anxiety show a wide range of variability in four independent datasets, with similar findings across datasets. Our results suggest that there is an urgent need for standardization in the field. We discuss the conceptual and empirical implications of these results and provide specific recommendations for future work.

Quantifying evidential value and selective reporting in recent and 10-year past psychophysiological literature: A pre-registered P-curve analysis

Selective reporting (i.e., only reporting significant findings as opposed to all analyses or results) is a questionable research practice that undermines the integrity of published research. Psychophysiology research may be susceptible to selective reporting, given the high number of decision points and methodological complexity in analyses of psychophysiology data. We aimed to assess the presence of selective reporting and evidential value (i.e., that significant results are due to true underlying effects) in recent and past psychophysiological research by utilizing p-curve analyses. Study protocols and methods were pre-registered on the Open Science Framework (OSF). P-values and the associated test statistics were extracted from articles in the most recent issue (as of January 2018) and 10-year previous counterpart issue of three major psychophysiology journals: Psychophysiology, International Journal of Psychophysiology, and Journal of Psychophysiology. Using the p-curve application, 10 primary p-curves were conducted: all recent articles, all past articles, recent articles split by journal, past articles split by journal, recent cognitive electrophysiology articles, and past cognitive electrophysiology articles. Evidential value and generally adequate average power (≥78% average power) were present in all p-curves, except those that only included articles from the Journal of Psychophysiology because of the small number of articles published in the journal. Findings provide some positive news and indicate that, generally, results were not selectively reported, and selective reporting may not be a primary issue for this sample of psychophysiological research. Future p-curve analyses examining sub-disciplines of psychophysiology are recommended.

Computational reproducibility of "Goal relevance and goal conduciveness appraisals lead to differential autonomic reactivity in emotional responding to performance feedback" (Kreibig, Gendolla, & Scherer, 2012): A guide and new evidence

The emerging field of the psychophysiology of motivation bears many new findings, but little replication. Using my own data (Kreibig, Gendolla, & Scherer, 2012), I test the reproducibility of this specific study, provide the necessary materials to make the study reproducible, and instantiate proper reproducibility practices that other researchers can use as a road map toward the same goal. In addition, based on re-analyses of the original data, I report new evidence for the motivational effects of emotional responding to performance feedback. Specifically, greater appraisal of goal relevance amplifies the emotional response to events appraised as conducive (i.e., effort mobilization), but not to those appraised as obstructive to a person's goals (i.e., effort withdrawal). I conclude by providing a ten-step road map of best practices to facilitate computational reproducibility for future studies.