Lessons from the first two years of operating a study registry

Why Most Research Findings About Psi Are False: The Replicability Crisis, the Psi Paradox and the Myth of Sisyphus

The replicability crisis in psychology has been influenced by the results of nine experiments conducted by Bem (2011) and presented as supporting the existence of precognition. In this paper, we hope to show how the debate concerning these experiments could be an opportunity to develop original thinking about psychology and replicability. After a few preliminary remarks about psi and scientific epistemology, we examine how psi results lead to a paradox which questions how appropriate the scientific method is to psi research. This paradox highlights a problem in the way experiments are conducted in psi research and its potential consequence on mainstream research in psychology. Two classical experiments - the Ganzfeld protocol and the Bem studies - are then analyzed in order to illustrate this paradox and its consequences. Mainstream research is also addressed in the broader context of the replication crisis, decline effect and questionable research practices. Several perspectives for future research are proposed in conclusion and underline the heuristic value of psi studies for psychology.

Bias in the evaluation of psychology studies: A comparison of parapsychology versus neuroscience

Research suggests that scientists display confirmation biases with regard to the evaluation of research studies, in that they evaluate results as being stronger when a study confirms their prior expectations. These biases may influence the peer review process, particularly for studies that present controversial findings. The purpose of the current study was to compare the evaluation of a parapsychology study versus a neuroscience study. One hundred participants with a background in psychology were randomly assigned to read and evaluate one of two virtually identical study abstracts (50 participants per group). One of the abstracts described the findings as if they were from a parapsychology study, whereas the other abstract described the findings as if they were from a neuroscience study. The results revealed that participants rated the neuroscience abstract as having stronger findings and as being more valid and reliable than the parapsychology abstract, despite the fact that the two abstracts were identical. Participants also displayed confirmation bias in their ratings of the parapsychology abstract, in that their ratings were correlated with their scores on transcendentalism (a measure of beliefs and experiences related to parapsychology, consciousness and reality). Specifically, higher transcendentalism was associated with more favorable ratings of the parapsychology abstract, whereas lower transcendentalism was associated with less favorable ratings. The findings suggest that individuals with a background in psychology need to be vigilant about potential biases that could impact their evaluations of parapsychology research during the peer review process.

Registered reports: an early example and analysis

The recent 'replication crisis' in psychology has focused attention on ways of increasing methodological rigor within the behavioral sciences. Part of this work has involved promoting 'Registered Reports', wherein journals peer review papers prior to data collection and publication. Although this approach is usually seen as a relatively recent development, we note that a prototype of this publishing model was initiated in the mid-1970s by parapsychologist Martin Johnson in the European Journal of Parapsychology (EJP). A retrospective and observational comparison of Registered and non-Registered Reports published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices. This paper aims both to bring Johnson's pioneering work to a wider audience, and to investigate the positive role that Registered Reports may play in helping to promote higher methodological and statistical standards.