The Function of Literature in Psychological Science

The recent reform debates in psychological science, prompted by a widespread crisis of confidence, have exposed and destabilized the so-called myth of self-correction, that is, the problem that most scientists perceive their disciplines as self-correcting without engaging in actual practices that correct the scientific record. In this paper, building on the idea of self-correction as a myth, I propose another myth common to psychological science: the myth of self-organization. The myth of self-organization is the idea that scientific literature will organize itself into something the community adding to it would recognize as systematic knowledge; while the actual members of those communities do not engage in effective ways of organizing it. I argue for the existence of the myth self-organization by taking a historical look at how the scientific literature was construed by psychologists during the 20th century. In my view, the literature, and behaviors of scientists related to it, becomes a social institution exerting influence over the science it belongs to. I conclude with a critical discussion of self-organization through the debates about preregistration and theory formalization in psychology’s reform movement.

Where are the Self-Correcting Mechanisms in Science?

It is often said that science is self-correcting, but the replication crisis suggests that self-correction mechanisms have fallen short. How can we know whether a particular scientific field has effective self-correction mechanisms, that is, whether its findings are credible? The usual processes that supposedly provide mechanisms for scientific self-correction, such as journal-based peer review and institutional committees, have been inadequate. We describe more verifiable indicators of a field’s commitment to self-correction. These fall under the broad headings of 1) transparency, which is already the subject of many reform efforts and 2) critical appraisal, which has received less attention and which we focus on here. Only by obtaining Observable Self-Correction Indicators (OSCIs) can we begin to evaluate the claim that “science is self-correcting.” We expect that the veracity of this claim varies across fields and subfields, and suggest that some fields, such as psychology and biomedicine, fall far short of an appropriate level of transparency and, especially, critical appraisal. Fields without robust, verifiable mechanisms for transparency and critical appraisal cannot reasonably be said to be self-correcting, and thus do not warrant the credibility often imputed to science as a whole.