The author–editor game

What is the sensitivity and specificity of the peer review process?

In this paper, we introduce the concepts of sensitivity and specificity to mathematically describe the accuracy of the peer review process. Sensitivity refers to the probability that the final decision for a manuscript would be acceptance, provided the manuscript meets the journal standards required for publication (i.e., true positive rate). Specificity refers to the probability that the final decision would be rejection, provided the work does not meet the standards required for publication (i.e., true negative rate). Therefore, in the peer review process, sensitivity measures the ability to correctly accept manuscripts that meet the required standards (true positives) and specificity measures the ability to correctly reject manuscripts that do not meet those quality standards required for publication (true negatives). Sensitivity and specificity values can inform the editor under what conditions the outcome of a peer review process becomes more precise and, therefore, if this does not occur, when the editor must improve the analysis involved in processing the information received from reviewers' reports. Sensitivity and specificity understood in this way can promote the ethical conduct of peer review processes and improve the validity of manuscript editorial decisions.

Challenges in peer review: how to guarantee the quality and transparency of the editorial process in scientific journals

The editorial process of scientific journals is complex but essential for the dissemination of scientific knowledge. The quality of the process depends on the authors, editors and reviewers, who must have the necessary experience and knowledge to ensure the quality of the published articles. One of the most significant challenges scientific journals face today is the peer review of manuscripts. Editors are responsible for coordinating and overseeing the entire editorial process, from manuscript submission to final publication, and ensuring that articles meet ethical and scientific integrity standards. Editors are also in charge of selecting appropriate reviewers. However, the latter is becoming difficult due to the increasing refusal of expert reviewers to participate in the editorial process. The reasons for it are diverse, but the lack of recognition for review work and reviewer fatigue in the most sought-after reviewers are among the most important. Some of the measures that could be taken to alleviate the problem concern the possibility of professionalizing peer review.

Fraud, specialization, and efficiency in peer review

Reviewers are humans and might be affected by cognitive biases when information overload comes into play. In fact, no amount of scientific training will completely mask the human impulses to partisanship. And the consequence is that authors may receive incorrect editorial decisions in their submissions to peer-reviewed journals. For instance, the journal editor issues a substantial revision when in fact a moderate one would suffice. This would be over-revision in peer review. In this situation, there exists a fraud cost if the journal editor tries to request the author to make a substantial revision when in fact a moderate one would be sufficient. Thus, in this article, we identify a set of conditions under which the peer review process involves equilibrium fraud and over-revision. An equilibrium in peer review is efficient if the first peer-reviewed journal to which the author submits their research paper makes a truthful editorial decision, which the author accepts. When the fraud cost is sufficiently high, there exists an efficient equilibrium. Otherwise, when the fraud cost cannot sustain an efficient equilibrium, it may arise a specialization equilibrium in which the author first submits the manuscript to a top journal which makes a truthful editorial decision. This specialization equilibrium may explain why academic journals with higher quality standards more often attract authors who write articles of higher quality. Finally, when the fraud cost is not too large, we show that a new type of equilibrium emerges in our model, equilibria involving costly fraud, in which the first peer-reviewed journal to which the research paper is submitted always requests substantial revisions. If the review time and the probability of very serious concerns from reviewers were large, the author would prefer to send the research paper to one single peer-reviewed journal even if that would involve over-revision. In the fraud equilibrium, the author’s revision cost is high and independent of the true quality of the manuscript.

Intermediacy of publications

Citation networks of scientific publications offer fundamental insights into the structure and development of scientific knowledge. We propose a new measure, called intermediacy, for tracing the historical development of scientific knowledge. Given two publications, an older and a more recent one, intermediacy identifies publications that seem to play a major role in the historical development from the older to the more recent publication. The identified publications are important in connecting the older and the more recent publication in the citation network. After providing a formal definition of intermediacy, we study its mathematical properties. We then present two empirical case studies, one tracing historical developments at the interface between the community detection literature and the scientometric literature and one examining the development of the literature on peer review. We show both conceptually and empirically how intermediacy differs from main path analysis, which is the most popular approach for tracing historical developments in citation networks. Main path analysis tends to favour longer paths over shorter ones, whereas intermediacy has the opposite tendency. Compared to the main path analysis, we conclude that intermediacy offers a more principled approach for tracing the historical development of scientific knowledge.

The Game Between a Biased Reviewer and His Editor

This paper shows that, for a large range of parameters, the journal editor prefers to delegate the choice to review the manuscript to the biased referee. If the peer review process is informative and the review reports are costly for the reviewers, even biased referees with extreme scientific preferences may choose to become informed about the manuscript's quality. On the contrary, if the review process is potentially informative but the reviewer reports are not costly for the referees, the biased reviewer has no incentive to become informed about the manuscript. Furthermore, if the reports are costly for referees but the peer review processes are not potentially informative, the biased reviewers will never become informed. In this paper, we also present a web resource that helps editors to experiment with the review process as a device for information transmission.

The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures

This paper looks at peer review as a cooperation dilemma through a game-theory framework. We built an agent-based model to estimate how much the quality of peer review is influenced by different resource allocation strategies followed by scientists dealing with multiple tasks, i.e., publishing and reviewing. We assumed that scientists were sensitive to acceptance or rejection of their manuscripts and the fairness of peer review to which they were exposed before reviewing. We also assumed that they could be realistic or excessively over-confident about the quality of their manuscripts when reviewing. Furthermore, we assumed they could be sensitive to competitive pressures provided by the institutional context in which they were embedded. Results showed that the bias and quality of publications greatly depend on reviewer motivations but also that context pressures can have a negative effect. However, while excessive competition can be detrimental to minimising publication bias, a certain level of competition is instrumental to ensure the high quality of publication especially when scientists accept reviewing for reciprocity motives.

Quantifying the effect of editor-author relations on manuscript handling times

In this article we study to what extent the academic peer review process is influenced by social relations between the authors of a manuscript and the editor handling the manuscript. Taking the open access journal PlosOne as a case study, our analysis is based on a data set of more than 100,000 articles published between 2007 and 2015. Using available data on handling editor, submission and acceptance time of manuscripts, we study the question whether co-authorship relations between authors and the handling editor affect the manuscript handling time, i.e. the time taken between the submission and acceptance of a manuscript. Our analysis reveals (1) that editors handle papers co-authored by previous collaborators significantly more often than expected at random, and (2) that such prior co-author relations are significantly related to faster manuscript handling. Addressing the question whether these shorter manuscript handling times can be explained by the quality of publications, we study the number of citations and downloads which accepted papers eventually accumulate. Moreover, we consider the influence of additional (social) factors, such as the editor’s experience, the topical similarity between authors and editors, as well as reciprocal citation relations between authors and editors. Our findings show that, even when correcting for other factors like time, experience, and performance, prior co-authorship relations have a large and significant influence on manuscript handling times, speeding up the editorial decision on average by 19 days.

Preserving the Integrity of Citations and References by All Stakeholders of Science Communication

Citations to scholarly items are building bricks for multidisciplinary science communication. Citation analyses are currently influencing individual career advancement and ranking of academic and research institutions worldwide. This article overviews the involvement of scientific authors, reviewers, editors, publishers, indexers, and learned associations in the citing and referencing to preserve the integrity of science communication. Authors are responsible for thorough bibliographic searches to select relevant references for their articles, comprehend main points, and cite them in an ethical way. Reviewers and editors may perform additional searches and recommend missing essential references. Publishers, in turn, are in a position to instruct their authors over the citations and references, provide tools for validation of references, and open access to bibliographies. Publicly available reference lists bear important information about the novelty and relatedness of the scholarly items with the published literature. Few editorial associations have dealt with the issue of citations and properly managed references. As a prime example, the International Committee of Medical Journal Editors (ICMJE) issued in December 2014 an updated set of recommendations on the need for citing primary literature and avoiding unethical references, which are applicable to the global scientific community. With the exponential growth of literature and related references, it is critically important to define functions of all stakeholders of science communication in curbing the issue of irrational and unethical citations and thereby improve the quality and indexability of scholarly journals.