Traditional peer review and post-publication peer review

The relationship of polarity of post-publication peer review to citation count

Purpose

The purpose of this study is to investigate the relationship of the post-publication peer review (PPPR) polarity of a paper to that paper's citation count.

Design/methodology/approach

Papers with PPPRs from Publons.com as the experimental groups were manually matched 1:2 with the related papers without PPPR as the control group, by the same journal, the same issue (volume), the same access status (gold open access or not) and the same document type. None of the papers in the experimental group or control group received any comments or recommendations from ResearchGate, PubPeer or F1000. The polarity of the PPPRs was coded by using content analysis. A negative binomial regression analysis was conducted to examine the data by controlling the characteristics of papers.

Findings

The four experimental groups and their corresponding control groups were generated as follows: papers with neutral PPPRs, papers with both negative and positive PPPRs, papers with negative PPPRs and papers with positive PPPRs as well as four corresponding control groups (papers without PPPRs). The results are as follows: while holding the other variables (such as page count, number of authors, etc.) constant in the model, papers that received neutral PPPRs, those that received negative PPPRs and those that received both negative and positive PPPRs had no significant differences in citation count when compared to their corresponding control pairs (papers without PPPRs). Papers that received positive PPPRs had significantly greater citation count than their corresponding control pairs (papers without PPPRs) while holding the other variables (such as page count, number of authors, etc.) constant in the model.

Originality/value

Based on a broader range of PPPR sentiments, by controlling many of the confounding factors (including the characteristics of the papers and the effects of the other PPPR platforms), this study analyzed the relationship of various polarities of PPPRs to citation count.

Should Authors be Requested to Suggest Peer Reviewers?

As part of a continuous process to explore the factors that might weaken or corrupt traditional peer review, in this paper, we query the ethics, fairness and validity of the request, by editors, of authors to suggest peer reviewers during the submission process. One of the reasons for the current crisis in science pertains to a loss in trust as a result of a flawed peer review which is by nature biased unless it is open peer review. As we indicate, the fact that some editors and journals rely on authors' suggestions in terms of who should peer review their paper already instills a potential way to abuse the trust of the submission and publishing system. An author-suggested peer reviewer choice might also tempt authors to seek reviewers who might be more receptive or sympathetic to the authors' message or results, and thus favor the outcome of that paper. Authors should thus not be placed in such a potentially ethically compromising situation, especially as a mandatory condition for submission. However, the fact that they do not have an opt-out choice during the submission process-especially when using an online submission system that makes such a suggestion compulsory-may constitute a violation of authors' rights.

Reviewer selection biases editorial decisions on manuscripts

Many journals, including the Journal of Neurochemistry, enable authors to list peer reviewers as 'preferred' or 'opposed' suggestions to the editor. At the Journal of Neurochemistry, the handling editor (HE) may follow recommendations or select non-author-suggested reviewers (non-ASRs). We investigated whether selection of author-suggested reviewers (ASRs) influenced decisions on a paper, and whether differences might be related to a reviewer's, editor's or manuscript's geographical location. In this retrospective analysis, we compared original research articles submitted to the Journal of Neurochemistry from 2013 through 2016 that were either reviewed exclusively by non-ASRs, by at least one ASR, by at least one reviewer marked by the author as 'opposed' or none. Manuscript outcome, reviewer rating of manuscript quality, rating of the reviewers' performance by the editor (R-score), time to review, and the country of the editor, reviewers and manuscript author were analyzed using non-parametric rank-based comparisons, chi-square (χ2 ) analysis, multivariate linear regression, one-way analysis of variance, and inter-rater reliability determination. Original research articles that had been reviewed by at least one ASR stood a higher chance of being accepted (525/1006 = 52%) than papers that had been reviewed by non-ASRs only (579/1800 = 32%). An article was 2.4 times more likely to be accepted than rejected by an ASR compared to a non-ASR (Pearson's χ2 (1) = 181.3, p < 0.05). At decision, the editor did not simply follow the reviewers' recommendation but had a balancing role: Rates of recommendation from reviewers for rejection were 11.2% (139/1241) with ASRs versus 29.0% (1379/4755) with non-ASRs (this is a ratio of 0.39 where 1 means no difference between rejection rates for both groups), whereas the proportion of final decisions to reject was 24.7% (248/1006) versus 45.7% (822/1800) (a ratio of 0.54, considerably closer to 1). Recommendations by non-ASRs were more favorable for manuscripts from USA/Canada and Europe than for Asia/Pacific or Other countries. ASRs judged North American manuscripts most favorably, and judged papers generally more positively (mean: 2.54 on a 1-5 scale) than did non-ASRs (mean: 3.16) reviewers, whereas time for review (13.28 vs. 13.20 days) did not differ significantly between these groups. We also found that editors preferably assigned reviewers from their own geographical region, but there was no tendency for reviewers to judge papers from their own region more favorably. Our findings strongly confirm a bias toward lower rejection rates when ASRs assess a paper, which led to the decision to abandon the option to recommend reviewers at the Journal of Neurochemistry. Open Data: Materials are available on https://osf.io/jshg7/.