Assessing open science and citizen science in addictions and substance use research: A scoping review

BACKGROUND

The EU promotes 'Open Science' as a public good. Complementary to its implementation is Citizen Science, which redefines the relationship between the scientific community, civic society and the individual. Open Science and Citizen Science poses challenges for the substance use and addictions research community but may provide positive opportunities for future European addiction research. This paper explores both current barriers and potential facilitators for the implementation of Open Science and Citizen Science in substance use and addictions research.

METHODOLOGY

A scoping review was used to examine barriers and facilitators identified in the substance use and addiction research literature for the adoption of Open Science and Citizen Science.

RESULTS

'Technical' facilitators included the pre-registration of study protocols; publication of open-source datasets; open peer review and online tools. 'Motivational' facilitators included enhanced reputation; embracing co-creation; engaged citizenship and gamification. 'Economic' facilitators included the use of free tools and balanced remuneration of crowdworkers. 'Political' facilitators included better informed debates through the 'triple helix' approach and trust-generating transparency. 'Legal' facilitators included epidemiologically informed law enforcement; better policy surveillance and the validation of other datasets. 'Ethical' facilitators included the 'democratisation of science' and opportunities to explore new concepts of ethics in addiction research.

CONCLUSION

Open Science and Citizen Science in substance use and addictions research may provide a range of benefits in relation to the democratisation of science; transparency; efficiency and the reliability/validity of data. However, its implementation raises a range of research integrity and ethical issues that need be considered. These include issues related to participant recruitment; privacy; confidentiality; security; cost and industry involvement. Progressive journal policies to support Open Science practices; a shift in researcher norms; the use of free tools and the greater availability of methodological and ethical standards are likely to increase adoption in the field.

The limitations to our understanding of peer review

Peer review is embedded in the core of our knowledge generation systems, perceived as a method for establishing quality or scholarly legitimacy for research, while also often distributing academic prestige and standing on individuals. Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie. We identify core themes including editorial responsibility, the subjectivity and bias of reviewers, the function and quality of peer review, and the social and epistemic implications of peer review. The high-priority gaps are focused around increased accountability and justification in decision-making processes for editors and developing a deeper, empirical understanding of the social impact of peer review. Addressing this at the bare minimum will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record. We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review.

Guidelines for open peer review implementation

Open peer review (OPR) is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. As more journals move to implement and experiment with the myriad of innovations covered by this term, there is a clear need for best practice guidelines to guide implementation. This brief article aims to address this knowledge gap, reporting work based on an interactive stakeholder workshop to create best-practice guidelines for editors and journals who wish to transition to OPR. Although the advice is aimed mainly at editors and publishers of scientific journals, since this is the area in which OPR is at its most mature, many of the principles may also be applicable for the implementation of OPR in other areas (e.g., books, conference submissions).

Peer review in medical journals: Beyond quality of reports towards transparency and public scrutiny of the process

Published medical research influences health care providers and policy makers, guides patient management, and is based on the peer review process. Peer review should prevent publication of unreliable data and improve study reporting, but there is little evidence that these aims are fully achieved. In the blinded systems, authors and readers do not know the reviewers' identity. Moreover, the reviewers' reports are not made available to readers. Anonymous peer review poses an ethical imbalance toward authors, who are judged by masked referees, and to the medical community and society at large, in case patients suffer the consequences of acceptance of flawed manuscripts or erroneous rejection of important findings. Some general medical journals have adopted an open process, require reviewers to sign their reports, and links online pre-publication histories to accepted articles. This system increases editors' and reviewers' accountability and allows public scrutiny, consenting readers understand on which basis were decisions taken and by whom. Moreover, this gives credit to reviewers for their apparently thankless job, as online availability of signed and scored reports may contribute to researchers' academic curricula. However, the transition from the blind to the open system could pose problems to journals. Reviewers may be more difficult to find, and publishers or medical societies could resist changes that may affect editorial costs and journals' revenues. Nonetheless, also considering the risk of competing interests in the medical field, general and major specialty journals could consider testing the effects of open review on manuscripts regarding studies that may influence clinical practice.

Comparison of self-citation by peer reviewers in a journal with single-blind peer review versus a journal with open peer review

OBJECTIVE

Some peer reviewers may inappropriately, or coercively request that authors include references to the reviewers' own work. The objective of this study was to evaluate whether, compared to reviews for a journal with single-blind peer review, reviews for a journal with open peer review included (1) fewer self-citations; (2) a lower proportion of self-citations without a rationale; and (3) a lower ratio of proportions of citations without a rationale in self-citations versus citations to others' work.

METHODS

Peer reviews for published manuscripts submitted in 2012 to a single-blind peer review journal, the Journal of Psychosomatic Research, were previously evaluated (Thombs et al., 2015). These were compared to publically available peer reviews of manuscripts published in 2012 in an open review journal, BMC Psychiatry. Two investigators independently extracted data for both journals.

RESULTS

There were no significant differences between journals in the proportion of all reviewer citations that were self-citations (Journal of Psychosomatic Research: 71/225, 32%; BMC Psychiatry: 90/315, 29%; p=.50), or in the proportion of self-citations without a rationale (Journal of Psychosomatic Research: 15/71, 21%; BMC Psychiatry: 12/90, 13%; p=.21). There was no significant difference between journals in the proportion of self-citations versus citations to others' work without a rationale (p=.31).

CONCLUSION

Blind and open peer review methodologies have distinct advantages and disadvantages. The present study found that, in reasonably similar journals that use single-blind and open review, there were no substantive differences in the pattern of peer reviewer self-citations.