Framing COVID-19 Preprint Research as Uncertain: A Mixed-Method Study of Public Reactions

During the COVID-19 pandemic, journalists were encouraged to convey uncertainty surrounding preliminary scientific evidence, including mentioning when research is unpublished or unverified by peer review. To understand how public audiences interpret this information, we conducted a mixed method study with U.S. adults. Participants read a news article about preprint COVID-19 vaccine research in early April 2021, just as the vaccine was becoming widely available to the U.S. public. We modified the article to test two ways of conveying uncertainty (hedging of scientific claims and mention of preprint status) in a 2 × 2 between-participants factorial design. To complement this, we collected open-ended data to assess participants' understanding of the concept of a scientific preprint. In all, participants who read hedged (vs. unhedged) versions of the article reported less favorable vaccine attitudes and intentions and found the scientists and news reporting less trustworthy. These effects were moderated by participants' epistemic beliefs and their preference for information about scientific uncertainty. However, there was no impact of describing the study as a preprint, and participants' qualitative responses indicated a limited understanding of the concept. We discuss implications of these findings for communicating initial scientific evidence to the public and we outline important next steps for research and theory-building.

Characterization of Comments About bioRxiv and medRxiv Preprints

Importance

Preprints have been increasingly used in biomedical science, and a key feature of many platforms is public commenting. The content of these comments, however, has not been well studied, and it is unclear whether they resemble those found in journal peer review.

Objective

To describe the content of comments on the bioRxiv and medRxiv preprint platforms.

Design, Setting, and Participants

In this cross-sectional study, preprints posted on the bioRxiv and medRxiv platforms in 2020 were accessed through each platform's application programming interface on March 29, 2021, and a random sample of preprints containing between 1 and 20 comments was evaluated independently by 3 evaluators using an instrument to assess their features and general content.

Main Outcome and Measures

The numbers and percentages of comments from authors or nonauthors were assessed, and the comments from nonauthors were assessed for content. These nonauthor comments were assessed to determine whether they included compliments, criticisms, corrections, suggestions, or questions, as well as their topics (eg, relevance, interpretation, and methods). Nonauthor comments were also analyzed to determine whether they included references, provided a summary of the findings, or questioned the preprint's conclusions.

Results

Of 52 736 preprints, 3850 (7.3%) received at least 1 comment (mean [SD] follow-up, 7.5 [3.6] months), and the 1921 assessed comments (from 1037 preprints) had a median length of 43 words (range, 1-3172 words). The criticisms, corrections, or suggestions present in 694 of 1125 comments (61.7%) were the most prevalent content, followed by compliments (n = 428 [38.0%]) and questions (n = 393 [35.0%]). Criticisms usually regarded interpretation (n = 286), methodological design (n = 267), and data collection (n = 238), while compliments were mainly about relevance (n = 111) and implications (n = 72).

Conclusions and Relevance

In this cross-sectional study of preprint comments, topics commonly associated with journal peer review were frequent. However, only a small percentage of preprints posted on the bioRxiv and medRxiv platforms in 2020 received comments on these platforms. A clearer taxonomy of peer review roles would help to describe whether postpublication peer review fulfills them.

COVID-19 retracted publications on retraction watch: A systematic survey of their pre-prints and citations

Background

Studies related to the coronavirus disease 2019 (COVID-19) were frequently published as pre-prints prior to undergoing peer-review. However, several publications were later retracted due to ethical concerns or study misconduct. Although these studies have been retracted, the availability of their corresponding pre-prints has never been formally investigated, and may result in the spread of misinformation if they are being used to inform decision-making.

Methods

Our objective was to conduct a systematic survey of retracted COVID-19 publications listed on the Retraction Watch database as of August 15th, 2021. We assessed the availability of corresponding pre-prints for retracted publications, and documented the number of citations and online views.

Results

Our study included 140 retracted COVID-19 publications, and we could not retrieve corresponding pre-prints for 132 retracted publications in our study (94%). Although we were unable to find the majority of pre-prints, they had already been disseminated, with a maximal citation count of 593 and Altmetric score of 558,928.

Conclusion

While it is reassuring that most corresponding pre-prints could not be retrieved, our study highlights the need for online platforms and journals to employ quality assurance methods to prevent the spread of misinformation through citation of retracted papers.

Changes in health communication in the age of COVID-19: A study on the dissemination of preprints to the public

Introduction

Preprints have become an important tool for meeting the challenges of health communication in the context of COVID-19. They allow scientists to disseminate their results more quickly due to the absence of a peer review process. Preprints have been well-received by scientists, however, there have been concerns about the exposure of wider public audiences to preprints due in part to this lack of peer review.

Methods

The aim of this study is to examine the dissemination of preprints on medRxiv and bioRxiv during the COVID-19 pandemic using content analysis and statistical analysis.

Results

Our findings show that preprints have played an unprecedented role in disseminating COVID-19-related science results to the public.

Discussion

While the overall media coverage of preprints is unsatisfactory, digital native news media performed better than legacy media in reporting preprints, which means that we could make the most of digital native media to improve health communication. This study contributes to understanding how science communication has evolved in response to the COVID-19 pandemic and provides some practical recommendations.

PreprintMatch: A tool for preprint to publication detection shows global inequities in scientific publication

Preprints, versions of scientific manuscripts that precede peer review, are growing in popularity. They offer an opportunity to democratize and accelerate research, as they have no publication costs or a lengthy peer review process. Preprints are often later published in peer-reviewed venues, but these publications and the original preprints are frequently not linked in any way. To this end, we developed a tool, PreprintMatch, to find matches between preprints and their corresponding published papers, if they exist. This tool outperforms existing techniques to match preprints and papers, both on matching performance and speed. PreprintMatch was applied to search for matches between preprints (from bioRxiv and medRxiv), and PubMed. The preliminary nature of preprints offers a unique perspective into scientific projects at a relatively early stage, and with better matching between preprint and paper, we explored questions related to research inequity. We found that preprints from low income countries are published as peer-reviewed papers at a lower rate than high income countries (39.6% and 61.1%, respectively), and our data is consistent with previous work that cite a lack of resources, lack of stability, and policy choices to explain this discrepancy. Preprints from low income countries were also found to be published quicker (178 vs 203 days) and with less title, abstract, and author similarity to the published version compared to high income countries. Low income countries add more authors from the preprint to the published version than high income countries (0.42 authors vs 0.32, respectively), a practice that is significantly more frequent in China compared to similar countries. Finally, we find that some publishers publish work with authors from lower income countries more frequently than others.

Science in motion: A qualitative analysis of journalists' use and perception of preprints

This qualitative study explores how and why journalists use preprints-unreviewed research papers-in their reporting. Through thematic analysis of interviews conducted with 19 health and science journalists in the second year of the COVID-19 pandemic, it applies a theoretical framework that conceptualizes COVID-19 preprint research as a form of post-normal science, characterized by high scientific uncertainty and societal relevance, urgent need for political decision-making, and value-related policy considerations. Findings suggest that journalists approach the decision to cover preprints as a careful calculation, in which the potential public benefits and the ease of access preprints provided were weighed against risks of spreading misinformation. Journalists described viewing unreviewed studies with extra skepticism and relied on diverse strategies to find, vet, and report on them. Some of these strategies represent standard science journalism, while others, such as labeling unreviewed studies as preprints, mark a departure from the norm. However, journalists also reported barriers to covering preprints, as many felt they lacked the expertise or the time required to fully understand or vet the research. The findings suggest that coverage of preprints is likely to continue post-pandemic, with important implications for scientists, journalists, and the publics who read their work.

Preprint citation practice in PLOS

The role of preprints in the scientific production and their part in citations have been growing over the past 10 years. In this paper we study preprint citations in several different aspects: the progression of preprint citations over time, their relative frequencies in relation to the IMRaD structure of articles, their distributions over time, per preprint database and per PLOS journal. We have processed the PLOS corpus that covers 7 journals and a total of about 240,000 articles up to January 2021, and produced a dataset of 8460 preprint citation contexts that cite 12 different preprint databases. Our results show that preprint citations are found with the highest frequency in the Method section of articles, though small variations exist with respect to journals. The PLOS Computational Biology journal stands out as it contains more than three times more preprint citations than any other PLOS journal. The relative parts of the different preprint databases are also examined. While ArXiv and bioRxiv are the most frequent citation sources, bioRxiv’s disciplinary nature can be observed as it is the source of more than 70% of preprint citations in PLOS Biology, PLOS Genetics and PLOS Pathogens. We have also compared the lexical content of preprint citation contexts to the citation content to peer-reviewed publications. Finally, by performing a lexicometric analysis, we have shown that preprint citation contexts differ significantly from citation contexts of peer-reviewed publications. This confirms that authors make use of different lexical content when citing preprints compared to the rest of citations.

Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets

In this article, we investigate the surge in use of COVID-19-related preprints by media outlets. Journalists are a main source of reliable public health information during crises and, until recently, journalists have been reluctant to cover preprints because of the associated scientific uncertainty. Yet, uploads of COVID-19 preprints and their uptake by online media have outstripped that of preprints about any other topic. Using an innovative approach combining altmetrics methods with content analysis, we identified a diversity of outlets covering COVID-19-related preprints during the early months of the pandemic, including specialist medical news outlets, traditional news media outlets, and aggregators. We found a ubiquity of hyperlinks as citations and a multiplicity of framing devices for highlighting the scientific uncertainty associated with COVID-19 preprints. These devices were rarely used consistently (e.g., mentioning that the study was a preprint, unreviewed, preliminary, and/or in need of verification). About half of the stories we analyzed contained framing devices emphasizing uncertainty. Outlets in our sample were much less likely to identify the research they mentioned as preprint research, compared to identifying it as simply "research." This work has significant implications for public health communication within the changing media landscape. While current best practices in public health risk communication promote identifying and promoting trustworthy sources of information, the uptake of preprint research by online media presents new challenges. At the same time, it provides new opportunities for fostering greater awareness of the scientific uncertainty associated with health research findings.

A Comprehensive COVID-19 Daily News and Medical Literature Briefing to Inform Health Care and Policy in New Mexico: Implementation Study

BACKGROUND

On March 11, 2020, the New Mexico Governor declared a public health emergency in response to the COVID-19 pandemic. The New Mexico medical advisory team contacted University of New Mexico (UNM) faculty to form a team to consolidate growing information on severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and its disease to facilitate New Mexico's pandemic management. Thus, faculty, physicians, staff, graduate students, and medical students created the "UNM Global Health COVID-19 Intelligence Briefing."

OBJECTIVE

In this paper, we sought to (1) share how to create an informative briefing to guide public policy and medical practice and manage information overload with rapidly evolving scientific evidence; (2) determine the qualitative usefulness of the briefing to its readers; and (3) determine the qualitative effect this project has had on virtual medical education.

METHODS

Microsoft Teams was used for manual and automated capture of COVID-19 articles and composition of briefings. Multilevel triaging saved impactful articles to be reviewed, and priority was placed on randomized controlled studies, meta-analyses, systematic reviews, practice guidelines, and information on health care and policy response to COVID-19. The finalized briefing was disseminated by email, a listserv, and posted on the UNM digital repository. A survey was sent to readers to determine briefing usefulness and whether it led to policy or medical practice changes. Medical students, unable to partake in direct patient care, proposed to the School of Medicine that involvement in the briefing should count as course credit, which was approved. The maintenance of medical student involvement in the briefings as well as this publication was led by medical students.

RESULTS

An average of 456 articles were assessed daily. The briefings reached approximately 1000 people by email and listserv directly, with an unknown amount of forwarding. Digital repository tracking showed 5047 downloads across 116 countries as of July 5, 2020. The survey found 108 (95%) of 114 participants gained relevant knowledge, 90 (79%) believed it decreased misinformation, 27 (24%) used the briefing as their primary source of information, and 90 (79%) forwarded it to colleagues. Specific and impactful public policy decisions were informed based on the briefing. Medical students reported that the project allowed them to improve on their scientific literature assessment, stay current on the pandemic, and serve their community.

CONCLUSIONS

The COVID-19 briefings succeeded in informing and guiding New Mexico policy and clinical practice. The project received positive feedback from the community and was shown to decrease information burden and misinformation. The virtual platforms allowed for the continuation of medical education. Variability in subject matter expertise was addressed with training, standardized article selection criteria, and collaborative editing led by faculty.

Rapid Response in an Uncertain Environment: Study of COVID-19 Scientific Research Under the Parallel Model

Purpose

The rapid response of COVID-19 scientific research played a significant role in pandemic prevention and control but failed to block the spread of the pandemic rapidly. Besides the complexity of the virus, the effectiveness of control and prevention measures, and other factors, the adaptation of the mode of conducting scientific research is also crucial for the prevention and control of COVID-19. In this study, a parallel model was used to explore the effects of the rapid scientific response on COVID-19 to assess why pandemics continue to spread under rapid response.

Analysis

This study presents the response of scientific research based on country/region and publication dimensions after analyzing COVID-19 studies in the Web of Science and PubMed databases. Co-occurrence analysis of items was used to determine the generation rate of COVID-19 research under different topics to identify the reflected innovation model.

Results

More manifestations on rapid response of COVID-19 research, especially compared with the linear model of SARS research, showed that the COVID-19 research followed a parallel or concurrent innovation model.

Conclusion

Early multi-stakeholder partnership, convenient information sharing, and improved research competence promote the parallel model in COVID-19. Meanwhile, the uncertainty of the COVID-19 virus and the adverse effect of rapid response may limit the time efficiency of the parallel model. In conclusion, the rapid prevention and control of the pandemic cannot fully rely on scientific research but requires more combined effort under an uncertain global setting.

Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues

COVID-19-related (vs. non-related) articles appear to be more expeditiously processed and published in peer-reviewed journals. We aimed to evaluate: (i) whether COVID-19-related preprints were favored for publication, (ii) preprinting trends and public discussion of the preprints, and (iii) the relationship between the publication topic (COVID-19-related or not) and quality issues. Manuscripts deposited at bioRxiv and medRxiv between January 1 and September 27 2020 were assessed for the probability of publishing in peer-reviewed journals, and those published were evaluated for submission-to-acceptance time. The extent of public discussion was assessed based on Altmetric and Disqus data. The Retraction Watch Database and PubMed were used to explore the retraction of COVID-19 and non-COVID-19 articles and preprints. With adjustment for the preprinting server and number of deposited versions, COVID-19-related preprints were more likely to be published within 120 days since the deposition of the first version (OR = 1.96, 95% CI: 1.80-2.14) as well as over the entire observed period (OR = 1.39, 95% CI: 1.31-1.48). Submission-to-acceptance was by 35.85 days (95% CI: 32.25-39.45) shorter for COVID-19 articles. Public discussion of preprints was modest and COVID-19 articles were overrepresented in the pool of retracted articles in 2020. Current data suggest a preference for publication of COVID-19-related preprints over the observed period.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11192-021-04249-7.

Scientometric engineering: Exploring citation dynamics via arXiv eprints

Scholarly communications have been rapidly integrated into digitised and networked open ecosystems, where preprint servers have played a pivotal role in accelerating the knowledge transfer processes. However, quantitative evidence is scarce regarding how this paradigm shift beyond the traditional journal publication system has affected the dynamics of collective attention on science. To address this issue, we investigate the citation data of more than 1.5 million eprints on arXiv (https://arxiv.org) and analyse the long-term citation trend for each discipline involved. We find that the typical growth and obsolescence patterns vary across disciplines, reflecting different publication and communication practices. The results provide unique evidence on the attention dynamics shaped by the research community today, including the dramatic growth and fast obsolescence of Computer Science eprints, which has not been captured in previous studies relying on the citation data of journal papers. Subsequently, we develop a quantitatively-and-temporally normalised citation index with an approximately normal distribution, which is useful for comparing citational attention across disciplines and time periods. Further, we derive a stochastic model consistent with the observed quantitative and temporal characteristics of citation growth and obsolescence. The findings and the developed framework open a new avenue for understanding the nature of citation dynamics.

Peer Review

https://publons.com/publon/10.1162/qss_a_00174

eLife's new model and its impact on science communication

The eLife Early-Career Advisory Group discusses eLife's new peer review and publishing model, and how the whole process of scientific communication could be improved for the benefit of early-career researchers and the entire scientific community.

Crowdsourcing Global Perspectives in Ecology Using Social Media

Transparent, open, and reproducible research is still far from routine, and the full potential of open science has not yet been realized. Crowdsourcing–defined as the usage of a flexible open call to a heterogeneous group of individuals to recruit volunteers for a task –is an emerging scientific model that encourages larger and more outwardly transparent collaborations. While crowdsourcing, particularly through citizen- or community-based science, has been increasing over the last decade in ecological research, it remains infrequently used as a means of generating scientific knowledge in comparison to more traditional approaches. We explored a new implementation of crowdsourcing by using an open call on social media to assess its utility to address fundamental ecological questions. We specifically focused on pervasive challenges in predicting, mitigating, and understanding the consequences of disturbances. In this paper, we briefly review open science concepts and their benefits, and then focus on the new methods we used to generate a scientific publication. We share our approach, lessons learned, and potential pathways forward for expanding open science. Our model is based on the beliefs that social media can be a powerful tool for idea generation and that open collaborative writing processes can enhance scientific outcomes. We structured the project in five phases: (1) draft idea generation, (2) leadership team recruitment and project development, (3) open collaborator recruitment via social media, (4) iterative paper development, and (5) final editing, authorship assignment, and submission by the leadership team. We observed benefits including: facilitating connections between unusual networks of scientists, providing opportunities for early career and underrepresented groups of scientists, and rapid knowledge exchange that generated multidisciplinary ideas. We also identified areas for improvement, highlighting biases in the individuals that self-selected participation and acknowledging remaining barriers to contributing new or incompletely formed ideas into a public document. While shifting scientific paradigms to completely open science is a long-term process, our hope in publishing this work is to encourage others to build upon and improve our efforts in new and creative ways.

Preprinting is positively associated with early career researcher status in ecology and evolution

The usage of preprint servers in ecology and evolution is increasing, allowing research to be rapidly disseminated and available through open access at no cost. Early Career Researchers (ECRs) often have limited experience with the peer review process, which can be challenging when trying to build publication records and demonstrate research ability for funding opportunities, scholarships, grants, or faculty positions. ECRs face different challenges relative to researchers with permanent positions and established research programs. These challenges might also vary according to institution size and country, which are factors associated with the availability of funding for open access journals. We predicted that the career stage and institution size impact the relative usage of preprint servers among researchers in ecology and evolution. Using data collected from 500 articles (100 from each of two open access journals, two closed access journals, and a preprint server), we showed that ECRs generated more preprints relative to non-ECRs, for both first and last authors. We speculate that this pattern is reflective of the advantages of quick and open access research that is disproportionately beneficial to ECRs. There is also a marginal association between first author, institution size, and preprint usage, whereby the number of preprints tends to increase with institution size for ECRs. The United States and United Kingdom contributed the greatest number of preprints by ECRs, whereas non-Western countries contributed relatively fewer preprints. This empirical evidence that preprint usage varies with the career stage, institution size, and country helps to identify barriers surrounding large-scale adoption of preprinting in ecology and evolution.

Intellectual property protection for scholarly publishing in the Italian framework: a globally open approach for medical and life sciences authors

This paper aims at analyzing the importance of protection of intellectual property (IP) in biomedical scholarly publications, both for the author’s reputation and the dissemination of scientific knowledge. The laws that regulate IP are very complex and differ from country to country. We shall focus on the Italian framework though many considerations could be applied to foreign contexts. IP is very articulated, yet often ignored, that is worth paying attention to a correct copyright management can help researchers promote their Work and the community to benefit from it. In the scholarly publishing field, there are two main areas: traditional publishers and open-access publishers. The first group requires a fee to access the content they publish and usually ask the authors for a complete transfer of copyright. The possibility to negotiate terms with such publishers is often overlooked: scholarly authors tend to think they do not have leverage in the publishing cycle. The so-called addendum and professional figures, like librarians and attorneys, can help manage the authors’ intellectual property. On the other hand, open-access publishers give free access to published material, guaranteeing the protection of IP: thanks to the Creative Commons Licenses, the authors do not have to surrender their copyright to the publisher and can manage and control the use made out of their Work. Applying the principles put forth in this article implies enhancing research dissemination by increasing its impact and visibility. However, to achieve such a goal, it is necessary to protect intellectual property for the sake of authors, users, and scientific progress.

Preprint Servers in Kidney Disease Research: A Rapid Review

Preprint servers, such as arXiv and bioRxiv, have disrupted the scientific communication landscape by providing rapid access to research before peer review. medRxiv was launched as a free online repository for preprints in the medical, clinical, and related health sciences in 2019. In this review, we present the uptake of preprint server use in nephrology and discuss specific considerations regarding preprint server use in medicine. Distribution of kidney-related research on preprint servers is rising at an exponential rate. Survey of nephrology journals identified that 15 of 17 (88%) are publishing original research accepted submissions that have been uploaded to preprint servers. After reviewing 52 clinically impactful trials in nephrology discussed in the online Nephrology Journal Club (NephJC), an average lag of 300 days was found between study completion and publication, indicating an opportunity for faster research dissemination. Rapid review of papers discussing benefits and risks of preprint server use from the researcher, publisher, or end user perspective identified 53 papers that met criteria. Potential benefits of biomedical preprint servers included rapid dissemination, improved transparency of the peer review process, greater visibility and recognition, and collaboration. However, these benefits come at the risk of rapid spread of results not yet subjected to the rigors of peer review. Preprint servers shift the burden of critical appraisal to the reader. Media may be especially at risk due to their focus on "late-breaking" information. Preprint servers have played an even larger role when late-breaking research results are of special interest, such as during the global coronavirus disease 2019 pandemic. Coronavirus disease 2019 has brought both the benefits and risks of preprint servers to the forefront. Given the prominent online presence of the nephrology community, it is poised to lead the medicine community in appropriate use of preprint servers.

Preprint manuscripts and servers in the era of coronavirus disease 2019

RATIONALE, AIMS, AND OBJECTIVES

To both examine the impact of preprint publishing on health sciences research and survey popular preprint servers amidst the current coronavirus disease 2019 (COVID-19) pandemic.

METHODS

The authors queried three biomedical databases (MEDLINE, Web of Science, and Google Scholar) and two preprint servers (MedRxiv and SSRN) to identify literature pertaining to preprints. Additionally, they evaluated 12 preprint servers featuring COVID-19 research through sample submission of six manuscripts.

RESULTS

The realm of health sciences research has seen a dramatic increase in the presence and importance of preprint publications. By posting manuscripts on preprint servers, researchers are able to immediately communicate their findings, thereby facilitating prompt feedback and promoting collaboration. In doing so, they may also reduce publication bias and improve methodological transparency. However, by circumventing the peer-review process, academia incurs the risk of disseminating erroneous or misinterpreted data and suffering the downstream consequences. Never have these issues been better highlighted than during the ongoing COVID-19 pandemic. Researchers have flooded the literature with preprint publications as stopgaps to meet the desperate need for knowledge about the disease. These unreviewed articles initially outnumbered those published in conventional journals and helped steer the mainstream scientific community at the start of the pandemic. In surveying select preprint servers, the authors discovered varying usability, review practices, and acceptance polices.

CONCLUSION

While vital in the rapid dispensation of science, preprint manuscripts promulgate their conclusions without peer review and possess the capacity to misinform. Undoubtedly part of the future of science, conscientious consumers will need to appreciate not only their utility, but also their limitations.

The collective wisdom in the COVID-19 research: Comparison and synthesis of epidemiological parameter estimates in preprints and peer-reviewed articles

OBJECTIVES

We aimed to explore the collective wisdom of preprints related to COVID-19 by comparing and synthesizing them with results of peer-reviewed publications.

METHODS

PubMed, Google Scholar, medRxiv, bioRxiv, arXiv, and SSRN were searched for papers regarding the estimation of four epidemiological parameters of COVID-19: the basic reproduction number, incubation period, infectious period, and case-fatality-rate. Distributions of parameters and timeliness of preprints and peer-reviewed papers were compared. Four parameters in two groups were synthesized by bootstrapping, and their validities were evaluated by simulated cumulative cases of the susceptible-exposed-infectious-recovered-dead-cumulative (SEIRDC) model.

RESULTS

A total of 106 papers were included for analysis. The distributions of four parameters in two literature groups were close, and the timeliness of preprints was better. Synthesized estimates of the basic reproduction number (3.18, 95% CI 2.85-3.53), incubation period (5.44 days, 95% CI 4.98-5.99), infectious period (6.25 days, 95% CI 5.09-7.51), and case-fatality-rate (4.51%, 95% CI 3.41%-6.29%) were obtained. Simulated cumulative cases of the SEIRDC model matched well with the onset cases in China.

CONCLUSIONS

The validity of the COVID-19 parameter estimations of the preprints was on par with that of peer-reviewed publications, and synthesized results of literatures could reduce the uncertainty and be used for epidemic decision-making.

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

BACKGROUND

Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader's ability to independently interpret data and reproduce findings.

METHODS

In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals.

RESULTS

Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication.

CONCLUSIONS

Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Systematic review of bibliometric studies on SARS-CoV-2

Objective: To perform a systematic review of articles that evaluated the scientific production on SARS-CoV-2 through bibliometric analyzes. Methods: Scopus, Web of Science and Google Scholar databases were used. After applying the pre-established inclusion criteria, 30 articles were included. Results. The total number of articles found in the bibliometric studies on SARS-CoV-2 varied widely from 153 to 21,395 articles and an average equal to 4,279 (± 5,510). A total of 17 countries published within the scope of this study, but only six published more than one article, emphasizing authors from Chinese institutions (17%). Scopus was the most used database in bibliometric studies (50%, n = 15). The articles used 72 different keywords with emphasis on: COVID-19 (15%), SARS-CoV-2 (12%) and 2019-nCoV (9%). Conclusion. We are facing an unprecedented scenario of information about SARS-CoV-2 and this has required a collective scientific effort reflected in the daily publication of hundreds of studies (articles, pre-prints, clinical guides, protocols). Bibliometric methods are being increasingly used by the scientific community to systematize this information. Therefore, the systematic review carried out in this study provided an overview of the bibliometric literature on the SARS-CoV-2 virus.

Lessons of COVID-19: A roadmap for post-pandemic science

The response to the COVID-19 crisis across most research institutions mandated ceasing nonessential research activities in order to minimize the spread of the virus in our communities. With minimal notice, experiments were terminated, cell lines were frozen, mouse colonies were culled, and trainees were prevented from performing bench research. Still, despite the interruption of experimental productivity, the shutdown has proven for many PIs and trainees that doing and thinking science are not activities that are bound to the laboratory. Furthermore, the shutdowns have solidified important emerging trends and forced us to further innovate to get the most out of working remotely. We hope that some of these innovations, hard-gained in this difficult time, will persist and develop into new paradigms-lessons that will improve our science and our relationship to the climate and community beyond the current pandemic.

International authorship and collaboration across bioRxiv preprints

Preprints are becoming well established in the life sciences, but relatively little is known about the demographics of the researchers who post preprints and those who do not, or about the collaborations between preprint authors. Here, based on an analysis of 67,885 preprints posted on bioRxiv, we find that some countries, notably the United States and the United Kingdom, are overrepresented on bioRxiv relative to their overall scientific output, while other countries (including China, Russia, and Turkey) show lower levels of bioRxiv adoption. We also describe a set of 'contributor countries' (including Uganda, Croatia and Thailand): researchers from these countries appear almost exclusively as non-senior authors on international collaborations. Lastly, we find multiple journals that publish a disproportionate number of preprints from some countries, a dynamic that almost always benefits manuscripts from the US.

The relationship between bioRxiv preprints, citations and altmetrics

A potential motivation for scientists to deposit their scientific work as preprints is to enhance its citation or social impact. In this study we assessed the citation and altmetric advantage of bioRxiv, a preprint server for the biological sciences. We retrieved metadata of all bioRxiv preprints deposited between November 2013 and December 2017, and matched them to articles that were subsequently published in peer-reviewed journals. Citation data from Scopus and altmetric data from Altmetric.com were used to compare citation and online sharing behavior of bioRxiv preprints, their related journal articles, and nondeposited articles published in the same journals. We found that bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles. Regression analysis reveals that this advantage is not explained by multiple explanatory variables related to the articles’ publication venues and authorship. Further research will be required to establish whether such an effect is causal in nature. bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has subsequently been published in a journal. bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.

Releasing a preprint is associated with more attention and citations for the peer-reviewed article

Preprints in biology are becoming more popular, but only a small fraction of the articles published in peer-reviewed journals have previously been released as preprints. To examine whether releasing a preprint on bioRxiv was associated with the attention and citations received by the corresponding peer-reviewed article, we assembled a dataset of 74,239 articles, 5,405 of which had a preprint, published in 39 journals. Using log-linear regression and random-effects meta-analysis, we found that articles with a preprint had, on average, a 49% higher Altmetric Attention Score and 36% more citations than articles without a preprint. These associations were independent of several other article- and author-level variables (such as scientific subfield and number of authors), and were unrelated to journal-level variables such as access model and Impact Factor. This observational study can help researchers and publishers make informed decisions about how to incorporate preprints into their work.

Releasing a preprint is associated with more attention and citations for the peer-reviewed article

Preprints in biology are becoming more popular, but only a small fraction of the articles published in peer-reviewed journals have previously been released as preprints. To examine whether releasing a preprint on bioRxiv was associated with the attention and citations received by the corresponding peer-reviewed article, we assembled a dataset of 74,239 articles, 5,405 of which had a preprint, published in 39 journals. Using log-linear regression and random-effects meta-analysis, we found that articles with a preprint had, on average, a 49% higher Altmetric Attention Score and 36% more citations than articles without a preprint. These associations were independent of several other article- and author-level variables (such as scientific subfield and number of authors), and were unrelated to journal-level variables such as access model and Impact Factor. This observational study can help researchers and publishers make informed decisions about how to incorporate preprints into their work.

Increasing the accessibility and impact of justice-related student and practitioner research

Much good quality research by pre-doctoral students and case-work focused practitioners remains unpublished. However, their findings could contribute to the evidence base underpinning science and practice within international justice system contexts. There are two main challenges to making findings accessible: reaching all criminal justice stakeholders, and encouraging collaborative efforts in research addressing 'real world' problems. This article presents the rationale for a new, open access repository. The aim is to share good quality pre-doctoral and practitioner criminal justice research across traditional disciplinary and international borders. Such a repository should be easy to use, well maintained and sustainable. Its reach, value and impact also need to be measurable. We present the major considerations relating to the operation and workflow of such a repository, and outline the potential value, benefits and limitations. Our research suggests that the proposed repository could foster interdisciplinary and collaborative work to benefit global justice systems and societies.

Preprints and Scholarly Communication: Adoption, Practices, Drivers and Barriers

Background: Since 2013, there has been a dramatic increase in the number of preprint servers available online. To date, little is known about the position of researchers, funders, research performing organisations and other stakeholders with respect to this fast-paced landscape. In this article, we explore the benefits and challenges of preprint posting, along with issues such as infrastructure and financial sustainability. We also discuss the definition of a 'preprint' in different communities, and the impact this has on further uptake. Methods: This study is based on 38 detailed semi-structured interviews of key stakeholders based on a purposive heterogeneous sampling approach. Interviews were undertaken between October 2018 and January 2019. These were recorded, transcribed and subjected to thematic analysis to identify trends. Interview questions were designed based on Innovation Diffusion Theory, which is also used to interpret the results of this study. Results: Our study is the first using empirical data to understand the new wave of preprint servers and found that early and fast dissemination is the most appealing feature of the practice. The main concerns are related to the lack of quality assurance and the 'Ingelfinger rule'. We identified trust as an essential enabler of preprint posting and stress the enabling role of Twitter in showcasing preprints and enabling comments on these. Conclusions: The preprints landscape is evolving fast and disciplinary communities are at different stages in the innovation diffusion process. The landscape is characterised by significant experimentation, which leads to the conclusion that a one-size-fits-all approach to preprints is not feasible. Cooperation and active engagement between the stakeholders involved will play an important role in the future. In our paper, we share questions for the further development of the preprints landscape, with the most important being whether preprint posting will develop as a publisher- or researcher-centric practice.

Quantifying the Growth of Preprint Services Hosted by the Center for Open Science

A wide range of disciplines are building preprint services—web-based systems that enable publishing non peer-reviewed scholarly manuscripts before publication in a peer-reviewed journal. We have quantitatively surveyed nine of the largest English language preprint services offered by the Center for Open Science (COS) and available through an Application Programming Interface. All of the services we investigate also permit the submission of postprints, non-typeset versions of peer-reviewed manuscripts. Data indicates that all services are growing, but with submission rates below more mature services (e.g., bioRxiv). The trend of the preprint-to-postprint ratio for each service indicates that recent growth is a result of more preprint submissions. The nine COS services we investigate host papers that appear in a range of peer-reviewed journals, and many of these publication venues are not listed in the Directory of Open Access Journals. As a result, COS services function as open repositories for peer-reviewed papers that would otherwise be behind a paywall. We further analyze the coauthorship network for each COS service, which indicates that the services have many small connected components, and the largest connected component encompasses only a small percentage of total authors on each service. When comparing the papers submitted to each service, we observe topic overlap measured by keywords self-assigned to each manuscript, indicating that search functionalities would benefit from cutting across the boundaries of a single service. Finally, though annotation capabilities are integrated into all COS services, it is rarely used by readers. Our analysis of these services can be a benchmark for future studies of preprint service growth.

Rxivist.org: Sorting biology preprints using social media and readership metrics

Preprints have arrived. In increasing numbers, researchers across the life sciences are embracing the once-niche practice, shaking off decades of reluctance and posting hundreds of papers per week to preprint servers, sharing their findings with the community before embarking on the weary march through peer review. However, there are limited methods for individuals sifting through this avalanche of research to identify the preprints that are most relevant to their interests. Here, we describe Rxivist.org, a website that indexes all preprints posted to bioRxiv.org, the largest preprint server in the life sciences, and allows users to filter and sort papers based on download metrics and Twitter activity over a variety of categories and time periods. In this work, we hope to make it easier for readers to find relevant research on bioRxiv and to improve the visibility of preprints currently being read and discussed online.

This Community Page article describes Rxivist.org, a new website that indexes all preprints posted to bioRxiv.org and allows users to filter and sort papers based on download metrics and Twitter activity over a variety of categories and time periods.

Tracking the popularity and outcomes of all bioRxiv preprints

The growth of preprints in the life sciences has been reported widely and is driving policy changes for journals and funders, but little quantitative information has been published about preprint usage. Here, we report how we collected and analyzed data on all 37,648 preprints uploaded to bioRxiv.org, the largest biology-focused preprint server, in its first five years. The rate of preprint uploads to bioRxiv continues to grow (exceeding 2,100 in October 2018), as does the number of downloads (1.1 million in October 2018). We also find that two-thirds of preprints posted before 2017 were later published in peer-reviewed journals, and find a relationship between the number of downloads a preprint has received and the impact factor of the journal in which it is published. We also describe Rxivist.org, a web application that provides multiple ways to interact with preprint metadata.

On the value of preprints: An early career researcher perspective

Peer-reviewed journal publication is the main means for academic researchers in the life sciences to create a permanent public record of their work. These publications are also the de facto currency for career progress, with a strong link between journal brand recognition and perceived value. The current peer-review process can lead to long delays between submission and publication, with cycles of rejection, revision, and resubmission causing redundant peer review. This situation creates unique challenges for early career researchers (ECRs), who rely heavily on timely publication of their work to gain recognition for their efforts. Today, ECRs face a changing academic landscape, including the increased interdisciplinarity of life sciences research, expansion of the researcher population, and consequent shifts in employer and funding demands. The publication of preprints, publicly available scientific manuscripts posted on dedicated preprint servers prior to journal-managed peer review, can play a key role in addressing these ECR challenges. Preprinting benefits include rapid dissemination of academic work, open access, establishing priority or concurrence, receiving feedback, and facilitating collaborations. Although there is a growing appreciation for and adoption of preprints, a minority of all articles in life sciences and medicine are preprinted. The current low rate of preprint submissions in life sciences and ECR concerns regarding preprinting need to be addressed. We provide a perspective from an interdisciplinary group of ECRs on the value of preprints and advocate their wide adoption to advance knowledge and facilitate career development.