The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape

Ivermectin repurposing for COVID-19: pharmacological and bibliometric analysis

Since the onset of the COVID-19 pandemic in March 2020, researchers worldwide have sought effective drugs to prevent and manage SARS-CoV-2 and its spectrum of symptoms. Ivermectin, originally developed as an anthelmintic for controlling parasitic infections in humans and animals, has drawn attention based on the hypothesis that it inhibits viral replication. In Austria, ivermectin usage peaked in November 2021, following promotion by the right-wing Freedom Party of Austria (FPÖ) as an alternative treatment to vaccination, resonating strongly within anti-vaccine and skeptical communities. The topic is also very present in the United States of America due to the re-election of D. Trump as US President and the designation of R. Kennedy as the United States' Secretary of Health and Human Services. To critically examine the controversial use of ivermectin for COVID-19 and publication trends during the pandemic, this study analysed all publications listed in PubMed from 1 January 2020 to 31 December 2022 using the keywords 'ivermectin' and 'COVID-19', resulting in a dataset of 353 publications. These publications were assessed for scientific quality, methodological rigour and bias, with particular focus on the influence of social and political dynamics on publication practices, as well as the prevalence of preprints, citation trends and the role of funding sources. Our study shows that many highly cited studies on ivermectin display methodological weaknesses and data gaps, contributing to the propagation of hypotheses lacking substantial empirical support. This analysis underscores the necessity of rigorous quality control during crises and highlights the long-term risks posed to scientific databases and public health by methodologically deficient research.

Predictions of PM2.5 using air pollutants and meteorological factors with COVID-19 cases in Malaysia and Indonesia: a comparative study using feature selection and robust regression

The study examines the relationship between air quality, meteorological factors, and COVID-19 cases in Cheras, Kuala Lumpur, and Kelapa Gading, North Jakarta. Analyzing data from 2020 and 2021, the research found notable correlations: COVID-19 cases in Cheras were positively associated with relative humidity (RH) and carbon monoxide (CO) but negatively with ozone (O₃) and RH in different years. In Kelapa Gading, COVID-19 cases were positively correlated with pollutants like sulfur dioxide (SO₂) and CO, while ambient temperature (AT) showed a negative correlation. The enforcement of social restrictions notably reduced air pollution, affecting COVID-19 spread. Predictive models for PM2.5 levels using robust regression techniques showed strong performance in Kuala Lumpur (R² > 0.9) but exhibited overfitting tendencies in Jakarta, suggesting the need for a longer study period for more accurate results.

The importance of rigor in pharmacy research: Challenges and solutions

Scientific rigor broadly refers to upholding basic principles within the conduct of research. Various threats associated with rigor exist in today's research environment, such as the replication crisis, the increasing prevalence of misconduct, and a loss of public trust in regulatory and educational institutions. The purpose of this commentary is to identify problems and solutions associated with research rigor, with a focus on pharmacy research. Problems exist at many levels, including within variable research training/funding, institutional pressures associated with career advancement, and norms associated with academic publishing. However, solutions are possible as methods of harm reduction, including (but not limited to) focused initiatives supporting rigor, team-based approaches to research that include diverse interested parties, and a reimagining of what constitutes value within science. Pharmacists and pharmacy researchers are called upon to uphold research rigor as a professional and ethical responsibility.

Post-publication peer review and the identification of methodological and reporting issues in COVID-19 trials: a qualitative study

OBJECTIVES

We aimed to determine to what extent systematic reviewers and post-preprint and post-publication peer review identified methodological and reporting issues in COVID-19 trials that could be easily resolved by the authors.

DESIGN

Qualitative study.

DATA SOURCES

COVID-NMA living systematic review (covid-nma.com), PubPeer, medRxiv, Research Square, SSRN.

METHODS

We considered randomised controlled trials (RCTs) in COVID-NMA that evaluated pharmacological treatments for COVID-19 and retrieved systematic reviewers' assessments of the risk of bias and outcome reporting bias. We also searched for commentary data on PubPeer and preprint servers up to 6 November 2023. We employed qualitative content analysis to develop themes and domains of methodological and reporting issues identified by commenters.

RESULTS

We identified 500 eligible RCTs. Systematic reviewers identified methodological and reporting issues in 446 (89%) RCTs. In 391 (78%) RCTs, the issues could be easily resolved by the trial authors; issues included incomplete reporting (49%), selection of the reported results (52%) and no access to the pre-specified plan (25%). Alternatively, 74 (15%) RCTs had received at least one comment on PubPeer or preprint servers, totalling 348 comments. In 46 (9%) RCTs, the issues identified by post-preprint and post-publication peer review comments could be easily resolved by the trial authors; the issues were related to incomplete reporting (6%), errors (5%), statistical analysis (3%), inconsistent reporting of methods and analyses (2%), spin (2%), selection of the reported results (1%) and no access to the raw data/pre-specified plan (1%).

CONCLUSIONS

Without changing their process, systematic reviewers identified issues in most RCTs that could be easily resolved by the trial authors; however, the lack of an established author feedback mechanism represents a wasted opportunity for facilitating improvement and enhancing the overall manuscript quality. On the other hand, despite the existing feedback loop to authors present in post-publication peer review, it demonstrated limited effectiveness in identifying methodological and reporting issues.

Predicting the replicability of social and behavioural science claims in COVID-19 preprints

Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgements from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise ('beginners') updated their estimates and confidence in their judgements significantly more than groups with greater task expertise ('experienced'). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners' average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant and their judgements on the full set of claims were correlated (r(98) = 0.48, P < 0.001). These results suggest that both beginners and more-experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of 'fast science' under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.

Public understanding of preprints: How audiences make sense of unreviewed research in the news

News reporting of preprints became commonplace during the COVID-19 pandemic, yet the extent to which the public understands what preprints are is unclear. We sought to fill this gap by conducting a content analysis of 1702 definitions of the term "preprint" that were generated by the US general population and college students. We found that only about one in five people were able to define preprints in ways that align with scholarly conceptualizations of the term, although participants provided a wide array of "other" definitions of preprints that suggest at least a partial understanding of the term. Providing participants with a definition of preprints in a news article helped improve preprint understanding for the student sample, but not for the general population. Our findings shed light on misperceptions that the public has about preprints, underscoring the importance of better education about the nature of preprint research.

Comparison of journal and top publisher self-citation rates in COVID-19 research

INTRODUCTION

Journal self-citation contributes to the overall citation count of a journal and to some metrics like the impact factor. However, little is known about the extent of journal self-citations in COVID-19 research. This study aimed to determine the journal self-citations in COVID-19 research and to compare them according to the type of publication and publisher.

METHODS

Data in COVID-19 research extracted from the Web of Science Core Collection 2020-2023 was collected and further analyzed with InCites. The journals with the highest self-citation rates and self-citation per publication were identified. Statistical comparisons were made according to the type of publication and publishers, as well as with other major infectious diseases.

RESULTS

The median self-citation rate was 4.0% (IQR 0-11.7%), and the median journal self-citation rate was 5.9% (IQR 0-12.5%). 1,859 journals (13% of total coverage) had self-citation rates at or above 20%, meaning that more than one in five references are journal self-citations. There was a positive and statistically significant correlation of self-citations with the other indicators, including number of publications, citations, and self-citations per publication (p<0.001). Editorial materials contributed more to journal SC with a median self-citation rate of 5%, which was statistically higher than other documents such as articles, letters or reviews (p<0.001). Among the top twelve publishers, the Multidisciplinary Digital Publishing Institute had a median self-citation rate of 8.33% and was statistically higher than the rest (p<0.001). Self-citation rates for COVID-19 were lower than tuberculosis and HIV/AIDS, but self-citations per publication of these diseases were statistically lower than those for COVID-19 (p<0.001).

CONCLUSION

Some journals from the Web of Science Core Collection displayed exorbitant journal self-citation patterns during the period 2020-2023. Approximately, one in every five paper references in COVID-19 is a journal self-citation. Types of publication such as editorials engage in this practice more frequently than others, suggesting that in COVID-19 research, self-citing non-citable items could potentially contribute to inflate journal impact factors during the pandemic.

Impact of Regulatory Risk Communication on Thrombosis With Thrombocytopenia Syndrome for COVID-19 Adenovirus Vector Vaccines on European Healthcare Professionals

PURPOSE

The European Medicines Agency (EMA) issued regulatory actions and communications in 2021 on thrombosis with thrombocytopenia syndrome (TTS) associated with adenovirus vector vaccines Vaxzevria or Jcovden. This study aimed to evaluate the impact of these actions on awareness, knowledge and implementation in practises of healthcare professionals (HCP).

METHODS

Web-based cross-sectional surveys were conducted on HCPs engaged in the vaccination, monitoring or counselling about the vaccines. We measured awareness and knowledge of the risk of TTS and their implementation of recommendations in practise. Descriptive and qualitative analyses were conducted. This study took place in Greece, Latvia, Netherlands, Portugal and Slovenia.

RESULTS

We surveyed 1659 HCPs. From these, 914 were included in the analysis. Most were aware about the reports of TTS associated with COVID-19 adenovirus vector vaccines, with countries reporting percentages between 85% and 97%. Mainstream media, health authorities and peers were the main sources of TTS risk information. Most HCPs were able to identify key symptoms from TTS but were less familiar with minor symptoms. Guidelines from health authorities on COVID-19 vaccination impacted on professional practise of 55%-77% of HCPs. The reported use of product information across countries was moderate, with some variations.

CONCLUSIONS

Awareness about and knowledge of TTS risk from COVID-19 adenoviral vector vaccines were high among HCPs. HCPs reported a clear preference for national guidelines as source of risk information and the implementation of product information remained moderate across countries.

COVID-19 vaccine evidence monitoring assisted by artificial Intelligence: An emergency system implemented by the Public Health Agency of Canada to capture and describe the trajectory of evolving pandemic vaccine literature

Background

The COVID-19 pandemic resulted in a rapid accumulation of novel vaccine research evidence. As a means to monitor this evidence, the Public Health Agency of Canada (PHAC) created the Evidence eXtraction Team for Research Analysis (EXTRA), which contributed to situational awareness in Canada through a bibliographic repository used to support decision-making by the National Advisory Committee on Immunization. We describe the process by which this literature was identified and catalogued, and provide an overview of characteristics in the identified literature.

Methods

To expedite the process, PHAC leveraged an artificial intelligence (AI) tool to assist in the screening and selection of relevant articles. Literature search results were initially screened by AI, then manually reviewed for relevance. Relevant articles were tagged using controlled vocabulary and stored in a bibliographic repository. This repository was analyzed to identify trends in vaccine research over time according to several key characteristics.

Results

As of December 31, 2023, EXTRA's repository contained 19,050 articles relevant to PHAC's immunization mandate. The majority of these articles (63.9 %) were identified between August 2021 and January 2023, with an average of 20 relevant articles added daily during this period. Nearly 14,000 articles reported on mRNA vaccines. Safety outcomes were most frequently reported (n = 8,289), followed by immunogenicity (n = 7,269) and efficacy/effectiveness (n = 3,246). COVID-19 vaccine literature output started to decrease in mid-2023, two years after the initial dramatic increase in mid-2021.

Conclusions

This hybrid (AI and human) approach was critical for PHAC situational awareness and the development of timely vaccine guidance in Canada during the COVID-19 pandemic. Given the volume of data and analyses required, the AI-augmented processes made this massive undertaking manageable. Analysis of COVID-19 vaccine research patterns supports projections of research volume, type, and rate that will help predict resourcing and information needs to plan future emergency vaccine guidance activities.

An international, cross-sectional survey of preprint attitudes among biomedical researchers

Background

Preprints are scientific manuscripts that are made available on open-access servers but are not yet peer-reviewed. Although preprints are becoming more prevalent, uptake is not optimal. Understanding researchers' opinions and attitudes toward preprints is valuable to optimize their use. Understanding knowledge gaps and researchers' attitudes toward preprinting can assist stakeholders, such as journals, funding agencies, and universities, to use preprints more effectively. Here, we aimed to collect perceptions and behaviours regarding preprints across an international sample of biomedical researchers.

Methods

Corresponding authors of articles published in biomedical research journals were identified from a random sample of journals from the MEDLINE database. Their names and email addresses were extracted to invite them to our anonymous, cross-sectional survey, which asked participants questions about their knowledge, attitudes, and opinions regarding preprinting.

Results

The survey was completed by 730 respondents providing a response rate of 3.20% and demonstrated a wide range of attitudes and opinions about preprints with authors from various disciplines and career stages worldwide. Most respondents were familiar with the concept of preprints but most had not previously posted one. The lead author of the project and journal policy had the greatest impact on decisions to post a preprint, whereas employers/research institutes had the least impact. Supporting open science practices was the highest ranked incentive, while increasing authors' visibility was the highest ranked motivation for publishing preprints.

Conclusions

Although many biomedical researchers recognize the benefits of preprints, there is still hesitation among others to engage in this practice. This may be due to the general lack of peer review of preprints and little enthusiasm from external organizations such as journals, funding agencies, and universities. Future work is needed to determine optimal ways to improve researchers' attitudes through modifications to current preprint systems and policies.

How the pandemic affected psychological research

In the wake of the COVID-19 pandemic, many journals swiftly changed their editorial policies and peer-review processes to accelerate the provision of knowledge about COVID-related issues to a wide audience. These changes may have favoured speed at the cost of accuracy and methodological rigour. In this study, we compare 100 COVID-related articles published in four major psychological journals between 2020 and 2022 with 100 non-COVID articles from the same journal issues and 100 pre-COVID articles published between 2017 and 2019. Articles were coded with regard to design features, sampling and recruitment features, and openness and transparency practices. Even though COVID research was, by and large, more 'observational' in nature and less experimentally controlled than non- or pre-COVID research, we found that COVID-related studies were more likely to use 'stronger' (i.e. more longitudinal and fewer cross-sectional) designs, larger samples, justify their sample sizes based on a priori power analysis, pre-register their hypotheses and analysis plans and make their data, materials and code openly available. Thus, COVID-related psychological research does not appear to be less rigorous in these regards than non-COVID research.

An analysis of the effects of sharing research data, code, and preprints on citations

Calls to make scientific research more open have gained traction with a range of societal stakeholders. Open Science practices include but are not limited to the early sharing of results via preprints and openly sharing outputs such as data and code to make research more reproducible and extensible. Existing evidence shows that adopting Open Science practices has effects in several domains. In this study, we investigate whether adopting one or more Open Science practices leads to significantly higher citations for an associated publication, which is one form of academic impact. We use a novel dataset known as Open Science Indicators, produced by PLOS and DataSeer, which includes all PLOS publications from 2018 to 2023 as well as a comparison group sampled from the PMC Open Access Subset. In total, we analyze circa 122'000 publications. We calculate publication and author-level citation indicators and use a broad set of control variables to isolate the effect of Open Science Indicators on received citations. We show that Open Science practices are adopted to different degrees across scientific disciplines. We find that the early release of a publication as a preprint correlates with a significant positive citation advantage of about 20.2% (±.7) on average. We also find that sharing data in an online repository correlates with a smaller yet still positive citation advantage of 4.3% (±.8) on average. However, we do not find a significant citation advantage for sharing code. Further research is needed on additional or alternative measures of impact beyond citations. Our results are likely to be of interest to researchers, as well as publishers, research funders, and policymakers.

Altmetric coverage of health research in Ireland 2017-2023: a protocol for a cross-sectional analysis

Background

Scientific publications have been growing exponentially, contributing to an oversaturated information environment. Quantifying a research output's impact and reach cannot be solely measured by traditional metrics like citation counts as these have a lag time and are largely focused on an academic audience. There is increasing recognition to consider 'alternative metrics' or altmetrics to measure more immediate and broader impacts of research. Better understanding of altmetrics can help researchers better navigate evolving information environments and changing appetites for different types of research.

Objectives

Our study aims to: 1) analyse the amount and medium of Altmetric coverage of health research produced by Irish organisations (2017 - 2023), identifying changes over time and 2) investigate differences in the amount of coverage between clinical areas (e.g., nutrition vs. neurology).

Methods

Using Altmetric institutional access, we will gather data on research outputs published 1 January 2017 through 31 December 2023 from active Irish organisations with Research Organisation Registry (ROR) IDs. Outputs will be deduplicated and stratified by their Australian and New Zealand Standard Research Classification relating to ≥1 field of health research: Biological Sciences, Biomedical and Clinical Sciences, Chemical Sciences, Health Sciences, and Psychology. We will clean data using R and perform descriptive analyses, establishing counts and frequencies of coverage by clinical area and medium (e.g., traditional news, X, etc.); data will be plotted on a yearly and quarterly basis where appropriate.

Results and Conclusions

Improved understanding of one's information environment can help researchers better navigate their local landscapes and identify pathways for more effective communication to the public. All R code will be made available open-source, allowing researchers to adapt it to evaluate their local landscapes.

An evaluation of the preprints produced at the beginning of the 2022 mpox public health emergency

BACKGROUND

Preprints are scientific articles that have not undergone the peer-review process. They allow the latest evidence to be rapidly shared, however it is unclear whether they can be confidently used for decision-making during a public health emergency. This study aimed to compare the data and quality of preprints released during the first four months of the 2022 mpox outbreak to their published versions.

METHODS

Eligible preprints (n = 76) posted between May to August 2022 were identified through an established mpox literature database and followed to July 2024 for changes in publication status. Quality of preprints and published studies was assessed by two independent reviewers to evaluate changes in quality, using validated tools that were available for the study design (n = 33). Tools included the Newcastle-Ottawa Scale; Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2); and JBI Critical Appraisal Checklists. The questions in each tool led to an overall quality assessment of high quality (no concerns with study design, conduct, and/or analysis), moderate quality (minor concerns) or low quality (several concerns). Changes in data (e.g. methods, outcomes, results) for preprint-published pairs (n = 60) were assessed by one reviewer and verified by a second.

RESULTS

Preprints and published versions that could be evaluated for quality (n = 25 pairs) were mostly assessed as low quality. Minimal to no change in quality from preprint to published was identified: all observational studies (10/10), most case series (6/7) and all surveillance data analyses (3/3) had no change in overall quality, while some diagnostic test accuracy studies (3/5) improved or worsened their quality assessment scores. Among all pairs (n = 60), outcomes were often added in the published version (58%) and less commonly removed (18%). Numerical results changed from preprint to published in 53% of studies, however most of these studies (22/32) had changes that were minor and did not impact main conclusions of the study.

CONCLUSIONS

This study suggests the minimal changes in quality, results and main conclusions from preprint to published versions supports the use of preprints, and the use of the same critical evaluation tools on preprints as applied to published studies, in decision-making during a public health emergency.

When are predictions useful? A new method for evaluating epidemic forecasts

BACKGROUND

COVID-19 will not be the last pandemic of the twenty-first century. To better prepare for the next one, it is essential that we make honest appraisals of the utility of different responses to COVID. In this paper, we focus specifically on epidemiologic forecasting. Characterizing forecast efficacy over the history of the pandemic is challenging, especially given its significant spatial, temporal, and contextual variability. In this light, we introduce the Weighted Contextual Interval Score (WCIS), a new method for retrospective interval forecast evaluation.

METHODS

The central tenet of the WCIS is a direct incorporation of contextual utility into the evaluation. This necessitates a specific characterization of forecast efficacy depending on the use case for predictions, accomplished via defining a utility threshold parameter. This idea is generalized to probabilistic interval-form forecasts, which are the preferred prediction format for epidemiological modeling, as an extension of the existing Weighted Interval Score (WIS).

RESULTS

We apply the WCIS to two forecasting scenarios: facility-level hospitalizations for a single state, and state-level hospitalizations for the whole of the United States. We observe that an appropriately parameterized application of the WCIS captures both the relative quality and the overall frequency of useful forecasts. Since the WCIS represents the utility of predictions using contextual normalization, it is easily comparable across highly variable pandemic scenarios while remaining intuitively representative of the in-situ quality of individual forecasts.

CONCLUSIONS

The WCIS provides a pragmatic utility-based characterization of probabilistic predictions. This method is expressly intended to enable practitioners and policymakers who may not have expertise in forecasting but are nevertheless essential partners in epidemic response to use and provide insightful analysis of predictions. We note that the WCIS is intended specifically for retrospective forecast evaluation and should not be used as a minimized penalty in a competitive context as it lacks statistical propriety. Code and data used for our analysis are available at https://github.com/maximilian-marshall/wcis .

A SONAR report on Nirmatrelvir/ritonavir-associated rebound COVID-19: Using new databases for evaluating new diseases

INTRODUCTION

In May 2022, the Centers for Disease Control and Prevention disseminated an alert advising that "a few" persons with Nirmatrelvir/ritonavir (NM/R)-associated rebound of COVID-19 infection had been identified. Three case reports appearing as pre-print postings described the first cases. Analyses in March 2023 by NM/R's manufacturer and the Food and Drug Administration (FDA) reported no association between NM/R and COVID-19 rebound in a large phase 3 randomized clinical trial. Our study evaluated if social media databases or electronically disseminated new articles might provide insights related to the putative new toxicity, NM/R-associated COVID-19 rebound.

METHODS

Information on NM/R-associated COVID-19 rebound cases was abstracted from preprint postings of non-peer-reviewed manuscripts, social media websites, electronically disseminated print and television media reports, a new FDA adverse event database for drugs that received Emergency Use Approval, and news articles in scientific journals.

RESULTS

Thirty-five persons experienced presumed or documented NM/R-associated COVID-19 rebound, based on information described in preprint services (n = 27), Twitter postings and related news articles (n = 7), and news articles without related Twitter reports (n = 1). These reports included information on dates of initial COVID-19 illness and rebound onset, COVID-19 testing, vaccine status, presentation, and outcome. A new FDA safety database identified 12,500 possible cases of this toxicity, but the quality of these data was poor. Preprint postings preceded peer-reviewed publications describing the same cases by four months. Social media websites including Instagram, Reddit, YouTube, the Center for Disease Control and Prevention's (CDC) Health Alert Network, CDC Twitter, and Facebook did not provide clinically meaningful information on individual cases.

CONCLUSION

Preprint services and Twitter facilitated identification of the largest case series of NM/R-associated COVID-19 rebound. The cases were reported in non-peer-reviewed media several weeks prior to the first peer-reviewed electronically disseminated publication of one person with this diagnosis.

Describing the Landscape of Medical Education Preprints on MedRxiv: Current Trends and Future Recommendations

PURPOSE

A preprint is a version of a research manuscript posted to a preprint server prior to peer review. Preprints enable authors to quickly and openly share research, afford opportunities for expedient feedback, and enable immediate listing of research on grant and promotion applications. In medical education, most journals welcome preprints, which suggests that preprints play a role in the field's discourse. Yet, little is known about medical education preprints, including author characteristics, preprint use, and ultimate publication status. This study provides an overview of preprints in medical education to better understand their role in the field's discourse.

METHOD

The authors queried medRxiv, a preprint repository, to identify preprints categorized as "medical education" and downloaded related metadata. CrossRef was queried to gather information on preprints later published in journals. Data were analyzed using descriptive statistics.

RESULTS

Between 2019 and 2022, 204 preprints were classified in medRxiv as "medical education," with most deposited in 2021 (n = 76; 37.3%). On average, preprint full-texts were downloaded 1,875.2 times, and all were promoted on social media. Preprints were authored, on average, by 5.9 authors. Corresponding authors were based in 41 countries, with 45.6% in the United States, the United Kingdom, and Canada. Almost half (n = 101; 49.5%) became published articles in predominantly peer-reviewed journals. Preprints appeared in 65 peer-reviewed journals, with BMC Medical Education (n = 9; 8.9%) most represented.

CONCLUSIONS

Medical education research is being deposited as preprints, which are promoted, heavily accessed, and subsequently published in peer-reviewed journals, including medical education journals. Considering the benefits of preprints and the slowness of medical education publishing, it is likely that preprint depositing will increase and preprints will be integrated into the field's discourse. The authors propose next steps to facilitate responsible and effective creation and use of preprints.

Preprint pointers from a long COVID scoping review: considerations for source selection and searching

This paper describes the search approach for preprints for a post COVID-19 condition (i.e., long COVID) scoping review, including source selection, search strategy development, challenges, and insights throughout a project life cycle. With the growth of medical preprints since the COVID-19 pandemic, information professionals and researchers should be aware that preprints are possible sources of evidence and be prepared to manage them in evidence reviews for COVID-19 topics and beyond. Preprints are not peer-reviewed but can include important evidence about emerging topics. Because of the importance of preprints to the scoping review, a preprint search of Europe PubMed Central (PMC) was added. Europe PMC and similar aggregators combine multiple preprint servers and often have Boolean search, but sometimes limited search functionalities or few export options. Strategy translation encountered challenges such as varying and inconsistent terminology for post-COVID-19 condition, a complex search, and negotiating large numbers of preprints with resource constraints. Europe PMC identified additional preprints for inclusion due to additional preprint server coverage. It was helpful to limit the preprint search to the title and abstract fields, and to run an extra Internet search for publication of included study preprints. Challenges and potential solutions are summarized to support those conducting preprint searches for COVID-19 and other topics.

Do good things really come to those who wait? An analysis of the average time of acceptance in Brazilian surgery journals

PURPOSE

To analyze the average time between submission and acceptance of national journals in seven Brazilian surgery journals from 2017 to 2022.

METHODS

It consists of a cross-sectional and observational study with a quantitative approach to analyze the acceptance time of articles approved by Brazilian journals on general surgery and its subspecialties, including Acta Cirúrgica Brasileira, Jornal Vascular Brasileiro, Arquivos Brasileiros de Cirurgia de Digestiva, Revista do Colégio Brasileiro de Cirurgiões, Journal of Coloproctology, Revista Brasileira de Cirurgia Plástica, and International Brazilian Journal of Urology.

RESULTS

The journals with the lowest average waiting times were Revista do Colégio Brasileiro de Cirurgiões, Acta Cirúrgica Brasileira, and Journal of Coloproctology, respectively, and, with the lowest interquartile range there is Acta Cirúrgica Brasileira. There was no significant difference between the pre-pandemic and pandemic periods. The study designs with the highest and lowest means were, respectively, ideas and innovations - also with the highest interquartile range - and expert opinion, while with the lowest interquartile range was technical skill.

CONCLUSIONS

The acceptance time for articles in Brazilian surgery journals is extremely variable. Identifying these discrepancies highlights the importance of understanding editorial processes and seeking ways to improve consistency and efficiency in reviewing articles.

Altmetric Attention Scores and Citations of Published Research With or Without Preprints

This cross-sectional study assesses how frequently research articles published in the clinical journals with high impact factors are preprinted and whether preprinting is associated with changes in media attention and citation counts.

Messing with Merton: The intersection between open science practices and Mertonian values

Although adherence to Mertonian values of science (i.e., communism, universalism, organized skepticism, disinterestedness) is desired and promoted in academia, such adherence can cause friction with the normative structures and practices of Open Science. Mertonian values and Open Science practices aim to improve the conduct and communication of research and are promoted by institutional actors. However, Mertonian values remain mostly idealistic and contextualized in local and disciplinary cultures and Open Science practices rely heavily on third-party resources and technology that are not equally accessible to all parties. Furthermore, although still popular, Mertonian values were developed in a different institutional and political context. In this article, we argue that new normative structures for science need to look beyond nostalgia and consider aspirations and outcomes of Open Science practices. To contribute to such a vision, we explore the intersection of several Open Science practices with Mertonian values to flesh out challenges involved in upholding these values. We demonstrate that this intersection becomes complicated when the interests of numerous groups collide and contrast. Acknowledging and exploring such tensions informs our understanding of researchers' behavior and supports efforts that seek to improve researchers' interactions with other normative structures such as research ethics and integrity frameworks.

Features of databases that supported searching for rapid evidence synthesis during COVID-19: implications for future public health emergencies

BACKGROUND

As evidence related to the COVID-19 pandemic surged, databases, platforms, and repositories evolved with features and functions to assist users in promptly finding the most relevant evidence. In response, research synthesis teams adopted novel searching strategies to sift through the vast amount of evidence to synthesize and disseminate the most up-to-date evidence. This paper explores the key database features that facilitated systematic searching for rapid evidence synthesis during the COVID-19 pandemic to inform knowledge management infrastructure during future global health emergencies.

METHODS

This paper outlines the features and functions of previously existing and newly created evidence sources routinely searched as part of the NCCMT's Rapid Evidence Service methods, including databases, platforms, and repositories. Specific functions of each evidence source were assessed as they pertain to searching in the context of a public health emergency, including the topics of indexed citations, the level of evidence of indexed citations, and specific usability features of each evidence source.

RESULTS

Thirteen evidence sources were assessed, of which four were newly created and nine were either pre-existing or adapted from previously existing resources. Evidence sources varied in topics indexed, level of evidence indexed, and specific searching functions.

CONCLUSION

This paper offers insights into which features enabled systematic searching for the completion of rapid reviews to inform decision makers within 5-10 days. These findings provide guidance for knowledge management strategies and evidence infrastructures during future public health emergencies.

"Spine to the future"-A narrative review of anatomy engagement

Anatomy has been integral to medical and health education for centuries, it has also had a significant role in wider public life, as an educational resource, a link to their health, and also as a darker deterrent. Historically, public engagement in anatomy is hallmarked by public dissections of convicted criminals across the globe. Artists, specifically non-medical men, such as Leonardo da Vinci, are reported to have participated in public dissection. Dissection would later rekindle public interest in anatomy as graverobbing led to the reform and regulation of anatomy in many countries. In recent years, there has been growing interest from the public in learning more about their bodies as health and well-being become of paramount importance, particularly following the COVID-19 pandemic. Anatomy sits in a prime position to direct and instigate conversations around health, well-being, and body image. Every human on earth possesses a perfect resource to look at and learn about. Models, art-based anatomical activities, and crafts provide active learning opportunities for the wider public around anatomy. Most recently, apps, games, and extended reality provide novel and insightful learning opportunities for the public relating to the body. Finally, training and resources must also be made available from institutions and professional bodies to anatomists to enable them to deliver engagement in an already congested and educationally heavy schedule. This resurgence of interest in anatomical public engagement sees anatomy re-enter the public spotlight, with more appropriate resources and educational settings to offer engagement with the aim of benefiting the public.

Demystifying Open Science in health psychology and behavioral medicine: a practical guide to Registered Reports and Data Notes

Open Science practices are integral to increasing transparency, reproducibility, and accessibility of research in health psychology and behavioral medicine. Drives to facilitate Open Science practices are becoming increasingly evident in journal editorial policies, including the establishment of new paper formats such as Registered Reports and Data Notes. This paper provides: (i) an overview of the current state of Open Science policies within health psychology and behavioral medicine, (ii) a call for submissions to an Article Collection of Registered Reports and Data Notes as new paper formats within the journal of Health Psychology & Behavioral Medicine, (iii) an overview of Registered Reports and Data Notes, and (iv) practical considerations for authors and reviewers of Registered Reports and Data Notes.

Exploring the current dynamics of preprints

Preprints are non-peer-reviewed and publicly available articles for open and transparent research communication. Preprint servers host the submission of such manuscripts, and despite the presence of established preprint servers, their numbers have continued to rise in recent times. A steep increasing pattern in posted preprints and their accommodating servers has been observed over the last decade. In this article, we explored the global trends in the preprint adoption and its involvement in promoting open and transparent research findings across various domains. We further emphasized the importance of preprinting, highlighting its significant impact during the pandemic through effective information sharing, and advocating for its broader integration in scholarly communication.

Endorsement of open science practices by dental journals: A meta-research study

OBJECTIVES

This study evaluates the endorsement of open science practices by dental journals.

MATERIALS AND METHODS

This was a meta-research study that included journals listed in the 2021 Journal Citation Reports under Dentistry. A comprehensive evaluation was performed by accessing journal websites to ascertain the availability of publicly accessible instructions to authors in Portuguese, English, or Spanish. A researcher extracted information from the "Instructions for Authors" section, encompassing the journal's impact factor, mention of any reporting guidelines, details on data sharing, acceptance of articles in preprint format, and information regarding study protocol registration. Descriptive data analysis was conducted using the Stata 14.0 program, and an Open Science Score (OSS) (ranging from 0 to 100 %) was calculated for each journal by considering five open science practices. Pearson's correlation test was conducted to determine the relationship between the OSS score and journal impact factor.

RESULTS

Ninety journals were included in the study. Most journals (70 %) indicated the mandatory use of reporting guidelines, while 60 % recommended data sharing. Conversely, 46.7 % did not provide information on study protocol registration, and 44.4 % stipulated them as mandatory for authors. Regarding preprints, 50 % of the journals did not provide any information, but 46.7 % confirmed their acceptance. The mean OSS was 52.9 % (standard deviation 26.2). There was a weak correlation (Pearson's correlation coefficient of 0.221) between the journal impact factor and OSS (P-value=0.036).

CONCLUSION

This study found varying degrees of endorsement of open science practices among dental journals.

CLINICAL SIGNIFICANCE

Dental practitioners rely on high-quality, evidence-based research for informed decision-making. By assessing the endorsement of open science practices, our study contributes to improving the quality and reliability of dental research, ultimately enhancing the evidence base for clinical practice.

Now Is the Time to Strengthen Government-Academic Data Infrastructures to Jump-Start Future Public Health Crisis Response

During public health crises, the significance of rapid data sharing cannot be overstated. In attempts to accelerate COVID-19 pandemic responses, discussions within society and scholarly research have focused on data sharing among health care providers, across government departments at different levels, and on an international scale. A lesser-addressed yet equally important approach to sharing data during the COVID-19 pandemic and other crises involves cross-sector collaboration between government entities and academic researchers. Specifically, this refers to dedicated projects in which a government entity shares public health data with an academic research team for data analysis to receive data insights to inform policy. In this viewpoint, we identify and outline documented data sharing challenges in the context of COVID-19 and other public health crises, as well as broader crisis scenarios encompassing natural disasters and humanitarian emergencies. We then argue that government-academic data collaborations have the potential to alleviate these challenges, which should place them at the forefront of future research attention. In particular, for researchers, data collaborations with government entities should be considered part of the social infrastructure that bolsters their research efforts toward public health crisis response. Looking ahead, we propose a shift from ad hoc, intermittent collaborations to cultivating robust and enduring partnerships. Thus, we need to move beyond viewing government-academic data interactions as 1-time sharing events. Additionally, given the scarcity of scholarly exploration in this domain, we advocate for further investigation into the real-world practices and experiences related to sharing data from government sources with researchers during public health crises.

Confidence score: a data-driven measure for inclusive systematic reviews considering unpublished preprints

OBJECTIVES

COVID-19, since its emergence in December 2019, has globally impacted research. Over 360 000 COVID-19-related manuscripts have been published on PubMed and preprint servers like medRxiv and bioRxiv, with preprints comprising about 15% of all manuscripts. Yet, the role and impact of preprints on COVID-19 research and evidence synthesis remain uncertain.

MATERIALS AND METHODS

We propose a novel data-driven method for assigning weights to individual preprints in systematic reviews and meta-analyses. This weight termed the "confidence score" is obtained using the survival cure model, also known as the survival mixture model, which takes into account the time elapsed between posting and publication of a preprint, as well as metadata such as the number of first 2-week citations, sample size, and study type.

RESULTS

Using 146 preprints on COVID-19 therapeutics posted from the beginning of the pandemic through April 30, 2021, we validated the confidence scores, showing an area under the curve of 0.95 (95% CI, 0.92-0.98). Through a use case on the effectiveness of hydroxychloroquine, we demonstrated how these scores can be incorporated practically into meta-analyses to properly weigh preprints.

DISCUSSION

It is important to note that our method does not aim to replace existing measures of study quality but rather serves as a supplementary measure that overcomes some limitations of current approaches.

CONCLUSION

Our proposed confidence score has the potential to improve systematic reviews of evidence related to COVID-19 and other clinical conditions by providing a data-driven approach to including unpublished manuscripts.

Health communication on social media at the early stage of the pandemic: Examining health professionals' COVID-19 related tweets

Focusing on health professionals' tweets regarding COVID-19, this study examines whether and how those tweets are unique based on their identity as health experts. The data revealed that the infusion of health communication with political opinions, whether pro- or against certain political parties or health policies, reflects values and may deviate from the original purpose of health communication. In addition, sentiment analysis countered the intuitive thought that health experts merely fulfill their role as neutral encyclopedias without excessively carrying sentiment. We conclude by reflecting on the meaning of health communication in relation to the political stances of professionals.

Framing COVID-19 Preprint Research as Uncertain: A Mixed-Method Study of Public Reactions

During the COVID-19 pandemic, journalists were encouraged to convey uncertainty surrounding preliminary scientific evidence, including mentioning when research is unpublished or unverified by peer review. To understand how public audiences interpret this information, we conducted a mixed method study with U.S. adults. Participants read a news article about preprint COVID-19 vaccine research in early April 2021, just as the vaccine was becoming widely available to the U.S. public. We modified the article to test two ways of conveying uncertainty (hedging of scientific claims and mention of preprint status) in a 2 × 2 between-participants factorial design. To complement this, we collected open-ended data to assess participants' understanding of the concept of a scientific preprint. In all, participants who read hedged (vs. unhedged) versions of the article reported less favorable vaccine attitudes and intentions and found the scientists and news reporting less trustworthy. These effects were moderated by participants' epistemic beliefs and their preference for information about scientific uncertainty. However, there was no impact of describing the study as a preprint, and participants' qualitative responses indicated a limited understanding of the concept. We discuss implications of these findings for communicating initial scientific evidence to the public and we outline important next steps for research and theory-building.

How the COVID-19 pandemic signaled the demise of Antarctic exceptionalism

This paper explores how the COVID-19 pandemic affected science and tourism activities and their governance in the Antarctic and Southern Ocean. The pandemic reduced the ability of Antarctic Treaty Parties to make decisions on policy issues and placed a considerable burden on researchers. Tourism was effectively suspended during the 2020-2021 Antarctic season and heavily reduced in 2021-2022 but rebounded to record levels in 2022-2023. The pandemic stimulated reflection on practices to facilitate dialog, especially through online events. Opportunities arose to integrate innovations developed during the pandemic more permanently into Antarctic practices, in relation to open science, reducing operational greenhouse gas footprints and barriers of access to Antarctic research and facilitating data sharing. However, as well as the long-term impacts arising directly from the pandemic, an assemblage of major geopolitical drivers are also in play and, combined, these signal a considerable weakening of Antarctic exceptionalism in the early Anthropocene.

Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study

OBJECTIVES

Numerous systematic reviews (SRs) have been published in the first months of the COVID-19 pandemic and clinical trials were designed rapidly highlighting the importance of informative implications for research (IfRs) sections in SRs. IfR is one item of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 checklist and the Cochrane Handbook suggests considering population, intervention, control, outcome (PICO) and Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) domains when developing IfR. We aimed (1) to assess whether SRs on COVID-19 treatments included any IfR statements and, for SRs with an IfR statement, (2) to examine which elements informed the IfR statement.

STUDY DESIGN AND SETTING

We conducted a metaresearch study based on SRs on COVID-19 treatment identified in the Living OVerview of the Evidence COVID-19 database in May 2021 as part of another research project (CRD42021240423). We defined an IfR statement as at least one sentence that contained at least one bit of information that could be informative for planning future research. We extracted any IfR statements anywhere in the SRs on predefined IfR variables, in particular PICO elements, study design, and concepts underlying GRADE domains. Three authors extracted data independently after piloting the data extraction form. We resolved discrepancies in weekly discussions to ensure a high-quality data extraction.

RESULTS

We included 326 SRs, of which 284 SRs (87.1%) stated IfR. Of these 284 SRs, 201 (70.8%) reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses and 66 (23.2%) using GRADE. IfR statements (n = 284) addressing PICO were unstructured and commonly reported 'population' (n = 195, 68.7%), 'intervention' (n = 242, 85.2%), and 'outcome' (n = 127, 44.7%) but not 'control' (n = 29, 10.2%). Concepts underlying GRADE domains were infrequently reported in IfR statements of SRs (n = 284): 'risk of bias' (n = 14, 4.9%), 'imprecision' (n = 8, 2.8%), 'inconsistency' (n = 7, 2.5%), 'publication bias' (n = 3, 1.1%), and 'indirectness' (n = 1, 0.4%). Additional IfR elements mentioned in IfR were 'better reporting' of future studies (n = 17, 6.0%) and 'standardization of procedures in clinical trials' (n = 12, 4.2%).

CONCLUSION

Almost 90% of SRs on COVID-19 treatments reported IfR. IfR statements addressing PICO were unstructured across SRs and concepts underlying GRADE were rarely reported to inform IfR. Further work is needed to assess generalizability beyond COVID-19 and to define more precisely which IfR elements should be considered, and how they should be reported in SRs of interventions. Until then, considering PICO elements and concepts underlying GRADE to derive IfR seems to be a sensible starting point.

Making science public: a review of journalists' use of Open Access research

Science journalists are uniquely positioned to increase the societal impact of open research outputs by contextualizing and communicating findings in ways that highlight their relevance and implications for non-specialist audiences. Yet, it is unclear to what degree journalists use open research outputs, such as open access publications or preprints, in their reporting; what factors motivate or constrain this use; and how the recent surge in openly available research seen during the COVID-19 pandemic has affected this. This article examines these questions through a review of relevant literature published from 2018 onwards-particularly literature relating to the COVID-19 pandemic-as well as seminal articles outside the search dates. We find that research that explicitly examines journalists' engagement with open access publications or preprints is scarce, with existing literature mostly addressing the topic tangentially or as a secondary concern, rather than a primary focus. Still, the limited body of evidence points to several factors that may hamper journalists' use of these outputs and thus warrant further exploration. These include an overreliance on traditional criteria for evaluating scientific quality; concerns about the trustworthiness of open research outputs; and challenges using and verifying the findings. We also find that, while the COVID-19 pandemic encouraged journalists to explore open research outputs such as preprints, the extent to which these explorations will become established journalistic practices remains unclear. Furthermore, we note that current research is overwhelmingly authored and focused on the Global North, and the United States specifically. We conclude with recommendations for future research that attend to issues of equity and diversity, and more explicitly examine the intersections of open access and science journalism.

Medical Doctors' Perceptions of the Media Coverage during the Covid-19 Pandemic: A Case Study in Stockholm

The strain on healthcare systems including emergency departments increased substantially during the Covid-19 pandemic,negatively affecting healthcare workers and their well-being. The emotional distress experienced by healthcare staff during the pandemic was worsened by confusion and conspiracy theories that circulated in the news and online media. Reports on the pandemic and general consumption of media intensified as the public's demand for information increased. There is limited research on how doctors perceived media coverage, and how they were affected in their work. This study aimed to explore how medical doctors in emergency departments perceived the media coverage during the Covid-19 pandemic. Twelve doctors at two different emergency departments in Stockholm, Sweden, participated. Interview questions on media were asked as part of a more extensive questionnaire. Informants' responses were analysed qualitatively. The results indicate that doctors to some extent used media as a source of information, due to limited access to knowledge about the virus. Results further suggest that media coverage triggered fear of infection, caused worry and job strain. The doctors percieved that the media coverage on Covid-19 affected patient-seeking behaviour as well as the doctor-patient relationship. The findings can be relevant in preparation for future pandemics and considered in development of policy for media and emergency departments.

Open reproducible scientometric research with Alexandria3k

Considerable scientific work involves locating, analyzing, systematizing, and synthesizing other publications, often with the help of online scientific publication databases and search engines. However, use of online sources suffers from a lack of repeatability and transparency, as well as from technical restrictions. Alexandria3k is a Python software package and an associated command-line tool that can populate embedded relational databases with slices from the complete set of several open publication metadata sets. These can then be employed for reproducible processing and analysis through versatile and performant queries. We demonstrate the software's utility by visualizing the evolution of publications in diverse scientific fields and relationships among them, by outlining scientometric facts associated with COVID-19 research, and by replicating commonly-used bibliometric measures and findings regarding scientific productivity, impact, and disruption.

Dissemination of Registered COVID-19 Clinical Trials (DIRECCT): a cross-sectional study

BACKGROUND

The results of clinical trials should be completely and rapidly reported during public health emergencies such as COVID-19. This study aimed to examine when, and where, the results of COVID-19 clinical trials were disseminated throughout the first 18 months of the pandemic.

METHODS

Clinical trials for COVID-19 treatment or prevention were identified from the WHO ICTRP database. All interventional trials with a registered completion date ≤ 30 June 2021 were included. Trial results, published as preprints, journal articles, or registry results, were located using automated and manual techniques across PubMed, Google Scholar, Google, EuropePMC, CORD-19, the Cochrane COVID-19 Study Register, and clinical trial registries. Our main analysis reports the rate of dissemination overall and per route, and the time from registered completion to results using Kaplan-Meier methods, with additional subgroup and sensitivity analyses reported.

RESULTS

Overall, 1643 trials with completion dates ranging from 46 to 561 days prior to the start of results searches were included. The cumulative probability of reporting was 12.5% at 3 months from completion, 21.6% at 6 months, and 32.8% at 12 months. Trial results were most commonly disseminated in journals (n = 278 trials, 69.2%); preprints were available for 194 trials (48.3%), 86 (44.3%) of which converted to a full journal article. Trials completed earlier in the pandemic were reported more rapidly than those later in the pandemic, and those involving ivermectin were more rapidly reported than other common interventions. Results were robust to various sensitivity analyses except when considering only trials in a "completed" status on the registry, which substantially increased reporting rates. Poor trial registry data on completion status and dates limits the precision of estimates.

CONCLUSIONS

COVID-19 trials saw marginal increases in reporting rates compared to standard practice; most registered trials failed to meet even the 12-month non-pandemic standard. Preprints were common, complementing journal publication; however, registries were underutilized for rapid reporting. Maintaining registry data enables accurate representation of clinical research; failing to do so undermines these registries' use for public accountability and analysis. Addressing rapid reporting and registry data quality must be emphasized at global, national, and institutional levels.

Comparison of COVID-19 Preprint and Peer-Reviewed Versions of Studies on Therapies for Critically Ill Patients

PURPOSE

Significant increases in the volume of preprint articles due to the COVID-19 pandemic, we examined the reliability of preprint articles compared to their peer-reviewed publications.

MATERIALS AND METHODS

Preprint articles evaluating experimental studies of select treatment options (anticoagulation, dexamethasone, hydroxychloroquine, remdesivir, and tocilizumab) for COVID-19 in the critically ill, available in a peer-reviewed publication were screened for inclusion within Altmetric (n = 2040). A total of 40 articles met inclusion criteria, with 21 being randomly selected for evaluation. The primary outcome of this evaluation was a change in a study's reported primary outcome or statistical significance between preprint and peer-reviewed articles. Secondary outcomes included changes in primary/secondary outcome effect size and change in study conclusion.

RESULTS

One article (4.8%, 95% CI 0.12%-23.8%) had a change in the primary outcome. Seven articles (33.3%, 95% CI 14.6%-57.0%) had a change in the primary outcome's effect measure. Five studies (23.8%, 95% CI 8.2%-47.2%) had changes in statistical significance of at least one secondary outcome. Four studies (19.0%, 95% CI 5.4%-41.9%) had a change in study conclusion.

CONCLUSIONS

In preprint articles of COVID-19 treatments, the provided primary outcome is generally reliable, while interpretation of secondary outcomes should be made with caution, while awaiting completion of the peer-review process.

How have mathematical models contributed to understanding the transmission and control of SARS-CoV-2 in healthcare settings? A systematic search and review

Since the onset of the COVID-19 pandemic, mathematical models have been widely used to inform public health recommendations regarding COVID-19 control in healthcare settings. The objective of this study was to systematically review SARS-CoV-2 transmission models in healthcare settings, and to summarize their contributions to understanding nosocomial COVID-19. A systematic search and review of published articles indexed in PubMed was carried out. Modelling studies describing dynamic inter-individual transmission of SARS-CoV-2 in healthcare settings, published by mid-February 2022 were included. Models have mostly focused on acute-care and long-term-care facilities in high-income countries. Models have quantified outbreak risk, showing great variation across settings and pandemic periods. Regarding surveillance, routine testing rather than symptom-based was highlighted as essential for COVID-19 prevention due to high rates of silent transmission. Surveillance impacts depended critically on testing frequency, diagnostic sensitivity, and turn-around time. Healthcare re-organization also proved to have large epidemiological impacts: beyond obvious benefits of isolating cases and limiting inter-individual contact, more complex strategies (staggered staff scheduling, immune-based cohorting) reduced infection risk. Finally, vaccination impact, while highly effective for limiting COVID-19 burden, varied substantially depending on assumed mechanistic impacts on infection acquisition, symptom onset and transmission. Modelling results form an extensive evidence base that may inform control strategies for future waves of SARS-CoV-2 and other viral respiratory pathogens. We propose new avenues for future models of healthcare-associated outbreaks, with the aim of enhancing their efficiency and contributions to decision-making.

Inter-rater reliability of the infectious disease modeling reproducibility checklist (IDMRC) as applied to COVID-19 computational modeling research

BACKGROUND

Infectious disease computational modeling studies have been widely published during the coronavirus disease 2019 (COVID-19) pandemic, yet they have limited reproducibility. Developed through an iterative testing process with multiple reviewers, the Infectious Disease Modeling Reproducibility Checklist (IDMRC) enumerates the minimal elements necessary to support reproducible infectious disease computational modeling publications. The primary objective of this study was to assess the reliability of the IDMRC and to identify which reproducibility elements were unreported in a sample of COVID-19 computational modeling publications.

METHODS

Four reviewers used the IDMRC to assess 46 preprint and peer reviewed COVID-19 modeling studies published between March 13th, 2020, and July 30th, 2020. The inter-rater reliability was evaluated by mean percent agreement and Fleiss' kappa coefficients (κ). Papers were ranked based on the average number of reported reproducibility elements, and average proportion of papers that reported each checklist item were tabulated.

RESULTS

Questions related to the computational environment (mean κ = 0.90, range = 0.90-0.90), analytical software (mean κ = 0.74, range = 0.68-0.82), model description (mean κ = 0.71, range = 0.58-0.84), model implementation (mean κ = 0.68, range = 0.39-0.86), and experimental protocol (mean κ = 0.63, range = 0.58-0.69) had moderate or greater (κ > 0.41) inter-rater reliability. Questions related to data had the lowest values (mean κ = 0.37, range = 0.23-0.59). Reviewers ranked similar papers in the upper and lower quartiles based on the proportion of reproducibility elements each paper reported. While over 70% of the publications provided data used in their models, less than 30% provided the model implementation.

CONCLUSIONS

The IDMRC is the first comprehensive, quality-assessed tool for guiding researchers in reporting reproducible infectious disease computational modeling studies. The inter-rater reliability assessment found that most scores were characterized by moderate or greater agreement. These results suggest that the IDMRC might be used to provide reliable assessments of the potential for reproducibility of published infectious disease modeling publications. Results of this evaluation identified opportunities for improvement to the model implementation and data questions that can further improve the reliability of the checklist.

Full publication of preprint articles in prevention research: an analysis of publication proportions and results consistency

There is concern that preprint articles will lead to an increase in the amount of scientifically invalid work available. The objectives of this study were to determine the proportion of prevention preprints published within 12 months, the consistency of the effect estimates and conclusions between preprint and published articles, and the reasons for the nonpublication of preprints. Of the 329 prevention preprints that met our eligibility criteria, almost half (48.9%) were published in a peer-reviewed journal within 12 months of being posted. While 16.8% published preprints showed some change in the magnitude of the primary outcome effect estimate, 4.4% were classified as having a major change. The style or wording of the conclusion changed in 42.2%, the content in 3.1%. Preprints on chemoprevention, with a cross-sectional design, and with public and noncommercial funding had the highest probabilities of publication. The main reasons for the nonpublication of preprints were journal rejection or lack of time. The reliability of preprint articles for evidence-based decision-making is questionable. Less than half of the preprint articles on prevention research are published in a peer-reviewed journal within 12 months, and significant changes in effect sizes and/or conclusions are still possible during the peer-review process.

Lessons Learned from the Connecticut Response to COVID-19 in Nursing Homes during the First 2 Years of the Pandemic

Nearly half of all SARS-CoV-2-related deaths in the United States occurred in long-term care facilities during the early pandemic. In Connecticut, statewide mitigation of this impact involved a collaboration between the Connecticut Department of Public Health and the Yale School of Public Health, alongside existing relationships with the long-term care industry and individual facilities. This close government-academic-industry collaboration facilitated the creation of a robust COVID-19 surveillance system that allowed for real-time analysis and identification of nursing homes where outbreak support was needed. The collaboration further facilitated vaccine and booster deployment to Connecticut nursing homes at a speed that outpaced much of the country. The impact of these interventions is demonstrated through COVID-19 case and death burdens among nursing home residents and the greater Connecticut population during each wave of the pandemic. We outline the evolution and impact of these alliances and how they enabled us to prioritize facilities, interventions, and the distribution of limited resources and training throughout the pandemic. We further detail lessons learned over the first 2 years of the pandemic. Such partnerships strengthen our ability to respond effectively to public health crises and should be created and/or maintained in the face of continued pandemic threats.

SARS-CoV-2 post-acute sequelae in previously hospitalised patients: systematic literature review and meta-analysis

BACKGROUND

Many individuals hospitalised with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection experience post-acute sequelae of SARS-CoV-2 infection (PASC), sometimes referred to as "long COVID". Our objective was to conduct a systematic literature review and meta-analysis to identify PASC-associated symptoms in previously hospitalised patients and determine the frequency and temporal nature of PASC.

METHODS

Searches of MEDLINE, Embase, Cochrane Library (2019-2021), World Health Organization International Clinical Trials Registry Platform and reference lists were performed from November to December 2021. Articles were assessed by two reviewers against eligibility criteria and a risk of bias tool. Symptom data were synthesised by random effects meta-analyses.

RESULTS

Of 6942 records, 52 studies with at least 100 patients were analysed; ∼70% were Europe-based studies. Most data were from the first wave of the pandemic. PASC symptoms were analysed from 28 days after hospital discharge. At 1-4 months post-acute SARS-CoV-2 infection, the most frequent individual symptoms were fatigue (29.3% (95% CI 20.1-40.6%)) and dyspnoea (19.6% (95% CI 12.8-28.7%)). Many patients experienced at least one symptom at 4-8 months (73.1% (95% CI 44.2-90.3%)) and 8-12 months (75.0% (95% CI 56.4-87.4%)).

CONCLUSIONS

A wide spectrum of persistent PASC-associated symptoms were reported over the 1-year follow-up period in a significant proportion of participants. Further research is needed to better define PASC duration and determine whether factors such as disease severity, vaccination and treatments have an impact on PASC.

The use and acceptability of preprints in health and social care settings: A scoping review

BACKGROUND

Preprints are open and accessible scientific manuscript or report that is shared publicly, through a preprint server, before being submitted to a journal. The value and importance of preprints has grown since its contribution during the public health emergency of the COVID-19 pandemic. Funders and publishers are establishing their position on the use of preprints, in grant applications and publishing models. However, the evidence supporting the use and acceptability of preprints varies across funders, publishers, and researchers. The scoping review explored the current evidence on the use and acceptability of preprints in health and social care settings by publishers, funders, and the research community throughout the research lifecycle.

METHODS

A scoping review was undertaken with no study or language limits. The search strategy was limited to the last five years (2017-2022) to capture changes influenced by COVID-19 (e.g., accelerated use and role of preprints in research). The review included international literature, including grey literature, and two databases were searched: Scopus and Web of Science (24 August 2022).

RESULTS

379 titles and abstracts and 193 full text articles were assessed for eligibility. Ninety-eight articles met eligibility criteria and were included for full extraction. For barriers and challenges, 26 statements were grouped under four main themes (e.g., volume/growth of publications, quality assurance/trustworthiness, risks associated to credibility, and validation). For benefits and value, 34 statements were grouped under six themes (e.g., openness/transparency, increased visibility/credibility, open review process, open research, democratic process/systems, increased productivity/opportunities).

CONCLUSIONS

Preprints provide opportunities for rapid dissemination but there is a need for clear policies and guidance from journals, publishers, and funders. Cautionary measures are needed to maintain the quality and value of preprints, paying particular attention to how findings are translated to the public. More research is needed to address some of the uncertainties addressed in this review.

Exploring the use of preprints in dentistry

OBJECTIVE

This study aims to assess the use, impact, and dissemination of preprints in dentistry.

METHODS

This is a meta-research study with a cross-sectional design. We included preprints published in dentistry, regardless of the year of publication. Searches were performed in the medRxiv.org and Preprints.org platforms and restricted to English. One researcher extracted the data, and another researcher verified data consistency. The following data were extracted: year of publication, country of the corresponding author, number of abstract and full-text views and downloads, Altmetric attention score, whether the preprint was mentioned in other servers such as Twitter and Publons, number of mentions in other servers, number of citations in the Dimensions database, and whether the preprint had already been published in a peer-reviewed journal. If already published, we extracted the journal's impact factor (JCR 2021) and the number of citations in the Dimensions database. We conducted a descriptive analysis of the extracted characteristics and explored relationships between metrics using the Spearman correlation.

RESULTS

We identified 276 preprints. Most of the studies were published between 2020 and 2022 (n = 229), especially those from ten countries. The most-cited preprint and published article are the same study. Only the correlation between the number of preprint citations and peer-reviewed article citations in the Dimensions database showed a large positive association (Spearman's rho = 0.5809).

CONCLUSION

Preprints gained popularity over the last several years due to the COVID-19 pandemic and reached a larger audience, especially on platforms such as Twitter.

CLINICAL SIGNIFICANCE

Preprint publishing allows faster dissemination of science for the benefit of society.

The experiences of COVID-19 preprint authors: a survey of researchers about publishing and receiving feedback on their work during the pandemic

The COVID-19 pandemic caused a rise in preprinting, triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experiences with preprinting their work and also with publishing their work in a peer-reviewed journal. Our research had the following objectives: 1. to learn about authors' experiences with preprinting, their motivations, and future intentions; 2. to consider preprints in terms of their effectiveness in enabling authors to receive feedback on their work; 3. to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. In our survey, 78% of the new adopters of preprinting reported the intention to also preprint their future work. The boost in preprinting may therefore have a structural effect that will last after the pandemic, although future developments will also depend on other factors, including the broader growth in the adoption of open science practices. A total of 53% of the respondents reported that they had received feedback on their preprints. However, more than half of the feedback was received through "closed" channels-privately to the authors. This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting "open" channels for preprint feedback. Almost a quarter of the feedback received by respondents consisted of detailed comments, showing the potential of preprint feedback to provide valuable comments on research. Respondents also reported that, compared to preprint feedback, journal peer review was more likely to lead to major changes to their work, suggesting that journal peer review provides significant added value compared to feedback received on preprints.