Open science saves lives: lessons from the COVID-19 pandemic

Inverting social innovation to transform health system responses to climate change adaptation and mitigation in the global south

Systems thinking is aimed at understanding and solving complex problems that cut across sectors, an approach that requires accurate, timely, and multisectoral data. Citizen-driven big data can advance systems thinking, considering the widespread use of digital devices. Using digital platforms, data from these devices can transform health systems to predict and prevent global health crises and respond rapidly to emerging crises by providing citizens with real-time support. For example, citizens can obtain real-time support to help with public health risks via a digital app, which can predict evolving risks. These big data can be aggregated and visualized on digital dashboards, which can provide decision-makers with advanced data analytics to facilitate jurisdiction-level rapid responses to evolving climate change impacts (e.g., direct public health crisis communication). In the context of climate change, digital platforms can strengthen rapid responses by integrating information across systems (e.g., food, health, and social services) via citizen big data. More importantly, these big data can be used for rapid decision-making,a paradigm-changing approach that can invert social innovation, which we define as co-conceptualizing societal solutions with vulnerable communities to improve economic development with a focus on community wellbeing. However, to foster equitable and inclusive digital partnerships that invert social innovation, it is critical to avoid top-down approaches that sometimes result when researchers in the Global North and South collaborate. Equitable Global South-North partnerships can be built by combining digital citizen science and community-based participatory research to ethically leverage citizen-driven big data for rapid responses across international jurisdictions.

Challenges in conducting clinical research in primary care dentistry

The integration of dentistry into primary health care is crucial for promoting patient well-being. However, clinical studies in dentistry face challenges, including issues with study design, transparency, and relevance to primary care. Clinical trials in dentistry often focus on specific issues with strict eligibility criteria, limiting the generalizability of findings. Randomized clinical trials (RCTs) face challenges in reflecting real-world conditions and using clinically relevant outcomes. The need for more pragmatic approaches and the inclusion of clinically relevant outcomes (CROs) is discussed, such as tooth loss or implant success. Solutions proposed include well-controlled observational studies, optimized data collection tools, and the integration of artificial intelligence (AI) for predictive modelling, computer-aided diagnostics and automated diagnosis. In this position paper advocates for more efficient trials with a focus on patient-centred outcomes, as well as the adoption of pragmatic study designs reflecting real-world conditions. Collaborative research networks, increased funding, enhanced data retrieval, and open science practices are also recommended. Technology, including intraoral scanners and AI, is highlighted for improving efficiency in dental research. AI is seen as a key tool for participant recruitment, predictive modelling, and outcome evaluation. However, ethical considerations and ongoing validation are emphasized to ensure the reliability and trustworthiness of AI-driven solutions in dental research. In conclusion, the efficient conduct of clinical research in primary care dentistry requires a comprehensive approach, including changes in study design, data collection, and analytical methods. The integration of AI is seen as pivotal in achieving these objectives in a meaningful and efficient way.

Immunotherapy and Cannabis: A Harmful Drug Interaction or Reefer Madness?

A retrospective (N = 140) and a prospective (N = 102) observational Israeli study by Bar-Sela and colleagues about cannabis potentially adversely impacting the response to immunotherapy have together been cited 202 times, including by clinical practice guidelines. There have also been concerns on PubPeer outlining irregularities and unverifiable information in their statistics and numerous errors in calculating percentages. This reanalysis attempted to verify the data analysis while including non-parametric statistics. The corrected prospective report contained 22 p-values, but only one (4.5%) could be verified despite the authors being transparent about the N and statistics employed. Cannabis users were significantly (p < 0.0025) younger than non-users, but this was not reported in the retrospective report. There were also errors in percentage calculations (e.g., 13/34 reported as 22.0% instead of 38.2%). Overall, these observational investigations, and especially the prospective, appear to contain gross inaccuracies which could impact the statistical decisions (i.e., significant findings reported as non-significant or vice-versa). Although it is mechanistically plausible that cannabis could have immunosuppressive effects which inhibit the response to immunotherapy, these two reports should be viewed cautiously. Larger prospective studies of this purported drug interaction that account for potential confounds (e.g., greater nicotine smoking among cannabis users) may be warranted.

Towards transparency: adoption of WHO best practices in clinical trial registration and reporting among top medical research funders in the USA

OBJECTIVE

To assess to what extent the clinical trial policies of the largest public and philanthropic funders of clinical research in the United States meet WHO best practices in trial registration and reporting.

METHODS

Public and philanthropic funders of clinical trials in the USA with >US$50 million annual spend were selected. The funders were assessed using an 11-item scoring tool based on WHO Joint Statement benchmarks. These 11 items fell into 4 categories, namely: trial registration, academic publication, monitoring and sanctions. An additional item captured whether and how funders referred to Consolidated Standards of Reporting Trials (CONSORT) within their trial policies. Each funder was independently assessed by two or three researchers. Funders were contacted to flag possible errors and omissions. Ambiguous or difficult-to-score items were settled by an independent adjudicator.

RESULTS

Fourteen funders were assessed. Our cross-sectional study found that, on average, funders have only implemented 4.1/11 (37%) of WHO best practices in clinical trial transparency. The most frequently adopted requirement was open access publishing (14/14 funders). The least frequently adopted were (1) requiring trial ID to appear in all publications (2/14 funders, 14%) and (2) making compliance reports public (2/14 funders, 14%). Public funders, on average, adopted more policy elements (5.2/11 items, 47%) than philanthropic funders (2.8/11 items, 25%). Only one funder's policy documents mentioned the CONSORT statement.

CONCLUSIONS

There is a significant variation between the number of best practice policy items adopted by medical research funders in the USA. Many funders fell significantly short of WHO Joint Statement benchmarks. Each funder could benefit from policy revision and strengthening.

Design and validation of a conceptual model regarding impact of open science on healthcare research processes

INTRODUCTION

The development and use of digital tools in various stages of research highlight the importance of novel open science methods for an integrated and accessible research system. The objective of this study was to design and validate a conceptual model of open science on healthcare research processes.

METHODS

This research was conducted in three phases using a mixed-methods approach. The first phase employed a qualitative method, namely purposive sampling and semi-structured interview guides to collect data from healthcare researchers and managers. Influential factors of open science on research processes were extracted for refining the components and developing the proposed model; the second phase utilized a panel of experts and collective agreement through purposive sampling. The final phase involved purposive sampling and Delphi technique to validate the components of the proposed model according to researchers' perspectives.

FINDINGS

From the thematic analysis of 20 interview on the study topic, 385 codes, 38 sub-themes, and 14 main themes were extracted for the initial proposed model. These components were reviewed by expert panel members, resulting in 31 sub-themes, 13 main themes, and 4 approved themes. Ultimately, the agreed-upon model was assessed in four layers for validation by the expert panel, and all the components achieved a score of > 75% in two Delphi rounds. The validated model was presented based on the infrastructure and culture layers, as well as supervision, assessment, publication, and sharing.

CONCLUSION

To effectively implement these methods in the research process, it is essential to create cultural and infrastructural backgrounds and predefined requirements for preventing potential abuses and privacy concerns in the healthcare system. Applying these principles will lead to greater access to outputs, increasing the credibility of research results and the utilization of collective intelligence in solving healthcare system issues.

How the COVID-19 pandemic signaled the demise of Antarctic exceptionalism

This paper explores how the COVID-19 pandemic affected science and tourism activities and their governance in the Antarctic and Southern Ocean. The pandemic reduced the ability of Antarctic Treaty Parties to make decisions on policy issues and placed a considerable burden on researchers. Tourism was effectively suspended during the 2020-2021 Antarctic season and heavily reduced in 2021-2022 but rebounded to record levels in 2022-2023. The pandemic stimulated reflection on practices to facilitate dialog, especially through online events. Opportunities arose to integrate innovations developed during the pandemic more permanently into Antarctic practices, in relation to open science, reducing operational greenhouse gas footprints and barriers of access to Antarctic research and facilitating data sharing. However, as well as the long-term impacts arising directly from the pandemic, an assemblage of major geopolitical drivers are also in play and, combined, these signal a considerable weakening of Antarctic exceptionalism in the early Anthropocene.

A pathway to strengthening open science: comments on the draft South African Ethics in Health Research Guidelines

The recently released draft South African Ethics in Health Research Guidelines: Principles, Processes and Structures (Draft Guidelines) by the National Health Research Ethics Council recognize open data and provide guiding principles for this in the context of health research in South Africa. While its inclusion is a positive development, there is room for improvement. Although the Draft Guidelines leverage the Draft National Policy on Data and Cloud, it lacks incorporation of other relevant government policies, notably the Draft National Open Science Policy, and fails to sufficiently detail the principles of open science and open access. This limited scope and lack of comprehensive definition and detailed guidance present challenges for researchers in conducting ethical and responsible health research in South Africa. It constrains the Draft Guidelines from fully aligning with national imperatives and from fostering African-centric approaches. To address these issues, it is recommended that the Draft Guidelines integrate broader policies and principles, enhance clarity through comprehensive definitions, provide detailed guidance on open access, and promote African-centric approaches. Implementing these solutions will strengthen the Draft Guidelines, aligning them with national visions of open science, and thereby harnessing the full potential of South Africa's diverse scientific community in advancing health research.

Accelerating the open research agenda to solve global challenges

Harnessing science-based policy is key to addressing global challenges like the biodiversity and climate crises. Open research principles underpin effective science-based policy, but the uptake of these principles is likely constrained by the politicisation, commoditisation and conflicting motives of stakeholders in the research landscape. Here, using the mission and vision statements from 129 stakeholders from across the research landscape, we explore alignment in open research principles between stakeholders. We find poor alignment between stakeholders, largely focussed around journals, societies and funders, all of which have low open research language-use. We argue that this poor alignment stifles knowledge flow within the research landscape, ultimately limiting the mobilisation of impactful science-based policy. We offer recommendations on how the research landscape could embrace open research principles to accelerate societies' ability to solve global challenges.

Clinical trials and their impact on policy during COVID-19: a review

Background

Of over 8,000 recorded randomised trials addressing COVID-19, around 80% were of treatments, and 17% have reported results. Approximately 1% were adaptive or platform trials, with 25 having results available, across 29 journal articles and 10 preprint articles.

Methods

We conducted an extensive literature review to address four questions about COVID-19 trials, particularly the role and impact of platform/adaptive trials and lessons learned.

Results

The key findings were: Q1. Social value in conducting trials and uptake into policy? COVID-19 drug treatments varied substantially and changed considerably, with drugs found effective in definitive clinical trials replacing unproven drugs. Dexamethasone has likely saved ½-2 million lives, and was cost effective across a range of countries and populations, whereas the cost effectiveness of remdesivir is uncertain. Published economic and health system impacts of COVID-19 treatments were infrequent. Q2. Issues with adaptive trial designs. Of the 77 platform trials registered, 6 major platform trials, with approximately 50 treatment arms, recruited ~135,000 participants with funding over $100 million. Q3. Models of good practice. Streamlined set-up processes such as flexible and fast-track funding, ethics, and governance approvals are vital. To facilitate recruitment, simple and streamlined research processes, and pre-existing research networks to coordinate trial planning, design, conduct and practice change are crucial to success. Q4. Potential conflicts to avoid? When treating patients through trials, balancing individual and collective rights and allocating scarce resources between healthcare and research are challenging. Tensions occur between commercial and non-commercial sectors, and academic and public health interests, such as publication and funding driven indicators and the public good.

Conclusion

There is a need to (i) reduce small, repetitive, single centre trials, (ii) increase coordination to ensure robust research conducted for treatments, and (iii) a wider adoption of adaptive/platform trial designs to respond to fast-evolving evidence landscape.

Making science public: a review of journalists' use of Open Access research

Science journalists are uniquely positioned to increase the societal impact of open research outputs by contextualizing and communicating findings in ways that highlight their relevance and implications for non-specialist audiences. Yet, it is unclear to what degree journalists use open research outputs, such as open access publications or preprints, in their reporting; what factors motivate or constrain this use; and how the recent surge in openly available research seen during the COVID-19 pandemic has affected this. This article examines these questions through a review of relevant literature published from 2018 onwards-particularly literature relating to the COVID-19 pandemic-as well as seminal articles outside the search dates. We find that research that explicitly examines journalists' engagement with open access publications or preprints is scarce, with existing literature mostly addressing the topic tangentially or as a secondary concern, rather than a primary focus. Still, the limited body of evidence points to several factors that may hamper journalists' use of these outputs and thus warrant further exploration. These include an overreliance on traditional criteria for evaluating scientific quality; concerns about the trustworthiness of open research outputs; and challenges using and verifying the findings. We also find that, while the COVID-19 pandemic encouraged journalists to explore open research outputs such as preprints, the extent to which these explorations will become established journalistic practices remains unclear. Furthermore, we note that current research is overwhelmingly authored and focused on the Global North, and the United States specifically. We conclude with recommendations for future research that attend to issues of equity and diversity, and more explicitly examine the intersections of open access and science journalism.

Research misconduct as a challenge for academic institutions and scientific journals

Research misconduct refers to a set of unethical practices in research and publication and is the main reason for retraction of articles published in the academic literature. Research misconduct has negative consequences and has generated public scepticism about research, which has led to increasing distrust in science. In this context, a joint effort by the scientific community, academic institutions, scientific journals and research funders is needed to identify and prevent research misconduct. In this paper, we will evaluate what has already been done and what is needed to do to better address research misconduct. The focus of this paper will be on the actions taken by academic institutions, as the first line of defence, and scientific journals, as the gatekeepers of science. However, scientific journals and academic institutions are only a part of a much larger and multistakeholder effort needed to address the challenges scientific research is facing.

Research integrity during the COVID-19 pandemic: Perspectives of health science researchers at an Academic Health Science Center

During the coronavirus disease 2019 pandemic, a complex mix of political pressure, social urgency, public panic, and scientific curiosity has significantly impacted the context of research and development. The goal of this study is to understand if and how researchers are shifting their practices and adjusting norms and beliefs regarding research ethics and integrity. We have conducted 31 interviews with Health Science Researchers at the University of Texas Medical Branch which were then analyzed using integrated deductive and inductive coding. We categorized participant views into four main areas: 1) limitations to the research design, 2) publication, 3) duplication of studies, and 4) research pipeline. Although certain researchers were in keeping to the status quo, more were willing to modify norms to address social need and urgency. Notably, they were more likely to opt for systemic change rather than modifications within their own research practices.

Reconciling the ideals of open science with data privacy in the context of health research in Nigeria: A legal analysis

Background

The Nigerian Revised National Policy on Science, Technology and Innovation 2022 formally proposed the adoption of open science principles among researchers and all other stakeholders in Nigeria. With regard to health research, open science would allow the dissemination and sharing of scientific data and other research outputs among health researchers, thereby accelerating the scientific process and the development of innovative solutions for health challenges. However, in this context, the existing privacy and data protection rules can profoundly affect the adoption and sustainability of open science in Nigeria. This is because researchers rely heavily on accessing and sharing personal information and the biological samples of human subjects. Therefore, this study identifies and examines possible legal issues and challenges arising from the existing privacy and data protection rules as researchers adopt an open science approach to health research in Nigeria.

Methods

This study adopts a qualitative approach, providing a legal analysis of existing national, regional and international instruments on privacy, data protection and health research in the context of open science in Nigeria. The study also draws on relevant research and non-research articles on open science, health research, privacy and data protection published in English. Searches for the articles were conducted on various sites through the Google search engine, using terms such as 'sharing of data', 'data privacy', 'principles of data protection', 'open science', etc.

Results/Conclusion

The study finds that the right to privacy and data protection could affect the promotion, adoption and sustainability of open science among health researchers in Nigeria, as well as their ability to engage in open collaborative research with their peers in Africa and other jurisdictions. To prevent infringements of the privacy and data protection rules by researchers and thereby ensure the adoption of open science approaches in health research, the study further identifies various legal options for researchers, including using personal data or material transfer arrangements, which, among other things, require recipients or users of human data or biological samples to commit not to re-identify human subjects.

Connectome-based modelling of neurodegenerative diseases: towards precision medicine and mechanistic insight

Neurodegenerative diseases are the most common cause of dementia. Although their underlying molecular pathologies have been identified, there is substantial heterogeneity in the patterns of progressive brain alterations across and within these diseases. Recent advances in neuroimaging methods have revealed that pathological proteins accumulate along specific macroscale brain networks, implicating the network architecture of the brain in the system-level pathophysiology of neurodegenerative diseases. However, the extent to which 'network-based neurodegeneration' applies across the wide range of neurodegenerative disorders remains unclear. Here, we discuss the state-of-the-art of neuroimaging-based connectomics for the mapping and prediction of neurodegenerative processes. We review findings supporting brain networks as passive conduits through which pathological proteins spread. As an alternative view, we also discuss complementary work suggesting that network alterations actively modulate the spreading of pathological proteins between connected brain regions. We conclude this Perspective by proposing an integrative framework in which connectome-based models can be advanced along three dimensions of innovation: incorporating parameters that modulate propagation behaviour on the basis of measurable biological features; building patient-tailored models that use individual-level information and allowing model parameters to interact dynamically over time. We discuss promises and pitfalls of these strategies for improving disease insights and moving towards precision medicine.

Translational Research and Communities

Impactful translational research requires new approaches to computational analysis and bioethics, both of which have been advanced by adoption of community-engagement strategies. Community knowledge and experience will hone data collection, research, and insights and accelerate the impact of derived translational applications to improve individual health, medical decision-making, and public health policy. In the context of translational research with big health data, meaningful community-researcher engagement will require developing and deploying coengagement tools across the research life cycle and developing approaches for novel coproduction.

Raising concerns on questionable ethics approvals - a case study of 456 trials from the Institut Hospitalo-Universitaire Méditerranée Infection

BACKGROUND

The practice of clinical research is strictly regulated by law. During submission and review processes, compliance of such research with the laws enforced in the country where it was conducted is not always correctly filled in by the authors or verified by the editors. Here, we report a case of a single institution for which one may find hundreds of publications with seemingly relevant ethical concerns, along with 10 months of follow-up through contacts with the editors of these articles. We thus argue for a stricter control of ethical authorization by scientific editors and we call on publishers to cooperate to this end.

METHODS

We present an investigation of the ethics and legal aspects of 456 studies published by the IHU-MI (Institut Hospitalo-Universitaire Méditerranée Infection) in Marseille, France.

RESULTS

We identified a wide range of issues with the stated research authorization and ethics of the published studies with respect to the Institutional Review Board and the approval presented. Among the studies investigated, 248 were conducted with the same ethics approval number, even though the subjects, samples, and countries of investigation were different. Thirty-nine (39) did not even contain a reference to the ethics approval number while they present research on human beings. We thus contacted the journals that published these articles and provide their responses to our concerns. It should be noted that, since our investigation and reporting to journals, PLOS has issued expressions of concerns for several publications we analyze here.

CONCLUSION

This case presents an investigation of the veracity of ethical approval, and more than 10 months of follow-up by independent researchers. We call for stricter control and cooperation in handling of these cases, including editorial requirement to upload ethical approval documents, guidelines from COPE to address such ethical concerns, and transparent editorial policies and timelines to answer such concerns. All supplementary materials are available.

Scoping review of the methodology of large health surveys conducted in Spain early on in the COVID-19 pandemic

Background

The use of health surveys has been key in the scientific community to promptly communicate results about the health impact of COVID-19. But what information was collected, where, when and how, and who was the study population?

Objective

To describe the methodological characteristics used in large health surveys conducted in Spain early on in the COVID-19 pandemic.

Methods

Scoping review. Inclusion criteria: observational studies published between January 2020 and December 2021, with sample sizes of over 2,000 persons resident in Spain. Databases consulted: PubMed, CINAHL, Literatura Latinoamericana y del Caribe en CC de la Salud, Scopus, PsycINFO, Embase, Sociological Abstracts, Dialnet and Web of Science Core Collection. We analyzed the characteristics of the literature references, methodologies and information gathered in the surveys selected. Fifty five studies were included.

Results

Sixty percentage of the studies included had mental health as their main topic and 75% were conducted on the general adult population. Thirteen percentage had a longitudinal design, 93% used the internet to gather information and the same percentage used non-probability sampling. Thirty percentage made some type of sampling correction to reduce coverage or non-response biases, but not selection biases. Sixty seven percentage did not state the availability of their data.

Conclusions

Consistent with the extensive use of non-probability sampling without any bias correction in the extraordinary setting created by COVID-19, quality population frameworks are required so that probability and representative samples can be extracted quickly to promptly address other health crises, as well as to reduce potential coverage, non-response and particularly selection biases by utilizing reweighting techniques. The low data accessibility despite the huge opportunity that COVID-19 provided for Open Science-based research is striking.

Transparency of COVID-19-related research: A meta-research study

BACKGROUND

We aimed to assess the adherence to five transparency practices (data availability, code availability, protocol registration and conflicts of interest (COI), and funding disclosures) from open access Coronavirus disease 2019 (COVID-19) related articles.

METHODS

We searched and exported all open access COVID-19-related articles from PubMed-indexed journals in the Europe PubMed Central database published from January 2020 to June 9, 2022. With a validated and automated tool, we detected transparent practices of three paper types: research articles, randomized controlled trials (RCTs), and reviews. Basic journal- and article-related information were retrieved from the database. We used R for the descriptive analyses.

RESULTS

The total number of articles was 258,678, of which we were able to retrieve full texts of 186,157 (72%) articles from the database Over half of the papers (55.7%, n = 103,732) were research articles, 10.9% (n = 20,229) were review articles, and less than one percent (n = 1,202) were RCTs. Approximately nine-tenths of articles (in all three paper types) had a statement to disclose COI. Funding disclosure (83.9%, confidence interval (CI): 81.7-85.8 95%) and protocol registration (53.5%, 95% CI: 50.7-56.3) were more frequent in RCTs than in reviews or research articles. Reviews shared data (2.5%, 95% CI: 2.3-2.8) and code (0.4%, 95% CI: 0.4-0.5) less frequently than RCTs or research articles. Articles published in 2022 had the highest adherence to all five transparency practices. Most of the reviews (62%) and research articles (58%) adhered to two transparency practices, whereas almost half of the RCTs (47%) adhered to three practices. There were journal- and publisher-related differences in all five practices, and articles that did not adhere to transparency practices were more likely published in lowest impact journals and were less likely cited.

CONCLUSION

While most articles were freely available and had a COI disclosure, adherence to other transparent practices was far from acceptable. A much stronger commitment to open science practices, particularly to protocol registration, data and code sharing, is needed from all stakeholders.

The rise of preprints in earth sciences

The rate of science information's spread has accelerated in recent years. In this context, it appears that many scientific disciplines are beginning to recognize the value and possibility of sharing open access (OA) online manuscripts in their preprint form. Preprints are academic papers that are published but have not yet been evaluated by peers. They have existed in research at least since the 1960s and the creation of ArXiv in physics and mathematics. Since then, preprint platforms-which can be publisher- or community-driven, profit or not for profit, and based on proprietary or free and open source software-have gained popularity in many fields (for example, bioRxiv for the biological sciences). Today, there are many platforms that are either disciplinary-specific or cross-domain, with exponential development over the past ten years. Preprints as a whole still make up a very small portion of scholarly publishing, but a large group of early adopters are testing out these value-adding tools across a much wider range of disciplines than in the past. In this opinion article, we provide perspective on the three main options available for earth scientists, namely EarthArXiv, ESSOAr/ESS Open Archive and EGUsphere.

Health data hubs: an analysis of existing data governance features for research

BACKGROUND

Digital transformation in healthcare and the growth of health data generation and collection are important challenges for the secondary use of healthcare records in the health research field. Likewise, due to the ethical and legal constraints for using sensitive data, understanding how health data are managed by dedicated infrastructures called data hubs is essential to facilitating data sharing and reuse.

METHODS

To capture the different data governance behind health data hubs across Europe, a survey focused on analysing the feasibility of linking individual-level data between data collections and the generation of health data governance patterns was carried out. The target audience of this study was national, European, and global data hubs. In total, the designed survey was sent to a representative list of 99 health data hubs in January 2022.

RESULTS

In total, 41 survey responses received until June 2022 were analysed. Stratification methods were performed to cover the different levels of granularity identified in some data hubs' characteristics. Firstly, a general pattern of data governance for data hubs was defined. Afterward, specific profiles were defined, generating specific data governance patterns through the stratifications in terms of the kind of organization (centralized versus decentralized) and role (data controller or data processor) of the health data hub respondents.

CONCLUSIONS

The analysis of the responses from health data hub respondents across Europe provided a list of the most frequent aspects, which concluded with a set of specific best practices on data management and governance, taking into account the constraints of sensitive data. In summary, a data hub should work in a centralized way, providing a Data Processing Agreement and a formal procedure to identify data providers, as well as data quality control, data integrity and anonymization methods.

FAIR-Checker: supporting digital resource findability and reuse with Knowledge Graphs and Semantic Web standards

The current rise of Open Science and Reproducibility in the Life Sciences requires the creation of rich, machine-actionable metadata in order to better share and reuse biological digital resources such as datasets, bioinformatics tools, training materials, etc. For this purpose, FAIR principles have been defined for both data and metadata and adopted by large communities, leading to the definition of specific metrics. However, automatic FAIRness assessment is still difficult because computational evaluations frequently require technical expertise and can be time-consuming. As a first step to address these issues, we propose FAIR-Checker, a web-based tool to assess the FAIRness of metadata presented by digital resources. FAIR-Checker offers two main facets: a "Check" module providing a thorough metadata evaluation and recommendations, and an "Inspect" module which assists users in improving metadata quality and therefore the FAIRness of their resource. FAIR-Checker leverages Semantic Web standards and technologies such as SPARQL queries and SHACL constraints to automatically assess FAIR metrics. Users are notified of missing, necessary, or recommended metadata for various resource categories. We evaluate FAIR-Checker in the context of improving the FAIRification of individual resources, through better metadata, as well as analyzing the FAIRness of more than 25 thousand bioinformatics software descriptions.

COVID-19 trials were not more likely to report intent to share individual data than non-COVID-19 trials in ClinicalTrials.gov

OBJECTIVES

To compare intent to share individual participant data (IPD) between COVID-19 and non-COVID-19 trials registered at ClinicalTrials.gov between 01/09/2020, and 01/03/2021. We also evaluated factors independently associated with intent to share IPD and whether intent to share IPD has improved as compared with the prepandemic period.

METHODS

We searched ClinicalTrials.gov for all interventional phase 3 studies registered between 01/09/2020, and 01/03/2021. Then, we identified COVID-19 trials and selected a random sample of non-COVID-19 trials with a ratio 2:1. We compared the intent to share IPD between these trials and with 292 trials registered between 01/12/2019, and 01/03/2020 (prepandemic period).

RESULTS

We included 148 COVID-19 trials and 296 non-COVID-19 trials. Intent to share IPD did not significantly differ between COVID-19 and non-COVID-19 trials (22.3% vs. 27.0%, P = 0.3). Intent to share IPD was independently associated with industry-sponsorship (odds ratio [OR] = 2.92; 95% confidence interval [CI]: 1.65-5.27) and location in the United States (OR = 2.93; 95% CI: 1.64-5.41) or the European Union (OR = 2.06; 95% CI: 1.03-4.19). The intent to share IPD has not significantly improved compared with the prepandemic period (P = 0.16).

CONCLUSION

Data-sharing intent at registration does not seem better for COVID-19 trials.

Building awareness and capacity of bioinformatics and open science skills in Kenya: a sensitize, train, hack, and collaborate model

We have applied the sensitize-train-hack-community model to build awareness of and capacity in bioinformatics in Kenya. Open science is the practice of science openly and collaboratively, with tools, techniques, and data freely shared to facilitate reuse and collaboration. Open science is not a mandatory curriculum course in schools, whereas bioinformatics is relatively new in some African regions. Open science tools can significantly enhance bioinformatics, leading to increased reproducibility. However, open science and bioinformatics skills, especially blended, are still lacking among students and researchers in resource-constrained regions. We note the need to be aware of the power of open science among the bioinformatics community and a clear strategy to learn bioinformatics and open science skills for use in research. Using the OpenScienceKE framework-Sensitize, Train, Hack, Collaborate/Community-the BOSS (Bioinformatics and Open Science Skills) virtual events built awareness and empowered researchers with the skills and tools in open science and bioinformatics. Sensitization was achieved through a symposium, training through a workshop and train-the-trainer program, hack through mini-projects, community through conferences, and continuous meet-ups. In this paper, we discuss how we applied the framework during the BOSS events and highlight lessons learnt in planning and executing the events and their impact on the outcome of each phase. We evaluate the impact of the events through anonymous surveys. We show that sensitizing and empowering researchers with the skills works best when the participants apply the skills to real-world problems: project-based learning. Furthermore, we have demonstrated how to implement virtual events in resource-constrained settings by providing Internet and equipment support to participants, thus improving accessibility and diversity.

Predictive stability, novel HPLC-MS analysis and semi-automatic compounding process for the emergency implementation of a production line of pancuronium in injectable solution

During the early months of the COVID-19 pandemic, the international medical product supply chain was tight, causing breaks in the availability of neuromuscular blocking agents essential for the treatment of patients in intensive care units. The present study describes the pharmaceutical development of an injectable 2 mg/mL solution of pancuronium bromide (PC) in a very short lapse of time. The sterile solution was compounded into a good manufacturing practice grade A clean room, filtered (0.2 µm) and filled into 10 mL type I glass, manually sealed with bromobutyl rubber stoppers. A novel HPLC-MS stability indicating method for pancuronium quantification and its degradation product was developed and validated. This fast, sensitive and straightforward method was used to study the stability of the formulation using a semi-predictive method, enabling a very fast attribution of a temporary shelf-life, which was confirmed by a classic prospective stability study. The production line and the analytical tools set-up were performed in six weeks and the semi-predictive stability study was conducted in 90 days, allowing us to predict a shelf life, which was successfully confirmed by prospective study. In conclusion, using innovative methods, we were able to rapidly overcome the shortage of a critical drug.

Structural under-reporting of informed consent, data handling and sharing, ethical approval, and application of Open Science principles as proxies for study quality conduct in COVID-19 research: a systematic scoping review

BACKGROUND

The COVID-19 pandemic required science to provide answers rapidly to combat the outbreak. Hence, the reproducibility and quality of conducting research may have been threatened, particularly regarding privacy and data protection, in varying ways around the globe. The objective was to investigate aspects of reporting informed consent and data handling as proxies for study quality conduct.

METHODS

A systematic scoping review was performed by searching PubMed and Embase. The search was performed on November 8th, 2020. Studies with hospitalised patients diagnosed with COVID-19 over 18 years old were eligible for inclusion. With a focus on informed consent, data were extracted on the study design, prestudy protocol registration, ethical approval, data anonymisation, data sharing and data transfer as proxies for study quality. For reasons of comparison, data regarding country income level, study location and journal impact factor were also collected.

RESULTS

972 studies were included. 21.3% of studies reported informed consent, 42.6% reported waivers of consent, 31.4% did not report consent information and 4.7% mentioned other types of consent. Informed consent reporting was highest in clinical trials (94.6%) and lowest in retrospective cohort studies (15.0%). The reporting of consent versus no consent did not differ significantly by journal impact factor (p=0.159). 16.8% of studies reported a prestudy protocol registration or design. Ethical approval was described in 90.9% of studies. Information on anonymisation was provided in 17.0% of studies. In 257 multicentre studies, 1.2% reported on data sharing agreements, and none reported on Findable, Accessible, Interoperable and Reusable data principles. 1.2% reported on open data. Consent was most often reported in the Middle East (42.4%) and least often in North America (4.7%). Only one report originated from a low-income country.

DISCUSSION

Informed consent and aspects of data handling and sharing were under-reported in publications concerning COVID-19 and differed between countries, which strains study quality conduct when in dire need of answers.

Assessing the publishing priorities and preferences among STEM researchers at a large R1 institution

The cost of academic publishing has increased substantially despite the ease with which information can be shared on the web. Open Access publishing is a key mechanism for amplifying research access, inclusivity, and impact. Despite this, shifting to a free-to-read publishing environment requires navigating complex barriers that vary by career status and publishing expectations. In this article, we investigate the motivations and preferences of researchers situated within our large research institution as a case study for publishing attitudes at similar institutions. We surveyed the publishing priorities and preferences of researchers at various career stages in STEM fields as they relate to openness, data practices, and assessment of research impact. Our results indicate that publishing preferences, data management experience and research impact assessment vary by career status and departmental approaches to promotion. We find that open access publishing is widely appreciated regardless of career status, but financial limitations and publishing expectations were common barriers to publishing in Open Access journals. Our findings shed light on publishing attitudes and preferences among researchers at a major R1 research institution, and offer insight into advocacy strategies that incentivize open access publishing.

Changes in health communication in the age of COVID-19: A study on the dissemination of preprints to the public

Introduction

Preprints have become an important tool for meeting the challenges of health communication in the context of COVID-19. They allow scientists to disseminate their results more quickly due to the absence of a peer review process. Preprints have been well-received by scientists, however, there have been concerns about the exposure of wider public audiences to preprints due in part to this lack of peer review.

Methods

The aim of this study is to examine the dissemination of preprints on medRxiv and bioRxiv during the COVID-19 pandemic using content analysis and statistical analysis.

Results

Our findings show that preprints have played an unprecedented role in disseminating COVID-19-related science results to the public.

Discussion

While the overall media coverage of preprints is unsatisfactory, digital native news media performed better than legacy media in reporting preprints, which means that we could make the most of digital native media to improve health communication. This study contributes to understanding how science communication has evolved in response to the COVID-19 pandemic and provides some practical recommendations.

Scientific Integrity Requires Publishing Rebuttals and Retracting Problematic Papers

Recently, an article by Seneff et al. entitled "Innate immunosuppression by SARS-CoV-2 mRNA vaccinations: The role of G-quadruplexes, exosomes, and MicroRNAs" was published in Food and Chemical Toxicology (FCT). Here, we describe why this article, which contains unsubstantiated claims and misunderstandings such as "billions of lives are potentially at risk" with COVID-19 mRNA vaccines, is problematic and should be retracted. We report here our request to the editor of FCT to have our rebuttal published, unfortunately rejected after three rounds of reviewing. Fighting the spread of false information requires enormous effort while receiving little or no credit for this necessary work, which often even ends up being threatened. This need for more scientific integrity is at the heart of our advocacy, and we call for large support, especially from editors and publishers, to fight more effectively against deadly disinformation.

Spatial and spatio-temporal epidemiological approaches to inform COVID-19 surveillance and control: a systematic review of statistical and modelling methods in Africa

OBJECTIVE

Various studies have been published to better understand the underlying spatial and temporal dynamics of COVID-19. This review sought to identify different spatial and spatio-temporal modelling methods that have been applied to COVID-19 and examine influential covariates that have been reportedly associated with its risk in Africa.

DESIGN

Systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

DATA SOURCES

Thematically mined keywords were used to identify refereed studies conducted between January 2020 and February 2022 from the following databases: PubMed, Scopus, MEDLINE via Proquest, CINHAL via EBSCOhost and Coronavirus Research Database via ProQuest. A manual search through the reference list of studies was also conducted.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Peer-reviewed studies that demonstrated the application of spatial and temporal approaches to COVID-19 outcomes.

DATA EXTRACTION AND SYNTHESIS

A standardised extraction form based on critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist was used to extract the meta-data of the included studies. A validated scoring criterion was used to assess studies based on their methodological relevance and quality.

RESULTS

Among 2065 hits in five databases, title and abstract screening yielded 827 studies of which 22 were synthesised and qualitatively analysed. The most common socioeconomic variable was population density. HIV prevalence was the most common epidemiological indicator, while temperature was the most common environmental indicator. Thirteen studies (59%) implemented diverse formulations of spatial and spatio-temporal models incorporating unmeasured factors of COVID-19 and the subtle influence of time and space. Cluster analyses were used across seven studies (32%) to explore COVID-19 variation and determine whether observed patterns were random.

CONCLUSION

COVID-19 modelling in Africa is still in its infancy, and a range of spatial and spatio-temporal methods have been employed across diverse settings. Strengthening routine data systems remains critical for generating estimates and understanding factors that drive spatial variation in vulnerable populations and temporal variation in pandemic progression.

PROSPERO REGISTRATION NUMBER

CRD42021279767.

Strategic priorities for reproducibility reform

Increasing the reproducibility of research should be a top priority. Great work is being done, but more work is needed to combine efforts and maximize our actions to enable true reproducibility reform.

The Chinese Open Science Network (COSN): Building an Open Science Community From Scratch

Open Science is becoming a mainstream scientific ideology in psychology and related fields. However, researchers, especially early-career researchers (ECRs) in developing countries, are facing significant hurdles in engaging in Open Science and moving it forward. In China, various societal and cultural factors discourage ECRs from participating in Open Science, such as the lack of dedicated communication channels and the norm of modesty. To make the voice of Open Science heard by Chinese-speaking ECRs and scholars at large, the Chinese Open Science Network (COSN) was initiated in 2016. With its core values being grassroots-oriented, diversity, and inclusivity, COSN has grown from a small Open Science interest group to a recognized network both in the Chinese-speaking research community and the international Open Science community. So far, COSN has organized three in-person workshops, 12 tutorials, 48 talks, and 55 journal club sessions and translated 15 Open Science-related articles and blogs from English to Chinese. Currently, the main social media account of COSN (i.e., the WeChat Official Account) has more than 23,000 subscribers, and more than 1,000 researchers/students actively participate in the discussions on Open Science. In this article, we share our experience in building such a network to encourage ECRs in developing countries to start their own Open Science initiatives and engage in the global Open Science movement. We foresee great collaborative efforts of COSN together with all other local and international networks to further accelerate the Open Science movement.

Exploring the challenges of and solutions to sharing personal genomic data for use in healthcare

Boosted by the COVID-19 pandemic, as well as the tightened General Data Protection Regulation (GDPR) legislation within the European Union (EU), individuals have become increasingly concerned about privacy. This is also reflected in how willing individuals are to consent to sharing personal data, including their health data. To understand this behaviour better, this study focuses on willingness to consent in relation to genomic data. The study explores how the provision of educational information relates to willingness to consent, as well as differences in privacy concerns, information sensitivity and the perceived trade-off value between individuals willing versus unwilling to consent to sharing their genomic data. Of the respondents, 65% were initially willing to consent, but after educational information 89% were willing to consent and only 11% remained unwilling to consent. Educating individuals about potential health benefits can thus help to correct the beliefs that originally led to the unwillingness to share genomic data.

RoboHapalytics: A Robot Assisted Haptic Controller for Immersive Analytics

Immersive environments offer new possibilities for exploring three-dimensional volumetric or abstract data. However, typical mid-air interaction offers little guidance to the user in interacting with the resulting visuals. Previous work has explored the use of haptic controls to give users tangible affordances for interacting with the data, but these controls have either: been limited in their range and resolution; were spatially fixed; or required users to manually align them with the data space. We explore the use of a robot arm with hand tracking to align tangible controls under the user's fingers as they reach out to interact with data affordances. We begin with a study evaluating the effectiveness of a robot-extended slider control compared to a large fixed physical slider and a purely virtual mid-air slider. We find that the robot slider has similar accuracy to the physical slider but is significantly more accurate than mid-air interaction. Further, the robot slider can be arbitrarily reoriented, opening up many new possibilities for tangible haptic interaction with immersive visualisations. We demonstrate these possibilities through three use-cases: selection in a time-series chart; interactive slicing of CT scans; and finally exploration of a scatter plot depicting time-varying socio-economic data.

Close to open-Factors that hinder and promote open science in ecology research and education

The Open Science (OS) movement is rapidly gaining traction among policy-makers, research funders, scientific journals and individual scientists. Despite these tendencies, the pace of implementing OS throughout the scientific process and across the scientific community remains slow. Thus, a better understanding of the conditions that affect OS engagement, and in particular, of how practitioners learn, use, conduct and share research openly can guide those seeking to implement OS more broadly. We surveyed participants at an OS workshop hosted by the Living Norway Ecological Data Network in 2020 to learn how they perceived OS and its importance in their research, supervision and teaching. Further, we wanted to know what OS practices they had encountered in their education and what they saw as hindering or helping their engagement with OS. The survey contained scaled-response and open-ended questions, allowing for a mixed-methods approach. We obtained survey responses from 60 out of 128 workshop participants (47%). Responses indicated that usage and sharing of open data and code, as well as open access publication, were the most frequent OS practices. Only a minority of respondents reported having encountered OS in their formal education. A majority also viewed OS as less important in their teaching than in their research and supervisory roles. The respondents' suggestions for what would facilitate greater OS engagement in the future included knowledge, guidelines, and resources, but also social and structural support. These are aspects that could be strengthened by promoting explicit implementation of OS practices in higher education and by nurturing a more inclusive and equitable OS culture. We argue that incorporating OS in teaching and learning of science can yield substantial benefits to the research community, student learning, and ultimately, to the wider societal objectives of science and higher education.

Open access and its potential impact on public health - A South African perspective

Traditionally, access to research information has been restricted through journal subscriptions. This means that research entities and individuals who were unable to afford subscription costs did not have access to journal articles. There has however been a progressive shift toward electronic access to journal publications and subsequently growth in the number of journals available globally. In the context of electronic journals, both open access and restricted access options exist. While the latter option is comparable to traditional, subscription-based paper journals, open access journal publications follow an "open science" publishing model allowing scholarly communications and outputs to be publicly available online at no cost to the reader. However, for readers to enjoy open access, publication costs are shifted elsewhere, typically onto academic institutions and authors. SARS-CoV-2, and the resulting COVID-19 pandemic have highlighted the benefits of open science through accelerated research and unprecedented levels of collaboration and data sharing. South Africa is one of the leading open access countries on the African continent. This paper focuses on open access in the South African higher education research context with an emphasis on our Institution and our own experiences. It also addresses the financial implications of open access and provides possible solutions for reducing the cost of publication for researchers and their institutions. Privacy in open access and the role of the Protection of Personal Information Act (POPIA) in medical research and secondary use of data in South Africa will also be discussed.

Why don't we share data and code? Perceived barriers and benefits to public archiving practices

The biological sciences community is increasingly recognizing the value of open, reproducible and transparent research practices for science and society at large. Despite this recognition, many researchers fail to share their data and code publicly. This pattern may arise from knowledge barriers about how to archive data and code, concerns about its reuse, and misaligned career incentives. Here, we define, categorize and discuss barriers to data and code sharing that are relevant to many research fields. We explore how real and perceived barriers might be overcome or reframed in the light of the benefits relative to costs. By elucidating these barriers and the contexts in which they arise, we can take steps to mitigate them and align our actions with the goals of open science, both as individual scientists and as a scientific community.

The roles of individual differences in time perspective, promotion focus, and innovativeness: Testing technology acceptance model

The goal of this study is to examine the roles of Zimbardo's time perspective along with other individual differences such as promotion focus and innovativeness in perceived ease of use, perceived usefulness, and attitude toward SNSs (social networking sites) in the technology acceptance model (TAM). A total of 234 participants joined this online study in South Korea. As predicted, past positive time perspective (TP) positively affected promotion focus and innovativeness, whereas past negative TP negatively affected them. Present hedonic TP positively affected innovativeness, and present fatalistic TP negatively affected promotion focus each. Future TP also positively related to promotion focus and innovativeness. In addition, simple and serial mediation effects of perceived ease of use and perceived usefulness independently and sequentially mediated the impact of TP on attitude toward SNSs. By considering TP along with promotion focus and innovativeness in conjunction with beliefs in the TAM, this study identifies psychological underpinnings of how individual differences affect technology adoption attitude and behavior. Research implications and future research suggestions will be discussed in detail.

Fostering openness in open science: An ethical discussion of risks and benefits

Transformation of science by embracing the concepts of open science presents a very attractive strategy to enhance the reliability of science. Open science policies embody the concepts of open data and open access that encompass sharing of resources, dissemination of ideas, and synergizing the collaborative forums of research. Despite the opportunities in openness, however, there are grave ethical concerns too, and they present a dual-use dilemma. Access to sensitive information is seen as a security risk, and it also possesses other concerns such as confidentiality, privacy, and affordability. There are arguments that open science can be harmful to marginalized groups. Through this study, we aim to discuss the opportunities of open science, as well as the ethical and security aspects, which require further deliberation before full-fledged acceptance in the science community.

Open Science in Africa: What policymakers should consider

As Open Science (OS) is being promoted as the best avenue to share and drive scientific discoveries at much lower costs and in transparent and credible ways, it is imperative that African governments and institutions take advantage of the momentum and build research infrastructures that are responsive to this movement. This paper aims to provide useful insight into the importance and implementation of OS policy frameworks. The paper uses a systematic review approach to review existing literature and analyse global OS policy development documents. The approach includes a review of existing OS policy frameworks that can guide similar work by African governments and institutions. This critical review also makes recommendations on key issues that Africa should consider in the process of OS policy development. These approaches can be widely used as further foundations for future developments in OS practices on the continent.

Open science and conflict of interest policies of medical and health sciences journals before and during the COVID-19 pandemic: A repeat cross-sectional study

Objectives

To audit the transparent and open science standards of health and medical sciences journal policies and explore the impact of the COVID-19 pandemic.

Design

Repeat cross-sectional study.

Setting

19 journals listed in Google Scholar's Top Publications for health and medical sciences.

Participants

Blood, Cell, Circulation, European Heart Journal, Gastroenterology, Journal of Clinical Oncology, Journal of the American College of Cardiology, Nature Genetics, Nature Medicine, Nature Neuroscience, Neuron, PLoS ONE, Proceedings of the National Academy of Sciences, Science Translational Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet, The Lancet Oncology, and The New England Journal of Medicine.

Main outcome measures

We used the Transparency and Openness Promotion (TOP) guideline and the International Committee of Medical Journal Editors (ICMJE) requirements for disclosing conflicts of interest (COIs) to evaluate journals standards.

Results

TOP scores slightly improved during the COVID-19 pandemic, from a median of 5 (IQR: 2–12.5) out of a possible 24 points in February 2020 to 7 (IQR: 4–12) in May 2021, but overall, scores were very low at both time points. Journal policies scored highest for their adherence to data transparency and scored lowest for preregistration of study protocols and analysis plans and the submission of replication studies. Most journals fulfilled all ICMJE provisions for reporting COIs before (84%; n  =  16) and during (95%; n  =  18) the COVID-19 pandemic.

Conclusions

The COVID-19 pandemic has highlighted the importance of practising open science. However, requirements for open science practices in audited policies were overall low, which may impede progress in health and medical research. As key stakeholders in disseminating research, journals should promote a research culture of greater transparency and more robust open science practices.

Transparency and reporting characteristics of COVID-19 randomized controlled trials

BACKGROUND

In the context of the COVID-19 pandemic, randomized controlled trials (RCTs) are essential to support clinical decision-making. We aimed (1) to assess and compare the reporting characteristics of RCTs between preprints and peer-reviewed publications and (2) to assess whether reporting improves after the peer review process for all preprints subsequently published in peer-reviewed journals.

METHODS

We searched the Cochrane COVID-19 Study Register and L·OVE COVID-19 platform to identify all reports of RCTs assessing pharmacological treatments of COVID-19, up to May 2021. We extracted indicators of transparency (e.g., trial registration, data sharing intentions) and assessed the completeness of reporting (i.e., some important CONSORT items, conflict of interest, ethical approval) using a standardized data extraction form. We also identified paired reports published in preprint and peer-reviewed publications.

RESULTS

We identified 251 trial reports: 121 (48%) were first published in peer-reviewed journals, and 130 (52%) were first published as preprints. Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items.

CONCLUSIONS

Transparency, completeness, and consistency of reporting of COVID-19 clinical trials were insufficient both in preprints and peer-reviewed publications. A comparison of paired reports published in preprint and peer-reviewed publication did not indicate major improvement.

Forecasting the publication and citation outcomes of COVID-19 preprints

Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals. In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10. On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server. For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints.