Promoting Open Science: A Holistic Approach to Changing Behaviour

A scoping review on what constitutes a good research culture

Background

The crisis in research culture is well documented, covering issues such as a tendency for quantity over quality, unhealthy competitive environments, and assessment based on publications, journal prestige and funding. In response, research institutions need to assess their own practices to promote and advocate for change in the current research ecosystem. The purpose of the scoping review was to explore ' What does the evidence say about the 'problem' with 'poor' research culture, what are the benefits of 'good' research culture, and what does 'good' look like?'

Aims

To examine the peer-reviewed and grey literature to explore the interplay between research culture, open research, career paths, recognition and rewards, and equality, diversity, and inclusion, as part of a larger programme of activity for a research institution.

Methods

A scoping review was undertaken. Six databases were searched along with grey literature. Eligible literature had relevance to academic research institutions, addressed research culture, and were published between January 2017 to May 2022. Evidence was mapped and themed to specific categories. The search strategy, screening and analysis took place between April-May 2022.

Results

1666 titles and abstracts, and 924 full text articles were assessed for eligibility. Of these, 253 articles met the eligibility criteria for inclusion. A purposive sampling of relevant websites was drawn from to complement the review, resulting in 102 records included in the review. Key areas for consideration were identified across the four themes of job security, wellbeing and equality of opportunity, teamwork and interdisciplinary, and research quality and accountability.

Conclusions

There are opportunities for research institutions to improve their own practice, however institutional solutions cannot act in isolation. Research institutions and research funders need to work together to build a more sustainable and inclusive research culture that is diverse in nature and supports individuals' well-being, career progression and performance.

Replication of the natural selection of bad science

This study reports an independent replication of the findings presented by Smaldino and McElreath (Smaldino, McElreath 2016 R. Soc. Open Sci. 3, 160384 (doi:10.1098/rsos.160384)). The replication was successful with one exception. We find that selection acting on scientist's propensity for replication frequency caused a brief period of exuberant replication not observed in the original paper due to a coding error. This difference does not, however, change the authors' original conclusions. We call for more replication studies for simulations as unique contributions to scientific quality assurance.

Challenges of Promoting Open Science within the NI4OS-Europe Project in Hungary

National Initiatives for Open Science in Europe (NI4OS-Europe) is a Horizon 2020 project related to the European Open Science Cloud (EOSC). One of the project objectives is promoting EOSC and open science in 15 Central and East European EU states and EU-associated countries. This paper describes the variety of promoting activities carried out in Hungary as part of the NI4OS-Europe project by the Governmental Agency for IT Development (KIFÜ). Identifying good practices will give us the chance to find the best communication channels and methods to promote open science and to manage expectations of funders, researchers and librarians. The audience diversity of organized NI4OS events was analyzed in this study. The anonymized dataset based on registration forms was filtered by profession. Results suggest that events are generally visited by more librarians than researchers. The only exception is the third forum where the main Hungarian research fund as co-organizer might have attracted researchers’ attention. This suggests that librarians are considered to be in charge of open science issues in general. Usage data of the open science news feed were also studied. The 130 posts between May 2021 and April 2022 and 2500 visitors until the end of June 2022 give us the chance to learn about the characteristics of the most visited posts. We can conclude that the focus of communication is on open and FAIR data management, while other areas receive less attention. The results show that despite more international posts being published, the target group is more interested in local information.

Sharing the Recipe: Reproducibility and Replicability in Research Across Disciplines

The open and transparent documentation of scientific processes has been established as a core antecedent of free knowledge. This also holds for generating robust insights in the scope of research projects. To convince academic peers and the public, the research process must be understandable and retraceable (reproducible), and repeatable (replicable) by others, precluding the inclusion of fluke findings into the canon of insights. In this contribution, we outline what reproducibility and replicability (R&R) could mean in the scope of different disciplines and traditions of research and which significance R&R has for generating insights in these fields. We draw on projects conducted in the scope of the Wikimedia "Open Science Fellows Program" (Fellowship Freies Wissen), an interdisciplinary, long-running funding scheme for projects contributing to open research practices. We identify twelve implemented projects from different disciplines which primarily focused on R&R, and multiple additional projects also touching on R&R. From these projects, we identify patterns and synthesize them into a roadmap of how research projects can achieve R&R across different disciplines. We further outline the ground covered by these projects and propose ways forward.

Research funders should be more transparent: a plea for open applications

Transparency is increasingly becoming the new norm and modus operandi of the global research enterprise. In this mini-review, we summarize ongoing initiatives to increase transparency in science and funding in particular. Based on this, we make a plea for the next step in funders' compliance with the principles of Open Science, suggesting the adoption of open applications. Our proposed model includes a plea for the publication of all submitted grant applications; open sharing of review reports, argumentations for funding decisions and project evaluation reports; and the disclosure of reviewers' and decision committee members' identities. In line with previous calls for transparency and the available evidence about these measures' effectiveness, we argue that open applications could lead to more diverse collaboration, recognition of research ideas, fairer procedures for grant allocation, more research on funding practices and increased trust in the funding allocation process.

Preprints as a driver of open science: Opportunities for Southeast Asia

Southeast Asia is an emerging force of open access scholarly output. For example, Indonesia is in a tight competition with United Kingdom as the largest publisher of open access journals and the second largest producer of open access articles in the world (according to DOAJ and the COKI OA Dashboard, respectively). However, this support for open practices is not yet reflected in institutional research policies in Southeast Asian countries, which still rely on criteria influenced by world university rankings that focus on publication outputs and do not incorporate elements related to research culture, integrity, or open science. Preprints have gained increasing attention across disciplines in the last few years, but they are still not included in institutional policies in SouthEast Asia. This paper discusses the potential for preprints to be a driving force for open science and for quality and integrity in scholarly outputs from Southeast Asia. There is a fledgling preprinting culture in the region, catalyzed by the RINarxiv preprint server in Indonesia and the Malaysia Open Science Platform. We argue that preprints have many advantages: opportunities for open access and for researchers to maintain copyright to their work, wide dissemination, encouraging feedback and critical thinking, and community governance. With these advantages, preprints can become a fast and open communication hub between researchers and all stakeholders in the research process. We recommend regulatory and practical steps to incorporate preprints into science policy and researchers' practices as an effort to promote research integrity, open data and reproducibility.

Open Science Knowledge Production: Addressing Epistemological Challenges and Ethical Implications

Open Science (OS) is envisioned to have a wide range of benefits including being more transparent, shared, accessible, and collaboratively developed than traditional science. Despite great enthusiasm, there are also several challenges with OS. In order to ensure that OS obtains its benefits, these challenges need to be addressed. Accordingly, the objective of this study is to provide an overview of one type of challenge, i.e., epistemological challenges with OS knowledge production, and their ethical implications. Based on a literature review, it (a) reveals factors undermining the envisioned benefits of OS, (b) identifies negative effects on knowledge production, and (c) exposes epistemological challenges with the various phases of the OS process. The main epistemic challenges are related to governance, framing, looping effects, proper data procurement, validation, replication, bias, and polarization. The ethical implications are injustice, reduced benefit (efficiency), increased harm (as a consequence of poor-quality science), deception and manipulation (reduced autonomy), and lack of trustworthiness. Accordingly, to obtain the envisioned benefits of OS, we need to address these epistemological challenges and their ethical implications.

A network of change: united action on research integrity

The last decade has seen renewed concern within the scientific community over the reproducibility and transparency of research findings. This paper outlines some of the various responsibilities of stakeholders in addressing the systemic issues that contribute to this concern. In particular, this paper asserts that a united, joined-up approach is needed, in which all stakeholders, including researchers, universities, funders, publishers, and governments, work together to set standards of research integrity and engender scientific progress and innovation. Using two developments as examples: the adoption of Registered Reports as a discrete initiative, and the use of open data as an ongoing norm change, we discuss the importance of collaboration across stakeholders.

Do German university medical centres promote robust and transparent research? A cross-sectional study of institutional policies

Background

In light of replication and translational failures, biomedical research practices have recently come under scrutiny. Experts have pointed out that the current incentive structures at research institutions do not sufficiently incentivise researchers to invest in robustness and transparency and instead incentivise them to optimize their fitness in the struggle for publications and grants. This cross-sectional study aimed to describe whether and how relevant policies of university medical centres in Germany support the robust and transparent conduct of research and how prevalent traditional metrics are.

Methods

For 38 German university medical centres, we searched for institutional policies for academic degrees and academic appointments as well as websites for their core facilities and research in general between December 2020 and February 2021. We screened the documents for mentions of indicators of robust and transparent research (study registration; reporting of results; sharing of research data, code and protocols; open access; and measures to increase robustness) and for mentions of more traditional metrics of career progression (number of publications; number and value of awarded grants; impact factors; and authorship order).

Results

While open access was mentioned in 16% of PhD regulations, other indicators of robust and transparent research were mentioned in less than 10% of institutional policies for academic degrees and academic appointments. These indicators were more frequently mentioned on the core facility and general research websites. Institutional policies for academic degrees and academic appointments had frequent mentions of traditional metrics.

Conclusions

References to robust and transparent research practices are, with a few exceptions, generally uncommon in institutional policies at German university medical centres, while traditional criteria for academic promotion and tenure still prevail.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00841-2.