The Hong Kong Principles for assessing researchers: Fostering research integrity

Education research is still the hardest science: a proposal for improving its trustworthiness and usability

In this essay, we argue that colleges of education, particularly those at research-intensive institutions, favor simplistic notions of scholarly impact and that this trend has concerning implications for the field, for researchers, and for the public at large. After describing the challenges and shortcomings of the current models of research assessment in education, we outline an alternative proposal in which trustworthiness and usability of research would complement traditional metrics of scholarly relevance. This proposal encourages a twofold approach to research assessment that involves (1) a more thorough analysis of the limitations and problems generated by the use of simplistic notions of scholarly impact, and (2) a commitment to the implementation of more equitable systems based on a broader range of assessment measures to assess faculty research contributions.

Transparent, Open, and Reproducible Prevention Science

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Practices for Research Integrity Promotion in Research Performing Organisations and Research Funding Organisations: A Scoping Review

Research integrity (RI) is a continuously developing concept, and increasing emphasis is put on creating RI promotion practices. This study aimed to map the existing RI guidance documents at research performing organisations (RPOs) and research funding organisations (RFOs). A search of bibliographic databases and grey literature sources was performed, and retrieved documents were screened for eligibility. The search of bibliographical databases and reference lists of selected articles identified a total of 92 documents while the search of grey literature sources identified 118 documents for analysis. The retrieved documents were analysed based on their geographical origin, research field and organisational origin (RPO or RFO) of RI practices, types of guidance presented in them, and target groups to which RI practices are directed. Most of the identified practices were developed for research in general, and are applicable to all research fields (n = 117) and medical sciences (n = 78). They were mostly written in the form of guidelines (n = 136) and targeted researchers (n = 167). A comprehensive search of the existing RI promotion practices showed that initiatives mostly come from RPOs while only a few RI practices originate from RFOs. This study showed that more RI guidance documents are needed for natural sciences, social sciences, and humanities since only a small number of documents was developed specifically for these research fields. The explored documents and the gaps in knowledge identified in this study can be used for further development of RI promotion practices in RPOs and RFOs.

Researchers' Perceptions of a Responsible Research Climate: A Multi Focus Group Study

The research climate plays a key role in fostering integrity in research. However, little is known about what constitutes a responsible research climate. We investigated academic researchers' perceptions on this through focus group interviews. We recruited researchers from the Vrije Universiteit Amsterdam and the Amsterdam University Medical Center to participate in focus group discussions that consisted of researchers from similar academic ranks and disciplinary fields. We asked participants to reflect on the characteristics of a responsible research climate, the barriers they perceived and which interventions they thought fruitful to improve the research climate. Discussions were recorded and transcribed at verbatim. We used inductive content analysis to analyse the focus group transcripts. We conducted 12 focus groups with 61 researchers in total. We identified fair evaluation, openness, sufficient time, integrity, trust and freedom to be mentioned as important characteristics of a responsible research climate. Main perceived barriers were lack of support, unfair evaluation policies, normalization of overwork and insufficient supervision of early career researchers. Possible interventions suggested by the participants centered around improving support, discussing expectations and improving the quality of supervision. Some of the elements of a responsible research climate identified by participants are reflected in national and international codes of conduct, such as trust and openness. Although it may seem hard to change the research climate, we believe that the realisation that the research climate is suboptimal should provide the impetus for change informed by researchers' experiences and opinions.

Nanocharacterization, Materials Modeling, and Research Integrity as Enablers of Sound Risk Assessment: Designing Responsible Nanotechnology

Nanotechnology, as a mature enabling technology, has great potential to boost societal welfare. However, nanomaterials' current and foreseen applications raise serious concerns about their impact on human health and the environment. These concerns emerge because a reliable risk assessment in nanotechnology is yet to be achieved. The reasons for such a shortcoming are the inherent difficulties in characterizing nanomaterials properties. The interaction of characterization with modeling is an open issue and, due to overarching concerns about the reliability of research results, usually framed within the context of research integrity. This essay explores the connection between these different, but deeply intertwined concerns and the way they enable the production of responsible nanotechnology, i.e., nanotechnology devoted to societal welfare.

Forest restoration or propaganda? The need for Transparency and Openness Promotion (TOP) scores to uphold research integrity

In a time of environmental crisis and ‘fake news’, there are calls for scientists to engage in public debate or advocacy. Some are wary, fearing that revealing subjective views poses a risk to scientific credibility or erodes trust in scholarly publishing. Others are less concerned, seeing it as their duty to society or an opportunity to boost their profile. Ideally, we need better checks and balances that allow scientists to contribute to public discourse without fear of compromising the credibility of their science, while avoiding subjective views influencing the outcomes of peer-reviewed research. For better or worse, scientists have personal views. The question is not whether they should be condoned or condemned, but how they should be managed in the context of scholarly publishing to maximise benefits and minimise negative outcomes. Using the recent contention around global tree ‘restoration’ potential as an example, I propose we score journals and articles based on the Transparency and Openness Promotion (TOP) guidelines and associated criteria. A high TOP score means readers have sufficient access to information to assess the objectivity and credibility of scientific publications and their authors. I show that current practice provides very little access to information, and readers are essentially being asked to have faith in the scholarly publication system. We must do better.

An inexpensive and easy-to-implement approach to a Quality Management System for an academic research lab

Background: Increasing concerns emerge regarding the limited success in reproducing data and translating research results into applications. This is a major problem for science, society and economy. Driven by industry or scientific networks, several attempts to combat this crisis are initiated. However, only few measures address the applicability and feasibility of implementation of actions into an academic research environment with limited resources.

Methods: Here we propose a strategy catalogue aiming for a quality management system suitable for many research labs, on the example of a cell culture focused laboratory. Our proposal is guided by its inexpensiveness and possibility of rapid installation.  For this we used eLabFTW, an electronic lab book, as hub for all other components of our Quality Management System (QMS) and digital storage of lab journals. We introduced Standard Operation Procedures (SOPs) as well as a managed bio bank for safer long-term storage of bio samples. Next, we set up a lab meeting as feedback mechanism for the QMS. Finally, we implemented an automated pipeline to be used for example for drug screens.

Results: With this effort we want to reduce individual differences in work techniques, to further improve the quality of our results. Although, just recently established, we can already observe positive outcomes in quality of experimental results, improvements in sample and data storage, stakeholder engagement and even promotion of new scientific discoveries.

Conclusions: We believe that our experiences can help to establish a road map to increase value and output of preclinical research in academic labs with limited budget and personnel.

When you are making plans to publish research, you need to plan for data sharing

Open data is another step on the pathway of strengthening medical research. Allowing access to data facilitates testing the reproducibility of research findings. It also allows for the testing of new hypotheses, the incorporation of individual level data into meta-analyses and the development of very large data sets in which to develop and test new algorithms. There are now many data repositories that researchers can use to share their protocols, syntax and data. There are strategies both for managing what other researchers do with publically available data and for rewarding researchers who share their data. There is a strong ethical argument for making data publically available and research participants are generally supportive of this approach.

Working 9 to 5, not the way to make an academic living: observational analysis of manuscript and peer review submissions over time

OBJECTIVE

To determine whether researchers are submitting manuscripts and peer reviews to BMJ journals out of hours and whether this has changed over time.

DESIGN

Observational study of research manuscripts and peer reviews submitted between 2012 and 2019 for which an author's address could be geocoded.

SETTING

Online BMJ submission systems for two large general medical journals.

MAIN OUTCOME MEASURES

Manuscript and peer review submissions on weekends, on national holidays, and by hour of day (to determine early mornings and late nights). Logistic regression was used to estimate the probability of manuscript and peer review submissions on weekends or holidays.

RESULTS

The analyses included more than 49 000 manuscript submissions and 76 000 peer reviews. Little change over time was seen in the average probability of manuscript or peer review submissions occurring on weekends or holidays. The levels of out of hours work were high, with average probabilities of 0.14 to 0.18 for work on the weekends and 0.08 to 0.13 for work on holidays compared with days in the same week. Clear and consistent differences were seen between countries. Chinese researchers most often worked at weekends and at midnight, whereas researchers in Scandinavian countries were among the most likely to submit during the week and the middle of the day.

CONCLUSION

The differences between countries that are persistent over time show that a "culture of overwork" is a literal thing, not just a figure of speech.