Messing with Merton: The intersection between open science practices and Mertonian values

Although adherence to Mertonian values of science (i.e., communism, universalism, organized skepticism, disinterestedness) is desired and promoted in academia, such adherence can cause friction with the normative structures and practices of Open Science. Mertonian values and Open Science practices aim to improve the conduct and communication of research and are promoted by institutional actors. However, Mertonian values remain mostly idealistic and contextualized in local and disciplinary cultures and Open Science practices rely heavily on third-party resources and technology that are not equally accessible to all parties. Furthermore, although still popular, Mertonian values were developed in a different institutional and political context. In this article, we argue that new normative structures for science need to look beyond nostalgia and consider aspirations and outcomes of Open Science practices. To contribute to such a vision, we explore the intersection of several Open Science practices with Mertonian values to flesh out challenges involved in upholding these values. We demonstrate that this intersection becomes complicated when the interests of numerous groups collide and contrast. Acknowledging and exploring such tensions informs our understanding of researchers' behavior and supports efforts that seek to improve researchers' interactions with other normative structures such as research ethics and integrity frameworks.

A multi-year science research or engineering experience in high school gives women confidence to continue in the STEM pipeline or seek advancement in other fields: A 20-year longitudinal study

There remains a large gender imbalance in the science, technology, engineering and mathematics (STEM) workforce deriving from a leaky pipeline where women start losing interest and confidence in science and engineering as early as primary school. To address this disparity, the Science Research & Engineering Program (SREP) at Hathaway Brown School was established in 1998 to engage and expose their all-female high school students to STEM fields through an internship-like multi-year research experience at partnering institutions. We compare data from existing Hathaway Brown School SREP alumnae records from 1998–2018 (n = 495) to Non-SREP students and national datasets (National Center for Educational Statistics, National Science Foundation, and US Census data) to assess how SREP participation may influence persistence in the STEM pipeline and whether SREP alumnae attribute differences in these outcomes to the confidence and skill sets they learned from the SREP experience. The results reveal that women who participate in the SREP are more likely to pursue a major in a STEM field and continue on to a STEM occupation compared to non-SREP students, national female averages, and national subsets. Participants attribute their outcomes to an increase in confidence, establishment of technical and professional skills, and other traits strengthened through the SREP experience. These data suggest that implementing similar experiential programs for women in science and engineering at the high school stage could be a promising way to combat the remaining gender gap in STEM fields.

Drawing lessons from the COVID-19 pandemic: science and epistemic humility should go together

During the COVID-19 pandemic, scientific experts advised governments for measures to be promptly taken; they also helped people to understand the situation. They carried out this role in the face of a worldwide emergency, when scientific understanding was still underway. Public scientific disputes also arose, creating confusion among people. This article highlights the importance of experts' epistemic stance under these circumstances. It suggests they should embrace the intellectual virtue of epistemic humility, regulating their epistemic behavior and communication accordingly. In so doing, they would also favour the functioning of the broad network of knowledge-based experts, which is required to properly address all the aspects of the global pandemic.

Science, misinformation and digital technology during the Covid-19 pandemic

Three interdependent factors are behind the current Covid-19 pandemic distorted narrative: (1) science´s culture of "publish or perish", (2) misinformation spread by traditional media and social digital media and (3) distrust of technology for tracing contacts and its privacy-related issues. In this short paper, I wish to tackle how these three factors have added up to give rise to a negative public understanding of science in times of a health crisis, such as the current Covid-19 pandemic and finally, how to confront all these problems.

Subversive affinities: Embracing soviet science in late 1940s Romania

This article discusses the appropriation of Soviet science in Romania during the late 1940s. To achieve this, I discuss various publications on biology, anthropology, heredity and genetics. In a climate of major political change, following the end of the Second World War, all scientific fields in Romania were gradually subjected to political pressures to adapt and change according to a new ideological context. Yet the adoption of Soviet science during the late 1940s was not a straightforward process of scientific acculturation. Whilst the deference to Soviet authors remained consistent through most of Romanian scientific literature at the time, what is perhaps less visible is the attempt to refashion Romanian science itself in order to serve the country's new political imaginary and social transformation. Some Romanian biologists and physicians embraced Soviet scientific theories as a demonstration of their loyalty to the newly established regime. Others, however, were remained committed to local and Western scientific traditions they deemed essential to the survival of their discipline. A critical reassessment of the late 1940s is essential to an understanding of these dissensions as well as of the overall political and institutional constraints shaping the development of a new politics of science in communist Romania.

Openness and trust in data-intensive science: the case of biocuration

Data-intensive science comes with increased risks concerning quality and reliability of data, and while trust in science has traditionally been framed as a matter of scientists being expected to adhere to certain technical and moral norms for behaviour, emerging discourses of open science present openness and transparency as substitutes for established trust mechanisms. By ensuring access to all available information, quality becomes a matter of informed judgement by the users, and trust no longer seems necessary. This strategy does not, however, take into consideration the networks of professionals already enabling data-intensive science by providing high-quality data. In the life sciences, biological data- and knowledge bases managed by expert biocurators have become crucial for data-intensive research. In this paper, I will use the case of biocurators to argue that openness and transparency will not diminish the need for trust in data-intensive science. On the contrary, data-intensive science requires a reconfiguration of existing trust mechanisms in order to include those who take care of and manage scientific data after its production.

Social attention of the top 50 scientific articles on gastric cancer: Bibliometric and altmetric analysis

PURPOSE

Bibliometric and Altmetric analyse the most influental publications and provide important perspectives regarding article impact. The Altmetric Score (AS) is an automatically calculated metric for monitoring social media attention. The hypothesis is that would citation number correlate positively with AS?

METHODS

Gastric cancer as a search term was entered into Thomson Reuter's Web of Science database to identify articles in the last decade. The 50 most cited articles were analysed by topic, journal, author, year, and AS.

RESULTS

By bibliometric criteria, it was observed that there were 63,154 articles published in the literature on gastric cancer in the last decade. The most cited article was made by Bang et al for the treatment of HER2-positive advanced gastric or gastro-oesophageal junction cancer. The study with the highest altmetric score was published by Bass et al and described the molecular evaluation of gastric cancer as part of the cancer genome atlas. The majority of the top 50 most cited article types were original scientific papers (n=39). The highest number of publications was published in the Journal of Clinical Oncology (n=9) and the most contributing country was the United States (n=22). Positive correlation was detected between the number of citations and AS. We revealed a negative correlation between AS and numbers of years since publication of the article.

CONCLUSION

The remarkable result of this study is that both the citation and AS of the articles provide important but different viewpoints.

Quantifying and contextualizing the impact of bioRxiv preprints through automated social media audience segmentation

Engagement with scientific manuscripts is frequently facilitated by Twitter and other social media platforms. As such, the demographics of a paper's social media audience provide a wealth of information about how scholarly research is transmitted, consumed, and interpreted by online communities. By paying attention to public perceptions of their publications, scientists can learn whether their research is stimulating positive scholarly and public thought. They can also become aware of potentially negative patterns of interest from groups that misinterpret their work in harmful ways, either willfully or unintentionally, and devise strategies for altering their messaging to mitigate these impacts. In this study, we collected 331,696 Twitter posts referencing 1,800 highly tweeted bioRxiv preprints and leveraged topic modeling to infer the characteristics of various communities engaging with each preprint on Twitter. We agnostically learned the characteristics of these audience sectors from keywords each user's followers provide in their Twitter biographies. We estimate that 96% of the preprints analyzed are dominated by academic audiences on Twitter, suggesting that social media attention does not always correspond to greater public exposure. We further demonstrate how our audience segmentation method can quantify the level of interest from nonspecialist audience sectors such as mental health advocates, dog lovers, video game developers, vegans, bitcoin investors, conspiracy theorists, journalists, religious groups, and political constituencies. Surprisingly, we also found that 10% of the preprints analyzed have sizable (>5%) audience sectors that are associated with right-wing white nationalist communities. Although none of these preprints appear to intentionally espouse any right-wing extremist messages, cases exist in which extremist appropriation comprises more than 50% of the tweets referencing a given preprint. These results present unique opportunities for improving and contextualizing the public discourse surrounding scientific research.

Consolidating RRI and Open Science: understanding the potential for transformative change

In European research and innovation policy, Responsible Research and Innovation (RRI) and Open Science (OS) encompass two co-existing sets of ambitions concerning systemic change in the practice of research and innovation. This paper is an exploratory attempt to uncover synergies and differences between RRI and OS, by interrogating what motivates their respective transformative agendas. We offer two storylines that account for the specific contexts and dynamics from which RRI and OS have emerged, which in turn offer entrance points to further unpacking what 'opening up' to society means with respect to the transformative change agendas that are implicit in the two agendas. We compare differences regarding the 'how' of opening up in light of the 'why' to explore common areas of emphasis in both OS and RRI. We argue that while both agendas align with mission-oriented narratives around grand societal challenges, OS tends to emphasize efficiency and technical optimisation over RRI's emphasis on normative concerns and democracy deficits, and that the two agendas thus contrast in their relative legitimate emphasis on doable outcomes versus desirable outcomes. In our conclusion, we reflect on the future outlook for RRI and OS' co-existence and uptake, and on what their respective ambitions for transformation might mean for science-society scholars and scholarship.

How Do Scientific Views Change? Notes From an Extended Adversarial Collaboration

There are few examples of an extended adversarial collaboration, in which investigators committed to different theoretical views collaborate to test opposing predictions. Whereas previous adversarial collaborations have produced single research articles, here, we share our experience in programmatic, extended adversarial collaboration involving three laboratories in different countries with different theoretical views regarding working memory, the limited information retained in mind, serving ongoing thought and action. We have focused on short-term memory retention of items (letters) during a distracting task (arithmetic), and effects of aging on these tasks. Over several years, we have conducted and published joint research with preregistered predictions, methods, and analysis plans, with replication of each study across two laboratories concurrently. We argue that, although an adversarial collaboration will not usually induce senior researchers to abandon favored theoretical views and adopt opposing views, it will necessitate varieties of their views that are more similar to one another, in that they must account for a growing, common corpus of evidence. This approach promotes understanding of others' views and presents to the field research findings accepted as valid by researchers with opposing interpretations. We illustrate this process with our own research experiences and make recommendations applicable to diverse scientific areas.

Living with COVID-19-triggered pseudoscience and conspiracies

OBJECTIVE

This piece of work proposes a way for the wise management of pseudoscience and conspiracy theories.

METHODS

This work encompassed a review of relevant literature and synthesized the critical thoughts on the proper management of pseudoscience and conspiracy theories.

RESULT

The coronavirus disease 2019 (COVID-19) overwhelmingly challenges the competency of the digital generation. Consequently, public had been left stranded, helpless and anxious, especially during the peak season of the pandemic. However, this crisis creates a conducive environment for pseudoscience and conspiracy theories to proliferate. Pseudoscience and conspiracy theories negatively impacted the effort made to contain COVID-19.

CONCLUSION

This piece of work, however, argues that although pseudoscience and conspiracy theories are real threats to conventional science, effort needs to be made to develop a sort of database to archive and curate them for downstream use.

Addressing selective reporting of experiments through predefined exclusion criteria

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we discuss how to balance clarity and thoroughness when reporting the results of research, and suggest that predefining the criteria for excluding experiments might help researchers to achieve this balance.

Filling in the gaps: The interpretation of curricula vitae in peer review

In this article, we study the use of curricula vitae (CV) for competitive funding decisions in science. The typically sober administrative style of academic résumés evokes the impression of straightforwardly conveyed, objective evidence on which to base comparisons of past achievements and future potentials. We instead conceptualize the evaluation of biographical evidence as a generative interplay between an historically grown, administrative infrastructure (the CV), and a situated evaluative practice in which the representational function of that infrastructure is itself interpreted and established. The use of CVs in peer review can be seen as a doubly comparative practice, where referees compare not only applicants (among each other or to an imagined ideal of excellence), but also their own experience-based understanding of practice and the conceptual assumptions that underpin CV categories. Empirically, we add to existing literature on peer review by drawing attention to self-correcting mechanisms in the reproduction of the scientific workforce. Conceptually, we distinguish three modalities of how the doubly comparative use of CVs can shape the assessment of applicants: calibration, branching out, and repair. The outcome of this reflexive work should not be seen as predetermined by situational pressures. In fact, bibliographic categories such as authorship of publications or performance metrics may themselves come to be problematized and reshaped in the process.

Strategic science communication as planned behavior: Understanding scientists' willingness to choose specific tactics

Strategic science communicators need to select tactics that can help them achieve both their short-term communication objectives and long-term behavioral goals. However, little previous research has sought to develop theory aimed at understanding what makes it more likely that a communicator will prioritize specific communication tactics. The current study aims to advance the development of a theory of strategic science communication as planned behavior based on the Integrated Behavioral Model. It does so in the context of exploring Canadian scientists' self-reported willingness to prioritize six different tactics as a function of attitudinal, normative, and efficacy beliefs. The results suggest that scientists' beliefs about ethicality, norms, response efficacy, and self-efficacy, are all meaningful predictors of willingness to prioritize specific tactics. Differences between scientists in terms of demographics and related variables provide only limited benefit in predicting such willingness.

Null Hypothesis Testing ≠ Scientific Inference: A Critique of the Shaky Premise at the Heart of the Science and Values Debate, and a Defense of Value-Neutral Risk Assessment

Many philosophers and statisticians argue that risk assessors are morally obligated to evaluate the probabilities and consequences of methodological error, and to base their decisions of whether to adopt a given parameter value, model, or hypothesis on those considerations. This argument is couched within the rubric of null hypothesis testing, which I suggest is a poor descriptive and normative model for risk assessment. Risk regulation is not primarily concerned with evaluating the probability of data conditional upon the null hypothesis, but rather with measuring risks, estimating the consequences of available courses of action and inaction, formally characterizing uncertainty, and deciding what to do based upon explicit values and decision criteria. In turn, I defend an ideal of value-neutrality, whereby the core inferential tasks of risk assessment-such as weighing evidence, estimating parameters, and model selection-should be guided by the aim of correspondence to reality. This is not to say that value judgments be damned, but rather that they should be accounted for within a structured approach to decision analysis, rather than embedded within risk assessment in an informal manner.

Objectivity in science and law: A shared rescue strategy

The ideal of objectivity is in crisis in science and the law, and yet it continues to do important work in both practices. This article describes that crisis and develops a shared rescue strategy for objectivity in both domains. In a recent article, Inkeri Koskinen (2018) attempts to bring unity to the fragmented discourse on objectivity in the philosophy of science with a risk account of objectivity. To put it simply, she argues that we call practitioners, processes, and products of science objective when they identify and manage certain important epistemic risks. We endorse this view and attempt to tailor Koskinen's strategy to the problem of objectivity in the legal context. To do so, we develop a novel notion of phronetic risk, and argue that we call practitioners, processes, and products of law objective when they identify and manage certain important epistemic and/or phronetic risks. Our attempt to rescue objectivity is especially important for work at the intersection of law and psychiatry. For that intersection represents a place where skeptical worries about objectivity in science and law work in tandem to pose serious critical challenges to contemporary practice; and our rescue strategy represents a promising way to negotiate those challenges.

A personal view on traits useful for success in science: Daniel S. Lehrman Award Lecture

As the 2018 recipient of the Daniel S. Lehrman Lifetime Achievement Award from the Society for Behavioral Neuroendocrinology, I was asked to give a short lecture in the Young Investigator Symposium of the combined Society for Behavioral Neuroendocrinology/International Congress on Neuroendocrinology meeting in Toronto. This lecture focused on one person's thoughts on what it takes to be successful in an academic science career. In this paper, I elaborate on success, on traits that may be useful for success in science, and on where the field of Behavioral Neuroendocrinology may be going.

Changing times for science and the public: Science journalists' roles for the responsible communication of science

The increasing number of corrections in the scientific record and the debate about reproducibility affect journalists’ reporting about science and thereby public opinion on scientists and research.

Multi-institutional study of GRE scores as predictors of STEM PhD degree completion: GRE gets a low mark

The process of selecting students likely to complete science, technology, engineering and mathematics (STEM) doctoral programs has not changed greatly over the last few decades and still relies heavily on Graduate Record Examination (GRE) scores in most U.S. universities. It has been long debated whether the GRE is an appropriate selection tool and whether overreliance on GRE scores may compromise admission of students historically underrepresented in STEM. Despite many concerns about the test, there are few studies examining the efficacy of the GRE in predicting PhD completion and even fewer examining this question in STEM fields. For the present study, we took advantage of a long-lived collaboration among institutions in the Northeast Alliance for Graduate Education and the Professoriate (NEAGEP) to gather comparable data on GRE scores and PhD completion for 1805 U.S./Permanent Resident STEM doctoral students in four state flagship institutions. We found that GRE Verbal (GRE V) and GRE Quantitative (GRE Q) scores were similar for women who completed STEM PhD degrees and those who left programs. Remarkably, GRE scores were significantly higher for men who left than counterparts who completed STEM PhD degrees. In fact, men in the lower quartiles of GRE V or Q scores finished degrees more often than those in the highest quartile. This pattern held for each of the four institutions in the study and for the cohort of male engineering students across institutions. GRE scores also failed to predict time to degree or to identify students who would leave during the first year of their programs. Our results suggests that GRE scores are not an effective tool for identifying students who will be successful in completing STEM doctoral programs. Considering the high cost of attrition from PhD programs and its impact on future leadership for the U.S. STEM workforce, we suggest that it is time to develop more effective and inclusive admissions strategies.

Psychometrics in action, science as practice

Practitioners in health sciences education and assessment regularly use a range of psychometric techniques to analyse data, evaluate models, and make crucial progression decisions regarding student learning. However, a recent editorial entitled "Is Psychometrics Science?" highlighted some core epistemological and practical problems in psychometrics, and brought its legitimacy into question. This paper attempts to address these issues by applying some key ideas from history and philosophy of science (HPS) discourse. I present some of the conceptual developments in HPS that have bearing on the psychometrics debate. Next, by shifting the focus onto what constitutes the practice of science, I discuss psychometrics in action. Some incorrectly conceptualize science as an assemblage of truths, rather than an assemblage of tools and goals. Psychometrics, however, seems to be an assemblage of methods and techniques. Psychometrics in action represents a range of practices using specific tools in specific contexts. This does not render the practice of psychometrics meaningless or futile. Engaging in debates about whether or not we should regard psychometrics as 'scientific' is, however, a fruitless enterprise. The key question and focus should be whether, on what grounds, and in what contexts, the existing methods and techniques used by psychometricians can be justified or criticized.

High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison

Countries, research institutions, and scholars are interested in identifying and promoting high-impact and transformative scientific research. This paper presents a novel set of text- and citation-based metrics that can be used to identify high-impact and transformative works. The 11 metrics can be grouped into seven types: Radical-Generative, Radical-Destructive, Risky, Multidisciplinary, Wide Impact, Growing Impact, and Impact (overall). The metrics are exemplified, validated, and compared using a set of 10,778,696 MEDLINE articles matched to the Science Citation Index ExpandedTM. Articles are grouped into six 5-year periods (spanning 1983-2012) using publication year and into 6,159 fields constructed using comparable MeSH terms, with which each article is tagged. The analysis is conducted at the level of a field-period pair, of which 15,051 have articles and are used in this study. A factor analysis shows that transformativeness and impact are positively related (ρ = .402), but represent distinct phenomena. Looking at the subcomponents of transformativeness, there is no evidence that transformative work is adopted slowly or that the generation of important new concepts coincides with the obsolescence of existing concepts. We also find that the generation of important new concepts and highly cited work is more risky. Finally, supporting the validity of our metrics, we show that work that draws on a wider range of research fields is used more widely.

The genesis of a conspiracy theory: Why do people believe in scientific conspiracy theories and how do they spread?

Conspiracy theories about science can do real damage if they affect matters of health and disease. Understanding the psychology of a conspiracy theorist along with openness about how scientists come to conclusions could help to combat harmful theories.

Scientific progress despite irreproducibility: A seeming paradox

It appears paradoxical that science is producing outstanding new results and theories at a rapid rate at the same time that researchers are identifying serious problems in the practice of science that cause many reports to be irreproducible and invalid. Certainly, the practice of science needs to be improved, and scientists are now pursuing this goal. However, in this perspective, we argue that this seeming paradox is not new, has always been part of the way science works, and likely will remain so. We first introduce the paradox. We then review a wide range of challenges that appear to make scientific success difficult. Next, we describe the factors that make science work-in the past, present, and presumably also in the future. We then suggest that remedies for the present practice of science need to be applied selectively so as not to slow progress and illustrate with a few examples. We conclude with arguments that communication of science needs to emphasize not just problems but the enormous successes and benefits that science has brought and is now bringing to all elements of modern society.

Issues with data and analyses: Errors, underlying themes, and potential solutions

Some aspects of science, taken at the broadest level, are universal in empirical research. These include collecting, analyzing, and reporting data. In each of these aspects, errors can and do occur. In this work, we first discuss the importance of focusing on statistical and data errors to continually improve the practice of science. We then describe underlying themes of the types of errors and postulate contributing factors. To do so, we describe a case series of relatively severe data and statistical errors coupled with surveys of some types of errors to better characterize the magnitude, frequency, and trends. Having examined these errors, we then discuss the consequences of specific errors or classes of errors. Finally, given the extracted themes, we discuss methodological, cultural, and system-level approaches to reducing the frequency of commonly observed errors. These approaches will plausibly contribute to the self-critical, self-correcting, ever-evolving practice of science, and ultimately to furthering knowledge.

The preregistration revolution

Progress in science relies in part on generating hypotheses with existing observations and testing hypotheses with new observations. This distinction between postdiction and prediction is appreciated conceptually but is not respected in practice. Mistaking generation of postdictions with testing of predictions reduces the credibility of research findings. However, ordinary biases in human reasoning, such as hindsight bias, make it hard to avoid this mistake. An effective solution is to define the research questions and analysis plan before observing the research outcomes-a process called preregistration. Preregistration distinguishes analyses and outcomes that result from predictions from those that result from postdictions. A variety of practical strategies are available to make the best possible use of preregistration in circumstances that fall short of the ideal application, such as when the data are preexisting. Services are now available for preregistration across all disciplines, facilitating a rapid increase in the practice. Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.

The new watchdogs' vision of science: A roundtable with Ivan Oransky ( Retraction Watch) and Brandon Stell ( PubPeer)

On March 3rd, 2016, the authors of this note hosted a conference entitled 'Destabilized Science' at the University of California, Los Angeles, to which we invited two representatives of core actors within the new science watchdog pack: Ivan Oransky, co-founder in 2010 of Retraction Watch, and Brandon Stell, co-founder in 2012 of PubPeer. After the formal conference, we organized a roundtable to discuss these invitees' experience and their vision of contemporary science. Mario Biagioli (University of California, Davis), Michael Chwe (UCLA) and Aaron Panofsky (UCLA) participated to the conversation. An edited transcript of the discussion and a short podcast version are being published on Transmissions ( ssstransmissions.org ) the new blog associated with Social Studies of Science.

Black and White and Shades of Gray [State of the Art]

"Moderation in all things" is a popular saying that many of us have heard all our lives. Still, a good number of people seem to have forgotten the sentiment behind this advice. Instead of looking for the good that exists within the bad and the bad that dwells within the good, people are choosing to line up behind one extreme or another. Nuances are being ignored in favor of strong positions on either side of the middle. This has led inevitably to polarization, partisanship, and balkanization in our society. For some reason, moderation has been forgotten.

Enhancing Scientific Foundations to Ensure Reproducibility: A New Paradigm

Progress in science is dependent on a strong foundation of reliable results. The publish or perish paradigm in research, coupled with an increase in retracted articles from the peer-reviewed literature, is beginning to erode the trust of both the scientific community and the public. The NIH is combating errors by requiring investigators to follow new guidelines addressing scientific premise, experimental design, biological variables, and authentication of reagents. Herein, we discuss how implementation of NIH guidelines will help investigators proactively address pitfalls of experimental design and methods. Careful consideration of the variables contributing to reproducibility helps ensure robust results. The NIH, investigators, and journals must collaborate to ensure that quality science is funded, explored, and published.

(I Can't Get No) Saturation: A simulation and guidelines for sample sizes in qualitative research

I explore the sample size in qualitative research that is required to reach theoretical saturation. I conceptualize a population as consisting of sub-populations that contain different types of information sources that hold a number of codes. Theoretical saturation is reached after all the codes in the population have been observed once in the sample. I delineate three different scenarios to sample information sources: "random chance," which is based on probability sampling, "minimal information," which yields at least one new code per sampling step, and "maximum information," which yields the largest number of new codes per sampling step. Next, I use simulations to assess the minimum sample size for each scenario for systematically varying hypothetical populations. I show that theoretical saturation is more dependent on the mean probability of observing codes than on the number of codes in a population. Moreover, the minimal and maximal information scenarios are significantly more efficient than random chance, but yield fewer repetitions per code to validate the findings. I formulate guidelines for purposive sampling and recommend that researchers follow a minimum information scenario.

Implementation of Good Laboratory Practices (GLP) in basic scientific research: Translating the concept beyond regulatory compliance

The principles of Good Laboratory Practices (GLPs) are mainly intended for the laboratories performing studies for regulatory compliances. However, today GLP can be applied to broad disciplines of science to cater to the needs of the experimental objectives, generation of quality data and assay reproducibility. Considering its significance, it can now be applied in academics; industries as well as government set ups throughout the world. GLP is the best way to promote the reliability, reproducibility of the test data and hence facilitates the international acceptability. Now it is high time to translate and implement the concept of GLP beyond regulatory studies. Thus, it can pave the way for better understanding of scientific problems and help to maintain a good human and environmental health. Through this review, we have made an attempt to explore the uses of GLP principles in different fields of science and its acceptability as well as looking for its future perspectives.

Can cancer researchers accurately judge whether preclinical reports will reproduce?

There is vigorous debate about the reproducibility of research findings in cancer biology. Whether scientists can accurately assess which experiments will reproduce original findings is important to determining the pace at which science self-corrects. We collected forecasts from basic and preclinical cancer researchers on the first 6 replication studies conducted by the Reproducibility Project: Cancer Biology (RP:CB) to assess the accuracy of expert judgments on specific replication outcomes. On average, researchers forecasted a 75% probability of replicating the statistical significance and a 50% probability of replicating the effect size, yet none of these studies successfully replicated on either criterion (for the 5 studies with results reported). Accuracy was related to expertise: experts with higher h-indices were more accurate, whereas experts with more topic-specific expertise were less accurate. Our findings suggest that experts, especially those with specialized knowledge, were overconfident about the RP:CB replicating individual experiments within published reports; researcher optimism likely reflects a combination of overestimating the validity of original studies and underestimating the difficulties of repeating their methodologies.

Science is supposed to be self-correcting. However, the efficiency with which science self-corrects depends in part on how well scientists can anticipate whether particular findings will hold up over time. We examined whether expert researchers could accurately forecast whether mouse experiments in 6 prominent preclinical cancer studies conducted by the Reproducibility Project: Cancer Biology would reproduce original effects. Experts generally overestimated the likelihood that replication studies would reproduce the effects observed in original studies. Experts with greater publication impact (as measured by h-index) provided more accurate forecasts, but experts did not consistently perform better than trainees, and topic-specific expertise did not improve forecast skill. Our findings suggest that experts tend to overestimate the reproducibility of original studies and/or they underappreciate the difficulty of independently repeating laboratory experiments from original protocols.

Good enough practices in scientific computing

Computers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don't know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.