Meta-research: Evaluation and Improvement of Research Methods and Practices

A new taxonomy was developed for overlap across 'overviews of systematic reviews': A meta-research study of research waste

Multiple 'overviews of reviews' conducted on the same topic ("overlapping overviews") represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesized reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD-10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD-10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time, and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied, which do not need to be duplicated.

Design and methodological characteristics of studies using observational routinely collected health data for investigating the link between cancer and neurodegenerative diseases: protocol for a meta-research study

INTRODUCTION

Health services generate large amounts of routine health data (eg, administrative databases, disease registries and electronic health records), which have important secondary uses for research. Increases in the availability and the ability to access and analyse large amounts of data represent a major opportunity for conducting studies on the possible relationships between complex diseases. The objective of this study will be to evaluate the design, methods and reporting of studies conducted using observational routinely collected health data for investigating the link between cancer and neurodegenerative diseases.

METHODS AND ANALYSIS

This is the protocol for a meta-research study. We registered the study protocol within the Open Science Framework: https://osf.io/h2qjg. We will evaluate observational studies (eg, cohort and case-control) conducted using routinely collected health data for investigating the associations between cancer and neurodegenerative diseases (such as Alzheimer's disease, amyotrophic lateral sclerosis/motor neuron disease, Huntington's disease, multiple sclerosis and Parkinson's disease). The following electronic databases will be searched (from their inception onwards): MEDLINE, Embase and Web of Science Core Collection. Screening and selection of articles will be conducted by at least two researchers. Potential discrepancies will be resolved via discussion. Design, methods and reporting characteristics in each article will be extracted using a standardised data extraction form. Information on general, methodological and transparency items will be reported. We will summarise our findings with tables and graphs (eg, bar charts, forest plots).

ETHICS AND DISSEMINATION

Due to the nature of the proposed study, no ethical approval will be required. We plan to publish the full study in an open access peer-reviewed journal and disseminate the findings at scientific conferences and via social media. All data will be deposited in a cross-disciplinary public repository.

Using COVID-19 Pandemic as a Prism: A Systematic Review of Methodological Approaches and the Quality of Empirical Studies on Physical Activity Behavior Change

Background

The COVID-19 pandemic has highlighted the importance of scientific endeavors. The goal of this systematic review is to evaluate the quality of the research on physical activity (PA) behavior change and its potential to contribute to policy-making processes in the early days of COVID-19 related restrictions.

Methods

We conducted a systematic review of methodological quality of current research according to PRISMA guidelines using Pubmed and Web of Science, of articles on PA behavior change that were published within 365 days after COVID-19 was declared a pandemic by the World Health Organization (WHO). Items from the JBI checklist and the AXIS tool were used for additional risk of bias assessment. Evidence mapping is used for better visualization of the main results. Conclusions about the significance of published articles are based on hypotheses on PA behavior change in the light of the COVID-19 pandemic.

Results

Among the 1,903 identified articles, there were 36% opinion pieces, 53% empirical studies, and 9% reviews. Of the 332 studies included in the systematic review, 213 used self-report measures to recollect prepandemic behavior in often small convenience samples. Most focused changes in PA volume, whereas changes in PA types were rarely measured. The majority had methodological reporting flaws. Few had very large samples with objective measures using repeated measure design (pre and during the pandemic). In addition to the expected decline in PA duration, these studies show that many of those who were active prepandemic, continued to be active during the pandemic.

Conclusions

Research responded quickly at the onset of the pandemic. However, most of the studies lacked robust methodology, and PA behavior change data lacked the accuracy needed to guide policy makers. To improve the field, we propose the implementation of longitudinal cohort studies by larger organizations such as WHO to ease access to data on PA behavior, and suggest those institutions set clear standards for this research. Researchers need to ensure a better fit between the measurement method and the construct being measured, and use both objective and subjective measures where appropriate to complement each other and provide a comprehensive picture of PA behavior.

Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement

There is a confidence crisis in many scientific disciplines, in particular disciplines researching human behavior, as many effects of original experiments have not been replicated successfully in large-scale replication studies. While human-robot interaction (HRI) is an interdisciplinary research field, the study of human behavior, cognition and emotion in HRI plays also a vital part. Are HRI user studies facing the same problems as other fields and if so, what can be done to overcome them? In this article, we first give a short overview of the replicability crisis in behavioral sciences and its causes. In a second step, we estimate the replicability of HRI user studies mainly 1) by structural comparison of HRI research processes and practices with those of other disciplines with replicability issues, 2) by systematically reviewing meta-analyses of HRI user studies to identify parameters that are known to affect replicability, and 3) by summarizing first replication studies in HRI as direct evidence. Our findings suggest that HRI user studies often exhibit the same problems that caused the replicability crisis in many behavioral sciences, such as small sample sizes, lack of theory, or missing information in reported data. In order to improve the stability of future HRI research, we propose some statistical, methodological and social reforms. This article aims to provide a basis for further discussion and a potential outline for improvements in the field.

Transparent, Open, and Reproducible Prevention Science

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Comparison between 2000 and 2018 on the reporting of statistical significance and clinical relevance in physiotherapy clinical trials in six major physiotherapy journals: a meta-research design

DESIGN

Meta-research.

OBJECTIVE

To compare the prevalence of reporting p values, effect estimates and clinical relevance in physiotherapy randomised controlled trials (RCTs) published in the years 2000 and 2018.

METHODS

We performed a meta-research study of physiotherapy RCTs obtained from six major physiotherapy peer-reviewed journals that were published in the years 2000 and 2018. We searched the databases Embase, Medline and PubMed in May 2019, and extracted data on the study characteristics and whether articles reported on statistical significance, effect estimates and confidence intervals for baseline, between-group, and within-group differences, and clinical relevance. Data were presented using descriptive statistics and inferences were made based on proportions. A 20% difference between 2000 and 2018 was regarded as a meaningful difference.

RESULTS

We found 140 RCTs: 39 were published in 2000 and 101 in 2018. Overall, there was a high prevalence (>90%) of reporting p values for the main (between-group) analysis, with no difference between years. Statistical significance testing was frequently used for evaluating baseline differences, increasing from 28% in 2000 to 61.4% in 2018. The prevalence of reporting effect estimates, CIs and the mention of clinical relevance increased from 2000 to 2018 by 26.6%, 34% and 32.8% respectively. Despite an increase in use in 2018, over 40% of RCTs failed to report effect estimates, CIs and clinical relevance of results.

CONCLUSION

The prevalence of using p values remains high in physiotherapy research. Although the proportion of reporting effect estimates, CIs and clinical relevance is higher in 2018 compared to 2000, many publications still fail to report and interpret study findings in this way.

Target and actual sample sizes for studies from two trial registries from 1999 to 2020: an observational study

Objectives

To investigate differences between target and actual sample sizes, and what study characteristics were associated with sample sizes.

Design

Observational study.

Setting

The large trial registries of clinicaltrials.gov (starting in 1999) and ANZCTR (starting in 2005) through to 2021.

Participants

Over 280 000 interventional studies excluding studies that were withheld, terminated for safety reasons or were expanded access.

Main outcome measures

The actual and target sample sizes, and the within-study ratio of the actual to target sample size.

Results

Most studies were small: the median actual sample sizes in the two databases were 60 and 52. There was a decrease over time in the target sample size of 9%–10% per 5 years, and a larger decrease of 18%–21% per 5 years for the actual sample size. The actual-to-target sample size ratio was 4.1% lower per 5 years, meaning more studies (on average) failed to hit their target sample size.

Conclusion

Registered studies are more often under-recruited than over-recruited and worryingly both target and actual sample sizes appear to have decreased over time, as has the within-study gap between the target and actual sample size. Declining sample sizes and ongoing concerns about underpowered studies mean more research is needed into barriers and facilitators for improving recruitment and accessing data.

A data-driven framework for mapping domains of human neurobiology

Functional neuroimaging has been a mainstay of human neuroscience for the past 25 years. Interpretation of fMRI data has often occurred within knowledge frameworks crafted by experts, which have the potential to amplify biases that limit the replicability of findings. Here, we employ a computational approach to derive a data-driven framework for neurobiological domains that synthesizes the texts and data of nearly 20,000 human neuroimaging articles. Across multiple levels of domain specificity, the structure-function links within domains better replicate in held-out articles than those mapped from dominant frameworks in neuroscience and psychiatry. We further show that the data-driven framework partitions the literature into modular subfields, for which domains serve as generalizable prototypes of structure-function patterns in single articles. The approach to computational ontology we present here is the most comprehensive characterization of human brain circuits quantifiable with fMRI and may be extended to synthesize other scientific literatures.

Educating for Responsible Research Practice in Biomedical Sciences: Towards Learning Goals

New developments in the field of biomedicine can have extensive implications for society. To steer research efforts in a responsible direction, biomedical scientists should contribute to a forward-looking ethical, and societal evaluation of new developments. However, the question remains how to equip students sufficiently with the skills they need to contribute to this evaluation. In this paper, we examine how the four dimensions of Responsible Research and Innovation (anticipation, reflexivity, inclusivity, and responsiveness) inform the identification of learning goals and teaching approaches that contribute to developing these skills in biomedical scientists. We suggest that these educational approaches focus on the skills to anticipate intended and unintended outcomes, reflect on the epistemological and moral aspects of research practice, and be inclusive of the variety of voices in society. We argue that if these dimensions are properly integrated into biomedical curricula, they will help students develop the attitudinal aspects necessary for becoming responsive, and prepare them for implementing the dimensions of responsible research into their daily practice. This paper focuses specifically on skills biomedical scientists need for the responsible conduct of research. Therefore, our analysis results, at least in part, in domain-specific recommendations. We invite educators from other disciplines to do the same exercise, as we believe this could lead to tailored educational approaches by which students from various disciplinary backgrounds learn how they each have a role in contributing to socially robust and morally responsible research practice.

Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017

To gain insight into changes of scholarly journals' recommendations, we conducted a systematic review of studies that analysed journals' Instructions to Authors (ItAs). We summarised results of 153 studies, and meta-analysed how often ItAs addressed: 1) authorship, 2) conflicts of interest, 3) data sharing, 4) ethics approval, 5) funding disclosure, and 6) International Committee of Medical Journal Editors' Uniform Requirements for Manuscripts. For each topic we found large between-study heterogeneity. Here, we show six factors that explained most of that heterogeneity: 1) time (addressing of topics generally increased over time), 2) country (large differences found between countries), 3) database indexation (large differences found between databases), 4) impact factor (topics were more often addressed in highest than in lowest impact factor journals), 5) discipline (topics were more often addressed in Health Sciences than in other disciplines), and 6) sub-discipline (topics were more often addressed in general than in sub-disciplinary journals).

Understanding Replication in a Way That Is True to Science

In this article, I critically examine a number of widely held beliefs about the nature of replication and its place in science, with particular reference to psychology. In doing so, I present a number of underappreciated understandings of the nature of science more generally. I contend that some contributors to the replication debates overstate the importance of replication in science and mischaracterize the relationship between direct and conceptual replication. I also claim that there has been a failure to appreciate sufficiently the variety of legitimate replication practices that scientists engage in. In this regard, I highlight the tendency to pay insufficient attention to methodological triangulation as an important strategy for justifying empirical claims. I argue, further, that the replication debates tend to overstate the closeness of the relationship between replication and theory construction. Some features of this relationship are spelt out with reference to the hypothetico-deductive and the abductive accounts of scientific method. Additionally, an evaluation of the status of replication in different characterizations of scientific progress is undertaken. I maintain that viewing replication as just one element of the wide array of scientific endeavors leads to the conclusion that it is not as prominent in science as is often claimed.

Psychology Exceptionalism and the Multiple Discovery of the Replication Crisis

This article outlines what we call the “narrative of psychology exceptionalism” in commentaries on the replication crisis: many thoughtful commentaries link the current crisis to the specificity of psychology’s history, methods, and subject matter, but explorations of the similarities between psychology and other fields are comparatively thin. Historical analyses of the replication crisis in psychology further contribute to this exceptionalism by creating a genealogy of events and personalities that shares little in common with other fields. We aim to rebalance this narrative by examining the emergence and evolution of replication discussions in psychology alongside their emergence and evolution in biomedicine. Through a mixed-methods analysis of commentaries on replication in psychology and the biomedical sciences, we find that these conversations have, from the early years of the crisis, shared a common core that centers on concerns about the effectiveness of traditional peer review, the need for greater transparency in methods and data, and the perverse incentive structure of academia. Drawing on Robert Merton’s framework for analyzing multiple discovery in science, we argue that the nearly simultaneous emergence of this narrative across fields suggests that there are shared historical, cultural, or institutional factors driving disillusionment with established scientific practices.

Placebo response and effect in randomized clinical trials: meta-research with focus on contextual effects

BACKGROUND

Contextual effects (i.e., placebo response) refer to all health changes resulting from administering an apparently inactive treatment. In a randomized clinical trial (RCT), the overall treatment effect (i.e., the post-treatment effect in the intervention group) can be regarded as the true effect of the intervention plus the impact of contextual effects. This meta-research was conducted to examine the average proportion of the overall treatment effect attributable to contextual effects in RCTs across clinical conditions and treatments and explore whether it varies with trial contextual factors.

METHODS

Data was extracted from trials included in the main meta-analysis from the latest update of the Cochrane review on "Placebo interventions for all clinical conditions" (searched from 1966 to March 2008). Only RCTs reported in English having an experimental intervention group, a placebo comparator group, and a no-treatment control group were eligible.

RESULTS

In total, 186 trials (16,655 patients) were included. On average, 54% (0.54, 95%CI 0.46 to 0.64) of the overall treatment effect was attributable to contextual effects. The contextual effects were higher for trials with blinded outcome assessor and concealed allocation. The contextual effects appeared to increase proportional to the placebo effect, lower mean age, and proportion of females.

CONCLUSION

Approximately half of the overall treatment effect in RCTs seems attributable to contextual effects rather than to the specific effect of treatments. As the study did not include all important contextual factors (e.g., patient-provider interaction), the true proportion of contextual effects could differ from the study's results. However, contextual effects should be considered when assessing treatment effects in clinical practice.

TRIAL REGISTRATION

PROSPERO CRD42019130257 . Registered on April 19, 2019.

The h-index is no longer an effective correlate of scientific reputation

The impact of individual scientists is commonly quantified using citation-based measures. The most common such measure is the h-index. A scientist’s h-index affects hiring, promotion, and funding decisions, and thus shapes the progress of science. Here we report a large-scale study of scientometric measures, analyzing millions of articles and hundreds of millions of citations across four scientific fields and two data platforms. We find that the correlation of the h-index with awards that indicate recognition by the scientific community has substantially declined. These trends are associated with changing authorship patterns. We show that these declines can be mitigated by fractional allocation of citations among authors, which has been discussed in the literature but not implemented at scale. We find that a fractional analogue of the h-index outperforms other measures as a correlate and predictor of scientific awards. Our results suggest that the use of the h-index in ranking scientists should be reconsidered, and that fractional allocation measures such as h-frac provide more robust alternatives.

Meta-research in pharmacy: Time for a look in the mirror

The volume of the biomedical literature continues to expand at a substantial rate. The research literature surrounding pharmaceutical services is no different. Due in part to events in the recent past, researchers, consumers, funders, and policymakers have raised concerns about the credibility, transparency, and potential waste in the global research enterprise. Meta-research, or research on research, provides a way to examine the efficiency, quality, and potential bias in the overall research ecosystem. The field of meta-research is a relatively new but rapidly growing field that has seen many applications in biomedical research. Applications in pharmacy research, however, are still developing. The goals of this commentary are to introduce pharmacy researchers to the concept of meta-research, discuss several examples of meta-research in pharmacy, and motivate the importance of sustained meta-research efforts in pharmacy.

Meta-research publications in dentistry: a review

The present scoping review has the objective of providing an overview of meta-research in dentistry. A search of the PubMed database was performed for the period 11 October 2014 to 10 October 2019. Study selection and data extraction were performed independently by one author; prior to this, a random sample of 10% of the retrieved titles and abstracts were independently screened by two authors, achieving agreement of >80% on eligibility for initial inclusion, corresponding to good agreement. The following information was extracted from the full text of each article: meta-research area of interest; study design; type of studies evaluated in the meta-research; type of methodology used in assessment of the primary research; conflicts of interest reported; sponsorships reported; dental discipline; journal of publication; country of the first author; number of citations; and impact factor. A total of 7800 documents were initially retrieved. After analysis of the title/abstract and the full text of each article, and a snowballing procedure, 155 meta-research studies were identified and included. The 'methods' and 'reporting' meta-research areas were the most prevalent, with 73 (47%) and 61 (40%) studies, respectively. General dentistry, and orthodontics and dentofacial orthopaedics were the dental specialties with the greatest number/proportion of included studies with 45 (29%) and 28 (18%) studies, respectively. These findings may help to prioritize future meta-research in dentistry, consequently avoiding unnessecary investigations, and increasing the value of oral and dental research.

Psychiatric neuroimaging research in Brazil: historical overview, current challenges, and future opportunities

The last four decades have witnessed tremendous growth in research studies applying neuroimaging methods to evaluate pathophysiological and treatment aspects of psychiatric disorders around the world. This article provides a brief history of psychiatric neuroimaging research in Brazil, including quantitative information about the growth of this field in the country over the past 20 years. Also described are the various methodologies used, the wealth of scientific questions investigated, and the strength of international collaborations established. Finally, examples of the many methodological advances that have emerged in the field of in vivo neuroimaging are provided, with discussion of the challenges faced by psychiatric research groups in Brazil, a country of limited resources, to continue incorporating such innovations to generate novel scientific data of local and global relevance.

Current Issues on Research Conducted to Improve Women's Health

There are varied lessons to be learned regarding the current methodological approaches to women’s health research. In the present scheme of growing medical literature and inflation of novel results claiming significance, the sheer amount of information can render evidence-based practice confusing. The factors that classically determined the impact of discoveries appear to be losing ground: citation count and publication rates, hierarchy in author lists according to contribution, and a journal’s impact factor. Through a comprehensive literature search on the currently available data from theses, opinion, and original articles and reviews on this topic, we seek to present to clinicians a narrative synthesis of three crucial axes underlying the totality of the research production chain: (a) critical advances in research methodology, (b) the interplay of academy and industry in a trial conduct, and (c) review- and publication-associated developments. We also provide specific recommendations on the study design and conduct, reviewing the processes and dissemination of data and the conclusions and implementation of findings. Overall, clinicians and the public should be aware of the discourse behind the marketing of alleged breakthrough research. Still, multiple initiatives, such as patient review and strict, supervised literature synthesis, have become more widely accepted. The “bottom-up” approach of a wide dissemination of information to clinicians, together with practical incentives for stakeholders with competing interests to collaborate, promise to improve women’s healthcare.

AR. Perspectives on Medical Education Meta-Research Special Issue: A call for papers exploring how research is performed, communicated, verified and rewarded

None

Correction to: perspectives on medical education meta-research special issue: a call for papers exploring how research is performed, communicated, verified and rewarded

A Correction to this paper has been published: https://doi.org/10.1007/s40037-020-00627-8

Countering medical nihilism by reconnecting facts and values

A pessimistic strain of thought is fomenting in the health studies literature regarding the status of medicine. Ioannidis's (2005) now famous finding that "most published research findings are false" and Stegenga's (2018) book-length argument for medical nihilism are examples of this. In this paper, we argue that these positions are incorrect insofar as they rest on an untenable account of the nature of facts. Proper attention to fallibilism and the social organization of knowledge, as well as Bayesian probabilities in medical reasoning, prompt us to ask why the cynics expect the results of quantitative studies to be incontrovertibly true in the first place. While we agree with Ioannidis and others' identified flaws in the medical research enterprise, and encourage rectification, we conclude that medical nihilism is not the natural outcome of the current state of research.

Mapping the past, present and future research landscape of paternal effects

BACKGROUND

Although in all sexually reproducing organisms an individual has a mother and a father, non-genetic inheritance has been predominantly studied in mothers. Paternal effects have been far less frequently studied, until recently. In the last 5 years, research on environmentally induced paternal effects has grown rapidly in the number of publications and diversity of topics. Here, we provide an overview of this field using synthesis of evidence (systematic map) and influence (bibliometric analyses).

RESULTS

We find that motivations for studies into paternal effects are diverse. For example, from the ecological and evolutionary perspective, paternal effects are of interest as facilitators of response to environmental change and mediators of extended heredity. Medical researchers track how paternal pre-fertilization exposures to factors, such as diet or trauma, influence offspring health. Toxicologists look at the effects of toxins. We compare how these three research guilds design experiments in relation to objects of their studies: fathers, mothers and offspring. We highlight examples of research gaps, which, in turn, lead to future avenues of research.

CONCLUSIONS

The literature on paternal effects is large and disparate. Our study helps in fostering connections between areas of knowledge that develop in parallel, but which could benefit from the lateral transfer of concepts and methods.

Protocol for meta-research on the evidence informing micronutrient dietary reference intakes for pregnant and lactating women

Nutrient reference values are important parameters that guide nutrition and public health work globally. Micronutrient requirements during the peri-conception period are generally increased, which is essential in ensuring maternal, fetal, and neonatal health. Nevertheless, the current dietary reference intakes (DRIs) may be limited in terms of the methods used and the populations included, particularly the DRIs for pregnancy and lactation. In this proposed review, we will examine the methods (rigor of design, utilization of molecular methods, and presence of modern methods) and the population (inclusion of women, and in particular, pregnant and lactating people) in the studies used to inform the current DRIs. We will apply meta-science methods to this review, which involves formally reviewing the current evidence, and identifying opportunities to improve how we fund, perform, evaluate, and incorporate nutrition science into public health programs for better outcomes.

Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research

Background: As part of a coordinated effort to expand research activity around rigor, reproducibility, and transparency (RRT) across scientific disciplines, a team of investigators at the Indiana University School of Public Health-Bloomington hosted a workshop in October 2019 with international leaders to discuss key opportunities for RRT research. Objective: The workshop aimed to identify research priorities and opportunities related to RRT. Design: Over two-days, workshop attendees gave presentations and participated in three working groups: (1) Improving Education & Training in RRT, (2) Reducing Statistical Errors and Increasing Analytic Transparency, and (3) Looking Outward: Increasing Truthfulness and Accuracy of Research Communications. Following small-group discussions, the working groups presented their findings, and participants discussed the research opportunities identified. The investigators compiled a list of research priorities, which were circulated to all participants for feedback. Results: Participants identified the following priority research questions: (1) Can RRT-focused statistics and mathematical modeling courses improve statistics practice?; (2) Can specialized training in scientific writing improve transparency?; (3) Does modality (e.g. face to face, online) affect the efficacy RRT-related education?; (4) How can automated programs help identify errors more efficiently?; (5) What is the prevalence and impact of errors in scientific publications (e.g., analytic inconsistencies, statistical errors, and other objective errors)?; (6) Do error prevention workflows reduce errors?; (7) How do we encourage post-publication error correction?; (8) How does 'spin' in research communication affect stakeholder understanding and use of research evidence?; (9) Do tools to aid writing research reports increase comprehensiveness and clarity of research reports?; and (10) Is it possible to inculcate scientific values and norms related to truthful, rigorous, accurate, and comprehensive scientific reporting? Conclusion: Participants identified important and relatively unexplored questions related to improving RRT. This list may be useful to the scientific community and investigators seeking to advance meta-science (i.e. research on research).

Population characteristics as important contextual factors in rheumatological trials: an exploratory meta-epidemiological study from an OMERACT Working Group

OBJECTIVES

To explore whether trial population characteristics modify treatment responses across various interventions, comparators and rheumatic conditions.

METHODS

In this meta-epidemiological study, we included trials from systematic reviews available from the Cochrane Musculoskeletal Group published up to 23 April 2019 in Cochrane Library with meta-analyses of five or more randomised controlled trials (RCTs) published from year 2000. From trial reports, we extracted data on 20 population characteristics. For characteristics with sufficient data (ie, available for ≥2/3 of the trials), we performed multilevel meta-epidemiological analyses.

RESULTS

We identified 19 eligible systematic reviews contributing 187 RCTs (212 comparisons). Only age and sex were explicitly reported in ≥2/3 of the trials. Using information about the country of the trials led to sufficient data for five further characteristics, that is, 7 out of 20 (35%) protocolised characteristics were analysed. The meta-regressions showed effect modification by economic status, place of residence, and, nearly, from healthcare system (explaining 4.8%, 0.9% and 1.5% of the between-trial variation, respectively). No effect modification was demonstrated from age, sex, patient education/health literacy or predominant religion.

CONCLUSIONS

This study demonstrates the scarce reporting of most population characteristics, hampering investigation of their impact with meta-research. Our sparse results suggest that place of residence (ie, continent of the trial), economic status (based on World Bank classifications) and healthcare system (based on WHO index for health system performance) may be important in explaining the variation in treatment response across trials. There is an urgent need for consistent reporting of important population characteristics in trials.

PROSPERO REGISTRATION NUMBER

CRD42019127642.

Is “the time ripe” for quantitative research on misconduct in science?

Misconduct in science is a timely and substantively important problem in the social study of science. But in the absence of comprehensive and reliable data needed for analysis, formidable obstacles stand in the way of its being studied quantitively. Accessible databases, including government data, are flawed, while undertaking new data collection presents its own problems. First, little is known about biases in official government reports. Second, official reports exclude classes of malfeasance other than fabrication, falsification, and plagiarism of evidence (FFP). Third, while drawing on official data is expedient, available official information is thin; it tells little about miscreants and fails to identify potential causes of their actions and the environments in which misconduct occurred. Fourth, it also fails the test of permitting estimates to be made of populations at risk, making it impossible to calculate incidence. A healthy dose of skepticism is in order in evaluating both the findings of current quantitative studies and of proposals for remediation.

The importance and need of more meta-research studies in medical ethics journals

Meta-research aims to research the process of research itself, and as such, it can be used to study threats to research integrity and potentially better understand them. This commentary assesses the prevalence of meta-research studies published in medical ethics journals to understand their representation. The retrieved articles were organized by meta-research area: methods, reporting, reproducibility, evaluation, and incentives. An analysis of articles published in the last two years in the 16 highest ranked medical ethics journals suggests that meta-research studies seemed to be not well represented in these journals. In this sample, less than 2% of the articles were classified as meta-research studies. Furthermore, some meta-research areas such as reproducibility and incentives are relatively unexplored currently in medical ethics journals. This commentary discusses the meaning of the specific meta-research areas and provides some examples of articles in medical research ethics that fit into each meta-research area. As a conclusion of this commentary, some action should be taken to encourage more meta-research publications in medical ethics journals for their potential to advance medical ethics science.

Scientometrics Approach to Research in Ovine Mastitis from 1970 to 2019 (with a Complete List of Relevant Literature References)

The present study is a scientometrics evaluation of refereed publications on bacterial mastitis in sheep; the objectives were the evaluation of the relevant papers and the presentation of quantitative characteristics regarding their scientific content and bibliometric details. The Web of Science platform was used with search terms: [mastitis OR *mammary infection*] AND [sheep OR ewe* OR ovine] for papers from 1970 tο 2019; only ‘articles’, ‘reviews’, ‘proceedings papers’, or ‘data papers’ were evaluated, whilst documents related solely to contagious agalactia, mammary aspects of lentiviral infections, or infections of the teats and the udder skin were excluded. Finally, 580 papers were considered in detail. The number of published papers increased from 8 during the 1970s to 273 during the 2010s. These papers originated from 43 countries (most from Greece or Spain, n = 87 from each) and 240 institutions (145 universities and 95 other establishments), of which 35 produced ≥ 5 papers each. Most papers present original studies (n = 539) with a few reviews (n = 41). The original papers refer to dairy (n = 428), meat (n = 113), or wool (n = 1) production systems and present field (n = 329), laboratory (n = 163), or experimental (n = 67) work; the papers report aetiology (n = 146), risk factors (n = 100), pathogenesis (n = 92), diagnosis (n = 88), effects (n = 66), treatment (n = 50), control (n = 36), or descriptive epidemiology (n = 32) of the disease. Papers related to dairy production present more field and fewer experimental work than papers related to meat production; also, in papers describing work performed in dairy sheep, studies about aetiology, risk factors, and diagnosis of the disease predominate, whilst in papers performed in meat sheep, studies about aetiology, pathogenesis, and effects/diagnosis are reported more often. The papers were published in 175 scientific journals (most in Small Ruminant Research, n = 90, or Journal of Dairy Science, n = 54). On average, the papers received 16.8 total citations and 1.6 yearly citations (h-index = 47). Most papers were published in Scimago classification Q1 (n = 240) or Q2 (n = 230) journals and received 23.4 or 15.4 total citations, respectively. Reviews received more citations than original papers; among the latter, papers with work referring to dairy production received more yearly citations than papers referring to meat production; no differences in citations were seen according to type of work or mastitis aspect covered. Most citations were received by papers from France. Papers published in Journal of Dairy Science or Small Ruminant Research received the most citations. In total, there were 1558 individual authors of the papers, with 24 authors having co-authored > 10 papers each (max: 73 papers); on average, there were 5.2 co-authors per paper (min–max: 1–25). Average number of co-authors progressively increased from 2.1 in the 1970s to 6.3 in the 2010s, with original papers having a higher number of co-authors than reviews: 5.3 and 3.7, respectively. Papers from France had highers number of co-authors (7.9). The findings of this first ever scientometrics study into ovine mastitis indicate that the disease has not been studied as other sheep diseases and that future studies in it should be directed to its control.

Methods used to assess outcome consistency in clinical studies: A literature-based evaluation

Evaluation studies of outcomes used in clinical research and their consistency are appearing more frequently in the literature, as a key part of the core outcome set (COS) development. Current guidance suggests such evaluation studies should use systematic review methodology as their default. We aimed to examine the methods used. We searched the Core Outcome Measures in Effectiveness Trials (COMET) database (up to May 2019) supplementing it with additional resources. We included evaluation studies of outcome consistency in clinical studies across health subjects and used a subset of A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 (items 1-9) to assess their methods. Of 93 included evaluation studies of outcome consistency (90 full reports, three summaries), 91% (85/93) reported performing literature searches in at least one bibliographic database, and 79% (73/93) was labelled as a "systematic review". The evaluations varied in terms of satisfying AMSTAR 2 criteria, such that 81/93 (87%) had implemented PICO in the research question, whereas only 5/93 (6%) had included the exclusions list. None of the evaluation studies explained how inconsistency of outcomes was detected, however, 80/90 (88%) concluded inconsistency in individual outcomes (66%, 55/90) or outcome domains (20%, 18/90). Methods used in evaluation studies of outcome consistency in clinical studies differed considerably. Despite frequent being labelled as a "systematic review", adoption of systematic review methodology is selective. While the impact on COS development is unknown, authors of these studies should refrain from labelling them as "systematic review" and focus on ensuring that the methods used to generate the different outcomes and outcome domains are reported transparently.

Reproducibility of animal research in light of biological variation

Context-dependent biological variation presents a unique challenge to the reproducibility of results in experimental animal research, because organisms' responses to experimental treatments can vary with both genotype and environmental conditions. In March 2019, experts in animal biology, experimental design and statistics convened in Blonay, Switzerland, to discuss strategies addressing this challenge. In contrast to the current gold standard of rigorous standardization in experimental animal research, we recommend the use of systematic heterogenization of study samples and conditions by actively incorporating biological variation into study design through diversifying study samples and conditions. Here we provide the scientific rationale for this approach in the hope that researchers, regulators, funders and editors can embrace this paradigm shift. We also present a road map towards better practices in view of improving the reproducibility of animal research.

Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices

As concern over the use of questionable research practices (QRPs) in academic science has increased over the last couple of decades, some reforms have been implemented and many others have been debated and recommended. While many of these proposals have merit, efforts to improve scientific practices are more likely to succeed when they are responsive to the prevailing views and concerns of scientists themselves. To date, there have been few efforts to solicit wide-ranging input from researchers on the topic of needed reforms. This article is a qualitative report of responses from federally funded scientists to the question of what should be done to address the problem of QRPs in their disciplines. Overall, participants were concerned about how institutional and career-oriented incentives encourage the use of QRPs. Compared to previous recommendations, participants had surprisingly little confidence in the ability of ethics training to improve research integrity.

The challenge of conducting qualitative research to understand the factors that influence equity in medical education: A scoping review

This article was migrated. The article was marked as recommended. Introduction There are national and international concerns about equity in basic and postgraduate medical education, especially about differential rates of access and attainment across groups of learners. Qualitative research has been increasingly used to understand the factors that influence equity but there are potential limitations to this understanding related to how the research has been conducted. The aim of the scoping review was to identify how qualitative research exploring the factors that influence equity in basic and postgraduate medical education has been conducted. The intention was to inform future research. Methods The electronic databases British Education Index, Campbell Library, CINAHL, Cochrane Library, EMBASE, ERIC, Google Scholar, Health Management Information Consortium (HMIC), MEDLINE, PsycINFO, Web of Science and medical education journals were searched to identify relevant published articles between 2008 and April 2019. Results Among 19,523 articles identified from the literature search, 72 full text articles were included in the review. Most studies had a focus on only one background characteristic and only two studies had a strengths-based focus on individuals. Recommendations for change was at the 'policy level' in ten studies and four studies had learner recommendations for change. No studies with a participatory approach were identified. Conclusion The approach to conducting previous qualitative research appears to limit greater understanding of the complexity of factors that influence equity. In response to this challenge, we recommend that future research widen the focus to consider the experiences and strengths of individual learners in addition to those identified by background characteristics. Future qualitative research is recommended to have a broad focus on both the 'policy level' and 'local level', especially from multiple perspectives. We also recommend greater collaboration of participants with researchers throughout the research process.

CoViD-19 Immunopathology and Immunotherapy

New evidence on the T-cell immuno-pathology in patient's with Corona Virus Disease 2019 (CoViD-19) was reported by Diao et al. in MedRxiv (doi: 10.1101/2020.02.18.20024364) [1]. It reports observations on 522 patients with confirmed CoViD-19 symptomatology, compared to 40 control subjects. In brief, notable T cytopoenia was recorded by flow cytometry in the CD4+ and the CD8+ populations, which were significantly yet inversely correlated with remarkably increased serum levels of the pro-inflammatory cytokines IL-6, IL-10 and TNF-a. Flow cytometry established a progressive increase in the expression of programmed cell death marker-1 (PD-1) and T cell immunoglobulin and mucin domain 3 (Tim-3) as patients (n=14) deteriorated from prodromal to symptomatic CoViD-19 requiring intensive care. Here, we interpret these observations of Diao et al from our current understanding of T cell immunophysiology and immunopathology following an immune challenge in the form of sustained viral infection, as is the case in CoViD-19, with emphasis on exhausted T cells (Tex). Recent clinical trials to rescue Tex show promising outcomes. The relevance of these interventions for the prevention and treatment of CoViD-19 is discussed. Taken together, the data of Diao et al could proffer the first glimpse of immunopathology and possible immunotherapy for patients with CoViD-19.

Clinical research in cancer palliative care: a metaresearch analysis

OBJECTIVE

This metaresearch of the clinicaltrials.gov database aims to evaluate how clinical research on palliative care is conducted within the setting of advanced cancer.

METHODS

Clinicaltrials.gov was searched to identify registered studies recruiting patients with cancer, and investigating issues relevant to palliative care. The European Organisation for Research and Treatment of Cancer QLQ-C15-PAL (Quality of Life in palliative cancer care patients) questionnaire was taken into account to define the research domains of interest. Studies investigating cancer-directed therapy, management of cancer treatment-related adverse events and diagnostic tests were excluded. Publication status was crosschecked using PubMed.

RESULTS

Of 3950 identified studies, 514 were included. The most frequent reason for exclusion was cancer-directed therapy (2491). In 2007-2012, 161 studies were registered versus 245 in 2013-2018. Included studies were interventional (84%) or observational (16%). Most studies were monocentric (60%), sponsored by academia (79%), and conducted in North America (57%) or Europe (25%). Seventy-nine per cent of studies evaluated a heterogeneous population (>1 tumour type). Interventional studies most frequently investigated systemic drugs (34%), behavioural interventions (29%) and procedures for pain (24%). Pain, quality of life and physical function were the most frequently studied research domains (188, 95 and 52 studies, respectively). The most applied primary outcome measures were efficacy/symptom control (61%), quality of life (14%) and feasibility (12%). Only 16% of the closed studies had published results in PubMed.

CONCLUSIONS

Our study describes the heterogeneous landscape of studies conducted to address the issues of patients with advanced cancer in palliative care. Albeit the observed increase in the number of studies over the last decade, the generalisation of the results brought by the existing trials is limited due to methodological issues and lack of reporting. A greater effort is needed to improve clinical research that supports evidence-based palliative cancer care.

Introducing our series: research synthesis and meta-research in biology

Research synthesis is the process of bringing together findings and attributes from different publications, for example, to give a more complete description of phenomena than is usually possible in a single work. We bring the Research Synthesis Series to BMC Biology to promote meta-analyses, other research syntheses including meta-research studies, and research synthesis methodologies in biology, facilitating their dissemination to broader communities.

Did awarding badges increase data sharing in BMJ Open? A randomized controlled trial

Sharing data and code are important components of reproducible research. Data sharing in research is widely discussed in the literature; however, there are no well-established evidence-based incentives that reward data sharing, nor randomized studies that demonstrate the effectiveness of data sharing policies at increasing data sharing. A simple incentive, such as an Open Data Badge, might provide the change needed to increase data sharing in health and medical research. This study was a parallel group randomized controlled trial (protocol registration: doi:10.17605/OSF.IO/PXWZQ) with two groups, control and intervention, with 80 research articles published in BMJ Open per group, with a total of 160 research articles. The intervention group received an email offer for an Open Data Badge if they shared their data along with their final publication and the control group received an email with no offer of a badge if they shared their data with their final publication. The primary outcome was the data sharing rate. Badges did not noticeably motivate researchers who published in BMJ Open to share their data; the odds of awarding badges were nearly equal in the intervention and control groups (odds ratio = 0.9, 95% CI [0.1, 9.0]). Data sharing rates were low in both groups, with just two datasets shared in each of the intervention and control groups. The global movement towards open science has made significant gains with the development of numerous data sharing policies and tools. What remains to be established is an effective incentive that motivates researchers to take up such tools to share their data.

An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)

Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis

Background

Randomised controlled trials (RCTs) provide the most reliable information to inform clinical practice and patient care. We aimed to map global clinical research publication activity through RCT-related articles in high-impact-factor medical journals over the past five decades.

Methods

We conducted a cross-sectional analysis of articles published in the highest ranked medical journals with an impact factor > 10 (according to Journal Citation Reports published in 2017). We searched PubMed/MEDLINE (from inception to December 31, 2017) for all RCT-related articles (e.g. primary RCTs, secondary analyses and methodology papers) published in high-impact-factor medical journals. For each included article, raw metadata were abstracted from the Web of Science. A process of standardization was conducted to unify the different terms and grammatical variants and to remove typographical, transcription and/or indexing errors. Descriptive analyses were conducted (including the number of articles, citations, most prolific authors, countries, journals, funding sources and keywords). Network analyses of collaborations between countries and co-words are presented.

Results

We included 39,305 articles (for the period 1965–2017) published in forty journals. The Lancet (n = 3593; 9.1%), the Journal of Clinical Oncology (n = 3343; 8.5%) and The New England Journal of Medicine (n = 3275 articles; 8.3%) published the largest number of RCTs. A total of 154 countries were involved in the production of articles. The global productivity ranking was led by the United States (n = 18,393 articles), followed by the United Kingdom (n = 8028 articles), Canada (n = 4548 articles) and Germany (n = 4415 articles). Seventeen authors who had published 100 or more articles were identified; the most prolific authors were affiliated with Duke University (United States), Harvard University (United States) and McMaster University (Canada). The main funding institutions were the National Institutes of Health (United States), Hoffmann-La Roche (Switzerland), Pfizer (United States), Merck Sharp & Dohme (United States) and Novartis (Switzerland). The 100 most cited RCTs were published in nine journals, led by The New England Journal of Medicine (n = 78 articles), The Lancet (n = 9 articles) and JAMA (n = 7 articles). These landmark contributions focused on novel methodological approaches (e.g. the “Bland-Altman method”) and trials on the management of chronic conditions (e.g. diabetes control, hormone replacement therapy in postmenopausal women, multiple therapies for diverse cancers, cardiovascular therapies such as lipid-lowering statins, antihypertensive medications, and antiplatelet and antithrombotic therapy).

Conclusions

Our analysis identified authors, countries, funding institutions, landmark contributions and high-impact-factor medical journals publishing RCTs. Over the last 50 years, publication production in leading medical journals has increased, with Western countries leading in research but with low- and middle-income countries showing very limited representation.

Metascience in Bioinformation

Metascience refers to the systematic process that uncovers, builds, evaluates, organizes and disseminates scientific advances. It is the principal tool at the disposal of the society to combat the debilitating effects of "false information" on health related data and its constituents.

The Systematic Review Data Repository (SRDR): descriptive characteristics of publicly available data and opportunities for research

BACKGROUND

Conducting systematic reviews ("reviews") requires a great deal of effort and resources. Making data extracted during reviews available publicly could offer many benefits, including reducing unnecessary duplication of effort, standardizing data, supporting analyses to address secondary research questions, and facilitating methodologic research. Funded by the US Agency for Healthcare Research and Quality (AHRQ), the Systematic Review Data Repository (SRDR) is a free, web-based, open-source, data management and archival platform for reviews. Our specific objectives in this paper are to describe (1) the current extent of usage of SRDR and (2) the characteristics of all projects with publicly available data on the SRDR website.

METHODS

We examined all projects with data made publicly available through SRDR as of November 12, 2019. We extracted information about the characteristics of these projects. Two investigators extracted and verified the data.

RESULTS

SRDR has had 2552 individual user accounts belonging to users from 80 countries. Since SRDR's launch in 2012, data have been made available publicly for 152 of the 735 projects in SRDR (21%), at a rate of 24.5 projects per year, on average. Most projects are in clinical fields (144/152 projects; 95%); most have evaluated interventions (therapeutic or preventive) (109/152; 72%). The most frequent health areas addressed are mental and behavioral disorders (31/152; 20%) and diseases of the eye and ocular adnexa (23/152; 15%). Two-thirds of the projects (104/152; 67%) were funded by AHRQ, and one-sixth (23/152; 15%) are Cochrane reviews. The 152 projects each address a median of 3 research questions (IQR 1-5) and include a median of 70 studies (IQR 20-130).

CONCLUSIONS

Until we arrive at a future in which the systematic review and broader research communities are comfortable with the accuracy of automated data extraction, re-use of data extracted by humans has the potential to help reduce redundancy and costs. The 152 projects with publicly available data through SRDR, and the more than 15,000 studies therein, are freely available to researchers and the general public who might be working on similar reviews or updates of reviews or who want access to the data for decision-making, meta-research, or other purposes.

Access to Clinical Trials for Adolescents and Young Adults With Cancer: A Meta-Research Analysis

BACKGROUND

The 18-year-old age limit for inclusion in clinical trials constitutes a hurdle for adolescents and young adults (AYAs) with cancer. We analyzed the impact of this age barrier on the access of AYAs to cancer trials and novel therapies.

METHODS

ClinicalTrials.gov was searched to identify all the trials including patients with 10 malignancies relevant for AYAs (January 2007 to July 2018). The trials were categorized as pediatric (patients <18 y), adult (≥18 y), and transitional (including adult and pediatric patients). Transitional trials with a lower limit between 12 and 18 years and an upper limit younger than 40 years were considered AYA-specific.

RESULTS

Of 2764 identified trials, 2176 were included: 79% adult, 19% transitional, 2% pediatric. Five trials were AYA-specific. The proportion of academic trials was higher for transitional (69%; 288 of 421) than for adult trials (48%; 832 of 1718) (P < .0001). The total number of new trials increased over the years (156 in 2007; 228 in 2017); however, the number of transitional trials remained stable. The availability of trials increased with age, with a major increase at age 18 years: at age 17 years, 20% (442 of 2176) of trials were potentially accessible vs 95% (2075 of 2176) at 18 years. For trials investigating targeted therapies, this increase was 460% (197 trials available at age 17 years; 901 at 18 years) and for immunotherapies, 1200% (55 at age 17 years; 658 at 18 years).

CONCLUSIONS

AYAs have limited access to cancer trials and innovative therapies, with no improvement over the last decade. The 18-years-old age limit continues to be a major hurdle. Our findings are consistent with the internationally supported idea that age inclusion criteria in oncological trials should be changed.

The NIH Open Citation Collection: A public access, broad coverage resource

Citation data have remained hidden behind proprietary, restrictive licensing agreements, which raises barriers to entry for analysts wishing to use the data, increases the expense of performing large-scale analyses, and reduces the robustness and reproducibility of the conclusions. For the past several years, the National Institutes of Health (NIH) Office of Portfolio Analysis (OPA) has been aggregating and enhancing citation data that can be shared publicly. Here, we describe the NIH Open Citation Collection (NIH-OCC), a public access database for biomedical research that is made freely available to the community. This dataset, which has been carefully generated from unrestricted data sources such as MedLine, PubMed Central (PMC), and CrossRef, now underlies the citation statistics delivered in the NIH iCite analytic platform. We have also included data from a machine learning pipeline that identifies, extracts, resolves, and disambiguates references from full-text articles available on the internet. Open citation links are available to the public in a major update of iCite (https://icite.od.nih.gov).

A standardized citation metrics author database annotated for scientific field

Citation metrics are widely used and misused. We have created a publicly available database of 100,000 top scientists that provides standardized information on citations, h-index, coauthorship-adjusted hm-index, citations to papers in different authorship positions, and a composite indicator. Separate data are shown for career-long and single-year impact. Metrics with and without self-citations and ratio of citations to citing papers are given. Scientists are classified into 22 scientific fields and 176 subfields. Field- and subfield-specific percentiles are also provided for all scientists who have published at least five papers. Career-long data are updated to end of 2017 and to end of 2018 for comparison.