Tracking self-citations in academic publishing

Self-citation pattern among world's top 2 % of the scientists

Although self-citation is a common practice among scholars, its impact and significance remain under scrutiny within the academic community. The current study aimed to provide a detailed analysis and ranking of the top 2 % scientists on self-citation, ranked by Stanford University researchers. Data extraction and organisation were performed between January and February 2024. Self-citation percentages were collected for authors belonging to 20 fields and 174 subfields using Excel spreadsheets. Entities were categorised into quartiles based on their self-citation percentages and rankings were assigned accordingly. Comparative analyses were conducted to assess the impact of self-citations on overall rankings. The study focused on data published by a group of researchers from Stanford University regarding the top 2 % of researchers, analysing the career-long records of 204,643 scholars and 210,198 scholars for the most recent year, 2022. Notably, in the single-year analysis, among the top 20 countries, approximately eight (40 %) exhibited self-citation percentages exceeding the average of 25.96 %. The self-citation percentages ranged from 4.47 % in Economics and Business to 20.88 % in Physics and Astronomy. Regarding career-long analysis, the percentage of self-citations ranged from 22.84 % in Poland to 41.31 % in Armenia, with significant drop in rankings among most entities when self-citations were excluded. These findings highlight the dramatic impact of self-citation exclusion on the rankings of the top 2 % of the researchers, underscoring the critical importance of accounting for self-citation in ranking assessments.

Citation manipulation through citation mills and pre-print servers

Citations are widely considered in scientists' evaluation. As such, scientists may be incentivized to inflate their citation counts. While previous literature has examined self-citations and citation cartels, it remains unclear whether scientists can purchase citations. Here, we compile a dataset of ~1.6 million profiles on Google Scholar to examine instances of citation fraud on the platform. We survey faculty at highly-ranked universities, and confirm that Google Scholar is widely used when evaluating scientists. We then engage with a citation-boosting service, and manage to purchase 50 citations while assuming the identity of a fictional author. Taken as a whole, our findings bring to light new forms of citation manipulation, and emphasize the need to look beyond citation counts.

A Bibliometric Analysis of the 100 Most Cited Articles on Problem-Based Learning in Medical Education

Problem-based learning (PBL) is an instructional approach used in medical education that is characterized by solving problems in small groups with tutor guidance. More than 50 years since PBL's inception, many questions remain to be addressed about its processes and learning outcomes. The purpose of the study was to examine the bibliometric characteristics of the 100 most cited articles on PBL in medical education and to identify landmark papers that have made significant contributions to PBL research. Results were systematically reviewed for citation frequency, publication year, journal, article type, article focus, authors, author collaboration, and country collaboration. The number of citations ranged from 81 to 3531 times cited with 31,041 total citations. The articles were contributed by 211 authors in 23 journals and most articles (68%) were published in Medical Education, Academic Medicine, and Medical Teacher. The majority of the articles (71%) originated from Netherlands, Canada, and the United States and six prolific authors were identified. Almost half of the articles are classified as empirical research. Article foci included theoretical foundations of PBL, curriculum design, learning outcomes and processes, tutors, assessment, guides to PBL implementation, commentaries, and student well-being. The strong author and country collaborations indicate continued global interest in the PBL instructional method, which is likely to remain an active topic of research as the evidence of its effectiveness over traditional instructional methods as well as its most impactful components is inconclusive.

Self-referencing rates in biological disciplines

The use of citation counts (among other bibliometrics) as a facet of academic research evaluation can influence citation behavior in scientific publications. One possible unintended consequence of this bibliometric is excessive self-referencing, where an author favors referencing their own publications over related publications from different research groups. Peer reviewers are often prompted by journals to determine whether references listed in the manuscript under review are unbiased, but there is no consensus on what is considered "excessive" self-referencing. Here, self-referencing rates are examined across multiple journals in the fields of biology, genetics, computational biology, medicine, pathology, and cell biology. Median self-referencing rates are between 8-13% across a range of journals within these disciplines. However, self-referencing rates vary as a function of total number of references, number of authors, author status/rank, author position, and total number of publications for each author. Importantly, these relationships exhibit interdisciplinary and journal-dependent differences that are not captured by examining broader fields in aggregate (e.g., Biology, Chemistry, Physics, etc.). These results provide useful statistical guidelines for authors, editors, reviewers, and journals when considering referencing practices for individual publications, and highlight the effects of additional factors influencing self-referencing rates.

Supporting grant reviewers through the scientometric ranking of applicants

INTRODUCTION

Comparing the scientific output of different researchers applying for a grant is a tedious work. In Hungary, to help reviewers to rapidly rank the scientific productivity of a researcher, a grant decision support tool was established and is available at www.scientometrics.org. In the present study, our goal was to assess the impact of this decision support tool on grant review procedures.

METHODS

The established, publicly available scientometric portal uses four metrics, including the H-index, the yearly citations without self-citations, the number of publications in the last five years, and the number of highly cited publications of a researcher within eleven independent scientific disciplines. Publication-age matched researchers are then ranked and the results are provided to grant reviewers. A questionnaire was completed by reviewers regarding utilization of the scientometric ranking system. The outcome of the grant selection was analyzed by comparing scientometric parameters of applying and funded applicants. We compared three grant allocation rounds before to two grant allocation rounds after the introduction of the portal.

RESULTS

The scientometric decision support tool was introduced in 2020 to assist grant selection in Hungary and all basic research grant applicants (n = 6,662) were screened. The average score of funded proposals compared to submitted proposals increased by 94% after the introduction of the ranking. Correlation between ranking scores and actual grant selection was strong in life and material sciences but some scientific panels had opposite correlation in social sciences and humanities. When comparing selection outcome to H-index across all applicants, both type I and type II errors decreased. All together 540 reviewers provided feedback representing all eleven scientific disciplines and 83.05% of the reviewers (especially younger reviewers) found the ranking useful.

CONCLUSIONS

The scientometric decision support tool can save time and increase transparency of grant review processes. The majority of reviewers found the ranking-based scientometric analysis useful when assessing the publication performance of an applicant.

Scientometric evaluation of highly cited scientists in the field of forensic science and legal medicine

A publically available database of the most highly cited scientists in all disciplines was used to identify people that belonged to the subject category “forensic science and legal medicine.” This bibliometric information was derived from Elsevier’s SCOPUS database containing eight million scientists with at least five articles as author or co-author. The top 100,000 most highly cited scientists were identified and ranked according to six citation metrics; total number of citations, H-index, H-index adjusted for co-authorship, citations to single-authored papers, citations to single or first author papers and, citations to single, first, or last-authored papers. The eight million entries in the SCOPUS database were sub-divided into 22 main subject categories and 176 sub-categories, one of which was legal and forensic medicine. The citation databases were provided as supplementary material in two articles published in PLoS Biology in 2019 and 2020. Among the top 100,000 most highly cited scientists, there were only 30 allocated to the legal and forensic medicine category, according to the 2019 PLoS Biology article. The updated database from 2020 also included the names of people within the top-cited 2% of their scientific discipline. This increased the number of forensic practitioners to 215 from a total of 10,158 individuals in this subject category. This article takes a closer look at these highly cited forensic scientists, the countries where they work, the particular research field in which they publish, and their composite citation scores with and without self-citations. The top ten most cited individuals in both databases (2019 and 2020) were the same and these should therefore be considered an elite group among all forensic practitioners.

Bibliometric-Enhanced Information Retrieval 10th Anniversary Workshop Edition

The Bibliometric-enhanced Information Retrieval workshop series (BIR) was launched at ECIR in 2014 [19] and it was held at ECIR each year since then. This year we organize the 10th iteration of BIR. The workshop series at ECIR and JCDL/SIGIR tackles issues related to academic search, at the crossroads between Information Retrieval, Natural Language Processing and Bibliometrics. In this overview paper, we summarize the past workshops, present the workshop topics for 2020 and reflect on some future steps for this workshop series.