Scientific influence is not always visible: The phenomenon of under-cited influential publications

Analysing academic paper ranking algorithms using test data and benchmarks: an investigation

Research on academic paper ranking has received great attention in recent years, and many algorithms have been proposed to automatically assess a large number of papers for this purpose. How to evaluate or analyse the performance of these ranking algorithms becomes an open research question. Theoretically, evaluation of an algorithm requires to compare its ranking result against a ground truth paper list. However, such ground truth does not exist in the field of scholarly ranking due to the fact that there does not and will not exist an absolutely unbiased, objective, and unified standard to formulate the impact of papers. Therefore, in practice researchers evaluate or analyse their proposed ranking algorithms by different methods, such as using domain expert decisions (test data) and comparing against predefined ranking benchmarks. The question is whether using different methods leads to different analysis results, and if so, how should we analyse the performance of the ranking algorithms? To answer these questions, this study compares among test data and different citation-based benchmarks by examining their relationships and assessing the effect of the method choices on their analysis results. The results of our experiments show that there does exist difference in analysis results when employing test data and different benchmarks, and relying exclusively on one benchmark or test data may bring inadequate analysis results. In addition, a guideline on how to conduct a comprehensive analysis using multiple benchmarks from different perspectives is summarised, which can help provide a systematic understanding and profile of the analysed algorithms.

“Sparking” and “Igniting” Key Publications of 2020 Nobel Prize Laureates

Purpose

This article aims to determine the percentage of “Sparking” articles among the work of this year’s Nobel Prize winners in medicine, physics, and chemistry.

Design/methodology/approach

We focus on under-cited influential research among the key publications as mentioned by the Nobel Prize Committee for the 2020 Noble Prize laureates. Specifically, we extracted data from the Web of Science, and calculated the Sparking Indices using the formulas as proposed by Hu and Rousseau in 2016 and 2017. In addition, we identified another type of igniting articles based on the notion in 2017.

Findings

In the fields of medicine and physics, the proportions of articles with sparking characteristics share 78.571% and 68.75% respectively, yet, in chemistry 90% articles characterized by “igniting”. Moreover, the two types of articles share more than 93% in the work of the Nobel Prize included in this study.

Research limitations

Our research did not cover the impact of topic, socio-political, and author’s reputation on the Sparking Indices.

Practical implications

Our study shows that the Sparking Indices truly reflect influence of the best research work, so it can be used to detect under-cited influential articles, as well as identifying fundamental work.

Originality/value

Our findings suggest that the Sparking Indices have good applicability for research evaluation.

Multimodal ensemble approach to identify and rank top-k influential nodes of scholarly literature using Twitter network

Scholarly literature is an immense network of activities, linked via collaborations or information propagation. Analysing such network can be leveraged by harnessing rich semantic meaning of scholarly graph. Identifying and ranking top- k influential nodes from various domains of scholarly literature using social media data are still infancy. Social networking sites like Twitter provide an opportunity to create inventive graph-based measures to identify and rank influential nodes such as scholars, articles, journal, information spreading media and academic institutions of scholarly literature. Many network-based models such as centrality measures have been proposed to identify influential nodes. The empirical annotation shows that centrality measures for finding influential nodes are high in computational complexity. In addition, notion of these measures have high variance, which signifies an influential node deviation with change in application and nature of information flows in the network. The research aims to propose an ensemble learning approach based on multimodal majority voting influence (MMMVI) to identify and weighted multimodal ensemble average influence (WMMEAI) to rank top- k influential nodes in Twitter network data set of well-known three influential nodes, that is, academic institution, scholar and journal. The empirical analysis has been accomplished to learn practicability and efficiency of the proposed approaches when compared with state-of-the-art approaches. The experimental result shows that the ensemble approach using surface learning models (SLMs) can lead to better identification and ranking of influential nodes with low computational complexity.

A warning for Chinese academic evaluation systems: short-term bibliometric measures misjudge the value of pioneering contributions

Publication citation-based research evaluation, even if only in support of peer review, is not everywhere, on every level, or for everyone suitable, because of differences in scientific research, patterns of research output, stages of scientific evolution, and merits-scientific or societal-of scientific results.

Slow reception and under-citedness in climate change research: A case study of Charles David Keeling, discoverer of the risk of global warming

The Keeling curve has become a chemical landmark, whereas the papers by Charles David Keeling about the underlying carbon dioxide measurements are not cited as often as can be expected against the backdrop of his final approval. In this bibliometric study, we analyze Keeling’s papers as a case study for under-citedness of climate change publications. Three possible reasons for the under-citedness of Keeling’s papers are discussed: (1) The discourse on global cooling at the starting time of Keeling’s measurement program, (2) the underestimation of what is often seen as “routine science”, and (3) the amount of implicit/informal citations at the expense of explicit/formal (reference-based) citations. Those reasons may have contributed more or less to the slow reception and the under-citedness of Keeling’s seminal works.