Measuring scientific contributions with modified fractional counting

COVID-19 Research in Communication Journals: A Structural Topic Modeling-Assisted Bibliometric Analysis

This article presents a bibliometric analysis of research on COVID-19 health communication. We reviewed and analyzed 1,851 articles published in 170 peer-reviewed communication journals between January 2020 and November 2022, to identify key bibliometric information and major research topics in this rapidly expanding field of research. The distribution of countries indicates that the United States is the most productive country, and researchers from Spain, China and the United Kingdom also play an important role. Health Communication is the most influential journal in terms of research productivity and impact. The analysis of highly cited references demonstrates the interdisciplinary nature of this research field. The topics generated by structural topic modeling show that scholars have responded to a variety of issues in COVID-19 communication, encompassing different levels of health communication, the effects of information dissemination, the impact on the general public as well as vulnerable populations, health preventive behaviors and communication technologies. This study aims to enhance researchers' understanding of the current state of this research field and provide insights for future studies.

Is big team research fair in national research assessments? The case of the UK Research Excellence Framework 2021

Collaborative research causes problems for research assessments because of the difficulty in fairly crediting its authors. Whilst splitting the rewards for an article amongst its authors has the greatest surface-level fairness, many important evaluations assign full credit to each author, irrespective of team size. The underlying rationales for this are labour reduction and the need to incentivise collaborative work because it is necessary to solve many important societal problems. This article assesses whether full counting changes results compared to fractional counting in the case of the UK's Research Excellence Framework (REF) 2021. For this assessment, fractional counting reduces the number of journal articles to as little as 10% of the full counting value, depending on the Unit of Assessment (UoA). Despite this large difference, allocating an overall grade point average (GPA) based on full counting or fractional counting gives results with a median Pearson correlation within UoAs of 0.98. The largest changes are for Archaeology (r=0.84) and Physics (r=0.88). There is a weak tendency for higher scoring institutions to lose from fractional counting, with the loss being statistically significant in 5 of the 34 UoAs. Thus, whilst the apparent over-weighting of contributions to collaboratively authored outputs does not seem too problematic from a fairness perspective overall, it may be worth examining in the few UoAs in which it makes the most difference.

An improved author-topic (AT) model with authorship credit allocation schemes

Authorship credit allocation schemes have attracted considerable research attention. However, no consensus about which one is the best has been attained until now, and limited evidence from practical tasks has been reported. Therefore, this study uses the author interest discovery task as a real-world task case to provide valuable insights into authorship credit allocation schemes and guidelines for further practical applications. For this purpose, a novel model, ATcredit, is proposed to strengthen the Author-Topic (AT) model with an authorship credit allocation scheme, and collapsed Gibbs sampling is used to approximate the posterior and estimate model parameters. Extensive experiments using the SynBio dataset reveal several interesting findings as follows. (a) Any scheme for allocating unequal authorship credits performs better than its equal-credit counterpart with our ATcredit model in terms of perplexity. (b) The fixed versions of four out of the six schemes work better than their flexible counterparts with our ATcredit model, regardless of the hyper-authorship strategy. (c) The variation coefficient of credit awards can serve as a criterion to decide whether the hyper-authorship strategy should be used. (d) When the number of authors in a scholarly article is less than three, the six authorship credit allocation schemes are similar to each other with our ATcredit model in terms of perplexity. (e) The harmonic counting scheme performs the best, followed by the arithmetic counting scheme, and the network-based counting scheme performs the worst with our ATcredit model in terms of perplexity. (f) The arithmetic counting scheme is similar to the harmonic counting scheme in terms of the normalised mutual information (NMI) of discovered interests, but the geometric counting scheme is different from the axiomatic and network-based counting schemes.

Quantifying global digital journalism research: a bibliometric landscape

Purpose

The purpose of this paper is to present a comprehensive analysis of the current status and development of the digital journalism field from 1987 to 2021 using the Dimensions database.

Design/methodology/approach

Using the Dimensions.ai database, 1734 articles were identified through search strategies which were published from 1987 to 2021. The downloaded results were analysed using specific parameters with the help of bibliometric and science mapping tools: Biblioshiny, VOSviewer and CiteSpace. The key contributions of the present comprehensive bibliometric study of the digital journalism field can be seen in terms of the following aspects: (1) Publication analysis from the perspectives of publication growth, key journals, contributing authors, institutions and countries done through Biblioshiny package. (2) Citation network analysis from the perspective of co-citation structure of papers, authors, countries and institutions done through VOSviewer. (3) Timeline analysis and keywords burst detection to identify hotspots and research trends in digital journalism with the help of CiteSpace.

Findings

The first paper with the keyword digital journalism was published in the year 1989. From 2011 onwards, there has been growth in digital journalism literature. The most popular journal in digital journalism studies is Digital Journalism, Journalism, Journalism Practice, Journalism Studies. Lewis, S.C. has contributed the most number of papers in digital journalism. Further, authors from the countries the USA, Spain, Brazil and UK have contributed immensely. The citation network of authors, institutions and countries contributing to digital journalism studies has also been explored in the study. Through burst analysis, hot topics in digital journalism were identified.

Originality/value

The paper provides a complete overview of the growth of digital journalism literature published from 1987 to 2021. The originality of this work lies in the triangulation of Biblioshiny, VOSviewer and CiteSpace software to present various aspects of bibliometric study. Findings of the study can help the researchers to identify areas as well as journals, authors, institutions working actively in the field of digital journalism.

Should open access lead to closed research? The trends towards paying to perform research

Open Access (OA) emerged as an important transition in scholarly publishing worldwide during the past two decades. So far, this transition is increasingly based on article processing charges (APC), which create a new paywall on the researchers’ side. Publishing is part of the research process and thereby necessary to perform research. This study analyses the global trends towards paying to perform research by combing observed trends in publishing from 2015 to 2020 with an APC price list. APC expenses have sharply increased among six countries with different OA policies: the USA, China, the UK, France, the Netherlands, and Norway. The estimated global revenues from APC among major publishers now exceed 2 billion US dollars annually. Mergers and takeovers show that the industry is moving towards APC-based OA as the more profitable business model. Research publishing will be closed to those who cannot make an institution or project money payment. Our results lead to a discussion of whether APC is the best way to promote OA.

Bibliometric analysis of articles on nanoemulsion and/or in-situ gel for ocular drug delivery system published during the 2011–2021 period

The bibliometric data were extracted from the Scopus database to investigate the conceptual framework of ocular nanoemulsion and/or in-situ gel drug delivery system using “ocular” AND “nanoemulsion” OR “in-situ gel” keywords. The data were evaluated with RStudio and VOSviewer program.

The results reveal that the publication trends tend to increase continually. India is the most impactful country, and the most constructive institution is Department of Pharmaceutics and Industrial Pharmacy, Faculty of Pharmacy, Cairo University. International Journal of Pharmaceutics is the top influential source. Ali A is the most prolific author. The title of the most impactful article was In-situ gelling systems based on Pluronic F127/Pluronic F68 formulations for ocular drug delivery”. “Controlled release” is the most popular keyword.

These results provide insights for stimulating research collaborations and revealing open issues of controlled-release ocular preparation to overcome an ocular barrier and enhance patient compliance.

On the possibilities of presenting the research performance of an institute over a long period of time: the case of the Institute of Information Science at the University of Graz in Austria

In this paper, we demonstrate how the research performance of a university institute (department) over a long period of time can be presented and evaluated. Using the example of an information science institute at a German-speaking university, namely the (former) Institute of Information Science at the University of Graz in Austria, we present the research performance of this institute over the entire duration of its existence (33 years) in different ways. In order to be able to contextualize its performance, we compare it with that of some related institutions from all over the world. Due to the high effort involved in collecting data and the lack of data availability, the comparison must be limited to a period of a few years and—with regard to the institutions from non-German-speaking countries—to the Web of Science as data source. In this international comparison, the institute in the focus of the study shows relatively poor results. As can be seen, the choice of the data source has a major influence on the evaluation results. Especially for institutes from non-English-speaking countries with publications in their respective national languages, an exclusive use of international databases, such as Web of Science or Scopus, cannot fully consider the whole research performance. The use of personal publication lists or local research databases seems almost indispensable in these cases. A major novelty of this article is the handling of a very long evaluation period and the discussion of different ways of subdividing it. With regard to the presentation of the results, in the case of a long observation period, not only should annual and overall results be presented, but also multi-year comparisons be performed. In this way, year-by-year fluctuations can be smoothed out, and longer-term developments can be well represented.

Examining the developments in scheduling algorithms research: A bibliometric approach

This study examined the developments in the field of Scheduling algorithms in the last 30 years (1992–2021) to help researchers gain new insight and uncover the emerging areas of growth for further research in this field. This study, therefore, carried out a bibliometric analysis of 12,644 peer-reviewed documents extracted from the Scopus database using the Bibliometrix R package for bibliometric analysis via the Biblioshiny web interface. The results of this study established the development status of the field of Scheduling Algorithms, the growth rate, and emerging thematic areas for further research, institutions, and country collaborations. It also identified the most impactful and leading authors, keywords, sources, and publications in this field. These findings can help both budding and established researchers to find new research focus and collaboration opportunities and make informed decisions as they research the field of scheduling algorithms and their applications.

•

Scheduling algorithms research has an annual scientific production growth rate of 16.62%.

•

The keyword “scheduling” is the most relevant keyword. It has the highest betweenness centrality of 101.94.

•

Buyya Rajkumar is the most productive author in scheduling algorithms research.

•

Edge Computing is the most discussed topic concerning scheduling algorithms research in 2020 and 2021.

•

Institutional collaborations have not been well-established in this field.

A Bibliometric Analysis of Robotic Surgery From 2001 to 2021

Introduction

Bibliometric analyses are a method of evaluating the quality of research output in a certain domain. Robotic surgery has made vast leaps during the past 20 years and this paper aimed to assess some of the main areas of research using this method.

Methods

A search was undertaken for documents published between 2001 and 2021 from the World of Science database, using the keywords ‘robotic surgery’, ‘robotic assisted surgery’ and ‘robotic-assisted surgery. Results were compared using numerous bibliometric methodologies, and stratified by source-specific metrics, author-specific metrics and country-specific metrics.

Results

The search yielded 3839 documents, from 879 different sources. Only 2% of sources were found to be within Bradford’s Zone 1 of research and the most relevant sources were from the field of urology. The Journal of Urology and Surgical Endoscopy and other Techniques ranked highly among metrics such as H, G, M index and total citations. The top-rated authors had a H index of 15 in the field of robotic surgery and the total citations reached a peak at 1342. The USA, Japan and Italy were the most productive nations and increased collaborative research is leading to a greater number of multiple-centre publications.

Conclusion

Research into robotic surgery is still in its infancy with further reviews of the literature and greater output through large randomised controlled trials in multiple centres through collaborative research needed.

Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation

The average Category Normalised Citation Impact (CNCI) of an institution’s publication output is a widely used indicator for research performance benchmarking. However, it combines all entity contributions, obscuring individual inputs and preventing clear insight and sound policy recommendations if it is not correctly understood. Here, variations (Fractional and Collaboration [Collab] CNCI)—which aim to address the obscurity problem—are compared to the Standard CNCI indicator for over 250 institutions, spread globally, covering a ten-year period using Web of Science data. Results demonstrate that both Fractional and Collab CNCI methods produce lower index values than Standard CNCI. Fractional and Collab results are often near-identical despite fundamentally different calculation approaches. Collab-CNCI, however, avoids assigning fractional credit (which is potentially incorrect) and is relatively easy to implement. As single metrics obscure individual inputs, institutional output is also deconstructed into five collaboration groups. These groups track the increasing international collaboration trend, particularly highly multi-lateral studies and the decrease in publications authored by single institutions. The deconstruction also shows that both Standard and Fractional CNCI increase with the level of collaboration. However, Collab-CNCI does not necessarily follow this pattern thus enabling the identification of institutions where, for example, their domestic single articles are their best performing group. Comparing CNCI variants and deconstructing portfolios by collaboration type is, when understood and used correctly, an essential tool for interpreting institutional performance and informing policy making.

What Is Wrong With the Current Evaluative Bibliometrics?

Bibliometric data are relatively simple and describe objective processes of publishing articles and citing others. It seems quite straightforward to define reasonable measures of a researcher's productivity, research quality, or overall performance based on these data. Why do we still have no acceptable bibliometric measures of scientific performance? Instead, there are hundreds of indicators with nobody knowing how to use them. At the same time, an increasing number of researchers and some research fields have been excluded from the standard bibliometric analysis to avoid manifestly contradictive conclusions. I argue that the current biggest problem is the inadequate rule of credit allocation for multiple authored articles in mainstream bibliometrics. Clinging to this historical choice excludes any systematic and logically consistent bibliometrics-based evaluation of researchers, research groups, and institutions. During the last 50 years, several authors have called for a change. Apparently, there are no serious methodologically justified or evidence-based arguments in the favor of the present system. However, there are intractable social, psychological, and economical issues that make adoption of a logically sound counting system almost impossible.

A bibliometric review on the drivers of environmental migration

A large body of literature exists arguing that numerous, complex factors result in environmental migration. Thus, in order to understand environmental migration, we must investigate how its drivers are defined, explained and interrelated. This study aims to produce a comprehensive analysis of the literature on the drivers of environmental migration and assess future opportunities for studying 'environmental migration'. We conduct a systematic literature search using the keywords 'environmental migration' and 'drivers' in Scopus and Web of Knowledge, analysing 146 publications. The findings are organised as a bibliometric analysis, including network analysis and evaluation of publication metrics. Results show that the literature on environmental migration drivers constitutes a relatively new, growing field largely developed in the USA. It is rooted in the wider environmental migration literature and strongly associated with the discourse of climate change impacts as driving factors. Typologies of 'migrants' are more prevalent than 'refugees' when referring to actors.

Toward internationalization: A bibliometric analysis of the social sciences in Mainland China from 1979 to 2018

The past 40 years have witnessed profound changes in the international competitiveness of Mainland China’s scientific research. Based on publication data from Chinese researchers in the Social Sciences Citation Index (SSCI) from the Web of Science (WoS), this study aims to provide a bird’s-eye view of how social science research in Mainland China has internationalized over the past four decades. The findings show that the number of social science articles published by Chinese authors in international journals has experienced a noticeable increase, and the collaboration networks of researchers from Mainland China have broadened, with the number of articles with a Chinese first author showing a strong upward trend. In addition, findings show that Chinese scholars are published in a wider range of journals, and there has been a steady increase in their appearance in higher impact journals (influenced in part by certain journals). Finally, different social science disciplines show various degrees of internationalization. This study provides a broad view from which to examine the internationalization process in Mainland China’s social science landscape in the last four decades, while also noting some of the possible explanations for these changes, thereby deepening our understanding of social science research stemming from the region.

The dominance of big teams in China’s scientific output

Modern science is dominated by scientific productions from teams. A recent finding shows that teams of both large and small sizes are essential in research, prompting us to analyze the extent to which a country’s scientific work is carried out by big or small teams. Here, using over 26 million publications from Web of Science, we find that China’s research output is more dominated by big teams than the rest of the world, which is particularly the case in fields of natural science. Despite the global trend that more papers are written by big teams, China’s drop in small team output is much steeper. As teams in China shift from small to large size, the team diversity that is essential for innovative work does not increase as much as that in other countries. Using the national average as the baseline, we find that the National Natural Science Foundation of China (NSFC) supports fewer small teams than the National Science Foundation (NSF) of the United States does, implying that big teams are preferred by grant agencies in China. Our finding provides new insights into the concern of originality and innovation in China, which indicates a need to balance small and big teams.

Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA

Purpose

Building on Leydesdorff, Bornmann, and Mingers (2019), we elaborate the differences between Tsinghua and Zhejiang University as an empirical example. We address the question of whether differences are statistically significant in the rankings of Chinese universities. We propose methods for measuring statistical significance among different universities within or among countries.

Design/methodology/approach

Based on z-testing and overlapping confidence intervals, and using data about 205 Chinese universities included in the Leiden Rankings 2020, we argue that three main groups of Chinese research universities can be distinguished (low, middle, and high).

Findings

When the sample of 205 Chinese universities is merged with the 197 US universities included in Leiden Rankings 2020, the results similarly indicate three main groups: low, middle, and high. Using this data (Leiden Rankings and Web of Science), the z-scores of the Chinese universities are significantly below those of the US universities albeit with some overlap.

Research limitations

We show empirically that differences in ranking may be due to changes in the data, the models, or the modeling effects on the data. The scientometric groupings are not always stable when we use different methods.

Practical implications

Differences among universities can be tested for their statistical significance. The statistics relativize the values of decimals in the rankings. One can operate with a scheme of low/middle/high in policy debates and leave the more fine-grained rankings of individual universities to operational management and local settings.

Originality/value

In the discussion about the rankings of universities, the question of whether differences are statistically significant, has, in our opinion, insufficiently been addressed in research evaluations.

Interpreting Bibliometric Data

Many academic analyses of good practice in the use of bibliometric data address only technical aspects and fail to account for and appreciate user requirements, expectations, and actual practice. Bibliometric indicators are rarely the only evidence put before any user group. In the present state of knowledge, it is more important to consider how quantitative evaluation can be made simple, transparent, and readily understood than it is to focus unduly on precision, accuracy, or scholarly notions of purity. We discuss how the interpretation of 'performance' from a presentation using accurate but summary bibliometrics can change when iterative deconstruction and visualization of the same dataset is applied. From the perspective of a research manager with limited resources, investment decisions can easily go awry at governmental, funding program, and institutional levels. By exploring select real-life data samples we also show how the specific composition of each dataset can influence interpretive outcomes.

National Lists of Scholarly Publication Channels: An Overview and Recommendations for Their Construction and Maintenance

Purpose

This paper presents an overview of different kinds of lists of scholarly publication channels and of experiences related to the construction and maintenance of national lists supporting performance-based research funding systems. It also contributes with a set of recommendations for the construction and maintenance of national lists of journals and book publishers.

Design/methodology/approach

The study is based on analysis of previously published studies, policy papers, and reported experiences related to the construction and use of lists of scholarly publication channels.

Findings

Several countries have systems for research funding and/or evaluation, that involve the use of national lists of scholarly publication channels (mainly journals and publishers). Typically, such lists are selective (do not include all scholarly or non-scholarly channels) and differentiated (distinguish between channels of different levels and quality). At the same time, most lists are embedded in a system that encompasses multiple or all disciplines. This raises the question how such lists can be organized and maintained to ensure that all relevant disciplines and all types of research are adequately represented.

Research limitation

The conclusions and recommendations of the study are based on the authors’ interpretation of a complex and sometimes controversial process with many different stakeholders involved.

Practical implications

The recommendations and the related background information provided in this paper enable mutual learning that may feed into improvements in the construction and maintenance of national and other lists of scholarly publication channels in any geographical context. This may foster a development of responsible evaluation practices.

Originality/value

This paper presents the first general overview and typology of different kinds of publication channel lists, provides insights on expert-based versus metrics-based evaluation, and formulates a set of recommendations for the responsible construction and maintenance of publication channel lists.

Bilateral Co-authorship Indicators Based on Fractional Counting

Purpose

In this contribution we provide two new co-authorship indicators based on fractional counting.

Design/methodology/approach

Based on the idea of fractional counting we reflect on what should be an acceptable indicator for co-authorship between two entities. From this reflection we propose an indicator, the co-authorship score, denoted as cs, using the harmonic mean. Dividing this new indicator by the classical co-authorship indicator based on full counting, leads to a co-authorship intensity indicator.

Findings

We show that the indicators we propose have many necessary or at least highly desirable properties for a proper cs-score. It is pointed out that the two new indicators can be used for countries, but also for institutions and other pairs of entities. A small example shows the feasibility of the co-authorship score and the co-authorship intensity indicator.

Research limitations

The indicators are not yet tested in real cases.

Practical implications

As the notions of co-authorship and collaboration have many aspects, we think that our contribution may help policy management to take yet another aspect into account as part of a multi-faceted description of research outcomes.

Originality/value

The indicators we propose cover yet another aspect of co-authorship.

The Norwegian Model in Norway

The “Norwegian Model” attempts to comprehensively cover all the peer-reviewed scholarly literatures in all areas of research in one single weighted indicator. Thereby, scientific production is made comparable across departments and faculties within and between research institutions, and the indicator may serve institutional evaluation and funding. This article describes the motivation for creating the model in Norway, how it was designed, organized and implemented, as well as the effects and experiences with the model. The article ends with an overview of a new type of bibliometric studies that are based on the type of comprehensive national publication data that the Norwegian Model provides.