Alternatives to the impact factor

Should the Eigenfactor Replace the Impact Factor?: A Bibliometric Study of Plastic, Reconstructive, and Aesthetic Surgery Journals

BACKGROUND

Despite the global acceptance of the impact factor (IF) by researchers and academic institutions as one, criticism has been voiced regarding its effectiveness in evaluating the unbiased impact of a journal. To overcome the limitations and drawbacks of the IF that were presented by the scientific community, they introduced several other citation-based bibliometric indices.

METHODS

All plastic and reconstructive surgery journals were analyzed, and bibliometric indices were extracted from the relevant source. Pearson correlation coefficients were used to determine correlations between the various indices.

RESULTS

All correlations in between the various bibliometric indices were found to be positively and statistically significant, ranging from moderately highly associated for the IF and Eigenfactor (r = 0.632) to very strongly associated (r = 0.962) for the IF and the Article Influence Score.

CONCLUSION

In the field of plastic and reconstructive surgery, the Eigenfactor and its derivative, the Article Influence Score, could potentially serve as better indices than the IF in demonstrating the overall picture. This information is based on the inclusion of a greater number of journals in the calculation and subtraction of self-citations, without compromising their correlation with other indices.The use of Eigenfactor and other additional bibliometrics cooperatively with the IF could provide the most extensive evaluation of a journal's scientific impact.

Bibliometric and Density Visualisation Mapping Analysis of Domestic Violence in Australia Research Output 1984-2019

Domestic violence is highly prevalent in Australia and has serious and complex impacts. This study aimed to analyse research outputs on domestic violence in Australia from the period of 1984 to 2019. Articles relevant to domestic violence in Australia that met specified inclusion criteria were retrieved using the Scopus database. Bibliometric analysis of the output was conducted to examine trends in publications. A trend of an increase in publications relating to domestic violence in Australia over time was identified, with the majority published in institutions located in densely populated capital cities. Significant diversity was found in the subject matter of highly cited articles, reflecting the far-reaching impacts of domestic violence. The increase in social attention to domestic violence over time was reflected in an increase in publications. Future research would benefit from examining trends in the reporting of domestic violence, and analysing the effectiveness of interventions for perpetrators and victims.

Twenty-year span of global acute pancreatitis trends: A bibliometric analysis

OBJECTIVE

This study aims to investigate the global research routine and trends of acute pancreatitis over the last twenty years based on the production, hotspots, and frontiers of published articles as well as to provide the global health system with a bibliometric reference.

METHODS

The Web of Science core collection database was retrieved for acute pancreatitis original articles and review articles published from January 1, 1999 to May 17, 2020. Duplicates and discrete papers were excluded. Articles were evaluated for several characteristics including number of citations, publication time, country of origin, institution, journal and authorship.

RESULTS

A total of 7001 articles originated from 94 countries and were published in 1263 journals. The China contributed most articles (1752) followed by USA (1214). The research was major published in specialized journals including the Pancreas (511) and pancreatology (351). Universities were the main institutions of science progress. High-impact articles focused on the fields of clinical medicine. A steady growth was observed in the last 20 years from 1999 to 2020.

CONCLUSION

This comprehensive bibliometric study indicates that severe acute pancreatitis and necrotizing pancreatitis are significant topic in the acute pancreatitis research. The structured information may be helpful in understanding research trends, and locating research hot spots and gaps in this domain.

Questioning the Impact of the Impact Factor. A Brief Review and Future Directions

Purpose: To provide a brief review of literature on the journal impact factors (JIF) and the newer research metrics being proposed or implemented.Methods: The authors performed a PubMed search of articles published in the English language on the journal impact factors. Data captured include historical perspectives, evolution, calculation, criticisms of JIF and their rebuttals, and organized efforts to address JIF issues, alternate research metrics, and future directions. Specific emphasis was laid on evaluating the criticisms, current lacunae, and the changing practice patterns.Results: One of the measures to assess the research impact of an article is the number of citations it receives. Hence, citation-based metrics are commonly used for such purposes. While editors and well-known scholars refrain from attributing article success to the journal's prominence, the same is not true for most authors. JIF is still one of the top factors when deciding on an article submission. JIF is today an acceptable objective and quantifiable measure of knowledge dissemination. However, JIF should not be used as a surrogate measure to assess an individual researcher or an individual article. The reverence to JIF in this regard needs to be questioned. While alternate metrics or altmetrics have their advantages and limitations, they nevertheless augur well an era where scientometrics are complementary to one another without undue reliance on a sole parameter.Conclusion: While there is no need to demonize the JIF, its role in the scholarly assessment should be scaled down. The over-reliance and undue hype surrounding it should be discouraged at multiple scientific levels.

Do journals raise their impact factor or SCImago ranking by self-citing in editorials? A bibliometric analysis of trauma and orthopaedic journals

BACKGROUND

Impact factor (IF) is widely accepted as a measure of a journal's quality but it can be influenced by self-citation. However, the SCImago Journal Rank (SJR) is based on journal prestige, excludes self-citation and considers quality of citations of a journal. This study aimed to investigate journal editors' use of self-citation and whether this correlated with IFs or SJR in trauma and orthopaedic (T&O) journals.

METHODS

T&O journals on the SJR database were identified. From each journal, data including country of publication, number of annual issues, IF, SJR indicator and citable articles were extracted. The editorial(s) of each issue in 2018 were reviewed. The total number of times the editors cited their own previous work or their journal (in the preceding 2 years, 2016-2017) were identified. Regression analyses were performed to investigate the association of editorial self-citation with journal IF or SJR indicator.

RESULTS

Of the 270 journals identified, 43 T&O journals with 151 editorials were included in the final analysis. A positive correlation between journal self-citation in the editorial and IF (P = 0.02) and SJR indicator (P = 0.02) was found. Citation by editors of their own publications within editorials also positively correlated with IF (P = 0.04) but not for SJR indicator (P = 0.19). There was a positive linear relationship between journal IF and SJR indicators (P < 0.01).

CONCLUSION

Editor self-citation influences the IF and SJR indicators in T&O journals. Therefore, these metrics should be considered in conjunction with other factors such as audience, topics included and international presence when evaluating journals.

Evaluating Journal Impact Factor: a systematic survey of the pros and cons, and overview of alternative measures

Background

Journal Impact Factor (JIF) has several intrinsic flaws, which highlight its inability to adequately measure citation distributions or indicate journal quality. Despite these flaws, JIF is still widely used within the academic community, resulting in the propagation of potentially misleading information. A critical review of the usefulness of JIF is needed including an overview of the literature to identify viable alternative metrics. The objectives of this study are: (1) to assess the usefulness of JIF by compiling and comparing its advantages and disadvantages; (2) to record the differential uses of JIF within research environments; and (3) to summarize and compare viable alternative measures to JIF.

Methods

Three separate literature search strategies using MEDLINE and Web of Science were completed to address the three study objectives. Each search was completed in accordance with PRISMA guidelines. Results were compiled in tabular format and analyzed based on reporting frequency.

Results

For objective (1), 84 studies were included in qualitative analysis. It was found that the recorded advantages of JIF were outweighed by disadvantages (18 disadvantages vs. 9 advantages). For objective (2), 653 records were included in a qualitative analysis. JIF was found to be most commonly used in journal ranking (n = 653, 100%) and calculation of scientific research productivity (n = 367, 56.2%). For objective (3), 65 works were included in qualitative analysis. These articles revealed 45 alternatives, which includes 18 alternatives that improve on highly reported disadvantages of JIF.

Conclusion

JIF has many disadvantages and is applied beyond its original intent, leading to inaccurate information. Several metrics have been identified to improve on certain disadvantages of JIF. Integrated Impact Indicator (I3) shows great promise as an alternative to JIF. However, further scientometric analysis is needed to assess its properties.

Journal impact factor: a bumpy ride in an open space

The journal impact factor (IF) is the leading method of scholarly assessment in today’s research world. An important question is whether or not this is still a constructive method. For a specific journal, the IF is the number of citations for publications over the previous 2 years divided by the number of total citable publications in these years (the citation window). Although this simplicity works to an advantage of this method, complications arise when answers to questions such as ‘What is included in the citation window’ or ‘What makes a good journal impact factor’ contain ambiguity. In this review, we discuss whether or not the IF should still be considered the gold standard of scholarly assessment in view of the many recent changes and the emergence of new publication models. We will outline its advantages and disadvantages. The advantages of the IF include promoting the author meanwhile giving the readers a visualization of the magnitude of review. On the other hand, its disadvantages include reflecting the journal’s quality more than the author’s work, the fact that it cannot be compared across different research disciplines, and the struggles it faces in the world of open access. Recently, alternatives to the IF have been emerging, such as the SCImago Journal & Country Rank, the Source Normalized Impact per Paper and the Eigenfactor Score, among others. However, all alternatives proposed thus far are associated with their own limitations as well. In conclusion, although IF contains its cons, until there are better proposed alternative methods, IF remains one of the most effective methods for assessing scholarly activity.

Impact factor correlations with Scimago Journal Rank, Source Normalized Impact per Paper, Eigenfactor Score, and the CiteScore in Radiology, Nuclear Medicine & Medical Imaging journals

INTRODUCTION

In the last decade, several journal's editors decided to publish alternative bibliometric indices parallel to the impact factor (IF): Scimago Journal Rank (SJR), Source Normalized Impact per Paper (SNIP), Eigenfactor Score (ES) and CiteScore™ (CiteScore); however, there is scarce information about the correlations among them. In this study, we aimed to evaluate the associations between this bibliometrics in the Radiology, Nuclear Medicine & Medical Imaging category of the Web of Knowledge. We hypothesized the IF did not show the best correlation with other metrics.

METHODS

Retrospective study. We used bibliometrics recorded from the 2017 publicly available versions of the Journal Citation Reports (JCR), SJR ( www.scimagojr.com ), SNIP ( www.journalindicators.com ), and CiteScore ( www.scopus.com ); we also included the Total Cites. We measured the correlations using the Spearman correlation coefficients (R_S) for all combinations of the bivariate pair, performed pairwise comparisons of the R_S values, and calculated the coefficients of determination. We also tested the statistical significance of the difference between r coefficients between groups. All analyses were conducted with the JMP Pro software.

RESULTS

The stronger bivariate correlations were represented by the ES↔Total Cites R_S = 0.968, p < 0.001, R² = 0.937; and the CiteScore↔SJR R_S = 0.911, p < 0.001, R² = 0.829. From 105 possible combinations of pairwise comparisons, 38 depicted a p value > 0.050 which would suggest interchangeability among bivariate correlations.

CONCLUSIONS

Our findings support our hypothesis that the IF does not show the best correlation between other metrics. Radiologists, interventional radiologist, or nuclear medicine doctors should have a clear understanding of the associations among the journal's bibliometrics for their decision-making during the manuscript submission phase.

The democratization of scientific publishing

Where should I submit my paper? This is a question that young scientists and trainees frequently ask. In this Commentary, we advise on how to make such a decision whilst balancing the risks and benefits. We argue that trying to publish in top tier journals may not always be the best option and that publishing in indexed, open access journals may expose research to the same or larger audiences. The value of research should not be judged according to the publishing journal's name, but rather from other measures of impact such as successful commercialization of new technologies, number of citations, and downloads. We also highlight the role of mentors, who have the responsibility to protect the long-term interests of their trainees by balancing the consequences of acceptances and rejections.

Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics

BACKGROUND

Understanding the impact of a publication by using bibliometric indices becomes an essential activity not only for universities and research institutes but also for individual academicians. This paper aims to provide a brief review of the current bibliometric tools used by authors and editors and proposes an algorithm to assess the relevance of the most common bibliometric tools to help the researchers select the fittest journal and know the trends of published submissions by using self-evaluation.

METHODS

We present a narrative review answering at least two related consecutive questions triggered by the topics mentioned above. How prestigious is a journal based on its most recent bibliometrics, so authors may choose it to submit their next manuscript? And, how can they self-evaluate/understand the impact of their whole publishing scientific life?

RESULTS

We presented the main relevant definitions of each bibliometrics and grouped them in those oriented to evaluated journals or individuals. Also, we share with our readers our algorithm to assess journals before manuscript submission.

CONCLUSIONS

Since there is a journal performance market and an article performance market, each one with its patterns, an integrative use of these metrics, rather than just the impact factor alone, might represent the fairest and most legitimate approach to assess the influence and importance of an acceptable research issue, and not only a sound journal in their respective disciplines.

Past, current, and future research on microalga-derived biodiesel: a critical review and bibliometric analysis

Microalga-derived biodiesel plays a crucial role in the sustainable development of biodiesel in recent years. Literature related to microalga-derived biodiesel had an increasing trend with the expanding research outputs. Based on the Science Citation Index Expanded (SCI-Expanded) of the Web of Science, a bibliometric analysis was conducted to characterize the body of knowledge on microalga-derived biodiesel between 1993 and 2016. From the 30 most frequently used author keywords, the following research hotspots are extracted: lipid preparation from different microalga species, microalga-derived lipid and environmental applications, lipid-producing microalgae cultivation, microalgae growth reactor, and microalga harvest and lipid extraction. Other keywords, i.e., microalga mixotrophic cultivation, symbiotic system between microalga and other oleaginous yeast, microalga genetic engineering, and other applications of lipid-producing microalga are future focal points of research. Graphical abstract.

Objective Methodology to Assess Meaningful Research Productivity by Orthopaedic Residency Departments: Validation Against Widely Distributed Ranking Metrics and Published Surrogates

The mission of any academic orthopaedic training program can be divided into 3 general areas of focus: clinical care, academic performance, and research. Clinical care is evaluated on clinical volume, patient outcomes, patient satisfaction, and becoming increasingly focused on data-driven quality metrics. Academic performance of a department can be used to motivate individual surgeons, but objective measures are used to define a residency program. Annual in-service examinations serve as a marker of resident knowledge base, and board pass rates are clearly scrutinized. Research productivity, however, has proven harder to objectively quantify. In an effort to improve transparency and better account for conflicts of interest, bias, and self-citation, multiple bibliometric measures have been developed. Rather than using individuals' research productivity as a surrogate for departmental research, we sought to establish an objective methodology to better assess a residency program's ability to conduct meaningful research. In this study, we describe a process to assess the number and quality of publications produced by an orthopaedic residency department. This would allow chairmen and program directors to benchmark their current production and make measurable goals for future research investment. The main goal of the benchmarking system is to create an "h-index" for residency programs. To do this, we needed to create a list of relevant articles in the orthopaedic literature. We used the Journal Citation Reports. This publication lists all orthopaedic journals that are given an impact factor rating every year. When we accessed the Journal Citation Reports database, there were 72 journals included in the orthopaedic literature section. To ensure only relevant, impactful journals were included, we selected journals with an impact factor greater than 0.95 and an Eigenfactor Score greater than 0.00095. After excluding journals not meeting these criteria, we were left with 45 journals. We performed a Scopus search over a 10-year period of these journals and created a database of articles and their affiliated institutions. We performed several iterations of this to maximize the capture of articles attributed to institutions with multiple names. Based off of this extensive database, we were able to analyze all allopathic US residency programs based on their quality research productivity. We believe this as a novel methodology to create a system by which residency program chairmen and directors can assess progress over time and accurate comparison with other programs.

Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of Radiology, Nuclear Medicine and Medical Imaging journals

INTRODUCTION

Because we believe the journal selection before a manuscript submission deserves further investigation in each medical specialty, we aimed to evaluate the predictive ability of seven bibliometrics in the Radiology, Nuclear Medicine and Medical Imaging category of the Web of Knowledge to calculate total citations over a 7-year period.

METHODS

A linear mixed effects design using random slopes and intercepts were performed on bibliometrics corresponding to 124 journals from 2007 to 2011, with their corresponding citations from 2009 to 2013, which appeared in the Journal Citations Report Science Edition.

RESULTS

The Eigenfactor Score, Article Influence Score, Cited Half-life, 5-years impact factor and Number of Articles are significant predictors of 2-year-ahead total citations (p ≤ 0.010 for all variables). The impact factor and Immediacy Index are not significant predictors. There was a significant global effect size (R² = 0.934; p < 0.001), which yielded a total variance of 93.4%.

CONCLUSIONS

Our findings support researchers' decision to stop the misuse of IF alone to evaluate journals. Radiologists and other researchers should review journal's bibliometrics for their decision-making during the manuscript submission phase. A re-ranking of journals using Eigenfactor Score, Article Influence Score, and Cited Half-life provides a better assessment of their significance and importance in particular disciplines.

The emerging landscape of scientific publishing

We present emerging models of publishing which have grown from the phenomenon of open access, the changing role of peer review in the scientific process and the new position of the impact factor. We juxtapose the new models of paid review, eponymous review, no review, post publication review and light review with the classic model which has dominated for a century, detailing advantages, problems and examples of each model to provide a comprehensive overview of the changing landscape of scientific publishing.

Comparative analysis of quantity and quality of biomedical publications in Gulf Cooperation Council countries from 2011-2013

Objectives:

To compare the research productivity of different Gulf Cooperation Council (GCC) countries in the field of biomedical sciences from 2011-2013.

Methods:

This is a retrospective study conducted in the College of Medicine, Alfaisal University, Riyadh, Saudi Arabia. Data on the biomedical publications originating from GCC countries published between January 2011 to December 2013 was searched via MEDLINE using PubMed. The total number of publications emanating from each country was normalized with the country’s population. The mean impact factor (IF) of all the publications in a year was calculated for comparative analysis.

Results:

A total of 11,000 publications were retrieved via MEDLINE using PubMed, out of which, 9222 were selected for analysis. A successive increase in the number of publications by every country was observed. The most striking increase in the number of publications was from Saudi Arabia. However, after normalization with population, the performance of Oman, Qatar, and Kuwait looks far better than Saudi Arabia in terms of research productivity. Data on mean IF showed that the overall mean IF of all GCC countries has remained largely unchanged except Oman. Although Oman had a comparatively low mean IF value in 2011, they recorded a tremendous improvement in successive years.

Conclusion:

All GCC countries underwent an increase in quantitative research productivity over the last 3 years. However, no increase in quality of research publications was noted based on the proxy reports of mean journal IF.

Analysis of self-citation and impact factor in dermatology journals

BACKGROUND

Concerns have been raised regarding the impact factor's (IF) accuracy and credibility, which may be affected by different factors, including self-citations.

OBJECTIVE

To investigate the self-citation rate (SCR) of dermatology journals and its relationship to the IF.

METHODS

Data on all dermatology journals listed in the Journal Citation Reports (JCR) were retrieved, and the following parameters were analyzed: IF, total publications used to calculate the IF, total citations used to calculate the IF, self-citations used to calculate the IF, SCR, and IF without self-citations (corrected IF).

RESULTS

The median SCR was 10.53% (0-50%), and the median IF and corrected IF, 1.54 (0.05-6.37) and 1.35 (0.03-5.84), respectively. There was an inverse correlation between the IF and the SCR. A statistically significant difference was noted in the SCR between general and subspecialty journals and between journals that offered a full English text and those that did not.

CONCLUSIONS

In general, the IF of dermatology journals is not influenced by the SCR. However, journals with a lower IF tend to have a higher SCR. Subspecialty journals and foreign language journals have a higher SCR than general dermatology and English language journals, respectively, probably owing to their limited distribution and the difficulty experienced by international authors in accessing references in specific languages.

Ascertaining the standard of journal using quality indices

In the modern era, most of the researchers want to share their work on a global platform so that they not only receive the due recognition but even their findings can be utilized by other professionals working in the same arena. In order to achieve this, the most common approach is to publish the research findings in an appropriate journal. However, the indicators which eventually determine the overall quality of a journal are variable and there is a great need that the contributors should understand the meaning and scope of each of these indicators. In conclusion, in order to establish the journal's quality, the researchers should obtain the information about the various indices from the journals' website or editorial board and then only submit their research work for publishing.

Citation analysis of otorhinolaryngology journals: follow-up study

OBJECTIVE

This study aimed to evaluate the changes in impact factors of otorhinolaryngology journals over the past 15 years.

METHOD

Using the online edition of Journal Citation Reports, standard (2-year) and 5-year impact factors were calculated for the leading 15 journals.

RESULTS

The results were compared with the impact factors for 1998. The average standard impact factor and 5-year impact factor increased by 2.72 and 2.05 fold respectively when compared with 1998. The average 2012 standard impact factor and 5-year impact factor were 1.82 and 1.99 respectively, reflecting a 9.3 per cent difference. The average 1998 standard impact factor and 5-year impact factor were 0.67 and 0.97 respectively, reflecting a 44.8 per cent difference. The Journal of the Association for Research in Otolaryngology had the highest standard and five-year impact factors.

CONCLUSION

These data may indicate changing clinical and research interests within our field, as well as increased speed and ease with which the internet has allowed citation. As a result, five-year intervals may no longer be necessary to adequately gauge journal impact.