The Journal Impact Factor: Moving Toward an Alternative and Combined Scientometric Approach

Advancing the impact of research through a dissemination-focused special interest group

BACKGROUND

The proverbial gap between research and translation to the real world is a complex and multi-factorial issue that persists and threatens the impact of research. Dissemination and implementation science emerged as significant contributors to knowledge translation. Much attention has been focused on implementation, with less developed methods and work dedicated on dissemination. Our academic research centre identified the need to better understand the intersection of ageing research and dissemination science for impactful and equitable dissemination of ageing research beyond the academic audience.

OBJECTIVE

We describe the purpose, deliverables, and plans of the Dissemination Special Interest Group (SIG) as a model for academic research centres to support and advance the dissemination efforts of their members.

SUMMARY OF KEY ARGUMENTS

In the long term, achieving robust dissemination will require restructuring academic and research incentives, alongside developing infrastructure and methods to assess the impact of dissemination efforts on the translation of ageing research findings. However, actionable efforts can be taken immediately for meaningful impact.

CONCLUSION

Our Dissemination SIG can serve as a model for advancing and supporting dissemination within other research centres, regardless of content and focus areas. More work is needed to develop infrastructure capable of assessing the reach and impact of dissemination efforts on the adoption of research findings.

Is journal prestige indicator equivalent of money for humans?

In this paper, we investigate the time evolution of the rank correlation coefficients of the prestige indicators of the journals (such as Scimago Journal Rank) over 21 years, for 9856 journals in various disciplines. The observed dependencies do not differ qualitatively from those observed for the time evolution of the ranking correlations in wealth rheology among the richest people recently found by Burda et al. [Entropy 23, 842 (2021)], but quantitatively the changes in rank correlations for human wealth are much stronger than in the coefficients measuring the prestige of journals. In terms of Kendall's rank correlation coefficients, disciplines "Agricultural and Biological Sciences" and "Earth and Planetary Science" are the most typical, meaning that the time evolution of their rank correlation coefficient based on Scimago Journal Rank is closest to the time evolution of an average over all disciplines. The time evolution of the mean value of the overlap ratio is closest to the time evolution of the overlap ratio for "Psychology" and "Physics and Astronomy" disciplines. The Kendall rank correlation coefficients and the overlap ratio decrease nearly linearly over time, except for the first year, where their decay is significantly faster. The analogy between human wealth and the prestige indicators of journals is strengthened by performing a Monte Carlo simulation of the time evolution of the probability distribution function of Scimago Journal Rank. The simulation is based on a simple model of economic systems. Due to the different coverage of the journals in Scimago Journal & Country Rank and Web of Science, the identically defined measures based on the data in these databases are different.

Preterm Birth Research in Southeast Asia and its Association with Socioeconomic Determinants and Burden of Disease: A Bibliometric Analysis

Objective

The aim of this study was to assess research productivity on preterm birth (PTB) in Southeast Asian (SEA) countries and its correlation with socioeconomic characteristics and burden of disease.

Methods

A systematic review of preterm birth publications by SEA authors indexed in Scopus, PubMed, ClinicalTrials. gov, and Cochrane was done. Case reports, cohorts, control trials, reviews and cost analysis studies done by SEA researches involving pathophysiology, diagnosis, management, and complications of preterm birth was included in the study while published letters to editors were excluded. The correlation of bibliometric indices, namely Scopus citations, and PlumX metrics indices (citations, usage, captures, mentions, and social media), with socioeconomic status and burden of preterm birth in SEA countries were analyzed by computing for the correlation coefficient (r) and p-value at an alpha of 0.05.

Results

Thailand had the highest number of publications and the highest count across all bibliometric indices among all countries in SEA. The percent gross domestic product (GDP) per capita allotted for research and development (R & D) had direct correlation with publications and captures while crude birth rates had indirect correlation with publications, citations, and captures. Neonatal mortality had indirect correlation with publications and captures.

Conclusion

Support for research and development is essential to increase research productivity in SEA, which in turn may help in finding solutions to decrease the rate of preterm birth in the region.

Trends and characteristics of employing cavitation technology for water and wastewater treatment with a focus on hydrodynamic and ultrasonic cavitation over the past two decades: A Scientometric analysis

Cavitation-based technologies have emerged as a sustainable and effective way to treat natural waters and wastewater, considering their increasing scarcity due to pollution and climate change. For this reason, this work aimed to conduct a scientometric analysis on the topic of cavitation for water and wastewater treatment during the last 20 years, from 2001 to August 2022. We focused on hydrodynamic and ultrasonic cavitation as the prevalent methods of inducing cavitation. Furthermore, an in-depth study on the main trends regarding the number of publications and citations, keywords co-occurrence and evolution, and countries' publication trends was carried out to investigate the future direction of this research topic. The data was gathered from the Web of Science database and analyzed by the Visualization Of Similarities software. This work focused on: i) publication and citation trends, ii) scientific categories, iii) countries' contribution to the topic of cavitation, iv) prominent journals, v) keyword co-occurrence and cluster analysis, and vi) keyword evolution analysis. Results showed a significant increase in publications during the past 5 years. The scientific categories with the highest number of publications were "environmental sciences" and "environmental engineering," with a combined share of 19.4 % of publications. Keywords evolution analysis showed that limited focus was given to topics related to "energy" and "energy efficiency" in the field of cavitation, but with the rising importance of each process's sustainability, the attention given to these concepts will increase in the future. Future directions for the topic of cavitation-related water and wastewater treatments will shift towards more environmentally friendly applications of hydrodynamic and ultrasonic cavitation as well as towards more green and sustainable approaches to address the increasing water pollution problems and shortage. Moreover, it will include other uses besides water treatment such as manufacturing nanomaterials food production and medicine.

SOCIAL MEDIA IN SCIENCE

Day by day, social media are entering our lives changing our every-day life and habits. This could not leave science unaffected. Over the last years, a lot of scientists have started using social media, exchanging ideas and sharing knowledge. Scientific journals have also followed being active participators in the social media world. In this opinion article we provide some arguments about the positive role of social media in the life of a scientist, highlighting at the same time the caveats that are encountered.  

Publication Times and Impact Factors (IFs) in Dentistry Journals

INTRODUCTION

 The speed of manuscript publication in reputable journals plays a crucial role in spreading scientific novelties and may influence the number of received citations. In the present study, the authors investigated the publication speed of dentistry journals. This is crucial for both authors, who desire rapid dissemination of their findings, and patients in need, who seek new therapies.

MATERIALS AND METHODS

 This was a cross-sectional bibliometric analysis of published dentistry journals. A list of dentistry journals featured in the 2021 Journal Citation Report was downloaded. A total of five random original articles were extracted from each of these journals. These articles were published between January and December 2020. Median and interquartile range (IQR) times from submission to acceptance, publication in print, online publication, time from acceptance to in print and online publication were calculated. The correlation between publication times and journal impact factor (IF) was examined.

RESULTS

 A total of 89 journals were included. Individual time from submission to acceptance (peer review time) ranged from 6 to 279 days, the combined median peer review time was 115 (80-159) days. The overall median time from acceptance to online or print publication was 17 (12-38) and 153 (92-249) days, respectively. Journals with available data concerning publication times tended to have higher IF than others. Only journals that did not have available time from acceptance to online publication had higher IF. There were negative correlations between times from submission (r = -0.442, p = 0.007), acceptance (r = -0.616, p < 0.001) to in-print publication, and IF. There were no correlations between IF and time from submission to acceptance, acceptance to online publication, and submission to online publication.

CONCLUSIONS

 Publication times availability was revealed to be an indicator of higher impacted journals, which is a potential new exponent of journal quality. Higher IF values were associated with shorter times from submission to acceptance and in-print publication, which is consistent with current editorial policies.

Research rating: Some technicalities

From time immemorial, the body of scientific knowledge has grown with incremental additions of research. Metrics-based research evaluation provides crucial information regarding research credibility that would be difficult to understand by means of individual expertise. h-index and its modifications give an approximate quantitative measure of research output. Furthermore, g-index, e-index, ħ-index and i10-index address various intricacies involving authorship. Altmetrics and Plum X metrics are newer usage metrics that put an additional weightage on the impact on social media, usage, capture and scholarly networking. Indirect evaluation of research can also be obtained from the Journal Impact Factor in which the research is published but with certain limitations. While the scientific community is still waiting for a unique one-stop solution based on a high-quality robust process to exert judgement on research, the Leiden Manifesto comprising ten principles for research assessment can act as a guiding tool for development of a comprehensive evaluation system.

Characteristics and Research Techniques Associated with the Journal Impact Factor and Other Key Metrics in Pharmacology Journals

In the present age, there is intense pressure on researchers to publish their research in ‘high-impact factor’ journals. It would be interesting to understand the trend of research publications in the field of pharmacology by exploring the characteristics of research articles, including research techniques, in relation to the journal’s key bibliometrics, particularly journal impact factor (JIF), the seemingly most mentioned metric. This study aimed to determine the characteristics and research techniques in relation to research articles in pharmacology journals with higher or lower JIF values. A cross-sectional study was conducted on primary research journals under the ‘Pharmacology and Pharmacy’ category. Analysis of 768 original research articles across 32 journals (with an average JIF of 2.565 ± 0.887) demonstrated that research studies involving molecular techniques, in vivo experiments on animals, and bioinformatics and computational modeling were significantly associated with a higher JIF value of the journal in which such contributions were published. Our analysis suggests that research studies involving such techniques/approaches are more likely to be published in higher-ranked pharmacology journals.

Bibliometric analysis of publication activity in the field of familial Mediterranean fever in 2010-2019: a Scopus-based study

Familial Mediterranean fever (FMF) is a systemic autoinflammatory disease, which presents with recurrent episodes of fever, serositis, synovial inflammation and various different clinical features. In parallel with the geographical distribution of FMF, the prevalence is notably higher in some population groups including Turks, Jews, Armenians and Arabs. The aim of this study was to assess FMF articles published between 2010 and 2019 bibliometrically and to present up-to-date data. This bibliometric research was conducted on the Scopus database. The key term ''familial Mediterranean fever'' was utilized to generate the publication list. The number of articles, active countries journals, prolific authors and citation data were determined. Visualization maps that revealed collaboration networks were also created. The number of FMF articles displayed a stable trend between 2010 and 2019 (p > 0.05). The top five countries according to the number of articles were Turkey (n = 473; 52.90%), Israel (n = 86; 9.61%), Japan (n = 39; 4.36%), Italy (n = 38; 4.25%) and the USA (n = 37; 4.13%. Italy, the USA and France were ranked as the top three countries for the average number of citations per article. Rheumatology International, Clinical Rheumatology, Clinical and Experimental Rheumatology, Annals of Rheumatic Diseases and Pediatric Rheumatology were the top five journals according to the number of FMF articles. The geographical distribution of FMF was seen to affect the article production of countries. Although Turkey dominated the field in the number of articles, it was not at the forefront in the average number of citations. To increase the production of articles in the field of FMF, it is necessary to support researchers in countries other than those which are already strongly active.

Questioning the Impact of the Impact Factor. A Brief Review and Future Directions

Purpose: To provide a brief review of literature on the journal impact factors (JIF) and the newer research metrics being proposed or implemented.Methods: The authors performed a PubMed search of articles published in the English language on the journal impact factors. Data captured include historical perspectives, evolution, calculation, criticisms of JIF and their rebuttals, and organized efforts to address JIF issues, alternate research metrics, and future directions. Specific emphasis was laid on evaluating the criticisms, current lacunae, and the changing practice patterns.Results: One of the measures to assess the research impact of an article is the number of citations it receives. Hence, citation-based metrics are commonly used for such purposes. While editors and well-known scholars refrain from attributing article success to the journal's prominence, the same is not true for most authors. JIF is still one of the top factors when deciding on an article submission. JIF is today an acceptable objective and quantifiable measure of knowledge dissemination. However, JIF should not be used as a surrogate measure to assess an individual researcher or an individual article. The reverence to JIF in this regard needs to be questioned. While alternate metrics or altmetrics have their advantages and limitations, they nevertheless augur well an era where scientometrics are complementary to one another without undue reliance on a sole parameter.Conclusion: While there is no need to demonize the JIF, its role in the scholarly assessment should be scaled down. The over-reliance and undue hype surrounding it should be discouraged at multiple scientific levels.

Research Trends on the Rotator Cuff Tendon: A Bibliometric Analysis of the Past 2 Decades

BACKGROUND

Clinical research on the rotator cuff tendon is increasing, and new approaches are being applied to rotator cuff disease. Considering the integration of research resources and research trends, it is necessary to conduct an analysis of recent research on the topic.

PURPOSE

To identity the research trends, influential journals, key researchers, and core countries of rotator cuff tendon research between 2000 and 2019.

STUDY DESIGN

Cross-sectional study.

METHODS

All the literature related to rotator cuff tendon research was retrieved from the Web of Science Core Collection on January 7, 2020. Qualitative and quantitative analyses were processed based on Web of Science and CiteSpace.

RESULTS

A total of 4131 studies, which included 3830 articles and 301 reviews, were obtained. There was an upward trend of studies on the topic, with small fluctuations in the past 2 decades. The United States had the most studies, and the number of studies from other countries increased over the study period. Most of the funding sources came from the United States. Articles in the Journal of Shoulder and Elbow Surgery had the most citations for rotator cuff research. Frontier topics, such as arthroscopic repair, mesenchymal stem cell, and "platelet-rich plasma, were identified. The number of citations in 2018 (r = 0.280; P = .005) and 2019 (r = 0.307; P = .002) had a weak positive correlation with publication date, indicating that the more recently published articles had a higher number of citations.

CONCLUSION

Valuable information on rotator cuff research based on bibliometric analysis was identified. Arthroscopic repair, mesenchymal stem cell, and platelet-rich plasma might be the research frontiers in this field, and researchers should focus on these topics in future studies.

Analysis of research trends in Korean dentistry journals by assigning MeSH to author keywords

Researchers seek to identify optimal journals for submission based on their studies but tend to rely on journal impact factors or scientific journal rankings. We investigated research trends by selecting high-frequency words from author keywords (AKs), analyzing subject areas, and performing quantitative data analysis of Korean dental journals. Consequently, we suggest a method for choosing journals that fit a specific subject area.We used a corpus of 9 Korean dentistry journals regarded in Korea as quality internationally approved journals. AKs occurring more than 10 times were assigned to Medical Subject Headings (MeSH) terms and subcategories, which were then categorized using the MeSH tree structure. KnowledgeMatrix Plus and VOSviewer were used to analyze network relationships, density, and clustering.The AKs were of 7527 types, 15,960 terms, and formed 54 clusters. The AKs with 10+ occurrence were 199 types, 4289 terms, and formed 9 clusters. Assigning the AKs with 10+ occurrence to MeSH terms led to expanding 732 types of AK terms into 249 types with 9 clusters and 4268 links. Core study areas over the past 10 years were facial asymmetry, a topic under oral surgery and medicine, and orthognathic surgery focused on mandibular fractures, followed by shear bond strength of zirconia. Analyzing 16 MeSH subject categories, we found that the "analytical, diagnostic and therapeutic techniques and equipment" category had the largest distribution of AKs (40.7%). This was followed by "diseases" (21.2%) and "anatomy" (14.90%). The orthognathic surgery cluster was the largest, followed by the shear bond strength cluster. Dental implants is a core area with strong links to high-occurrence words, such as cone-beam computed tomography and mandible, which were distributed in the order of The Journal of Advanced Prosthodontics (37.8%) and Journal of Periodontal & Implant Science (30.6%). Five clusters were closely packed in the center, 2 clusters were formed above the center, 1 cluster was formed below the center, and a cluster on the right was widespread.Cluster analysis using AKs and MeSH may be a good analytic method for researchers to determine expanding research areas and select optimal journals for paper submission.

SOCIAL MEDIA FOR MEDICAL JOURNALS

Social networks are now an integrated part of life in most digitally connected societies. Bringing scientific papers of interest to a defined audience using the appropriate channel might substantially contribute to the impact of a scientific discovery. Various media and metrics have come to the fore in strategizing dissemination of scientific information. This opinion piece offers insights from the social-media experience of digital editors of peer-reviewed journals from non-Anglophone countries.

Nutrition in times of Covid-19, how to trust the deluge of scientific information

PURPOSE OF REVIEW

The Covid-19 pandemic has daunted the world with its enormous impact on healthcare, economic recession, and psychological distress. Nutrition is an integral part of every person life care, and should also be mandatorily integrated to patient care under the Covid-19 pandemic. It is crucial to understand how the Covid-19 does develop and which risk factors are associated with negative outcomes and death. Therefore, it is of utmost importance to have studies that respect the basic tenets of the scientific method in order to be trusted. The goal of this review is to discuss the deluge of scientific data and how it might influence clinical reasoning and practice.

RECENT FINDINGS

A large number of scientific manuscripts are daily published worldwide, and the Covid-19 makes no exception. Up to now, data on Covid-19 have come from countries initially affected by the disease and mostly pertain either epidemiological observations or opinion papers. Many of them do not fulfil the essential principles characterizing the adequate scientific method.

SUMMARY

It is crucial to be able to critical appraise the scientific literature, in order to provide adequate nutrition therapy to patients, and in particular, to Covid-19 infected individuals.

Social media for research, education and practice in rheumatology

Online social networking offers numerous opportunities for continuing medical education, professional development, and scholarly collaboration. Available social media channels proved useful for expanding education and research perspectives, particularly in rapidly developing academic disciplines such as rheumatology. Although there are numerous advantages of social media, busy clinicians should be aware of some drawbacks related to misinformation, unethical promotion, and unprofessional behavior in globally expanding platforms. Filtering credible and expert-proven information by skilled users is, therefore, increasingly important. Enforcing ethical norms and advancing professional etiquette in the field is strongly advisable. This article overviews the advantages and shortcomings of social media and reflects on available platforms for education and research in rheumatology.

Getting the Word Out: New Approaches for Disseminating Public Health Science

The gap between discovery of public health knowledge and application in practice settings and policy development is due in part to ineffective dissemination. This article describes (1) lessons related to dissemination from related disciplines (eg, communication, agriculture, social marketing, political science), (2) current practices among researchers, (3) key audience characteristics, (4) available tools for dissemination, and (5) measures of impact. Dissemination efforts need to take into account the message, source, audience, and channel. Practitioners and policy makers can be more effectively reached via news media, social media, issue or policy briefs, one-on-one meetings, and workshops and seminars. Numerous “upstream” and “midstream” indicators of impact include changes in public perception or awareness, greater use of evidence-based interventions, and changes in policy. By employing ideas outlined in this article, scientific discoveries are more likely to be applied in public health agencies and policy-making bodies.

Journal impact factor: a bumpy ride in an open space

The journal impact factor (IF) is the leading method of scholarly assessment in today’s research world. An important question is whether or not this is still a constructive method. For a specific journal, the IF is the number of citations for publications over the previous 2 years divided by the number of total citable publications in these years (the citation window). Although this simplicity works to an advantage of this method, complications arise when answers to questions such as ‘What is included in the citation window’ or ‘What makes a good journal impact factor’ contain ambiguity. In this review, we discuss whether or not the IF should still be considered the gold standard of scholarly assessment in view of the many recent changes and the emergence of new publication models. We will outline its advantages and disadvantages. The advantages of the IF include promoting the author meanwhile giving the readers a visualization of the magnitude of review. On the other hand, its disadvantages include reflecting the journal’s quality more than the author’s work, the fact that it cannot be compared across different research disciplines, and the struggles it faces in the world of open access. Recently, alternatives to the IF have been emerging, such as the SCImago Journal & Country Rank, the Source Normalized Impact per Paper and the Eigenfactor Score, among others. However, all alternatives proposed thus far are associated with their own limitations as well. In conclusion, although IF contains its cons, until there are better proposed alternative methods, IF remains one of the most effective methods for assessing scholarly activity.

Comprehensive Researcher Achievement Model (CRAM): a framework for measuring researcher achievement, impact and influence derived from a systematic literature review of metrics and models

OBJECTIVES

Effective researcher assessment is key to decisions about funding allocations, promotion and tenure. We aimed to identify what is known about methods for assessing researcher achievements, leading to a new composite assessment model.

DESIGN

We systematically reviewed the literature via the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols framework.

DATA SOURCES

All Web of Science databases (including Core Collection, MEDLINE and BIOSIS Citation Index) to the end of 2017. ELIGIBILITY CRITERIA: (1) English language, (2) published in the last 10 years (2007-2017), (3) full text was available and (4) the article discussed an approach to the assessment of an individual researcher's achievements.

DATA EXTRACTION AND SYNTHESIS

Articles were allocated among four pairs of reviewers for screening, with each pair randomly assigned 5% of their allocation to review concurrently against inclusion criteria. Inter-rater reliability was assessed using Cohen's Kappa (ĸ). The ĸ statistic showed agreement ranging from moderate to almost perfect (0.4848-0.9039). Following screening, selected articles underwent full-text review and bias was assessed.

RESULTS

Four hundred and seventy-eight articles were included in the final review. Established approaches developed prior to our inclusion period (eg, citations and outputs, h-index and journal impact factor) remained dominant in the literature and in practice. New bibliometric methods and models emerged in the last 10 years including: measures based on PageRank algorithms or 'altmetric' data, methods to apply peer judgement and techniques to assign values to publication quantity and quality. Each assessment method tended to prioritise certain aspects of achievement over others.

CONCLUSIONS

All metrics and models focus on an element or elements at the expense of others. A new composite design, the Comprehensive Researcher Achievement Model (CRAM), is presented, which supersedes past anachronistic models. The CRAM is modifiable to a range of applications.

Improving Scientific Writing Skills and Publishing Capacity by Developing University-Based Editing System and Writing Programs

Scholarly article writing and publishing in international peer-reviewed journals can become an overwhelming task for many medical, nursing, and healthcare professionals in a university setting, especially in countries whose native language is not English. To help improve their scientific writing skills and publishing capacity, a university-based editing system and writing programs can be developed as educational platforms. These are delivered by a team of specialist editors composed of tenured faculty members who have a strong medical background and extensive experience in teaching courses on medical research, editing, writing, and publishing. For the editing system, the specialist editors provide comprehensive editing, personalized consultation, full editorial support after peer review, guidance with online submissions/resubmissions, and detailed editorial review at different stages of the manuscript writing. In addition, the specialist editors can develop writing programs such as medical writing and editing internships, academic courses in medical writing or research study designs and reporting standards, special interactive lectures and sessions on predatory publishing, seminars on updated editorial guidance of global editorial associations, academic visits on medical writing and editing, medical writing mentoring program, networking programs in scholarly communication, and publication resources in medical writing and scholarly publishing. These editing system and writing programs can serve as integrated platforms for improving scientific writing skills and publishing capacity by providing continuing education in medical writing, editing, publishing, and publication ethics.

Bibliometric Indices of Scientific Journals: Time to overcome the obsession and think beyond the Impact Factor

Journal Impact Factor (JIF) has been widely used for a long time as a proxy marker of journal prestige. However, off late, accuracy and reliability of JIF for evaluation of scientific journals has been increasingly questioned by numerous stakeholders in the field of scholarly publications. Having realized the perils and pitfalls of JIF, there is an increasing understanding among academia to develop and consider alternatives to the traditional JIF. It is possibly time for all concerned to understand the pros and cons of JIF to overcome this obsession. Limitations of JIF and possible alternative and emerging bibliometric indicators are being discussed in this article for the benefit of the readers of MJAFI.