Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas

Application of graph theory in the library domain—Building a faceted framework based on a literature review

Based on a literature review, we present a framework for structuring the application of graph theory in the library domain. Our goal is to provide both researchers and libraries with a standard tool to classify scientific work, at the same time allowing for the identification of previously underrepresented areas where future research might be productive. To achieve this, we compile graph theoretical approaches from the literature to consolidate the components of our framework on a solid basis. The extendable framework consists of multiple facets grouped into five categories whose elements can be arbitrarily combined. Libraries can benefit from these facets by using them as a point of reference for the (meta)data they offer. Further work on formally defining the framework’s categories as well as on integration of other graph-related research areas not discussed in this article (e.g. knowledge graphs) would be desirable and helpful in the future.

POSITIVE RESULTS BIAS AND IMPACT FACTOR IN RETINA CLINICAL TRIALS 2016 to 2019

PURPOSE

To assess for a positive results bias in recently published randomized controlled trials in the field of vitreoretinal disease.

METHODS

A bibliometric analysis was conducted examining randomized controlled trials published in the field of retina between January 1, 2016, and December 31, 2019. Studies were classified as positive result or negative result based on the statistical significance of their primary outcome. Publication date and sample size were documented. These variables were compared against Journal Citation Reports Impact Factor in the year of publication.

RESULTS

Two hundred and eighty-eight randomized controlled trials from 64 unique journals were included and analyzed. One hundred and eighty-five (64.2%) studies were classified as positive result, and 103 (35.8%) studies were classified as negative result. There was no association between impact factor and positive result. Studies classified as positive result had larger sample sizes, and higher sample size was associated with higher impact factor.

CONCLUSION

These results do not support the presence of a recent positive results bias in retina randomized controlled trials. This is reassuring, although several factors could be contributing to this finding including studies that were conducted but never submitted and selective reporting of outcomes. Thus, it will be important to remain cognizant of potential publication biases moving forward.

Relationship between an editor in chief's commentary publications and the impact factor of cardiovascular medicine journals

Journal impact factor (IF) inflation is suggested as a problem resulting from commentaries published by the editors in chief (EiCs) of their respective journals. However, it is unclear whether this is a systemic problem across the top thirty cardiovascular medicine journals. Therefore, the purpose of this investigation was to examine the relationship between the number of commentaries written by an EiC and their journal's IF and Eigenfactor (Ef). Utilizing Spearman rank partial correlations controlling for length of service as the EiC, significant moderate correlations were found between the number of commentaries and the number of first-author commentaries by the EiC and the IF of their journal (r=0.568, p=0.001 and r=0.504, p=0.005; respectively). A weak but still significant correlation was found between the number of commentaries by the EiC and the Ef of their journal (r=0.431, p=0.020). The reason for these correlations is unclear, and whether the methodology used to compute the IF and Ef should be modified needs further research.

Publication Trends of Nonobstetric Brachial Plexus Injury Research: A Bibliometric Analysis

OBJECTIVE

To evaluate publication trends in nonobstetric brachial plexus injury research.

METHODS

In September 2020, Scopus was searched for articles on nonobstretric brachial plexus injury. Citation count, year of publication, country of corresponding author and its income category, destiny journal and its 5-year impact factor (IF), and research type were retrieved.

RESULTS

The analysis comprised 1245 articles. Mean number of citations per article was 18.01 (95% confidence interval 16.46-19.55). Mean IF was 3.60 (95% confidence interval 3.25-3.95). The 5 most prolific journals had an IF <5. The journal with the highest number of articles was the Journal of Hand Surgery (American Volume) (n = 70, 5.6%). The most prolific country was the United States (n = 313, 25.1%). There were 913 articles (73.3%) from high-income countries, 246 (19.8%) from upper middle-income countries, and 68 (5.5%) from lower middle-income countries. No articles were from low-income countries. The representation of middle-income countries increased from 2.1% of published articles in 1980-1989 to 40.0% in 2010-2019. Primary research represented 64.0% (n = 797) of articles, while secondary research and case reports represented 13.0% (n = 162) and 23.0% (n = 286) of articles, respectively. Narrative reviews (n = 142, 11.4%;) and systematic reviews (n = 20, 1.6%) comprised articles from the secondary research group.

CONCLUSIONS

While high-income countries still represent the majority of publications, the contribution of researchers from middle-income countries is increasing. The most common destiny journals are field specific, with a relatively low IF. Although most articles are primary research, a representative portion have a shallow level of evidence (case reports and narrative reviews).

The Bibliometric Literature on Scopus and WoS: The Medicine and Environmental Sciences Categories as Case of Study

In a broad sense, science can be understood as the knowledge contained in scientific manuscripts published in scientific journals. Scientific databases index only those journals that reach certain quality standards. Therefore, research and dissemination of scientific knowledge are essential activities for the growth of science itself. The aim of this manuscript is to assess the situation of medicine and environmental sciences among the bibliometric literature and to put it in perspective with the overall bibliometric publications in all scientific fields. The main countries publishing bibliometric manuscripts are China, USA and Spain. The latter country is ranked three out of the top five institutions according to the Scopus and WoS databases. In both databases, the average scientific collaboration of the top 20 institutions offers the same result, 41%. According to Scopus, the main subject categories in which this research falls are social sciences (38%), computer science (26%) and medicine (23%), while the environmental sciences category has 8%. In the analysis of the Medicine category alone, it has been observed that 136 countries have contributions in this field. The main countries are the United States, China and the United Kingdom. In the field of medicine, the main areas studied were: Epidemiology, Pediatrics, Orthopedics, Cardiology, Neurosurgery, Radiology, Ophthalmology, Oncology, Plastic Surgery and Psychiatry. With respect to environmental sciences, less international dissemination has been found, with only 83 countries having worked in this field. The main ones are China, Spain and the United States. Regarding the top 10 institutions, it can be stated that only Spain and China are relevant. Spain focuses on sustainability and China on the environment. The result of an independent keyword analysis of all published bibliometric manuscripts has shown that the main clusters are: Mapping Science (29%), Research Productivity (23%), Medicine (20%), Environmental Sciences (12%), Psychology (7%), Nursing (6%) and Engineering (4%). In short, medicine and environmental sciences are the most relevant areas in the field of bibliometrics after social sciences and computer sciences.

Russian University Teachers’ Perceptions of Scientometrics

This article is devoted to the attitude of Russian university teachers toward scientometrics and its indicators, which have been imposed on them by university administrations and the state since 2012. In addition to substantiating the problem’s urgency, the article contains a brief critical outline of the main scientometric parameters and their application in practice in Russia. To evaluate this, 283 people from leading universities in Russia (included in Program 5-100) were questioned. As the study showed, faculties of Russian universities understand the specifics of scientometrics, relate to it relatively positively, and over the past years have been able to adapt to the new requirements of the administration regarding implementing scientometric tasks and standards. The higher the position and scholarly qualification of a respondent, the more complete the knowledge about scientometrics. Scholars in the humanities know and relate comparatively better to scientometrics than representatives of technical and general scientific specialties.

Journal Metrics in Psychiatry: What do the rankings tell us?

BACKGROUND

Metrics of journal's impact factor may suggest the journal's influence in a particular field, but they have been used inadvertently as a measure of the journal and individual publications' scientific quality.

METHODS

We assessed how scientific journals in the field of psychiatry and mental health are ranked (top 20) according to the scores of distinct metrics (Eigenfactor score, Google Scholar Metrics, Journal Citation Reports, Scimago Journal & Country Rank, and Source Normalized Impact per Paper), described their main characteristics and perfomed a spearman's correlation analyses to investigate to which extent these metrics are associated. We also discussed the limitations of dealing with these metrics and the rankings they provide as a proxy of the journal's quality.

RESULTS

Only 5 (12.5%) journals appear in all metrics (JAMA Psychiatry, American Journal of Psychiatry, Molecular Psychiatry, Schizophrenia Bulletin, and the Journal of Child Psychology and Psychiatry), more than one-third of the journals show up in only one and less than half (42.5%) appear in three or more. Only JAMA Psychiatry is in one of the first five positions of all metrics. No journal ranked in the same position across the metrics. On the other hand, we found the correlations between all the metrics were statistically significant.

LIMITATIONS

The metrics included are not exhaustive.

CONCLUSIONS

Although each metric provides a particular ranking, they are highly correlated. Rankings also change according to distinct subject categories in which they are assessed. We suggest less emphasis should be given to Journal Metrics to infer journal's quality.

Highly Cited Papers at the Spanish Domestic Level

This paper presents a methodological proposal based on the identification of highly cited papers (HCPs) at domestic-level in the Spanish Public University System (SUPE), in order to find the most outstanding publications in the local context. The principal aim is to detect different activity and impact profiles among Spanish universities and differentiate those institutions that play a more significant role. To determine which and how many are the highly cited papers at the domestic level (HCP-DL) collected in the Web of Science, three citation thresholds (1, 5, and 10%) were established. Thematic classification in Incites/Essential Science Indicators areas is used. The results show a preponderance of HCPs in the field of Space Science, while the polytechnic universities have high visibility in the Computer Science area. It has been observed that the presence of HCPs in a given area is involved with universities specialized in teaching and research activities. In absolute terms, the big non-specialized universities are major producers of HCPs and hold the leading positions in our results. However, when efficiency is analyzed in relative terms, some small, specialized universities reveal themselves to be more efficient at producing HCPs (% of HCPs or citations per HCP). We think that this methodology, due to its simplicity, its ease of calculation, and the knowledge it provides, can be very useful to analyze the national systems of any country, in order to know the impact and visibility of the research carried out in its scientific institutions or research areas.

Questioning the Impact of the Impact Factor. A Brief Review and Future Directions

Purpose: To provide a brief review of literature on the journal impact factors (JIF) and the newer research metrics being proposed or implemented.Methods: The authors performed a PubMed search of articles published in the English language on the journal impact factors. Data captured include historical perspectives, evolution, calculation, criticisms of JIF and their rebuttals, and organized efforts to address JIF issues, alternate research metrics, and future directions. Specific emphasis was laid on evaluating the criticisms, current lacunae, and the changing practice patterns.Results: One of the measures to assess the research impact of an article is the number of citations it receives. Hence, citation-based metrics are commonly used for such purposes. While editors and well-known scholars refrain from attributing article success to the journal's prominence, the same is not true for most authors. JIF is still one of the top factors when deciding on an article submission. JIF is today an acceptable objective and quantifiable measure of knowledge dissemination. However, JIF should not be used as a surrogate measure to assess an individual researcher or an individual article. The reverence to JIF in this regard needs to be questioned. While alternate metrics or altmetrics have their advantages and limitations, they nevertheless augur well an era where scientometrics are complementary to one another without undue reliance on a sole parameter.Conclusion: While there is no need to demonize the JIF, its role in the scholarly assessment should be scaled down. The over-reliance and undue hype surrounding it should be discouraged at multiple scientific levels.

The 100 most cited articles on lung cancer screening: a bibliometric analysis

Background

The number of citations of an article reflects its impact on the scientific community. The aim of this study was to identify and characterize the 100 most cited articles on lung cancer screening.

Methods

The 100 most cited articles on lung cancer screening published in all scientific journals were identified using the Web of Science database. Relevant data, including the number of citations, publication year, publishing journal and impact factor (IF), authorship and country of origin, article type and study design, screening modality, and main topic, were collected and analyzed.

Results

The 100 most cited articles were all English and published between 1973 and 2017, with 81 published after 2000. The mean number of citations was 292.90 (range 100–3,910). Sixty articles originated from the United States. These articles were published in 32 journals; there was a statistically significant positive correlation between journal IF and the number of citations (r=0.238, P=0.018). Seventy-nine articles were original research of which 37.9% were about results from randomized controlled trials (RCTs). The most common screening modalities in these articles were low-dose computed tomography (LDCT) (n=78), followed by chest X-ray radiography (CXR) and sputum cytology (n=11). The most common topic in these articles was screening test effectiveness.

Conclusions

Our study presents a detailed list and analysis of the 100 most cited articles published about lung cancer screening which provides insight into the historical developments and key contributions in this field.

The Scientific Impact Derived From the Disciplinary Profiles

The disciplinary profiles of the mean citation rates across 22 research areas were analyzed for 107 countries/territories that published at least 3,000 papers that exceeded the entrance thresholds for the Essential Science Indicators (ESI; Clarivate Analytics) during the period from January 1, 2009 to December 31, 2019. The matrix of pairwise differences between any two profiles was analyzed with a non-metric multidimensional scaling (MDS) algorithm, which recovered a two-dimensional geometric space describing these differences. These two dimensions, Dim1 and Dim2, described 5,671 pairwise differences between countries' disciplinary profiles with a sufficient accuracy (stress = 0.098). A significant correlation (r = 0.81, N = 107, p < 0.0001) was found between Dim1 and the Indicator of a Nation's Scientific Impact (INSI), which was computed as a composite of the average and the top citation rates. The scientific impact ranking of countries derived from the pairwise differences between disciplinary profiles seems to be more accurate and realistic compared with more traditional citation indices.

Legaltech and Lawtech: Global Perspectives, Challenges, and Opportunities

Legaltech refers to the application of new technologies to the world of law, to carry out tasks that, until recently, were performed by lawyers or other personnel working in law firms. From 2015 onwards the Lawtech alternative has emerged. In this work, the concepts of Legaltech and Lawtech have been analyzed by searching the two main scientific information databases such as Scopus and Wed of Science (WoS). There has been a clear trend to use the concept of Legaltech against Lawtech. Six clear research lines have been detected from the whole of the published documents regarding these concepts. These are the related to Computer Science, Justice, Legal profession, Legal design, Law firms, and Legal Education. It is proposed to use the term Legaltech to include all technological advances in the legal field. From the point of view of opportunities, the irruption of Legaltech will be able to offer accurate legal advice to the public, reducing the price of this and on the other hand, analyze large amounts of data that law firms and legal advisors will use to improve their management and increase their productivity. In short, Legaltech and Lawtech are opening up new opportunities in the legal sector encouraging technological innovation, giving greater access to legal services, even try to achieve the goal of universal access to justice.

A comprehensive analysis of the journal evaluation system in China

Journal evaluation systems reflect how new insights are critically reviewed and published, and the prestige and impact of a discipline’s journals is a key metric in many research assessment, performance evaluation, and funding systems. With the expansion of China’s research and innovation systems and its rise as a major contributor to global innovation, journal evaluation has become an especially important issue. In this paper, we first describe the history and background of journal evaluation in China and then systematically introduce and compare the most currently influential journal lists and indexing services. These are the Chinese Science Citation Database (CSCD), the Journal Partition Table (JPT), the AMI Comprehensive Evaluation Report (AMI), the Chinese S&T Journal Citation Report (CJCR), “A Guide to the Core Journals of China” (GCJC), the Chinese Social Sciences Citation Index (CSSCI), and the World Academic Journal Clout Index (WAJCI). Some other influential lists produced by government agencies, professional associations, and universities are also briefly introduced. Through the lens of these systems, we provide comprehensive coverage of the tradition and landscape of the journal evaluation system in China and the methods and practices of journal evaluation in China with some comparisons to how other countries assess and rank journals.

Mapping the Research Landscape of Nature-Based Solutions in Urbanism

Nature-based solutions (NBS) are an innovative concept that mimics the processes of natural ecosystems, popularized principally in the European Union. With a substantial body of literature amassed since the term’s inception in 2015, there is a need to systematically review existing literature to identify overarching gaps and trends, according to disciplinary focus, geographic scope, and key themes, and direct future research inquiry and policy recommendations. This review consists of bibliometric analysis and thematic analysis for NBS studies in urbanism. NBS studies were found to relate strongly with other concepts of ‘Ecosystem Services’, ‘Green Infrastructure’, ‘Climate Change’, and ‘Risk management and Resilience’, which align with four major thematic goals set by the European Commission. Within NBS scholarship, various sub-themes have emerged, namely, ‘Greening’, ‘Urban Development’, ‘Water’, ‘Wellbeing’, and ‘Governance’. Furthermore, we illustrate that the amount and thematic focus of NBS research have been unevenly distributed worldwide. Analysis of emerging trends shows a recent increase in topics, such as adaptive governance of NBS, and the incorporation of social justice in sustainability transitions. Based on an assessment of extant NBS literature, we offer some recommendations for the future direction of the research fields.

Urban Food Systems: A Bibliometric Review from 1991 to 2020

The increase of urbanization is affecting the urban food system (UFS) in many areas, primarily production, processing, and consumption. The upgrading of the urban food consumption structure not only puts forward higher food production requirements, but also poses a challenge to resource consumption and technological innovation. Considerable case or review studies have been conducted on UFS, but there is no bibliometric review attempting to provide an objective and comprehensive analysis of the existing articles. In this study, we selected 5360 research publications from the core Web of Science collection from 1991 to 2020, analyzing contributions of countries, institutions, and journals. In addition, based on keyword co-occurrence and clustering analyses, we evaluated the research hotspots of UFS. The results show that global research interest in UFS has increased significantly during these three decades. The USA, China, and the UK are the countries with the highest output and closest collaborations. UFS research involves multiple subject categories, with environmental disciplines becoming mainstream. Food security, food consumption, and food waste are the three main research areas. We suggest that food sustainability and resilience, food innovation, and comparative studies between cities should be given more attention in the future.

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB.

Nature of Science (NOS) Being Acquainted with Science of Science (SoS): Providing a Panoramic Picture of Sciences to Embody NOS for Pre-Service Teachers

Understanding about nature of science is important topic in science education as well as in pre-service science teacher education. In science education, Nature of Science (NOS), in its different forms of educational scaffoldings, seeks to provide with students an understanding of features of scientific knowledge and science in general, how scientific knowledge changes and becomes accepted, and what factors guide scientific activities. For a science teacher, deep and broad enough picture of sciences is therefore of importance. This study attempts to show that the research field called Science of Science (SoS) can significantly support building such a panoramic picture of sciences, and through that, significantly support NOS. The SoS approaches the structure and dynamics of science quantitatively, using scientific documents (e.g., publications, reports, books and monographs and patent applications) as trails to map the landscape of sciences. It is argued here that SoS may provide material and interesting cases for NOS, and in so doing enrich NOS in a similarly significant way as history, philosophy and sociology of science (HPSS) scholarship has done thus far. This study introduces several themes based on SoS that are of relevance for NOS as they were introduced and discussed in a pre-service science teachers’ course. The feedback from pre-service teachers shows that introducing SoS, with minimal additional philosophical interpretations and discussions, but simply as evidential facts and findings, sparks ideas and views that come very close to NOS themes and topics. Discussions related to nature of science, and specific educational NOS scaffoldings for it, can find a good companion in SoS; the latter providing facts and evidence of thee structure and dynamics of sciences, the former providing perspectives for interpretations.

Responsible Science Assessment: downplaying indexes, boosting quality

Scientists are facing enormous pressures posed by growing scientific communities and stagnant/reduced funding. In this scenario, mechanisms of knowledge achievement and management, as well as how recruitment, progression and evaluation are carried out should be reevaluated. We argue here that knowledge has become a profitable commodity and, as a consequence, excessive academic quantification, individual output assessment problems and abusive editorial market strategies have reached unsustainable levels. We propose to reinforce existing guidelines and to establish new ones to overcome these issues. Our proposal, the Initiative for Responsible Scientific Assessment (IRSA), has the main goal to strengthen and expand previous movements in the scientific community to promote higher quality research assessment, focused on better Science.

Bibliometric indicators to evaluate scientific activity

Bibliometric indicators have been devised to quantify scientific production and to try to evaluate its impact in the community. In general, bibliometric indicators can be classified according to whether the unit of analysis is the author (individual or group) or journal. The most widely used indicators for authors are those that measure an individual author's production, such as the Crown index or the h-index and its derivatives (e-index, h5-index, and the absolute or Ab-index, among others). The bibliometric indicators devised to try to evaluate journal quality are associated with Journal Citation Reports (e.g., impact factor, field-weighted citation impact, Eigenfactor, and article influence) or with Scopus (Scimago Journal Rank (SJR), source normalized impact per paper (SNIP), and CiteScore). This article describes the main bibliometric indicators, explains how they are calculated, and discusses their advantages and limitations.

Interpreting Bibliometric Data

Many academic analyses of good practice in the use of bibliometric data address only technical aspects and fail to account for and appreciate user requirements, expectations, and actual practice. Bibliometric indicators are rarely the only evidence put before any user group. In the present state of knowledge, it is more important to consider how quantitative evaluation can be made simple, transparent, and readily understood than it is to focus unduly on precision, accuracy, or scholarly notions of purity. We discuss how the interpretation of 'performance' from a presentation using accurate but summary bibliometrics can change when iterative deconstruction and visualization of the same dataset is applied. From the perspective of a research manager with limited resources, investment decisions can easily go awry at governmental, funding program, and institutional levels. By exploring select real-life data samples we also show how the specific composition of each dataset can influence interpretive outcomes.

Important citations identification by exploiting generative model into discriminative model

Although the citations between scientific documents are deemed as a vehicle for dissemination, inheritance and development of scientific knowledge, not all citations are well-positioned to be equal. A plethora of taxonomies and machine-learning models have been implemented to tackle the task of citation function and importance classification from qualitative aspect. Inspired by the success of kernel functions from resulting general models to promote the performance of the support vector machine (SVM) model, this work exploits the potential of combining generative and discriminative models for the task of citation importance classification. In more detail, generative features are generated from a topic model, citation influence model (CIM) and then fed to two discriminative traditional machine-learning models, SVM and RF (random forest), and a deep learning model, convolutional neural network (CNN), with other 13 traditional features to identify important citations. The extensive experiments are performed on two data sets with different characteristics. These three models perform better on the data set from one discipline. It is very possible that the patterns for important citations may vary by the fields, which disable machine-learning models to learn effectively the discriminative patterns from publications from multiple domains. The RF classifier outperforms the SVM classifier, which accords with many prior studies. However, the CNN model does not achieve the desired performance due to small-scaled data set. Furthermore, our CIM model–based features improve further the performance for identifying important citations.

A Study of eHealth from the Perspective of Social Sciences

The field of social sciences has become increasingly important in eHealth. Patients currently engage more proactively with health services. This means that eHealth is linked to many different areas of Social Sciences. The main purpose of this research is to analyze the state-of-the-art research on eHealth from the perspective of social sciences. To this end, a bibliometric analysis was conducted using the Web of Science database. The main findings show the evolution of publications, the most influential countries, the most relevant journals and papers, and the importance of the different areas of knowledge. Although there are some studies on eHealth within social sciences, most of them focus on very specific aspects and do not develop a holistic analysis. Thus, this paper contributes to academia by analyzing the state-of-the-art of research, as well as identifying the most relevant trends and proposing future lines of research such as the potential of eHealth as a professional training instrument, development of predictive models in eHealth, analysis of the eHealth technology acceptance model (TAM), efficient integration of eHealth within public systems, efficient budget management, or improvement in the quality of service for patients.

A Glimpse of the First Eight Months of the COVID-19 Literature on Microsoft Academic Graph: Themes, Citation Contexts, and Uncertainties

As scientists worldwide search for answers to the overwhelmingly unknown behind the deadly pandemic, the literature concerning COVID-19 has been growing exponentially. Keeping abreast of the body of literature at such a rapidly advancing pace poses significant challenges not only to active researchers but also to society as a whole. Although numerous data resources have been made openly available, the analytic and synthetic process that is essential in effectively navigating through the vast amount of information with heightened levels of uncertainty remains a significant bottleneck. We introduce a generic method that facilitates the data collection and sense-making process when dealing with a rapidly growing landscape of a research domain such as COVID-19 at multiple levels of granularity. The method integrates the analysis of structural and temporal patterns in scholarly publications with the delineation of thematic concentrations and the types of uncertainties that may offer additional insights into the complexity of the unknown. We demonstrate the application of the method in a study of the COVID-19 literature.

Citation inequality and the Journal Impact Factor: median, mean, (does it) matter?

Skewed citation distribution is a major limitation of the Journal Impact Factor (JIF) representing an outlier-sensitive mean citation value per journal The present study focuses primarily on this phenomenon in the medical literature by investigating a total of n = 982 journals from two medical categories of the Journal Citation Report (JCR). In addition, the three highest-ranking journals from each JCR category were included in order to extend the analyses to non-medical journals. For the journals in these cohorts, the citation data (2018) of articles published in 2016 and 2017 classified as citable items (CI) were analysed using various descriptive approaches including e.g. the skewness, the Gini coefficient, and, the percentage of CI contributing 50% or 90% of the journal’s citations. All of these measures clearly indicated an unequal, skewed distribution with highly-cited articles as outliers. The %CI contributing 50% or 90% of the journal’s citations was in agreement with previously published studies with median values of 13–18% CI or 44–60% CI generating 50 or 90% of the journal’s citations, respectively. Replacing the mean citation values (corresponding to the JIF) with the median to represent the central tendency of the citation distributions resulted in markedly lower numerical values ranging from − 30 to − 50%. Up to 39% of journals showed a median citation number of zero in one medical journal category. For the two medical cohorts, median-based journal ranking was similar to mean-based ranking although the number of possible rank positions was reduced to 13. Correlation of mean citations with the measures of citation inequality indicated that the unequal distribution of citations per journal is more prominent and, thus, relevant for journals with lower citation rates. By using various indicators in parallel and the hitherto probably largest journal sample, the present study provides comprehensive up-to-date results on the prevalence, extent and consequences of citation inequality across medical and all-category journals listed in the JCR.

The steering effects of citations and metrics

Purpose

This paper aims to understand the nature of citations and metrics in the larger system of knowledge production involving universities, funding agencies, publishers, and indexing and data analytic services.

Design/methodology/approach

First, the normative and social constructivist views of citations are reviewed to be understood as co-existing conditions. Second, metrics are examined through the processes of commensuration by tracing the meanings of metrics embedded in various kinds of documents and contexts. Third, the steering effects of citations and metrics on knowledge production are discussed. Finally, the conclusion addresses questions pertaining to the validity and legitimacy of citations as data and their implications for knowledge production and the conception of information.

Findings

The normative view of citations is understood as an ideal speech situation; the social constructivist view of citation is recognised in the system of knowledge production where citing motivations are influenced by epistemic, social and political factors. When organisational performances are prioritised and generate system imperatives, motives of competition become dominant in shaping citing behaviour, which can deviate from the norms and values in the academic lifeworld. As a result, citations and metrics become a non-linguistic steering medium rather than evidence of research quality and impact.

Originality/value

This paper contributes to the understanding of the nature of citations and metrics and their implications for the conception of information and knowledge production.

Uncertainty and the ranking of economics journals

Journal rankings often show significant changes compared to previous rankings. This gives rise to the question of how well estimated the rank of a journal is. In this contribution, we consider uncertainty in a ranking of economics journals. We use the invariant method of Pinski and Narin to rank the journals. We propose an uncertainty measure, which is based on a bootstrap approach. The measure is the average absolute change in rank, which we see as a reasonable uncertainty measure regarding rankings. We further calculate, based on the bootstrap method, 95% confidence interval for the observed values of the invariant method. We show that ranks of the highest, as well as the lowest, ranked journals are well estimated, while there is a high degree of uncertainty regarding the rank of many mid-ranked journals. The distribution of the underlying measure is useful for identifying groups of journals that are more or less of the same quality (from the point of view of the invariant measure). The journal with the highest observed value of the invariant measure, Journal of Political Economy, has the best performance and constitutes a singleton, whereas Quarterly Journal of Economics and Econometrica form the next group (there is a slight overlap between the two with respect to confidence intervals). The journals ranked between about 190–230 form another group in which there are no major quality differences between the journals, as the confidence intervals are overlapping.

Scientific Impact and Clinical Influence: Identifying Landmark Studies in Burns

Although many reviews describe significant advances in burn care, no studies have yet examined why these papers had such profound impact. Our objective was to identify the most highly cited, as well as the most clinically influential studies in burns, and describe their characteristics, to inform future research in the field. Web of Science was searched using keywords related to burns to identify the 100 most-cited burns papers. Study design, year and journal of publication, and subject of the paper were recorded. A mixed-methods approach was used to identify papers in burn research leading to change in clinical practice. Characteristics of these papers were compared with identify any factors predictive of future citations or clinical influence. The 100 highly cited papers were cited between 159 and 907 times. There was no correlation between total citations and journal impact factor, year of publication, or subject area. Level of evidence did not predict future citations or influence, but may be influenced by evolving research standards. Of 23 clinically influential studies, 6 were not among 100 most-cited. Using papers only from the 100 most-cited list was not sufficient to identify leading researchers in burns. Citation analysis is a beneficial, however not alone sufficient to identify landmark papers, particularly for multidisciplinary fields such as burns.

Output regularities of China's international collaboration research projects funded by NSFC

Purpose

The purpose of this paper is to provide theoretical basis and data support for researchers to choose appropriate international partners, provide a basis for Chinese research funding agencies, such as National Natural Science Foundation of China (NSFC) to formulate international research collaboration (IRC) strategies and policies and provide recommendations for the improvement of the internationalization level of China's basic scientific research.

Design/methodology/approach

Based on existing research, this study took output of “Major International (Regional) Joint Research Project” (MIJRP) funded by NSFC and participated by Chinese scholars in the meantime as the analysis object, proposed hypotheses and constructed the indicators of IRC and research output (RO). In addition, the mathematical statistics was used to compare the RO of China's IRC and nonIRC, and the statistical analysis model was used to measure the influence on RO of collaboration country's research capacity, research collaboration between China and US, scope of international research collaboration and reprint author country.

Findings

The RO of China's IRC is higher than that of nonIRC; research capacity of collaboration country has no inevitable effect on the RO of China's IRC; the RO of China's IRC participated by Americans is higher than that without American scholars; expanding the scope of China's IRC to some degree can increase RO; the RO of China's IRC led by foreigners is higher than that led by Chinese. In particular, China–US IRC and foreign scholars acting as the reprint author are two major factors for the RO of China's IRC.

Originality/value

Most of the traditional research on IRC are based on the co-author papers, and this study tried to analyze the characteristics and regularities on IRC from a new view of international collaboration projects, which can be a supplement to the traditional international collaboration research on co-author papers.