Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas

Obtaining interactions among science, technology, and research policy for developing an innovation strategy: A case study of supercapacitors

Comprehensive observations of science, technology, and research policy transactions are important for developing an innovation strategy. We propose a new method that combines the academic landscape and matrix analysis to understand the relationships among activities of three aspects of the technological landscape: science, technology, and research policy. First, we divided academic research into 28 knowledge domains by clustering a citation network of scientific papers. Next, we developed a new matrix classifying them into three groups: “mature technology,” “intermediate technology,” and “emerging technology.” The results showed that research domains in “emerging technology” showed a high rate of patent increase, indicating that they were commercializing rapidly. Finally, we identified the group that each country focused on, and this result reflected the countries' research policies. China and Singapore showed high rates, whereas Japan, France, and Germany had low values. This result reflects countries’ research policies and implies that specialty research areas differed by country. As above, our research result implies that academia, industry, and government have paid attention to knowledge domains in “emerging technology” and these are important for creating innovation. A supercapacitor, also known as an electric double layer capacitor or ultracapacitor, was selected as an example in our method. This research could help academic researchers, industrial companies, and policymakers in developing innovation strategies.

Global trends and prospects of blue carbon sinks: a bibliometric analysis

Blue carbon sinks (mangroves, saltmarshes, and seagrasses) are considered an effective nature-based approach for climate change mitigation. Despite growing interest, a systematic review of this topic is still scarce. This study evaluated 1348 blue carbon sink-related articles from 1990 to 2020 using bibliometric technology. The results from total of 85 countries, 1538 institutions, and 4492 authors indicated that blue carbon sink research shows the characteristics of rapid growth. The most active country, institution, and author were USA, Chinese Academy of Sciences, and Duarte C.M., respectively. Relatively close academic collaboration has formed in blue carbon science. Environmental Sciences was the most popular category with 590 papers. The percentages of articles related to mangroves, saltmarshes, and seagrasses were 63.87%, 40.36%, and 40.65%, respectively. Mangrove carbon sinks are the most popular topic, and stable isotope and remote sensing are the most researched technologies for mapping and quantifying blue carbon sinks. The threats to blue carbon sinks are complex and distinctive. Restoration, conservation, and management of blue carbon ecosystems aimed to improve their carbon sink capacity are becoming hot issues and should be further investigated in the future. These findings provide a scientific roadmap for further research in this field and will enable stakeholders to identify the research trend.

The Author Impact Factor as a Metric to Evaluate the Impact of Neurosurgical Researchers

OBJECTIVE

The objective of this study was to assess the Author Impact Factor (AIF) as a useful metric and as a complement to the h-index among neurosurgical researchers.

METHODS

The 5-year AIF and h-index were compared among 3 groups of researchers: 1) the 100 most prolific of all time within general neurosurgical journals ("Experienced"), 2) the 100 most prolific during the 2015-2019 period within general neurosurgical journals ("Trending Group"), and 3) the 100 postgraduation year 7 neurosurgical residents with the highest h-index ("Amateur").

RESULTS

The Amateur group had a lower median h-index than the Experienced (6 vs. 55; P < 0.001) and Trending (6 vs. 43; P < 0.001) groups. The highest h-index of the Amateur group (24) was lower than the first quartile of the Experienced (46.25) and Trending (26.00) groups. The Amateur group had a lower median 5-year AIF than the Experienced (2.15 vs. 3.17; P < 0.001) and Trending (2.15 vs. 2.85; P = 0.02) groups. Unlike the h-index, the gap between the 5-year AIF distribution of the Amateur group and other groups was not profound. Although there was a positive correlation between the metrics in the 3 groups, they did not proxy for each other. For instance, while the h-index of some experienced authors that have not published recently was high, their AIFs were zero. Also, some Amateur authors published very impactful articles and had a high 5-year AIF. However, since their number of publications is inevitably low, their h-index were low.

CONCLUSIONS

The AIF provides intuitive and complementary information to the h-index regarding the research output of neurosurgical authors.

The Top 50 Articles on Knee Posterolateral Corner Injuries

Purpose

To identify and characterize the most cited publications in orthopaedic research related to posterolateral corner (PLC) injuries of the knee.

Methods

The Science Citation Index Expanded was queried for PLC injury articles. The 50 most-cited studies from 1976 to 2021 were selected. Article characteristics, including number of citations, citation density, year of publication, source journal, country of origin, article type, article subtype, and level of evidence, were analyzed.

Results

The number of citations for individual articles ranged from 47 to 205. The 50 most cited articles were published in 16 journals. Eleven of the 50 articles (22%) were published in Arthroscopy-The Journal of Arthroscopic and Related Surgery. The largest proportion of the articles (n = 22, 44%) were classified as clinical, with the rest classified as reviews (n = 15, 30%), and basic science research (n = 13, 26%). The most common level of evidence for clinical articles was IV (14/22, 63.6%). Nine countries contributed to the top 50 articles, with the majority published in the United States (n = 35, 70%).

Conclusions

In the last 20 years, papers published on the diagnosis and surgical management of PLC injuries have a high citation frequency. Future high-quality research is needed to establish best-practice guidelines for the management of PLC injuries due to the low overall level of evidence of existing clinical studies.

Clinical Relevance

This research provides a comprehensive list for practitioners and may help educators identify articles to include in the curriculum for residents and fellows.

Investigating evolutionary trends and characteristics of renewable energy research in Africa: a bibliometric analysis from 1999 to 2021

Several countries across the African continent have been challenged with energy crises for decades. A growing number of studies have identified renewable energy as a sustainable way for Africa to address its persisting energy situation while combating climate change, as the continent has in abundance some of the common renewable energy resources. Little has been reported in the body of literature to quantitatively and qualitatively map the knowledge domain of this growing research field. In the current study, we conduct a bibliometric analysis on research documents extracted from the Web of Science Core Collection to identify trends and characteristics of the knowledge domain related to renewable energy in Africa from 1999 to 2021. Using two different software (VOSviewer and ITgInsight), we report the contribution of journals, countries, institutions, and authors and their collaboration patterns. We also perform co-citation and keyword analysis to identify the intellectual base and central themes of this research field. The results from the study revealed a growing interest in Africa's renewable energy, with about 90% of the total publication from within the last decade. Renewable & Sustainable Energy Reviews was identified as the most productive as well as the most influential journal in this field. The most contributing countries in this field were South Africa, USA, and Algeria. Centre de Developpement Des Energies Renouvelables, a research institute in Algeria, emerged as the most productive and influential institution. The analysis of research hotspots under different categories revealed that "solar energy," "CO2 emissions," and "rural electrification" are the topics that have gained maximum attention over the years. Keyword evolution analysis also identified "economic growth" and "green hydrogen production" as emerging topics that will play a major role in future studies. We conclude our work by providing specific suggestions and strategies to help bridge the gap which exists in the quantity and quality of renewable energy research between Africa and the rest of the world.

Mapping pathways to public understanding of climate science

Climate communication is a thriving research area spanning science, social science, and humanities. The field has grown explosively in recent years, necessitating increased efforts to synthesize and make sense of the resulting profusion of studies. To support scholars navigating this quickly evolving knowledge domain, we developed a knowledge map of the climate communication research landscape by applying network analysis and data visualization techniques to the metadata from 2995 publications on climate communication. The map reveals a dense web of connections among five distinct knowledge communities, indicating a tightly knit and intensely collaborative knowledge domain, and suggests new avenues for the application of climate communication knowledge, in particular to support climate services and co-production. The climate communication knowledge map answers the call for synoptic perspectives on areas of science communication while demonstrating a novel visual approach to knowledge synthesis for science communication domains.

Lexicons of Key Terms in Scholarly Texts and Their Disciplinary Differences: From Quantum Semantics Construction to Relative-Entropy-Based Comparisons

Complex networks are often used to analyze written text and reports by rendering texts in the form of a semantic network, forming a lexicon of words or key terms. Many existing methods to construct lexicons are based on counting word co-occurrences, having the advantage of simplicity and ease of applicability. Here, we use a quantum semantics approach to generalize such methods, allowing us to model the entanglement of terms and words. We show how quantum semantics can be applied to reveal disciplinary differences in the use of key terms by analyzing 12 scholarly texts that represent the different positions of various disciplinary schools (of conceptual change research) on the same topic (conceptual change). In addition, attention is paid to how closely the lexicons corresponding to different positions can be brought into agreement by suitable tuning of the entanglement factors. In comparing the lexicons, we invoke complex network-based analysis based on exponential matrix transformation and use information theoretic relative entropy (Jensen-Shannon divergence) as the operationalization of differences between lexicons. The results suggest that quantum semantics is a viable way to model the disciplinary differences of lexicons and how they can be tuned for a better agreement.

Improved PageRank and New Indices for Academic Impact Evaluation Using AI Papers as Case Studies

Evaluating academic papers and groups is important in scholar evaluation and literature retrieval. However, current evaluation indices, which pay excessive attention to the citation number rather than the citation importance and unidirectionality, are relatively simple. This study proposes new evaluation indices for papers and groups. First, an improved PageRank (PR) algorithm introducing citation importance is proposed to obtain a new citation-based paper index (CPI) via a pre-ranking and fine-tuning strategy. Second, to evaluate the paper’s influence inside and outside its research field, the focus citation-based paper index (FCPI) and diversity citation-based paper index (DCPI) are proposed based on topic similarity and diversity, respectively. Third, aside from the statistical indices for academic papers, we propose a foreign academic degree of dependence (FAD) to characterise the dependence between two academic groups. Finally, artificial intelligence (AI) papers from 2005 to 2019 are utilised for a case study.

Worldwide trends in the scientific production on soccer players market value, a bibliometric analysis using bibliometrix R-tool

Purpose

This study aims to identify the current state, the emergent research clusters, the key research topics and the configuration of collaboration in scientific production related to the market value of soccer players.

Design/methodology/approach

This article analyzes 52 articles published between 1985 and 2021 and from the Scopus and WoS databases.

Findings

The subject is of growing interest both in academic and practical areas. A variable that frequently appears as a determinant of market value is crowd wisdom. The largest cluster related to the co-citation level shows that the main issues about soccer player market value are player performance, team performance, and the determinants of the superstar formation. Spain and Germany stand out as essential countries both in literary production and citation rate. The network of collaborations is still low.

Research limitations/implications

This study is supported by databases being constantly updated, resulting in continuous variation in the number of indexed journals. Consequently, a bibliometric analysis regarding an emergent topic can, in fewer years, be subject to essential variations. Another limitation is that it has analyzed a particular topic using the most influential databases, and the global perspective could be improved with the incorporation of other different databases. Data regarding collaborations could be helpful for investigations or policies that propose to approach the topic supported by specialized groups. This study offers the possibility for future researchers to extend the databases used, the level of analysis, or focus on specific topics or variables affecting the soccer player market value.

Originality/value

This study contributes to knowing the current state of the soccer player market value research. Studies on such topics are relatively limited concerning the literature review.

Research on the discourse power evaluation of academic journals from the perspective of multiple fusion: Taking Medicine, General and Internal journals as an example

In the open science environment, this article evaluates the discourse power of academic journals from the perspective of multiple integration. It is conducive to scientific research management and provides a reference for enriching and perfecting the evaluation theory and indicators system of academic journals. Based on the theory of evaluation science, discourse power theory and communication theory, first, this article discusses the basic issues of the discourse power evaluation for academic journals. It defines the academic discourse power and the discourse power of academic journals. It is proposed that the discourse power of academic journals is composed of discourse influence and discourse leading. Discourse influence is composed of discourse influence ability and discourse power, and discourse leading is composed of social media discourse, news and policy discourse, encyclopaedia discourse, peer-review discourse and video discourse leading. This article explores the formation process and operation mechanism of the discourse power for academic journals. Then, this article constructs the discourse power evaluation model of academic journals. Second, this article integrates multi-source heterogeneous data, then adopts correlation analysis, integrated factor analysis, entropy weight method, Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method and two-dimensional four-quadrant mapping method to conduct empirical research on the discourse power evaluation of Medicine, General and Internal journals from the perspectives of multi-dimensions, multi-factors, multi-indicators and multi-methods fusion. The results show that the research on the discourse power evaluation for academic journals based on the theory, method and application logic is practical, comprehensive and reliable.

A stochastic generative model for citation networks among academic papers

We propose a stochastic generative model to represent a directed graph constructed by citations among academic papers, where nodes and directed edges represent papers with discrete publication time and citations respectively. The proposed model assumes that a citation between two papers occurs with a probability based on the type of the citing paper, the importance of cited paper, and the difference between their publication times, like the existing models. We consider the out-degrees of citing paper as its type, because, for example, survey paper cites many papers. We approximate the importance of a cited paper by its in-degrees. In our model, we adopt three functions: a logistic function for illustrating the numbers of papers published in discrete time, an inverse Gaussian probability distribution function to express the aging effect based on the difference between publication times, and an exponential distribution (or a generalized Pareto distribution) for describing the out-degree distribution. We consider that our model is a more reasonable and appropriate stochastic model than other existing models and can perform complete simulations without using original data. In this paper, we first use the Web of Science database and see the features used in our model. By using the proposed model, we can generate simulated graphs and demonstrate that they are similar to the original data concerning the in- and out-degree distributions, and node triangle participation. In addition, we analyze two other citation networks derived from physics papers in the arXiv database and verify the effectiveness of the model.

Co-Authorship Networks Analysis to Discover Collaboration Patterns among Italian Researchers

The study of the behaviors of large community of researchers and what correlations exist between their environment, such as grouping rules by law or specific institution policies, and their performance is an important topic since it affects the metrics used to evaluate the quality of the research. Moreover, in several countries, such as Italy, these metrics are also used to define the recruitment and funding policies. To effectively study these topics, we created a procedure that allow us to craft a large dataset of Italian Academic researchers, having the most important performance indices together with co-authorships information, mixing data extracted from the official list of academic researchers provided by Italian Ministry of University and Research and the Elsevier’s Scopus database. In this paper, we discuss our approach to automate the process of correct association of profiles and the mapping of publications reducing the use of computational resources. We also present the characteristics of four datasets related to specific research fields defined by the Italian Ministry of University and Research used to group the Italian researchers. Then, we present several examples of how the information extracted from these datasets can help to achieve a better understanding of the dynamics influencing scientist performances.

Joining the meta-research movement: A bibliometric case study of the journal Perspectives on Medical Education

PURPOSE

To conduct a bibliometric case study of the journal Perspectives on Medical Education (PME) to provide insights into the journal's inner workings and to "take stock" of where PME is today, where it has been, and where it might go.

METHODS

Data, including bibliographic metadata, reviewer and author details, and downloads, were collected for manuscripts submitted to and published in PME from the journal's Editorial Manager and Web of Science. Gender of authors and reviewers was predicted using Genderize.io. To visualize and analyze collaboration patterns, citation relationships and term co-occurrence social network analyses (SNA) were conducted. VOSviewer was used to visualize the social network maps.

RESULTS

Between 2012-2019 PME received, on average, 260 manuscripts annually (range = 73-402). Submissions were received from authors in 81 countries with the majority in the United States (US), United Kingdom, and the Netherlands. PME published 518 manuscripts with authors based in 31 countries, the majority being in the Netherlands, US, and Canada. PME articles were downloaded 717,613 times (mean per document: 1388). In total 1201 (55% women) unique peer reviewers were invited and 649 (57% women) completed reviews; 1227 (49% women) unique authors published in PME. SNA revealed that PME authors were quite collaborative, with most authoring articles with others and only a minority (n = 57) acting as single authors.

DISCUSSION

This case study provides a glimpse into PME and offers evidence for PME's next steps. In the future, PME is committed to growing the journal thoughtfully; diversifying and educating editorial teams, authors, and reviewers, and liberating and sharing journal data.

Co-citation Percentile Rank and JYUcite: a new network-standardized output-level citation influence metric and its implementation using Dimensions API

Judging value of scholarly outputs quantitatively remains a difficult but unavoidable challenge. Most of the proposed solutions suffer from three fundamental shortcomings: they involve (i) the concept of journal, in one way or another, (ii) calculating arithmetic averages from extremely skewed distributions, and (iii) binning data by calendar year. Here, we introduce a new metric Co-citation Percentile Rank (CPR), that relates the current citation rate of the target output taken at resolution of days since first citable, to the distribution of current citation rates of outputs in its co-citation set, as its percentile rank in that set. We explore some of its properties with an example dataset of all scholarly outputs from University of Jyväskylä spanning multiple years and disciplines. We also demonstrate how CPR can be efficiently implemented with Dimensions database API, and provide a publicly available web resource JYUcite, allowing anyone to retrieve CPR value for any output that has a DOI and is indexed in the Dimensions database. Finally, we discuss how CPR remedies failures of the Relative Citation Ratio (RCR), and remaining issues in situations where CPR too could potentially lead to biased judgement of value.

COVID-19 citation pandemic within the psychological knowledge domain

Presented study explores the knowledge domain of psychological research published in 2020 and 2021. Metadata from 156,942 psychology papers available in Scopus were analyzed using citation analysis and bibliographic mapping techniques. Having in mind the ubiquity of the COVID-19 pandemic and the numerous ways it has affected people's lives, the fact that COVID-19-related papers represent only 2% to 7% of the total academic production in psychology may seem rather low. However, these papers have attracted much more attention from the public than non-COVID papers. They were also cited two to eight times more frequently, depending on the measure used, and account for 16% to 19% of total citations to psychology papers. Results show that early-stage researchers and those who had fewer articles in Scopus have benefited more from publishing COVID papers. They have managed to boost their average citation rates to the level of their colleagues who were much longer active and previously had higher citedness. Results indicate that the authors citing behavior largely follows the cumulative advantage pattern. Psychological research in general is mainly focused on mental health, anxiety, depression, and stress. This trend is even more fostered due to the pandemic since some of these topics are often analyzed as typical emotional reactions to COVID-19. Other relevant issues are also very well covered, except for the question how scientific results are communicated to the public. The role of "hot" papers was elaborated from the perspective of research evaluation practice.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12144-022-03146-3.

Citation Oriented AuthorRank for Scientific Publication Ranking

It is now generally accepted that an article written by influential authors often deserves a higher ranking in information retrieval. However, it is a challenging task to determine an author’s relative influence since information about the author is, much of the time, inaccessible. Actually, in scientific publications, the author is an important metadata item, which has been widely used in previous studies. In this paper, we bring an optimized AuthorRank, which is a topic-sensitive algorithm calculated by citation context, into citation analysis for testing whether and how topical AuthorRank can replace or enhance classical PageRank for publication ranking. For this purpose, we first propose a PageRank with Priors (PRP) algorithm to rank publications and authors. PRP is an optimized PageRank algorithm supervised by the Labeled Latent Dirichlet Allocation (Labeled-LDA) topic model with full-text information extraction. We then compared four methods of generating an AuthorRank score, looking, respectively, at the first author, the last author, the most famous author, and the “average” author (of a publication). Additionally, two combination methods (Linear and Cobb–Douglas) of AuthorRank and PRP were compared with several baselines. Finally, as shown in our evaluation results, the performance of AuthorRank combined with PRP is better (p < 0.001) than other baselines for publication ranking.

Citation index analysis: a key or a trap?

[No abstract available]

A scholarly network of AI research with an information science focus: Global North and Global South perspectives

This paper primarily aims to provide a citation-based method for exploring the scholarly network of artificial intelligence (AI)-related research in the information science (IS) domain, especially from Global North (GN) and Global South (GS) perspectives. Three research objectives were addressed, namely (1) the publication patterns in the field, (2) the most influential articles and researched keywords in the field, and (3) the visualization of the scholarly network between GN and GS researchers between the years 2010 and 2020. On the basis of the PRISMA statement, longitudinal research data were retrieved from the Web of Science and analyzed. Thirty-two AI-related keywords were used to retrieve relevant quality articles. Finally, 149 articles accompanying the follow-up 8838 citing articles were identified as eligible sources. A co-citation network analysis was adopted to scientifically visualize the intellectual structure of AI research in GN and GS networks. The results revealed that the United States, Australia, and the United Kingdom are the most productive GN countries; by contrast, China and India are the most productive GS countries. Next, the 10 most frequently co-cited AI research articles in the IS domain were identified. Third, the scholarly networks of AI research in the GN and GS areas were visualized. Between 2010 and 2015, GN researchers in the IS domain focused on applied research involving intelligent systems (e.g., decision support systems); between 2016 and 2020, GS researchers focused on big data applications (e.g., geospatial big data research). Both GN and GS researchers focused on technology adoption research (e.g., AI-related products and services) throughout the investigated period. Overall, this paper reveals the intellectual structure of the scholarly network on AI research and several applications in the IS literature. The findings provide research-based evidence for expanding global AI research.

The effect of data sources on the measurement of open access: A comparison of Dimensions and the Web of Science

With the growing number of open access (OA) mandates, the accurate measurement of OA publishing is an important policy issue. Existing studies have provided estimates of the prevalence of OA publications ranging from 27.9% to 53.7%, depending on the data source and period of investigation. This paper aims at providing a comparison of the proportion of OA publishing as represented in two major bibliometric databases, Web of Science (WoS) and Dimensions, and assesses how the choice of database affects the measurement of OA across different countries. Results show that a higher proportion of publications indexed in Dimensions are OA than those indexed by WoS, and that this is particularly true for publications originating from outside North America and Europe. The paper concludes with a discussion of the cause and consequences of these differences, motivating the use of more inclusive databases when examining OA, especially for publications originating beyond North America and Europe.

Exploring the Role of International Research Collaboration in Building China’s World-Class Universities

This study explored the international research collaboration led by China’s world-class universities and its impact during the first construction cycle of the “Double First-Class” initiative (2016–2020). We collected international collaborative publications based on the Scopus database and examined the performance of international research collaboration in terms of quantity, impact, collaborative networks, and subject areas using scientometric indicators and social network analysis. We found that international collaboration accounts for only a quarter of the total output but is far above the overall level of scientific papers in terms of quality and impact. The United States, the United Kingdom, Australia, and Hong Kong remain China’s closest partners. Meanwhile, ties with Belt and Road partner countries have become stronger with the introduction of foreign policy. China’s medicine and multidisciplinary research have gained prominence in the context of the global health crisis. Thus, international research collaboration effectively improves research performance, deepens academic networks, and disseminates local issues and solutions to the world, thereby enabling China’s world-class universities to reconcile global engagement with Chinese characteristics.

What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives

The strategic relevance of innovation and scientific research has amplified the attention towards the definition of quality in research practice. However, despite the proliferation of evaluation metrics and procedures, there is a need to go beyond bibliometric approaches and to identify, more explicitly, what constitutes good research and which are its driving factors or determinants. This article reviews specialized research policy, science policy and scientometrics literature to extract critical dimensions associated with research quality as presented in a vast although fragmented theory background. A literature-derived framework of research quality attributes is, thus, obtained, which is subject to an expert feedback process, involving scholars and practitioners in the fields of research policy and evaluation. The results are represented by a structured taxonomy of 66 quality attributes providing a systemic definition of research quality. The attributes are aggregated into a three-dimensional framework encompassing research design (ex ante), research process (in-process) and research impact (ex post) perspectives. The main value of the study is to propose a literature-derived and comprehensive inventory of quality attributes and perspectives of evaluation. The findings can support further theoretical developments and research policy discussions on the ultimate drivers of quality and impact of scientific research. The framework can be also useful to design new exercises or procedures of research evaluation based on a multidimensional view of quality.

Ecotourism and sustainable development: a scientometric review of global research trends

With the increasing attention and awareness of the ecological environment, ecotourism is becoming ever more popular, but it still brings problems and challenges to the sustainable development of the environment. To solve such challenges, it is necessary to review literature in the field of ecotourism and determine the key research issues and future research directions. This paper uses scientometrics implemented by CiteSpace to conduct an in-depth systematic review of research and development in the field of ecotourism. Two bibliographic datasets were obtained from the Web of Science, including a core dataset and an expanded dataset, containing articles published between 2003 and 2021. Our research shows that ecotourism has been developing rapidly in recent years. The research field of ecotourism spans many disciplines and is a comprehensive interdisciplinary subject. According to the research results, the evolution of ecotourism can be roughly divided into three phases: human disturbance, ecosystem services and sustainable development. It could be concluded that it has entered the third stage of Shneider's four-stage theory of scientific discipline. The research not only identifies the main clusters and their advance in ecotourism research based on high impact citations and research frontier formed by citations, but also presents readers with new insights through intuitive visual images.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s10668-022-02190-0.

Comparing standard, collaboration and fractional CNCI at the institutional level: Consequences for performance evaluation

The average Category Normalised Citation Impact (CNCI) of an institution’s publication output is a widely used indicator for research performance benchmarking. However, it combines all entity contributions, obscuring individual inputs and preventing clear insight and sound policy recommendations if it is not correctly understood. Here, variations (Fractional and Collaboration [Collab] CNCI)—which aim to address the obscurity problem—are compared to the Standard CNCI indicator for over 250 institutions, spread globally, covering a ten-year period using Web of Science data. Results demonstrate that both Fractional and Collab CNCI methods produce lower index values than Standard CNCI. Fractional and Collab results are often near-identical despite fundamentally different calculation approaches. Collab-CNCI, however, avoids assigning fractional credit (which is potentially incorrect) and is relatively easy to implement. As single metrics obscure individual inputs, institutional output is also deconstructed into five collaboration groups. These groups track the increasing international collaboration trend, particularly highly multi-lateral studies and the decrease in publications authored by single institutions. The deconstruction also shows that both Standard and Fractional CNCI increase with the level of collaboration. However, Collab-CNCI does not necessarily follow this pattern thus enabling the identification of institutions where, for example, their domestic single articles are their best performing group. Comparing CNCI variants and deconstructing portfolios by collaboration type is, when understood and used correctly, an essential tool for interpreting institutional performance and informing policy making.

Sustained software development, not number of citations or journal choice, is indicative of accurate bioinformatic software

Background

Computational biology provides software tools for testing and making inferences about biological data. In the face of increasing volumes of data, heuristic methods that trade software speed for accuracy may be employed. We have studied these trade-offs using the results of a large number of independent software benchmarks, and evaluated whether external factors, including speed, author reputation, journal impact, recency and developer efforts, are indicative of accurate software.

Results

We find that software speed, author reputation, journal impact, number of citations and age are unreliable predictors of software accuracy. This is unfortunate because these are frequently cited reasons for selecting software tools. However, GitHub-derived statistics and high version numbers show that accurate bioinformatic software tools are generally the product of many improvements over time. We also find an excess of slow and inaccurate bioinformatic software tools, and this is consistent across many sub-disciplines. There are few tools that are middle-of-road in terms of accuracy and speed trade-offs.

Conclusions

Our findings indicate that accurate bioinformatic software is primarily the product of long-term commitments to software development. In addition, we hypothesise that bioinformatics software suffers from publication bias. Software that is intermediate in terms of both speed and accuracy may be difficult to publish—possibly due to author, editor and reviewer practises. This leaves an unfortunate hole in the literature, as ideal tools may fall into this gap. High accuracy tools are not always useful if they are slow, while high speed is not useful if the results are also inaccurate.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13059-022-02625-x).

An Analysis of the Worldwide Research on the Socio-Cultural Valuation of Forest Ecosystem Services

The socio-cultural valuation of forest ecosystem services is a useful tool to generate knowledge and help balance the different interests of stakeholders with respect to the management of these services. The aim of this study is to analyse the evolution of global research on the economic valuation of forest ecosystem services through a review of the existing literature on this topic. The results show that socio-cultural valuation has gained importance in recent years. There is a wide disparity between the countries conducting the research and those being studied. Inconsistency has been observed in the definition and classification of services provided by forests, as well as a lack of unanimity on the reference framework to be applied. The main methodological approaches in the socio-cultural valuation of forest services are participatory mapping, social media analysis, the Q method and free listing. For the collection of primary information, the dominant methodologies are focus groups, semi-structured interviews and online surveys. Finally, this study demonstrates that socio-cultural valuation has great potential to improve the legitimacy of forest ecosystem management decisions and to promote consensus building.

Rating a Researcher’s Cumulative Scholarly Output Based on Their Sequence Numbers in Multi-Authored Publications

As the academic world yields an ever-increasing research output in terms of journal papers, conference proceedings, and books, the rating of published works and authors becomes imperative. All the big citation databases and search engines are currently using cumulative output indices, such as h-index, i10-index, and g-index, which do not consider the number of co-authors or the researcher’s sequence number in the authors list of a publication. In this context, the article presents a novel computational approach for evaluating a researcher’s scholarly output by taking into account the total number of co-authors, the sequence number of the researcher in the authors list, and the number of citations received per year by an article. Arithmetic progression is applied to quantify the credit for each co-author of a publication. The respective credits of a researcher are then accumulated for all their publications to obtain the rating. The method yields a truer value of the researcher’s impact in terms of their scholarly activities. A global implementation of the metric presented in this work will curb the unethical practice of including the names of non-contributing researchers in the authors list and expecting reciprocity in return.

A model for cooperative scientific research inspired by the ant colony algorithm

Modern scientific research has become largely a cooperative activity in the Internet age. We build a simulation model to understand the population-level creativity based on the heuristic ant colony algorithm. Each researcher has two heuristic parameters characterizing the goodness of his own judgments and his trust on literature. We study how the distributions of contributor heuristic parameters change with the research problem scale, stage of the research problem, and computing power available. We also identify situations where path dependence and hasty research due to the pressure on productivity can significantly impede the long-term advancement of scientific research. Our work provides some preliminary understanding and guidance for the dynamical process of cooperative scientific research in various disciplines.

Publicación científica en acceso abierto: desafíos decoloniales para América Latina

A través de los conceptos de colonialidades, geopolítica y corpo-politica del conocimiento se reflexiona sobre algunos desafíos de la publicación científica latinoamericana en acceso abierto. Aunque América Latina es pionera en iniciativas de acceso abierto y en la creación de sistemas regionales cooperativos para compartir conocimiento como bien común, las revistas “internacionales”, refrendadas por factor de impacto, continúan a ser priorizadas en los sistemas de evaluación y financiamiento de la ciencia de la mayoría de los países de la región. Adicionalmente, estrategias de mercantilización del acceso abierto se hacen cada vez más penetrantes y amenazan con subvertir algunos de los propósitos iniciales del Movimiento de acceso abierto y crear brechas más profundas entre el Norte y el Sur. Por detrás de estos aspectos se sitúa la naturalización y perpetuación de jerarquías y exclusiones ontológicas y epistémicas con trasfondos de racismo sistémico que autores decoloniales caracterizan como colonialidades. Se requiere una desobediencia epistémica, como actitud decolonial, y una concertación de esfuerzos a nivel regional que transforme sistemas de evaluación, preserve el carácter público y académico de la ciencia y garantice equidad y justicia social

Exploring Natural Language Processing in Construction and Integration with Building Information Modeling: A Scientometric Analysis

The European Union (EU) aims to increase the efficiency and productivity of the construction industry. The EU suggests pairing Building Information Modeling with other digitalization technologies to seize the full potential of the digital transition. Meanwhile, industrial applications of Natural Language Processing (NLP) have emerged. The growth of NLP is affecting the construction industry. However, the potential of NLP and the combination of an NLP and BIM approach is still unexplored. The study tries to address this lack by applying a scientometric analysis to explore the state of the art of NLP in the AECO sector, and the combined applications of NLP and BIM. Science mapping is used to analyze 254 bibliographic records from Scopus Database analyzing the structure and dynamics of the domain by drawing a picture of the body of knowledge. NLP in AECO, and its pairing with BIM domain and applications, are investigated by representing: Conceptual, Intellectual, and Social structure. The highest number of NLP applications in AECO are in the fields of Project, Safety, and Risk Management. Attempts at combining NLP and BIM mainly concern the Automated Compliance Checking and semantic BIM enrichment goals. Artificial intelligence, learning algorithms, and ontologies emerge as the most widespread and promising technological drivers.

Introducing a bibliometric index based on factor analysis

This work applies a factor analysis with VARIMAX rotation to develop a bibliometric indicator, named the Weighted Factor Index, in order to derive a new classification for journals belonging to a certain category, alternative to the one provided by the Journal Impact Factor. For this, 16 metrics from three different databases (Web of Science, Scopus and SCImago Journal Rank) are considered. The Weighed Factor Index entails the advantage of incorporating and summarizing information from all the indicators; so as to test its performance, it was applied to rank journals belonging to the category Information Science & Library Science.

scite: A smart citation index that displays the context of citations and classifies their intent using deep learning

Citation indices are tools used by the academic community for research and research evaluation that aggregate scientific literature output and measure impact by collating citation counts. Citation indices help measure the interconnections between scientific papers but fall short because they fail to communicate contextual information about a citation. The use of citations in research evaluation without consideration of context can be problematic because a citation that presents contrasting evidence to a paper is treated the same as a citation that presents supporting evidence. To solve this problem, we have used machine learning, traditional document ingestion methods, and a network of researchers to develop a “smart citation index” called scite, which categorizes citations based on context. Scite shows how a citation was used by displaying the surrounding textual context from the citing paper and a classification from our deep learning model that indicates whether the statement provides supporting or contrasting evidence for a referenced work, or simply mentions it. Scite has been developed by analyzing over 25 million full-text scientific articles and currently has a database of more than 880 million classified citation statements. Here we describe how scite works and how it can be used to further research and research evaluation.

Characteristics and Research Techniques Associated with the Journal Impact Factor and Other Key Metrics in Pharmacology Journals

In the present age, there is intense pressure on researchers to publish their research in ‘high-impact factor’ journals. It would be interesting to understand the trend of research publications in the field of pharmacology by exploring the characteristics of research articles, including research techniques, in relation to the journal’s key bibliometrics, particularly journal impact factor (JIF), the seemingly most mentioned metric. This study aimed to determine the characteristics and research techniques in relation to research articles in pharmacology journals with higher or lower JIF values. A cross-sectional study was conducted on primary research journals under the ‘Pharmacology and Pharmacy’ category. Analysis of 768 original research articles across 32 journals (with an average JIF of 2.565 ± 0.887) demonstrated that research studies involving molecular techniques, in vivo experiments on animals, and bioinformatics and computational modeling were significantly associated with a higher JIF value of the journal in which such contributions were published. Our analysis suggests that research studies involving such techniques/approaches are more likely to be published in higher-ranked pharmacology journals.

Differences in Citation Patterns across Areas, Article Types and Age Groups of Researchers

The evaluation of research proposals and academic careers is subject to indicators of scientific productivity. Citations are critical signs of impact for researchers, and many indicators are based on these data. The literature shows that there are differences in citation patterns between areas. The scope and depth that these differences may have to motivate the extension of these studies considering types of articles and age groups of researchers. In this work, we conducted an exploratory study to elucidate what evidence there is about the existence of these differences in citation patterns. To perform this study, we collected historical data from Scopus. Analyzing these data, we evaluate if there are measurable differences in citation patterns. This study shows that there are evident differences in citation patterns between areas, types of publications, and age groups of researchers that may be relevant when carrying out researchers’ academic evaluation.

A Historical Review and Analysis of Impact Factors Across School Psychology Journals from 1977 to 2019

School psychology journals yield hundreds of articles each year. As these journals are often evaluated based on the impact factors they produce, the aim of this study was to provide a historically complete record of the five impact factor values for the generalist school psychology journals that yield them. This study identified impact factors beginning in 1977, 20 years earlier than previously reported, and ending in 2019. Across all years and journals, the average Journal Impact Factor (JIF) was about 1.0, the average Immediacy Index was less than 0.4, the average 5-year Impact Factor was about 2.3, the average original CiteScore was 1.8, and the average new CiteScore was about 3.0. Increases in values were evident across time, and the highest recorded values across journals are held by the Journal of School Psychology (for the JIF, 5-year Impact Factor, and both CiteScore metrics) and School Psychology Review (for the Immediacy Index). Most impact factors, with the exception of the Immediacy Index, were moderately to highly correlated. The new CiteScore values were always the highest, and Immediacy Index values were always the lowest. School psychology has added journals to the list of those indexed by major databases, and these journals have increased their impact over time.

Impact factor and citation metrics in phase III cancer trials

Purpose: Journal impact factor (IF) is often used to measure research quality and importance. We assessed trial factors associated with the publication of cancer trials in journals with higher IF and publications receiving higher citations.

Materials and Methods: Cancer-specific phase III RCTs were screened through https://clinicaltrials.gov. We identified trials with published primary endpoints, along with their corresponding journal IF and relative citation ratio (RCR).

Results: Seven-hundred ninety manuscripts were included in our study. Trials that met their primary endpoint were more commonly published in journals with higher IF (Median IF: positive trials 35.4 vs. negative trials 26.3, P < 0.001). Furthermore, trials that led to subsequent FDA drug approvals were also published in journals with higher IF (Median IF: 59.1 vs. 26.3 in trials not leading to FDA approvals, P < 0.001). When analyzing RCR, trial positivity (meeting primary endpoint) was not associated with increased citations on multivariable analysis (P = 0.56). Lastly, publications of trials leading to FDA approvals (P < 0.001), and publications of trials in journals with higher IF (P < 0.001) were associated with increased RCR.

Conclusions: Positive trials are commonly published in journals with high IF, but do not necessarily lead to increased citations. Moreover, trials published in journals with higher IF are more likely to receive increased citations.