The signs of science

Peer review in research assessment and data analysis of Italian publications in SSD M-STO/08 (Archival science, bibliography, library science)

Since the introduction of research assessment systems at institutional level in the 1980s, the ongoing debate on the roles and functions of peer review and bibliometrics has been vivid and lively. In the first part of the article, the main lines over time of this debate are traced, and a reflection on the epistemic functions of peer review and citations is proposed. In Italy, the first research assessment exercise (VTR) was based on peer review only, while the following ones (VQR) were based on different methods for bibliometric disciplines and non-bibliometric disciplines, namely bibliometric indicators and peer review. Starting from a data analysis on Italian publications, and using as a sample data from M-STO/08 (Archival science, bibliography and library science) area, the essay shows some trends and changes in publication habits in HSS. Conclusions open a perspective on revitalization of peer review as a solid qualitative method for research assessment.

What drives citations of frontier application publications?

A large body of literature exists on analysis of citation and reviews of application of efficiency frontier. However, the reviews that assessed the determinants of citation counts did not focus on frontier applications. We contribute to the literature by identifying the drivers of citations of frontier application publications on Ghana. We employed two-part mixture modelling with inverse hyperbolic sine (IHS) transformation of the second part, which was found to be more appropriate than single equation IHS transformation modelling, for our data. Use of stochastic frontier analysis or data envelopment analysis did not drive citations counts. However, quality of journals in which frontier application studies were published and accessibility of the journals to readers, drive citation counts. Authors, institutions and funders of studies on frontier applications may consider these over collaborations, in seeking growth in citation counts.

Investigating SSH Research and Publication Practices in Disciplinary and Institutional Contexts. A Survey-Based Comparative Approach in Two Universities

In this paper, we comparatively analyze, present and discuss the results from a survey on increasing the visibility of research achievements in the social sciences and humanities (SSH) that was carried out at the University of Vienna (Austria) and the University of Navarra (Spain) in 2016 and 2017. Covering four major topics—searching and finding literature, publishing, the visibility of research, and the assessment of research outputs—we ask the following questions: are there disciplinary differences to be identified, and how do they present themselves in the two institutional contexts? Discussing the results, we showcase how disciplinary and institutional traditions and contexts are important factors that influence research and publication practices in the SSH. Our results indicate that the practices of searching and finding literature as well as publication practices and behavior are shaped by disciplinary traditions and epistemic cultures. On the contrary, assessment and valuation of research outputs are influenced by institutional and national contexts in which SSH research is organized and carried out.

The geography of references in elite articles: Which countries contribute to the archives of knowledge?

This study asks the question on which national “shoulders” the world’s top-level research stands. Traditionally, the number of citations to national papers has been the evaluative measures of national scientific standings. We raise a different question: instead of analyzing the citations to a countries’ articles (the forward view), we examine references to prior publications from specific countries cited in the most elite publications (the backward—citing—view). “Elite publications” are operationalized as the top-1% most-highly cited articles. Using the articles published from 2004 to 2013, we examine the research referenced in these works. Our results confirm the well-known fact that China has emerged to become a major player in science. However, China still belongs to the low contributors when countries are ranked as contributors to the cited references in top-1% articles. Using this perspective, the results do not support a decreasing trend for the USA; in fact, the USA exceeds expectations (compared to its publication share) in terms of references in the top-1% articles. Switzerland, Sweden, and the Netherlands also appear at the top of the list. However, the results for Germany are lower than statistically expected.

Visualization of Disciplinary Profiles: Enhanced Science Overlay Maps

Purpose

The purpose of this study is to modernize previous work on science overlay maps by updating the underlying citation matrix, generating new clusters of scientific disciplines, enhancing visualizations, and providing more accessible means for analysts to generate their own maps.

Design/methodology/approach

We use the combined set of 2015 Journal Citation Reports for the Science Citation Index (n of journals = 8,778) and the Social Sciences Citation Index (n = 3,212) for a total of 11,365 journals. The set of Web of Science Categories in the Science Citation Index and the Social Sciences Citation Index increased from 224 in 2010 to 227 in 2015. Using dedicated software, a matrix of 227 × 227 cells is generated on the basis of whole-number citation counting. We normalize this matrix using the cosine function. We first develop the citing-side, cosine-normalized map using 2015 data and VOSviewer visualization with default parameter values. A routine for making overlays on the basis of the map (“wc15.exe”) is available at http://www.leydesdorff.net/wc15/index.htm.

Findings

Findings appear in the form of visuals throughout the manuscript. In Figures 1–9 we provide basemaps of science and science overlay maps for a number of companies, universities, and technologies.

Research limitations

As Web of Science Categories change and/or are updated so is the need to update the routine we provide. Also, to apply the routine we provide users need access to the Web of Science.

Practical implications

Visualization of science overlay maps is now more accurate and true to the 2015 Journal Citation Reports than was the case with the previous version of the routine advanced in our paper.

Originality/value

The routine we advance allows users to visualize science overlay maps in VOSviewer using data from more recent Journal Citation Reports.

Engaging scientometrics in information systems

Although scientometrics is seeing increasing use in Information Systems (IS) research, in particular for evaluating research efforts and measuring scholarly influence; historically, scientometric IS studies are focused primarily on ranking authors, journals, or institutions. Notwithstanding the usefulness of ranking studies for evaluating the productivity of the IS field's formal communication channels and its scholars, the IS field has yet to exploit the full potential that scientometrics offers, especially towards its progress as a discipline. This study makes a contribution by raising the discourse surrounding the value of scientometric research in IS, and proposes a framework that uncovers the multi-dimensional bases for citation behaviour and its epistemological implications on the creation, transfer, and growth of IS knowledge. Having identified 112 empirical research evaluation studies in IS, we select 44 substantive scientometric IS studies for in-depth content analysis. The findings from this review allow us to map an engaging future in scientometric research, especially towards enhancing the IS field's conceptual and theoretical development.

Can intellectual processes in the sciences also be simulated? The anticipation and visualization of possible future states

Socio-cognitive action reproduces and changes both social and cognitive structures. The analytical distinction between these dimensions of structure provides us with richer models of scientific development. In this study, I assume that (1) social structures organize expectations into belief structures that can be attributed to individuals and communities; (2) expectations are specified in scholarly literature; and (3) intellectually the sciences (disciplines, specialties) tend to self-organize as systems of rationalized expectations. Whereas social organizations remain localized, academic writings can circulate, and expectations can be stabilized and globalized using symbolically generalized codes of communication. The intellectual restructuring, however, remains latent as a second-order dynamics that can be accessed by participants only reflexively. Yet, the emerging "horizons of meaning" provide feedback to the historically developing organizations by constraining the possible future states as boundary conditions. I propose to model these possible future states using incursive and hyper-incursive equations from the computation of anticipatory systems. Simulations of these equations enable us to visualize the couplings among the historical-i.e., recursive-progression of social structures along trajectories, the evolutionary-i.e., hyper-incursive-development of systems of expectations at the regime level, and the incursive instantiations of expectations in actions, organizations, and texts.

Setting our bibliographic references free: towards open citation data

Purpose

– Citation data needs to be recognised as a part of the Commons – those works that are freely and legally available for sharing – and placed in an open repository. The paper aims to discuss this issue.

Design/methodology/approach

– The Open Citation Corpus is a new open repository of scholarly citation data, made available under a Creative Commons CC0 1.0 public domain dedication and encoded as Open Linked Data using the SPAR Ontologies.

Findings

– The Open Citation Corpus presently provides open access (OA) to reference lists from 204,637 articles from the OA Subset of PubMed Central, containing 6,325,178 individual references to 3,373,961 unique papers.

Originality/value

– Scholars, publishers and institutions may freely build upon, enhance and reuse the open citation data for any purpose, without restriction under copyright or database law.

Measuring researchers’ use of scholarly information through social bookmarking data: A case study of BibSonomy

This paper explores the possibility of using data from social bookmarking services to measure the use of information by academic researchers. Social bookmarking data can be used to augment participative methods (e.g. interviews and surveys) and other, non-participative methods (e.g. citation analysis and transaction logs) to measure the use of scholarly information. We use BibSonomy, a free resource-sharing system, as a case study. Results show that published journal articles are by far the most popular type of source bookmarked, followed by conference proceedings and books. Commercial journal publisher platforms are the most popular type of information resource bookmarked, followed by websites, records in databases and digital repositories. Usage of open access information resources is low in comparison with toll access journals. In the case of open access repositories, there is a marked preference for the use of subject-based repositories over institutional repositories. The results are consistent with those observed in related studies based on surveys and citation analysis, confirming the possible use of bookmarking data in studies of information behaviour in academic settings. The main advantages of using social bookmarking data are that is an unobtrusive approach, it captures the reading habits of researchers who are not necessarily authors, and data are readily available. The main limitation is that a significant amount of human resources is required in cleaning and standardizing the data.