Are book publications disappearing from scholarly communication in the social sciences and humanities?

The footprint of a metrics-based research evaluation system on Spain’s philosophical scholarship: An analysis of researchers’ perceptions

The use of bibliometric indicators in research evaluation has a series of complex impacts on academic inquiry. These systems have gradually spread into a wide range of locations and disciplines, including the humanities. The aim of this study is to examine their effects as perceived by philosophy and ethics researchers in Spain, a country where bibliometric indicators have long been used to evaluate research. The study uses a mixed approach combining quantitative and qualitative data from a self-administered questionnaire completed by 201 researchers and from 14 in-depth interviews with researchers selected according to their affiliation, professional category, gender, and area of knowledge. Results show that the evaluation system is widely perceived to affect university researchers in significant ways, particularly related to publication habits (document type and publication language), the transformation of research agendas and the neglect of teaching work, as well as increasing research misconduct and negatively affecting mental health. Although to a lesser extent, other consequences included increased research productivity and enhanced transparency and impartiality in academic selection processes.

Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland

The Academy of Finland (AKA), Finland’s major public research funding agency, uses a Web of Science (WoS) based bibliometric indicator to assess the performance of research it has funded. We use an alternative methodology to compare (1) the research performance and (2) the scholarly communication profile of AKA-funded research to the Finnish universities’ entire output across the major fields of arts and sciences. Our data consists of 142,742 publications (years 2015–2018) registered in the national information service, which integrates Current Research Information System (CRIS) data of 13 Finnish universities. Research performance is analyzed using the Finnish community-curated expert-based rating of publication channels (so-called JUFO). Our results show that compared to the Finnish universities’ entire output a larger share of AKA-funded research is published in leading JUFO rated journals and book publishers. JUFO and WoS-based indicators produced consonant results regarding the performance of AKA-funded research. Analysis of publication profiles shows that AKA-funded research is more focused than the universities’ output on using peer-reviewed publications, articles published in journals, English language, foreign publishers and open access publishing. We conclude that the CRIS-based publication data can support multidimensional assessments of research performance and scholarly communication profiles, potentially also in other countries and institutions. CRIS development and maintenance require multi-stakeholder commitment, resources and incentives to ensure data quality and coverage. To fully recognize diverse open science practices and to enable international comparisons, CRISs need further development and integration as data sources.

The Uptake and Impact of a Label for Peer-Reviewed Books

This article presents an analysis of the uptake of the GPRC label (Guaranteed Peer Reviewed Content label) since its introduction in 2010 until 2019. GPRC is a label for books that have been peer reviewed introduced by the Flemish publishers association. The GPRC label allows locally published scholarly books to be included in the regional database for the Social Sciences and Humanities which is used in the Flemish performance-based research funding system. Ten years after the start of the GPRC label, this is the first systematic analysis of the uptake of the label. We use a mix of qualitative and quantitative methods. Our two main data sources are the Flemish regional database for the Social Sciences and Humanities, which currently includes 2,580 GPRC-labeled publications, and three interviews with experts on the GPRC label. Firstly, we study the importance of the label in the Flemish performance-based research funding system. Secondly, we analyse the label in terms of its possible effect on multilingualism and the local or international orientation of publications. Thirdly, we analyse to what extent the label has been used by the different disciplines. Lastly, we discuss the potential implications of the label for the peer review process among book publishers. We find that the GPRC label is of limited importance to the Flemish performance-based research funding system. However, we also conclude that the label has a specific use for locally oriented book publications and in particular for the discipline Law. Furthermore, by requiring publishers to adhere to a formalized peer review procedure, the label affects the peer review practices of local publishers because not all book publishers were using a formal system of peer review before the introduction of the label and even at those publishers who already practiced peer review, the label may have required the publishers to make these procedures more uniform.

Ensuring Quality and Status: Peer Review Practices in Kriterium, A Portal for Quality-Marked Monographs and Edited Volumes in Swedish SSH

Although established forms of peer review are often criticized for being slow, secretive, and even unfair, they are repeatedly mentioned by academics as the most important indicator of quality in scholarly publishing. In many countries, the peer review of books is a less codified practice than that of journal articles or conference papers, and the processes and actors involved are far from uniform. In Sweden, the review process of books has seldom been formalized. However, more formal peer review of books has been identified as a response to the increasing importance placed on streamlined peer-reviewed publishing of journal articles in English, which has been described as a direct challenge to more pluralistic publication patterns found particularly in the humanities. In this study, we focus on a novel approach to book review, Kriterium, where an independent portal maintained by academic institutions oversees the reviewing of academic books. The portal administers peer reviews, providing a mark of quality through a process which involves reviewers, an academic coordinator, and an editorial board. The paper studies how this process functions in practice by exploring materials concerning 24 scholarly books reviewed within Kriterium. Our analysis specifically targets tensions identified in the process of reviewing books with a focus on three main themes, namely the intended audience, the edited volume, and the novel role of the academic coordinator. Moreover, we find that the two main aims of the portal–quality enhancement (making research better) and certification (displaying that research is of high quality)–are recurrent in deliberations made in the peer review process. Consequently, we argue that reviewing procedures and criteria of quality are negotiated within a broader discussion where more traditional forms of publishing are challenged by new standards and evaluation practices.

Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities

This article analyses the impact and visibility of scholarly journals in the humanities that are publishing in the national languages in Finland, Norway and Spain. Three types of publishers are considered: commercial publishers, scholarly society as publisher, and research organizations as publishers. Indicators of visibility and impact were obtained from Web of Science, SCOPUS, Google Metrics, Scimago Journal Rank and Journal Citation Report. The findings compiled show that in Spain the categories “History and Archaeology” and “Language and Literature” account for almost 70% of the journals analysed, while the other countries offer a more homogeneous distribution. In Finland, the scholarly society publisher is predominant, in Spain, research organization as publishers, mostly universities, have a greater weighting, while in Norway, the commercial publishers take centre stage. The results show that journals from Finland and Norway will have reduced possibilities in terms of impact and visibility, since the vernacular language appeals to a smaller readership. Conversely, the Spanish journals are more attractive for indexing in commercial databases. Distribution in open access ranges from 64 to 70% in Norwegian and Finish journals, and to 91% in Spanish journals. The existence of DOI range from 31 to 41% in Nordic journals to 60% in Spanish journals and has a more widespread bearing on the citations received in all three countries (journals with DOI and open access are cited more frequently).

Towards complexity-sensitive book metrics for scholarly monographs in national databases for research output

Purpose

This study investigates an approach to book metrics for research evaluation that takes into account the complexity of scholarly monographs. This approach is based on work sets – unique scholarly works and their within-work related bibliographic entities – for scholarly monographs in national databases for research output.

Design/methodology/approach

This study examines bibliographic records on scholarly monographs acquired from four European databases (VABB in Flanders, Belgium; CROSBI in Croatia; CRISTIN in Norway; COBISS in Slovenia). Following a data enrichment process using metadata from OCLC WorldCat and Amazon Goodreads, the authors identify work sets and the corresponding ISBNs. Next, on the basis of the number of ISBNs per work set and the presence in WorldCat, they design a typology of scholarly monographs: Globally visible single-expression works, Globally visible multi-expression works, Miscellaneous and Globally invisible works.

Findings

The findings show that the concept “work set” and the proposed typology can aid the identification of influential scholarly monographs in the social sciences and humanities (i.e. the Globally visible multi-expression works).

Practical implications

In light of the findings, the authors outline requirements for the bibliographic control of scholarly monographs in national databases for research output that facilitate the use of the approach proposed here.

Originality/value

The authors use insights from library and information science (LIS) to construct complexity-sensitive book metrics. In doing so, the authors, on the one hand, propose a solution to a problem in research evaluation and, on the other hand, bring to attention the need for a dialogue between LIS and neighbouring communities that work with bibliographic data.

Tracing the context in disciplinary classifications: A bibliometric pairwise comparison of five classifications of journals in the social sciences and humanities

Despite the centrality of disciplinary classifications in bibliometric analyses, it is not well known how the choice of disciplinary classification influences bibliometric representations of research in the social sciences and humanities (SSH). This is especially crucial when using data from national databases. Therefore, we examine the differences in the disciplinary profile of an article along with the absolute and relative number of articles across disciplines using five disciplinary classifications for journals. We use data on journal articles (2006–2015) from the national bibliographic databases VABB-SHW in Flanders (Belgium) and Cristin in Norway. Our study is based on pairwise comparisons of the local classifications used in these databases, the Web of Science subject categories, the Science-Metrix, and the ERIH PLUS journal classifications. For comparability, all classifications are mapped to the OECD Fields of Research and Development classification. The findings show that the choice of disciplinary classification can lead to over- or underestimation of the absolute number of publications per discipline. In contrast, if the focus is on the relative numbers, the choice of classification has practically no influence. These findings facilitate an informed choice of a disciplinary classification for journals in SSH when using data from national databases.

Open access at the national level: A comprehensive analysis of publications by Finnish researchers

Open access (OA) has mostly been studied by relying on publication data from selective international databases, notably Web of Science (WoS) and Scopus. The aim of our study is to show that it is possible to achieve a national estimate of the number and share of OA based on institutional publication data providing a comprehensive coverage of the peer-reviewed outputs across fields, publication types, and languages. Our data consists of 48,177 journal, conference, and book publications from 14 Finnish universities in 2016–2017, including information about OA status, as self-reported by researchers and validated by data-collection personnel through their Current Research Information System (CRIS). We investigate the WoS, Scopus, and DOI coverage, as well as the share of OA outputs between different fields, publication types, languages, OA mechanisms (gold, hybrid, and green), and OA information sources (DOAJ, Bielefeld list, and Sherpa/Romeo). We also estimate the role of the largest international commercial publishers compared to the not-for-profit Finnish national publishers of journals and books. We conclude that institutional data, integrated at national and international level, provides one of the building blocks of a large-scale data infrastructure needed for comprehensive assessment and monitoring of OA across countries, for example at the European level.

Multilingual publishing in the social sciences and humanities: A seven-country European study

We investigate the state of multilingualism across the social sciences and humanities (SSH) using a comprehensive data set of research outputs from seven European countries (Czech Republic, Denmark, Finland, Flanders [Belgium], Norway, Poland, and Slovenia). Although English tends to be the dominant language of science, SSH researchers often produce culturally and societally relevant work in their local languages. We collected and analyzed a set of 164,218 peer-reviewed journal articles (produced by 51,063 researchers from 2013 to 2015) and found that multilingualism is prevalent despite geographical location and field. Among the researchers who published at least three journal articles during this time period, over one-third from the various countries had written their work in at least two languages. The highest share of researchers who published in only one language were from Flanders (80.9%), whereas the lowest shares were from Slovenia (57.2%) and Poland (59.3%). Our findings show that multilingual publishing is an ongoing practice in many SSH research fields regardless of geographical location, political situation, and/or historical heritage. Here we argue that research is international, but multilingual publishing keeps locally relevant research alive with the added potential for creating impact.

National Lists of Scholarly Publication Channels: An Overview and Recommendations for Their Construction and Maintenance

Purpose

This paper presents an overview of different kinds of lists of scholarly publication channels and of experiences related to the construction and maintenance of national lists supporting performance-based research funding systems. It also contributes with a set of recommendations for the construction and maintenance of national lists of journals and book publishers.

Design/methodology/approach

The study is based on analysis of previously published studies, policy papers, and reported experiences related to the construction and use of lists of scholarly publication channels.

Findings

Several countries have systems for research funding and/or evaluation, that involve the use of national lists of scholarly publication channels (mainly journals and publishers). Typically, such lists are selective (do not include all scholarly or non-scholarly channels) and differentiated (distinguish between channels of different levels and quality). At the same time, most lists are embedded in a system that encompasses multiple or all disciplines. This raises the question how such lists can be organized and maintained to ensure that all relevant disciplines and all types of research are adequately represented.

Research limitation

The conclusions and recommendations of the study are based on the authors’ interpretation of a complex and sometimes controversial process with many different stakeholders involved.

Practical implications

The recommendations and the related background information provided in this paper enable mutual learning that may feed into improvements in the construction and maintenance of national and other lists of scholarly publication channels in any geographical context. This may foster a development of responsible evaluation practices.

Originality/value

This paper presents the first general overview and typology of different kinds of publication channel lists, provides insights on expert-based versus metrics-based evaluation, and formulates a set of recommendations for the responsible construction and maintenance of publication channel lists.

Adopting open access in the social sciences and humanities: evidence from a developing nation

Open Access (OA) publishing, with ambitious movements such as Plan S, is engendering radical changes among academic publishers. Emerging countries need to keep publishing as well as adopt open access to catch up with the changes. Using exclusive data from the Social Sciences & Humanities Peer Awards (SSHPA) database, the study employed both descriptive statistics and a Bayesian linear regression model to examine the journals and publishers in which Vietnamese social scientists published during the period 2008–2019, and the potential of pursuing the OA movement in Vietnam. We found an increasing diversification in the publishing sources of Vietnamese social science researchers with growth rates of 9.8% and 14.1% per annum in the number of publishers and journals, respectively. Given that the proportion of Gold OA articles had a fourfold increase over the examined period, it seems that the Vietnamese academic community is adopting OA. Furthermore, Bayesian analysis results hint at positive associations of internal and external collaborative power (number of domestic and foreign authors, respectively) with the decision to publish in OA (β_{b_TotalVN_OpenAccess} = 0.22; β_{b_TotalForeign_OpenAccess} = 0.15). The results and its implications suggest that Vietnamese policymakers and university director boards should facilitate as well as control the quality of the scientific publishing and the OA movement.

Researchers publishing monographs are more productive and more local-oriented

In this study, we investigate what share of researchers publish monographs across fields, gender and seniority. We acquired data from the Polish current research information system (POL-on) containing metadata of about 1,031,141 peer-reviewed publications from 67,415 Polish researchers, including 30,185 monographs from 2013 to 2016. The data are aggregated at the researcher level, which allows us to shed new light on publication patterns in all fields. We show that scholars who publish monographs also publish journal articles at the same time. This pattern is observed in all dimensions, i.e. research fields, gender and seniority. However, substantial differences between the fields are observed. Moreover, we show that researchers who publish monographs are also more productive in terms of the number of publications than researchers who did not publish any monographs. This result is independent of the publication counting method, i.e. fractional or whole counting. At the same time, scholars who publish monographs are more local-oriented in terms of the publication channels they choose.

El prestigio de las editoriales académicas con libros en Ciencias Sociales y Humanidades en Brasil

Resumen El libro es un importante medio de comunicación científica en Ciencias Sociales y Humanidades, es por ello por lo que se debe potenciar su correcta evaluación. Así, en el presente artículo se exponen las distintas metodologías de valoración de libros académicos que aplican las agencias de Brasil y de España y a continuación se ofrecen resultados de un estudio sobre el prestigio de las editoriales. El cuestionario por el que se han obtenido los datos fue enviado a 16.542 directores de grupos de investigación en Ciencias Sociales y Humanidades y se recibieron 1.977 respuestas (12%). Los resultados son novedosos, ya que es la primera vez que se realiza un estudio de percepción de la calidad de las editoriales en Brasil, a partir de la opinión de los especialistas. Como ejemplo del trabajo realizado, se presentan las editoriales con mayor prestigio en Educación y, como en el resto de las áreas, hay un núcleo pequeño de editoriales brasileñas que son las que concentran más votos. En el caso de las editoriales extranjeras, el prestigio no está tan concentrado, siendo Portugal, Francia y España los países de procedencia de estas editoriales. Estos resultados pueden servir de forma orientativa en los procesos de evaluación de la actividad científica en Brasil. Además, los editores pueden tener conocimiento de cómo son percibidos y para los autores también es importante conocer esta información cuando tengan que elegir una editorial para presentar sus manuscritos.

Do we need a book citation index for research evaluation?

Given the importance of books and book chapters as vehicles of knowledge in social sciences and humanities (SSH) disciplines, it has previously been thought that the application of citation metrics to the evaluation of these disciplines should also include, in addition to journal articles, citations from books and book chapters. The main argument supporting this claim is the belief that top cited authors in journal articles and in monographs form two distinct populations. In this article, we compare the rankings of the most cited authors in three SSH disciplines (sociology, philosophy, and history), obtained by counting citations in the journal articles covered in the Web of Science, and a large sample of books and book chapters covered in the book citation index. Contrary to what is often suggested, we show that adding book and book chapter citations to journal citations does not produce significantly different rankings than those obtained solely on the basis of citations in journal articles.