Taking scholarly books into account: current developments in five European countries

The Landscape of Scholarly Book Publishing in Croatia: Finding Pathways for Viable Open Access Models

(1) Background: Open access to scholarly works is globally recognized as a goal to be achieved as soon as possible; however, there is not yet a general understanding of how to achieve open access for books. In considering the most appropriate models of transition, an accurate and detailed insight into national and regional specifics can be of great importance. The aim of this research is to show the current state of scholarly book publishing in Croatia: recognising the key stakeholders, their characteristics, and the current level of open access to scholarly books. (2) Methods: The existing data from two different sources were used: the data about the public subsidies for book publishers by the Ministry of Science and Education and the data on published books from the Croatian Scientific Bibliography CROSBI, both for the period from 2018 to 2021. (3) Results: In the four-year period, 224 Croatian publishers were awarded subsidies to publish 2359 book titles. The majority of the publishers received support for only a small number of titles and relatively low amounts of subsidies. More than half of the titles are published by small private commercial publishers. However, the uptake of digital publishing among commercial publishers is very modest. Open access to scholarly books is almost entirely in the domain of non-commercial publishers. Most open access titles are available on the websites of their publishers. (4) Conclusions: The analysis of the data from these two sources have resulted in an overview of the current state of book publishing in Croatia. Such an overview provides a good basis for designing future measures and creating a national open science plan and can also be a useful contribution to international discussions.

Prestige of scholarly book publishers—An investigation into criteria, processes, and practices across countries

Numerous national research assessment policies aim to promote ‘excellence’ and incentivize scholars to publish their research in the most prestigious journals or with the most prestigious book publishers. We investigate the practicalities of assessing book outputs in different countries, concentrating on evaluation based on the prestige of book publishers (e.g. Denmark, Finland, Lithuania, Norway, Poland, and Spain). Additionally, we test whether such judgments are transparent and yield consistent results. We show inconsistencies in the levelling of publishers, such as cases where the same publisher is ranked as prestigious and not-so-prestigious in different states or within the same country in consequent years. Likewise, we find that verification of compliance with the mandatory prerequisites is not always possible because of the lack of transparency. Our findings support doubts about whether the assessment of books based on a judgement about their publishers yields acceptable outcomes. Even more, currently used rankings of publishers focus on evaluating the gatekeeping role of publishers but do not assess other essential stages in scholarly book publishing (e.g. dissemination or preservation). Our suggestion for future research is to develop approaches to evaluate books by accounting for the value added to every book at every publishing stage, which is vital for the quality of book outputs from research assessment and scholarly communication perspectives.

The Uptake and Impact of a Label for Peer-Reviewed Books

This article presents an analysis of the uptake of the GPRC label (Guaranteed Peer Reviewed Content label) since its introduction in 2010 until 2019. GPRC is a label for books that have been peer reviewed introduced by the Flemish publishers association. The GPRC label allows locally published scholarly books to be included in the regional database for the Social Sciences and Humanities which is used in the Flemish performance-based research funding system. Ten years after the start of the GPRC label, this is the first systematic analysis of the uptake of the label. We use a mix of qualitative and quantitative methods. Our two main data sources are the Flemish regional database for the Social Sciences and Humanities, which currently includes 2,580 GPRC-labeled publications, and three interviews with experts on the GPRC label. Firstly, we study the importance of the label in the Flemish performance-based research funding system. Secondly, we analyse the label in terms of its possible effect on multilingualism and the local or international orientation of publications. Thirdly, we analyse to what extent the label has been used by the different disciplines. Lastly, we discuss the potential implications of the label for the peer review process among book publishers. We find that the GPRC label is of limited importance to the Flemish performance-based research funding system. However, we also conclude that the label has a specific use for locally oriented book publications and in particular for the discipline Law. Furthermore, by requiring publishers to adhere to a formalized peer review procedure, the label affects the peer review practices of local publishers because not all book publishers were using a formal system of peer review before the introduction of the label and even at those publishers who already practiced peer review, the label may have required the publishers to make these procedures more uniform.

Ensuring Quality and Status: Peer Review Practices in Kriterium, A Portal for Quality-Marked Monographs and Edited Volumes in Swedish SSH

Although established forms of peer review are often criticized for being slow, secretive, and even unfair, they are repeatedly mentioned by academics as the most important indicator of quality in scholarly publishing. In many countries, the peer review of books is a less codified practice than that of journal articles or conference papers, and the processes and actors involved are far from uniform. In Sweden, the review process of books has seldom been formalized. However, more formal peer review of books has been identified as a response to the increasing importance placed on streamlined peer-reviewed publishing of journal articles in English, which has been described as a direct challenge to more pluralistic publication patterns found particularly in the humanities. In this study, we focus on a novel approach to book review, Kriterium, where an independent portal maintained by academic institutions oversees the reviewing of academic books. The portal administers peer reviews, providing a mark of quality through a process which involves reviewers, an academic coordinator, and an editorial board. The paper studies how this process functions in practice by exploring materials concerning 24 scholarly books reviewed within Kriterium. Our analysis specifically targets tensions identified in the process of reviewing books with a focus on three main themes, namely the intended audience, the edited volume, and the novel role of the academic coordinator. Moreover, we find that the two main aims of the portal–quality enhancement (making research better) and certification (displaying that research is of high quality)–are recurrent in deliberations made in the peer review process. Consequently, we argue that reviewing procedures and criteria of quality are negotiated within a broader discussion where more traditional forms of publishing are challenged by new standards and evaluation practices.

Towards complexity-sensitive book metrics for scholarly monographs in national databases for research output

Purpose

This study investigates an approach to book metrics for research evaluation that takes into account the complexity of scholarly monographs. This approach is based on work sets – unique scholarly works and their within-work related bibliographic entities – for scholarly monographs in national databases for research output.

Design/methodology/approach

This study examines bibliographic records on scholarly monographs acquired from four European databases (VABB in Flanders, Belgium; CROSBI in Croatia; CRISTIN in Norway; COBISS in Slovenia). Following a data enrichment process using metadata from OCLC WorldCat and Amazon Goodreads, the authors identify work sets and the corresponding ISBNs. Next, on the basis of the number of ISBNs per work set and the presence in WorldCat, they design a typology of scholarly monographs: Globally visible single-expression works, Globally visible multi-expression works, Miscellaneous and Globally invisible works.

Findings

The findings show that the concept “work set” and the proposed typology can aid the identification of influential scholarly monographs in the social sciences and humanities (i.e. the Globally visible multi-expression works).

Practical implications

In light of the findings, the authors outline requirements for the bibliographic control of scholarly monographs in national databases for research output that facilitate the use of the approach proposed here.

Originality/value

The authors use insights from library and information science (LIS) to construct complexity-sensitive book metrics. In doing so, the authors, on the one hand, propose a solution to a problem in research evaluation and, on the other hand, bring to attention the need for a dialogue between LIS and neighbouring communities that work with bibliographic data.

Open access at the national level: A comprehensive analysis of publications by Finnish researchers

Open access (OA) has mostly been studied by relying on publication data from selective international databases, notably Web of Science (WoS) and Scopus. The aim of our study is to show that it is possible to achieve a national estimate of the number and share of OA based on institutional publication data providing a comprehensive coverage of the peer-reviewed outputs across fields, publication types, and languages. Our data consists of 48,177 journal, conference, and book publications from 14 Finnish universities in 2016–2017, including information about OA status, as self-reported by researchers and validated by data-collection personnel through their Current Research Information System (CRIS). We investigate the WoS, Scopus, and DOI coverage, as well as the share of OA outputs between different fields, publication types, languages, OA mechanisms (gold, hybrid, and green), and OA information sources (DOAJ, Bielefeld list, and Sherpa/Romeo). We also estimate the role of the largest international commercial publishers compared to the not-for-profit Finnish national publishers of journals and books. We conclude that institutional data, integrated at national and international level, provides one of the building blocks of a large-scale data infrastructure needed for comprehensive assessment and monitoring of OA across countries, for example at the European level.

National Lists of Scholarly Publication Channels: An Overview and Recommendations for Their Construction and Maintenance

Purpose

This paper presents an overview of different kinds of lists of scholarly publication channels and of experiences related to the construction and maintenance of national lists supporting performance-based research funding systems. It also contributes with a set of recommendations for the construction and maintenance of national lists of journals and book publishers.

Design/methodology/approach

The study is based on analysis of previously published studies, policy papers, and reported experiences related to the construction and use of lists of scholarly publication channels.

Findings

Several countries have systems for research funding and/or evaluation, that involve the use of national lists of scholarly publication channels (mainly journals and publishers). Typically, such lists are selective (do not include all scholarly or non-scholarly channels) and differentiated (distinguish between channels of different levels and quality). At the same time, most lists are embedded in a system that encompasses multiple or all disciplines. This raises the question how such lists can be organized and maintained to ensure that all relevant disciplines and all types of research are adequately represented.

Research limitation

The conclusions and recommendations of the study are based on the authors’ interpretation of a complex and sometimes controversial process with many different stakeholders involved.

Practical implications

The recommendations and the related background information provided in this paper enable mutual learning that may feed into improvements in the construction and maintenance of national and other lists of scholarly publication channels in any geographical context. This may foster a development of responsible evaluation practices.

Originality/value

This paper presents the first general overview and typology of different kinds of publication channel lists, provides insights on expert-based versus metrics-based evaluation, and formulates a set of recommendations for the responsible construction and maintenance of publication channel lists.

El prestigio de las editoriales académicas con libros en Ciencias Sociales y Humanidades en Brasil

Resumen El libro es un importante medio de comunicación científica en Ciencias Sociales y Humanidades, es por ello por lo que se debe potenciar su correcta evaluación. Así, en el presente artículo se exponen las distintas metodologías de valoración de libros académicos que aplican las agencias de Brasil y de España y a continuación se ofrecen resultados de un estudio sobre el prestigio de las editoriales. El cuestionario por el que se han obtenido los datos fue enviado a 16.542 directores de grupos de investigación en Ciencias Sociales y Humanidades y se recibieron 1.977 respuestas (12%). Los resultados son novedosos, ya que es la primera vez que se realiza un estudio de percepción de la calidad de las editoriales en Brasil, a partir de la opinión de los especialistas. Como ejemplo del trabajo realizado, se presentan las editoriales con mayor prestigio en Educación y, como en el resto de las áreas, hay un núcleo pequeño de editoriales brasileñas que son las que concentran más votos. En el caso de las editoriales extranjeras, el prestigio no está tan concentrado, siendo Portugal, Francia y España los países de procedencia de estas editoriales. Estos resultados pueden servir de forma orientativa en los procesos de evaluación de la actividad científica en Brasil. Además, los editores pueden tener conocimiento de cómo son percibidos y para los autores también es importante conocer esta información cuando tengan que elegir una editorial para presentar sus manuscritos.

El futuro de las editoriales universitarias en España

En este trabajo los autores presentan una prospección sobre la evolución de las editoriales universitarias españolas a partir de un estudio Delphi realizado a cerca de una treintena de responsables de estos servicios editoriales, miembros de la Unión de Editoriales Universitarias Españolas (UNE). Los principales resultados obtenidos ponen de manifiesto la necesidad de un mayor reconocimiento del papel de las editoriales universitarias en el control y la promoción de la calidad de la edición académica. La adaptación al modelo digital, la gestión del open access, la flexibilización de los modelos de gestión para garantizar una mayor orientación al marketing y una mejor comercialización y distribución se plantean también como algunos de los retos más importantes.

Do we need a book citation index for research evaluation?

Given the importance of books and book chapters as vehicles of knowledge in social sciences and humanities (SSH) disciplines, it has previously been thought that the application of citation metrics to the evaluation of these disciplines should also include, in addition to journal articles, citations from books and book chapters. The main argument supporting this claim is the belief that top cited authors in journal articles and in monographs form two distinct populations. In this article, we compare the rankings of the most cited authors in three SSH disciplines (sociology, philosophy, and history), obtained by counting citations in the journal articles covered in the Web of Science, and a large sample of books and book chapters covered in the book citation index. Contrary to what is often suggested, we show that adding book and book chapter citations to journal citations does not produce significantly different rankings than those obtained solely on the basis of citations in journal articles.

Normalizing Book Citations in Google Scholar: A Hybrid Cited-side Citing-side Method

Purpose

To design and test a method for normalizing book citations in Google Scholar.

Design/methodology/approach

A hybrid citing-side, cited-side normalization method was developed and this was tested on a sample of 285 research monographs. The results were analyzed and conclusions drawn.

Findings

The method was technically feasible but required extensive manual intervention because of the poor quality of the Google Scholar data.

Research limitations

The sample of books was limited and also all were from one discipline —business and management. Also, the method has only been tested on Google Scholar, it would be useful to test it on Web of Science or Scopus.

Practical limitations

Google Scholar is a poor source of data although it does cover a much wider range citation sources that other databases.

Originality/value

This is the first method that has been developed specifically for normalizing books which have so far not been able to be normalized.

The Flemish Performance-based Research Funding System: A Unique Variant of the Norwegian Model

The BOF-key is the performance-based research funding system that is used in Flanders, Belgium. In this paper we describe the historical background of the system, its current design and organization, as well as its effects on the Flemish higher education landscape. The BOF-key in its current form relies on three bibliometric parameters: publications in Web of Science, citations in Web of Science, and publications in a comprehensive regional database for SSH publications. Taken together, the BOF-key forms a unique variant of the Norwegian model: while the system to a large extent relies on a commercial database, it avoids the problem of inadequate coverage of the SSH. Because the bibliometric parameters of the BOF-key are reused in other funding allocation schemes, their overall importance to the Flemish universities is substantial.

The Norwegian Model in Norway

The “Norwegian Model” attempts to comprehensively cover all the peer-reviewed scholarly literatures in all areas of research in one single weighted indicator. Thereby, scientific production is made comparable across departments and faculties within and between research institutions, and the indicator may serve institutional evaluation and funding. This article describes the motivation for creating the model in Norway, how it was designed, organized and implemented, as well as the effects and experiences with the model. The article ends with an overview of a new type of bibliometric studies that are based on the type of comprehensive national publication data that the Norwegian Model provides.

Redesigning the Model of Book Evaluation in the Polish Performance-based Research Funding System

Purpose

This study aims to present the key systemic changes in the Polish book evaluation model to focus on the publisher list, as inspired by the Norwegian Model.

Design/methodology/approach

In this study we reconstruct the framework of the 2010 and 2018 models of book evaluation in Poland within the performance-based research funding system.

Findings

For almost 20 years the book evaluation system in Poland has been based on the verification of various technical criteria (e.g. length of the book). The new 2018 model is based on the principle of prestige inheritance (a book is worth as much as its publisher is) and is inspired by the publisher list used in the Norwegian Model. In this paper, we argue that this solution may be a more balanced policy instrument than the previous 2010 model in which neither the quality of the publisher nor the quality of the book played any role in the evaluation.

Research limitations

We work from the framework of the 2018 model of book evaluation specified in the law on higher education and science from 20 July 2018, as implementation acts are not available yet.

Practical implications

This study may provide a valuable point of reference on how structural reforms in the research evaluation model were implemented on a country level. The results of this study may be interesting to policy makers, stakeholders and researchers focused on science policy.

Originality/value

This is the very first study that presents the new framework of the Polish research evaluation model and policy instruments for scholarly book evaluation. We describe what motivated policy makers to change the book evaluation model, and what arguments were explicitly raised to argue for the new solution.

Estimación del valor educativo de los libros académicos que no están en inglés: el caso de España

Este estudio examina el valor educativo de 15.117 libros escritos en español y publicados por editoriales españolas en disciplinas de ciencias sociales y humanidades en el período 2002-2011, en base a las menciones que reciben desde guías docentes en línea. Se implementó un método para identificar automáticamente las menciones y filtrar los resultados. El chequeo manual de las 52.716 menciones encontradas estimó una precision del 99,5% para filtrar las menciones falsas y del 74,7% para identificar las menciones correctas. Una quinta parte de los libros (2.849; 19%) se mencionaron al menos una vez en guías académicas en línea, y casi todos ellos (95%) han sido publicados por un tercio de las editoriales incluídas en el estudio. Un análisis detallado de los 23 libros más recomendados en guías docentes mostró que la mayoría son monografías de humanidades con un solo autor, escritos originalmente en español. Las menciones procedieron de 379 dominios web, mayoritariamente de sitios web de universidades españolas. En conclusión, es posible crear indicadores a partir de las menciones en guías docentes para evaluar el valor educativo de los libros en español, aunque se requieren chequeos manuales si los valores se usan para evaluar libros individuales.

How arbitrary are the weights assigned to books in performance-based research funding? An empirical assessment of the weight and size of monographs in Flanders

Purpose

The purpose of this paper is to present an empirical assessment of the weight assigned to monographs in the publication indicator of the performance-based research funding system (PRFS) in Flanders, Belgium. By relating publication weight to publication size the authors offer an alternative perspective on the production of scholars who publish monographs. This perspective on weights is linked to the aggregation level at which PRFS indicators are used: the national/regional one as opposed to the local one. In Flanders as elsewhere the publication indicator designed for funding distribution between universities has sometimes trickled down to institutions, their faculties and departments.

Design/methodology/approach

As an alternative indicator of scholarly production the authors propose the median number of pages of a publication type. Measuring the size of publications allows to compare the weight ratio between monographs and journal articles in the publication indicator to their size ratio in the VABB-SHW database. The authors compare two levels, one of four universities and one of 16 disciplines.

Findings

Median publication size differences between disciplines are much larger than those between universities. This indicates that an increase of monographs’ weight in the publication indicator would hardly affect funding distribution at the regional level. Disciplines with a relatively large share of monographs, however, would contribute more to the publication indicator. Hence an increase of monographs’ weight might provide a better balance between fields and between publication types.

Originality/value

This paper presents a thought experiment regarding the weight assigned to different publication types in the publication indicator of the Flemish PRFS: what would happen if this weight were replaced by the median number of pages of a publication type? In doing so, we highlight that such weighting schemes play an important role in finding a balance between fields of research. The sizeable differences between weight and size ratios offer a new and critical perspective on the weighting schemes currently used in PRFS, also in other countries.

Are book publications disappearing from scholarly communication in the social sciences and humanities?

Purpose

The purpose of this paper is to analyze the evolution in terms of shares of scholarly book publications in the social sciences and humanities (SSH) in five European countries, i.e. Flanders (Belgium), Finland, Norway, Poland and Slovenia. In addition to aggregate results for the whole of the social sciences and the humanities, the authors focus on two well-established fields, namely, economics & business and history.

Design/methodology/approach

Comprehensive coverage databases of SSH scholarly output have been set up in Flanders (VABB-SHW), Finland (VIRTA), Norway (NSI), Poland (PBN) and Slovenia (COBISS). These systems allow to trace the shares of monographs and book chapters among the total volume of scholarly publications in each of these countries.

Findings

As expected, the shares of scholarly monographs and book chapters in the humanities and in the social sciences differ considerably between fields of science and between the five countries studied. In economics & business and in history, the results show similar field-based variations as well as country variations. Most year-to-year and overall variation is rather limited. The data presented illustrate that book publishing is not disappearing from an SSH.

Research limitations/implications

The results presented in this paper illustrate that the polish scholarly evaluation system has influenced scholarly publication patterns considerably, while in the other countries the variations are manifested only slightly. The authors conclude that generalizations like “performance-based research funding systems (PRFS) are bad for book publishing” are flawed. Research evaluation systems need to take book publishing fully into account because of the crucial epistemic and social roles it serves in an SSH.

Originality/value

The authors present data on monographs and book chapters from five comprehensive coverage databases in Europe and analyze the data in view of the debates regarding the perceived detrimental effects of research evaluation systems on scholarly book publishing. The authors show that there is little reason to suspect a dramatic decline of scholarly book publishing in an SSH.

The culture of orphaned texts

Purpose

The purpose of this paper is to contribute to the body of knowledge on how research evaluation in different national and organisational contexts affects, often in unintended ways, research and publication practices. In particular, it looks at the development of book publication in the social sciences and humanities (SSH) in the Czech Republic since 2004, when a performance-based system of evaluation was introduced, up to the present.

Design/methodology/approach

The paper builds upon ethnographic research complemented by the analysis of Czech science policy documents, data available in the governmental database “Information Register of R&D results” and formal and informal interviews with expert evaluators and other stakeholders in the research system. It further draws on the authors’ own experience as scholars, who have also over the years participated in a number of evaluation procedures as peers and experts.

Findings

The number of books published by researchers in SSH based at Czech institutions has risen considerably in reaction to the pressure for productivity that is inscribed into the evaluation methodology and has resulted in the rise of in-house publishing by researchers’ own research institution, “fake internationalisation” using foreign low-quality presses as the publication venue, and the development of a culture of orphaned books that have no readers.

Practical implications

In the Czech Republic robust and internationally harmonised bibliometric data regarding books would definitely help to create a form of research evaluation that would stimulate meaningful scholarly book production. At the same time, better-resourced and better-designed peer review evaluation is needed.

Originality/value

This is the first attempt to analyse in detail the conditions and consequences the Czech performance-based research evaluation system has for SSH book publication. The paper demonstrates that often discussed harming of SSH and book-writing in particular by performance-based IF-centred research evaluation does not necessarily manifest in declining numbers of publications. On the contrary, the number of books published may increase at the cost of producing more texts of questionable scholarly quality.

Do prestigious Spanish scholarly book publishers have more teaching impact?

Purpose

The purpose of this paper is to assess the educational value of prestigious and productive Spanish scholarly publishers based on mentions of their books in online scholarly syllabi.

Design/methodology/approach

Syllabus mentions of 15,117 books from 27 publishers were searched for, manually checked and compared with Microsoft Academic (MA) citations.

Findings

Most books published by Ariel, Síntesis, Tecnos and Cátedra have been mentioned in at least one online syllabus, indicating that their books have consistently high educational value. In contrast, few books published by the most productive publishers were mentioned in online syllabi. Prestigious publishers have both the highest educational impact based on syllabus mentions and the highest research impact based on MA citations.

Research limitations/implications

The results might be different for other publishers. The online syllabus mentions found may be a small fraction of the syllabus mentions of the sampled books.

Practical implications

Authors of Spanish-language social sciences and humanities books should consider general prestige when selecting a publisher if they want educational uptake for their work.

Originality/value

This is the first study assessing book publishers based on syllabus mentions.

The diversity of monographs: changing landscape of book evaluation in Poland

Purpose

The purpose of this paper is to determine the characteristic patterns of monographs in the humanities, social sciences and hard sciences published by Polish scholars. The study provides a comprehensive overview of the Polish book evaluation system to explain how monographs are assessed and illustrate how changes in the definitions of the types of scholarly book publications influence publication patterns.

Design/methodology/approach

This paper analyses bibliographic records of 42,307 monographs published by Polish scholars in the humanities and science fields from 2009 to 2016. Through a bibliometric analysis, the paper investigates the characteristic patterns of the monographs, including authorship, publication language and length, across three fields.

Findings

The present study demonstrates that changes in the definitions of scholarly book publications in Poland have significantly influenced the characteristic patterns of monographs. The analysis of the characteristic patterns across three fields reveals that the monographs are different in terms of all characteristics. In the entire period, 85.3 percent monographs were written in Polish, 10.1 percent in English, 1.4 percent in German, 1.1 percent in Russian and 2.1 percent in 39 other languages. The most significant changes are observed in authorship patterns.

Originality/value

This work offers empirical findings on the characteristic patterns of monographs in the humanities, social sciences and hard sciences from a non-English speaking country. It discusses a unique model of book assessment and shows certain consequences of various overly formalized procedures of evaluation. Thus, the study identifies the major challenges and implications of using highly formalized procedures for book evaluation.