Making research and evaluation more useful and more interesting for information services and their users: A guide for students and practitioners

Reflections on the recent increase in the number of cross-sectional surveys received by the editorial team of the journal indicated that potential contributors might consider other research techniques, in addition to, or instead of a survey. In this article, Christine Urquhart discusses some different research designs, and different research methods that may help students and practitioners find useful answers to questions about professional practice beyond the standard survey. Researchers could consider research designs such as quasi-experimental techniques, controlled before-after studies, and interrupted time series. The basic principles of such methods are outlined and some examples cited. Other research techniques outlined include those that research subjects might find more interesting to do, such as conjoint analysis and vignettes.

The Upsurge of Impact Factors in Pediatric Journals Post COVID-19 Outbreak: A Cross-Sectional Study

Background

Impact factor (IF) is a quantitative tool designed to evaluate scientific journals' excellence. There was an unprecedented upsurge in biomedical journals' IF in 2020, perhaps contributed by the increased number of publications since the COVID-19 outbreak. We conducted a cross-sectional study (2018–2020) to analyze recent trends in standard bibliometrics (IF, Eigenfactor, SNIP) of pediatric journals. We also estimated reference and publication counts of biomedical journals since publication volume determines the number of citations offered and IF.

Methods

Various bibliometrics of pediatric journals and reference/publication volumes of biomedical journals were compared between 2020 vs. 2019 and 2019 vs. 2018. We also compared open access (OA) and subscription journals' trends. Finally, we estimated IF changes in the journals of a different specialty, pulmonology.

Results

The study included 164 pediatric and 4,918 biomedical journals (OA = 1,473, subscription = 3,445). Pediatric journals' IFs had increased significantly in 2020 [median (IQR) = 2.35 (1.34)] vs. 2019 [1.82 (1.22)] (Wilcoxon: p-value < 0.001). IFs were unchanged between 2018 and 2019. Eigenfactor remained stable between 2018 and 2020, while SNIP increased progressively. Reference/publication volumes of biomedical journals escalated between 2018 and 2020, and OA journals experienced faster growth than subscription journals. IFs of pulmonary journals also increased considerably in 2020 vs. 2019.

Conclusions

We report an upsurge in pediatric journals' IF, perhaps contributed by a sudden increase in publication numbers in 2020. Therefore, considering this limitation, IF should be cautiously used as the benchmark of excellence. Unlike IF, Eigenfactor remained stable between 2018 and 2020. Similar changes in IF were also observed among the journals of another specialty, pulmonology.

Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection

The knowledge and stance of researchers regarding bibliometric indicators is a field of study that has gained weight in recent decades. In this paper we address this issue for the little explored areas of philosophy and ethics, and applied to a context, in this case Spain, where bibliometric indicators are widely used in evaluation processes. The study combines data from a self-administered questionnaire completed by 201 researchers and from 14 in-depth interviews with researchers selected according to their affiliation, professional category, gender and area of knowledge. The survey data suggest that researchers do not consider bibliometric indicators a preferred criterion of quality, while there is a fairly high self-perception of awareness of a number of indicators. The qualitative data points to a generalised perception of a certain rejection of the specific use of indicators, with four main positions being observed: (1) disqualification of the logic of metrics, (2) scepticism about the possibility of assessing quality with quantitative methods, (3) complaints about the incorporation of methods that are considered to belong to other disciplines, and (4) criticism of the consequences that this generates in the discipline of philosophy.

Researchers' attitudes towards the h-index on Twitter 2007-2020: criticism and acceptance

The h-index is an indicator of the scientific impact of an academic publishing career. Its hybrid publishing/citation nature and inherent bias against younger researchers, women, people in low resourced countries, and those not prioritizing publishing arguably give it little value for most formal and informal research evaluations. Nevertheless, it is well-known by academics, used in some promotion decisions, and is prominent in bibliometric databases, such as Google Scholar. In the context of this apparent conflict, it is important to understand researchers’ attitudes towards the h-index. This article used public tweets in English to analyse how scholars discuss the h-index in public: is it mentioned, are tweets about it positive or negative, and has interest decreased since its shortcomings were exposed? The January 2021 Twitter Academic Research initiative was harnessed to download all English tweets mentioning the h-index from the 2006 start of Twitter until the end of 2020. The results showed a constantly increasing number of tweets. Whilst the most popular tweets unapologetically used the h-index as an indicator of research performance, 28.5% of tweets were critical of its simplistic nature and others joked about it (8%). The results suggest that interest in the h-index is still increasing online despite scientists willing to evaluate the h-index in public tending to be critical. Nevertheless, in limited situations it may be effective at succinctly conveying the message that a researcher has had a successful publishing career.