A science impact framework to measure impact beyond journal metrics

Overview of health research in Greenland from 2001 to 2020

AIMS

Health research in Greenlanders is challenged by a small population, few dedicated researchers and limited funding. Some research areas have proven impactful but gaps in knowledge are unknown. We aimed to quantify health research in Greenland, providing insights for researchers, policymakers and funding agencies to guide future research.

METHODS

We identified all peer-reviewed health research publications with the term 'Greenland***' in title or abstract from 2001 to 2020 through the platform entitled The Lens. Our bibliometric analysis included number of publications, publication rate, research types, research topics, citations and author affiliations, as well as funding.

RESULTS

From 2001 to 2020, a total of 640 health research publications concerning Greenland or Greenlanders were identified, with an average annual publication growth rate of 2.9%. The major health research topics studied in Greenland were infectious diseases, environmental health and general health. Several research fields, including rheumatology, nephrology and dermatology among others proved understudied at best. Three Danish universities contributed with 33% of all publications, with the International Journal of Circumpolar Health (N=176, 28%) being the top journal destination. The most productive author had 119 article authorships during the period.

CONCLUSION

During the past 20 years, Greenland's annual publication rate has been steadily increasing, with an average of 32 articles about health research published each year, which is a higher publication rate than world average. The health research in Greenland is primarily epidemiologic with infectious disease being the most studied research area. Most publications were anchored in Denmark and centred around a few highly productive authors.

Assessment of Public Health Impact of 20 Non-Research HIV Demonstration Projects by Use of the CDC Science Impact Framework, United States, 2018-2022

Compared with traditional measures of scientific impact, the CDC Science Impact Framework more broadly, directly, and quickly assesses impact of public health science. For 20 CDC-funded HIV prevention projects that were conducted during 2018-2022, health departments documented impact, based on CDC Science Impact Framework domains: Disseminating Science, Creating Awareness, Catalyzing Action, Effecting Change , and Shaping the Future . Health departments reported 282 impacts: the most common were new partnerships (n = 17, Catalyzing Action ), capacity building (n = 16, Effecting Change ), new projects or initiatives begun (n = 15, Shaping the Future ), new collaborations (n = 14, Catalyzing Action ), improved program (n = 13, Shaping the Future ), new guidelines or practices (n = 13, Effecting Change ), and informed persons affected by work (n = 13, Creating Awareness ). Health departments documented substantial impact with a simple, timely, and broad approach. Demonstrating impact is important for community-based organizations, funders, and others interested in public health and helps them better understand the value of public health.

Bibliometric top ten healthcare-related ChatGPT publications in the first ChatGPT anniversary

Since its public release on November 30, 2022, ChatGPT has shown promising potential in diverse healthcare applications despite ethical challenges, privacy issues, and possible biases. The aim of this study was to identify and assess the most influential publications in the field of ChatGPT utility in healthcare using bibliometric analysis. The study employed an advanced search on three databases, Scopus, Web of Science, and Google Scholar, to identify ChatGPT-related records in healthcare education, research, and practice between November 27 and 30, 2023. The ranking was based on the retrieved citation count in each database. The additional alternative metrics that were evaluated included (1) Semantic Scholar highly influential citations, (2) PlumX captures, (3) PlumX mentions, (4) PlumX social media and (5) Altmetric Attention Scores (AASs). A total of 22 unique records published in 17 different scientific journals from 14 different publishers were identified in the three databases. Only two publications were in the top 10 list across the three databases. Variable publication types were identified, with the most common being editorial/commentary publications (n=8/22, 36.4%). Nine of the 22 records had corresponding authors affiliated with institutions in the United States (40.9%). The range of citation count varied per database, with the highest range identified in Google Scholar (1019-121), followed by Scopus (242-88), and Web of Science (171-23). Google Scholar citations were correlated significantly with the following metrics: Semantic Scholar highly influential citations (Spearman's correlation coefficient ρ=0.840, p<0.001), PlumX captures (ρ=0.831, p<0.001), PlumX mentions (ρ=0.609, p=0.004), and AASs (ρ=0.542, p=0.009). In conclusion, despite several acknowledged limitations, this study showed the evolving landscape of ChatGPT utility in healthcare. There is an urgent need for collaborative initiatives by all stakeholders involved to establish guidelines for ethical, transparent, and responsible use of ChatGPT in healthcare. The study revealed the correlation between citations and alternative metrics, highlighting its usefulness as a supplement to gauge the impact of publications, even in a rapidly growing research field.

Supporting Time-Critical Decision Making: The Creation of a Nurse-Led Rapid Literature Review Service

Due to shifting priorities and unforeseen challenges, nurse leaders often lack sufficient time and resources to systematically review and appraise the available literature in search of the best evidence to guide decisions. A nurse-led rapid review service can produce accelerated knowledge synthesis and contextualized translation of evidence in a resource-efficient manner. This article describes a nurse-led rapid review service implemented at a large academic medical center and provides a reproducible process to guide other healthcare organizations in developing similar programs.

Certainty of Evidence Assessment in Systematic Reviews Published by High-Impact Sports Science Journals: A Meta-epidemiological Study

BACKGROUND

Assessing certainty of evidence is a key element of any systematic review. The aim of this meta-epidemiology study was to understand the frequency and ways with which certainty of evidence is assessed in contemporary systematic reviews published in high-impact sports science journals.

METHODS

We searched PubMed and relevant journal web sites from 1 August 2016 to 11 October 2022 for systematic reviews published in the top-ten highest-impact journals within the 2020 Journal Citation Report for the Sports Sciences category. Pairs of independent reviewers screened items using a priori established criteria.

RESULTS

Of 1250 eligible documents, 258 (20.6%) assessed the certainty of evidence, defined as using two or more distinct domains to provide an overall rating of the trustworthiness of findings across studies. Nine methods were cited for assessing certainty, with the most common being the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (61.6%). The proportion of systematic reviews assessing certainty of evidence appeared to increase over the 6-year timeframe analyzed. Across all reviews analyzed, a large majority addressed the domains of risk of bias, imprecision, and inconsistency of the results. Other certainty domains including indirectness/applicability were less commonly assessed.

DISCUSSION

Only one in five recent contemporary systematic reviews in the field of exercise and sports science assessed certainty of evidence. Organizational and institutional education on methods for assessing evidence may help further increase uptake of these methods and improve both the quality and clinical impact of systematic reviews in the field.

Regaining creativity in science: insights from conversation

The 'early modern' (Renaissance) workshop was predicated on the idea that informal, open-ended cooperation enables participants to experience difference and develop new insights, which can lead to new ways of thinking and doing. This paper presents the insights that emerged from a conversation event that brought wide-ranging voices together from different domains in science, and across the arts and industry, to consider science leadership as we look to the future in a time of interlocking crises. The core theme identified was a need to regain creativity in science; in the methods of scientific endeavours, in the way science is produced and communicated, and in how science is experienced in society. Three key challenges for re-establishing a culture of creativity in science emerged: (i) how scientists communicate what science is and what it is for, (ii) what scientists value, and (iii) how scientists create and co-create science with and for society. Furthermore, the value of open-ended and ongoing conversation between different perspectives as a means of achieving this culture was identified and demonstrated.

Hybrid Gold Open Access Citation Advantage in Clinical Medicine: Analysis of Hybrid Journals in the Web of Science

Biomedical fields have seen a remarkable increase in hybrid Gold open access articles. However, it is uncertain whether the hybrid Gold open access option contributes to a citation advantage, an increase in the citations of articles made immediately available as open access regardless of the article’s quality or whether it involves a trending topic of discussion. This study aimed to compare the citation counts of hybrid Gold open access articles to subscription articles published in hybrid journals. The study aimed to ascertain if hybrid Gold open access publications yield an advantage in terms of citations. This cross-sectional study included the list of hybrid journals under 59 categories in the ‘Clinical Medicine’ group from Clarivate’s Journal Citation Reports (JCR) during 2018–2021. The number of citable items with ‘Gold Open Access’ and ‘Subscription and Free to Read’ in each journal, as well as the number of citations of those citable items, were extracted from JCR. A hybrid Gold open access citation advantage was computed by dividing the number of citations per citable item with hybrid Gold open access by the number of citations per citable item with a subscription. A total of 498, 636, 1009, and 1328 hybrid journals in the 2018 JCR, 2019 JCR, 2020 JCR, and 2021 JCR, respectively, were included in this study. The citation advantage of hybrid Gold open access articles over subscription articles in 2018 was 1.45 (95% confidence interval (CI), 1.24–1.65); in 2019, it was 1.31 (95% CI, 1.20–1.41); in 2020, it was 1.30 (95% CI, 1.20–1.39); and in 2021, it was 1.31 (95% CI, 1.20–1.42). In the ‘Clinical Medicine’ discipline, the articles published in the hybrid journal as hybrid Gold open access had a greater number of citations when compared to those published as a subscription, self-archived, or otherwise openly accessible option.

Eliminating Sleep Health Disparities and Achieving Health Equity: Seven Areas for Action in the Behavioral Sleep Medicine Community

Racial and ethnically minoritized and under-resourced populations do not reap the same benefits of sufficient sleep as their white counterparts resulting in insufficient sleep and sleep health disparities. Research exploring these disparities have documented a plethora of factors including social determinants of health, community violence, and structural issues - all of which are associated with adverse sleep. There are robust evidence base behavioral intervention that can be leveraged to improve sleep health among racial and ethnic groups. However, EBIs are not well leveraged. In 2021, with participation from members of the society of behavioral sleep medicine, we conducted this report to bring together the field of behavioral sleep medicine including researchers, clinicians and trainees to discuss gaps and opportunities at the intersection of the COVID-19 pandemic, systemic racism, and sleep health. The goals were anchored around seven recommendations toward reducing disparities in the near-term and longer-term approaches to eliminating disparities. Furthermore, we acknowledge that reducing and eliminating disparities in sleep health requires a multifaceted approach that includes a focus on individual, community, health care and societal levels of influence with participation from diverse partners including federal, state and local.

What is the societal impact of university research? A policy-oriented review to map approaches, identify monitoring methods and success factors

The discussion on the 'societal impact of university research' has emerged within the recent decade with different utilisation from the British REF impact stories to the EU Framework Programmes' criteria. The fundamental idea is that (university) research should contribute to addressing societal challenges, such as climate change, and not just pursue 'scientific excellence'. Several approaches have been developed, from case studies of the 'productive interactions' that co-create knowledge with societal stakeholders to the so-called 'altmetrics' tracking online dissemination of research outputs. Nonetheless, these experiences do not always point in the same direction, and the notion of 'societal impact' seems still unclearly defined. Given the growing policy pressure on universities to address societal challenges, this paper reviews the scientific literature on the so-called 'societal impact' of university research. By querying three scientific archives (Web of Science, Scopus and ScienceDirect), 135 relevant scientific publications are selected and discussed, looking for conceptualisations, monitoring methods and success factors. The review maps a value-laden discussion arguing for longer-term, multi-dimensional perspectives on university research impacts beyond bibliometric indicators. Findings highlight a progressive shift from 'attribution', i.e., looking for causal relationships between research and societal changes, to 'contribution', acknowledging researchers' efforts to engage with societal challenges.

Beyond Citation Metrics: A Pilot Scoping Review to Rapidly Characterize Article Impact

BACKGROUND

Systematic reviews and meta-analyses (SRMAs) support evidence-based practice in laboratory medicine. However, there is little in the literature to address the challenge of globally characterizing the impact of SRMA findings and recommendations. We propose that characterization of citing studies through the structured methods of scoping review provides a meaningful window.

CONTENT

A scoping review method characterized the impact of a test utilization SRMA's findings and recommendations through use of a 17-item structured charting template. Following charting, we used descriptive content analysis to identify patterns of utilization, while incorporating current conceptual thinking on healthcare quality improvement.

SUMMARY

Rapid scoping reviews support descriptive characterization of SRMA impact and linkage of studies to evolving concepts in the quality improvement literature. The approach may be especially appealing to communities of practice, e.g., to support reflective practice. This pilot suggests that primary studies that have cited the 2018 SRMA have utilized some, but not all, aspects of the 2018 SRMA.

Applying the CDC Science Impact Framework to the results of the National Institute for Occupational Safety and Health and the Bureau of Labor Statistics 2001 survey of respirator use and practices

During 2001-2002, the National Institute for Occupational Safety and Health (NIOSH), at the United States Centers for Disease Control and Prevention, collaborated with the Bureau of Labor Statistics (BLS) at the United States Department of Labor to conduct a voluntary survey of U.S. employers regarding the use of respiratory protective devices. In 2003, the survey results were jointly published by NIOSH and BLS. This study highlights and evaluates the scientific impact of the 2001-2002 survey by using the Science Impact Framework which provides a historical tracking method with five domains of influence. The authors conducted interviews with original project management as well as a thorough document review and qualitative content analysis of published papers, books, presentations, and other relevant print media. A semi-structured and cross-vetted coding was applied across the five domains: Disseminating Science, Creating Awareness, Catalyzing Action, Effecting Change, and Shaping the Future. The 2001-2002 survey findings greatly enhanced understanding and awareness of respirator use in occupational settings within the United States. It also led to similar surveys in other countries, regulatory initiatives by the Occupational Safety and Health Administration and Mine Safety and Health Administration, and ultimately to a renewed partnership between NIOSH and BLS to collect contemporary estimates of respirator use in the workplace within the United States.