Social Media and Academic Impact: Do Early Tweets Correlate with Future Citations?

OBJECTIVE

This study aims to determine whether social media platforms (SMPs) can influence article impact as measured by citations.

METHODS

This is a cross-sectional study that analyzed articles published in the top ten otolaryngology journals by the Eigenfactor score in January 2015. Total accumulated twitter mentions and citations were recorded in 2021. The main outcomes examined the difference in citations, tweets, article types, and author counts accumulated over a 5-year period for all articles that were either tweeted or non-tweeted.

RESULTS

3094 articles were included for analysis. The average article was cited 11.2 ± 13.2 times and tweeted 2.10 ± 4.0 times. Sixty-four percent of the articles had at least one tweet. Over the study period, there was a statistically significant difference in mean number of citations between tweeted articles (12.1 ± 15.0) vs non-tweeted articles (9.6 ± 10.5) citations, representing a 26% difference (P < .001). Review articles had the highest mean citations (19.4 ± 23.4), while editorials had the lowest mean citations (2.8 ± 6.9). Tweets peaked in the year of publication, but citations continued to rise in the subsequent years. Tweeted articles' peak citation rate change was +1.27 mean citations per year, compared to +.99 mean citations per year in non-tweeted articles. The mean author count in tweeted articles (5.40 ± 3.1) was not significantly different than mean author count in non-tweeted articles (5.19 ± 2.65, P = .0794).

CONCLUSION

These data suggest a moderate correlation between tweets and article citations, but a clear difference in the number of citations in articles tweeted vs those with no tweets. Thus, dissemination of knowledge may be impacted by social media platforms such as Twitter.

Corrigendum: Corrigendum: Evaluative altmetrics: is there evidence for its application to research evaluation?

[This corrects the article DOI: 10.3389/frma.2023.1295959.].

Corrigendum: Evaluative altmetrics: is there evidence for its application to research evaluation?

[This corrects the article DOI: 10.3389/frma.2023.1188131.].

Evaluation of the correlation between Altmetric attention score and citation number of top 50 articles in hip fractures: a cross-sectional study

Background

Publications quality evaluation gets more attention nowadays, because of its impact on researchers ranking and academic journals. Beside traditional bibliometric tools, altemetric metrics have been introduced as tools to evaluate the dissemination of a study by the number of views, mentions, and posts on different websites and social medias.

Method

In this study, the authors evaluate the correlation between citation number as a traditional tool and altmetric attention score (AAS) as a new method. Scopus database was searched to find the 50 most cited manuscripts on "hip fractures" title from January 2015 to December 2020. After excluding irrelevant subjects, AAS of included articles was collected from the Altmetric.com website. At the last stage, the data were analyzed using statistical tests.

Results

According to statistical analysis, R 2 was 0.121, and the P-value was 0.017, which shows a weak but statistically significant relationship between citation and AAS. The relationship between the number of mentions on Twitter and the AAS was linear.The differences observed between the two groups were significant only in "Readers on Mendeley" and "Dimensions". Results shown that the impact factor of the journal and the AAS of articles had no significant relationship (R 2=0.001, P-value=0.986).

Conclusion

Findings showed that social media does not seem to be ineffective in disseminating published articles. It has also been shown that Twitter can play a significant role in the propagation of articles on social networks. It is not unreasonable to say that the accessibility of a journal affects the dissemination of an article on social media. In the end, the authors found that the impact factor of the journal could not significantly affect the AAS.

Post or perish? Social media strategies for disseminating orthopedic research

Social media usage, particularly Twitter, among scientists in academia has increased in recent years. However, Twitter's use in scholarly post-publication dissemination of orthopaedic research and musculoskeletal advocacy remains low. To enhance usage of Twitter among musculoskeletal researchers, this article reviews data supporting the professional benefits of using the platform to disseminate scholarly works. Next, we provide a linear workflow for Tweet curation, discuss the importance of data-driven decision making behind tweet curation and posting, and propose new guidelines for professional Twitter usage. Since this workflow may not eliminate all the identified barriers and new institutionalized shifts in policies regarding curation and consumption of social media on Twitter, we also briefly introduce and explore using other social media platforms. We hope this information will be persuasive and compelling to those in the orthopedic research field and be broadly applicable to others in related scientific fields who wish to disseminate findings and engage a public audience on social media. In addition, we encourage the Orthopedic Research Society (ORS) and Journal of Orthopedic Research (JOR) communities to take advantage of the many tools curated by the Wiley editorial office and the ORS social media committee to increase dissemination of their scholarly works online. Twitter and social media can assist in accomplishing our mission of creating a world without musculoskeletal limitations via the timely dissemination of orthopedic information. However, this can only be accomplished if the orthopedic research community has a unified and strong online presence actively engaged in orthopaedic research findings and news.

Evaluative altmetrics: is there evidence for its application to research evaluation?

Introduction

Altmetrics have been demonstrated as a promising tool for analyzing scientific communication on social media. Nevertheless, its application for research evaluation remains underdeveloped, despite the advancement of research in the study of diverse scientific interactions.

Methods

This paper develops a method for applying altmetrics in the evaluation of researchers, focusing on a case study of the Environment/Ecology ESI field publications by researchers at the University of Granada. We considered Twitter as a mirror of social attention, news outlets as media, and Wikipedia as educational, exploring mentions from these three sources and the associated actors in their respective media, contextualizing them using various metrics.

Results

Our analysis evaluated different dimensions such as the type of audience, local attention, engagement generated around the mention, and the profile of the actor. Our methodology effectively provided dashboards that gave a comprehensive view of the different instances of social attention at the author level.

Discussion

The use of altmetrics for research evaluation presents significant potential, as shown by our case study. While this is a novel method, our results suggest that altmetrics could provide valuable insights into the social attention that researchers garner. This can be an important tool for research evaluation, expanding our understanding beyond traditional metrics.

Global COVID-19 Policy Engagement With Scientific Research Information: Altmetric Data Study

BACKGROUND

Previous studies on COVID-19 scholarly articles have primarily focused on bibliometric characteristics, neglecting the identification of institutional actors that cite recent scientific contributions related to COVID-19 in the policy domain, and their locations.

OBJECTIVE

The purpose of this study was to assess the online citation network and knowledge structure of COVID-19 research across policy domains over 2 years from January 2020 to January 2022, with a particular emphasis on geographical frequency. Two research questions were addressed. The first question was related to who has been the most active in policy engagement with science and research information sharing during the COVID-19 pandemic, particularly in terms of countries and organization types. The second question was related to whether there are significant differences in the types of coronavirus research shared among countries and continents.

METHODS

The Altmetric database was used to collect policy report citations of scientific articles for 3 topic terms (COVID-19, COVID-19 vaccine, and COVID-19 variants). Altmetric provides the URLs of policy agencies that have cited COVID-19 research. The scientific articles used for Altmetric citations are extracted from journals indexed by PubMed. The numbers of COVID-19, COVID-19 vaccine, and COVID-19 variant research outputs between January 1, 2020, and January 31, 2022, were 216,787, 16,748, and 2777, respectively. The study examined the frequency of citations based on policy institutional domains, such as intergovernmental organizations, national and domestic governmental organizations, and nongovernmental organizations (think tanks and academic institutions).

RESULTS

The World Health Organization (WHO) stood out as the most notable institution citing COVID-19-related research outputs. The WHO actively sought and disseminated information regarding the COVID-19 pandemic. The COVID-19 vaccine citation network exhibited the most extensive connections in terms of degree centrality, 2-local eigenvector centrality, and eigenvector centrality among the 3 key terms. The Netherlands, the United States, the United Kingdom, and Australia were the countries that sought and shared the most information on COVID-19 vaccines, likely due to their high numbers of COVID-19 cases. Developing nations, although gaining quicker access to COVID-19 vaccine information, appeared to be relatively isolated from the enriched COVID-19 pandemic content in the global network.

CONCLUSIONS

The global scientific network ecology during the COVID-19 pandemic revealed distinct types of links primarily centered around the WHO. Western countries demonstrated effective networking practices in constructing these networks. The prominent position of the key term "COVID-19 vaccine" demonstrates that nation-states align with global authority regardless of their national contexts. In summary, the citation networking practices of policy agencies have the potential to uncover the global knowledge distribution structure as a proxy for the networking strategy employed during a pandemic.

Video abstracts are associated with an increase in research reports citations, views and social attention: a cross-sectional study

Video abstracts have been proposed as a tool to disseminate research through to social networks. However, its association with metrics of research dissemination has not been adequately investigated, particularly in the field of medical research. The aim of this study was to assess the association between video abstracts and citations, views and Altmetric Attention Score (AAS) of research papers. A cross-sectional study of research reports published in the New England Journal of Medicine (NEJM) over a 3-year period was conducted. An inverse binomial regression was used to assess factors associated with citations, views and AAS. The model included the presence of video abstracts as well as other independent covariables as potential confounding factors. 500 research reports were included in the analysis and 152 benefited from a video abstract. The median time from publication was 3.0 (2.2 to 3.6) years and 72% were RCTs. Research reports published with a video abstract were associated with an increase in citations (IRR 1.15), although this estimate came with uncertainty ranging from virtually no effect to a worthwhile effect (95% CI 0.98 to 1.35). There were also associated with a worthwhile increase in views (IRR 1.35, 95% CI 1.18 to 1.54) as well as with an increase in AAS (IRR 1.25, 95% CI 1.08 to 1.44). To conclude, video abstracts are associated with a worthwhile increase in the number of views of research reports. They are also associated with an increase in citations and social attention, although the association may be small.

Could the altmetrics wave bring a flood of confusion for anatomists?

Altmetrics are non-traditional metrics that can capture downloads, social media shares, and other modern measures of research impact and reach. Despite most of the altmetrics literature focusing on evaluating the relationship between research outputs and academic impact/influence, the perceived and actual value of altmetrics among academicians remains nebulous and inconsistent. This work proposes that ambiguities surrounding the value and use of altmetrics may be explained by a multiplicity of altmetrics definitions communicated by journal publishers. A root cause analysis was initiated to compare altmetrics definitions between anatomy and medical education journal publishers' websites and to determine the comparability of the measurement and platform sources used for computing altmetrics values. A scoping content analysis of data from across eight publishers' websites revealed wide variability in definitions and heterogeneity among altmetrics measurement sources. The incongruencies among publishers' altmetrics definitions and their value demonstrate that publishers may be one of the root cause of ambiguity perpetuating confusion around the value and use of altmetrics. This review highlights the need to more deeply explore the root causes of altmetrics ambiguities within academia and makes a compelling argument for establishing a ubiquitous altmetrics definition that is concise, clear, and specific.

Altmetric and Bibliographic Analysis of the Most Popular Articles on Vitreoretinal Surgery Between 2010 and 2020

Purpose Altmetrics is a web-based metrics method that measures the online dissemination of an article and the interactions it receives. We aimed to perform a bibliometric and altmetrics analysis of the 100 most cited articles (T100) on vitreoretinal surgery (VRS) published between 2010 and 2020. Methods A detailed search including terms regarding VRS in the Web of Science database was conducted. Articles were examined for bibliometric data and altmetrics. Results T100 articles had citation numbers ranging from 55 to 368 and altmetrics score (AS) values ranging from 0 to 125, and there was no statistically significant correlation between them. AS was weakly correlated with publication year. A statistically weak negative correlation was also found between AS and percent of citable open access, number of years since publication (NYsP), and average citations per year (ACpY). Conclusion AS values of articles on VRS were found to be low. Apparently, this was because the issue was of a highly specific and technical nature. However, the existence of articles with a zero AS value despite high citation numbers suggests that journals and authors do not yet attach enough importance to altmetrics. Altmetric analysis is not a reliable indicator for evaluating the scientific value of an article, and it cannot be a substitute for traditional metrics but it can provide perspective on the social impact of articles.

Evaluating the impact of a CTSA program from 2008 to 2021 through bibliometrics, social network analysis, and altmetrics

Introduction

We evaluate a CTSA program hub by applying bibliometrics, social network analysis (SNA), and altmetrics and examine the changes in research productivity, citation impact, research collaboration, and CTSA-supported research topics since our pilot study in 2017.

Methods

The sampled data included North Carolina Translational and Clinical Science Institute (NC TraCS)-supported publications produced between September 2008 and March 2021. We applied measures and metrics from bibliometrics, SNA, and altmetrics to the dataset. In addition, we analyzed research topics and correlations between different metrics.

Results

1154 NC TraCS-supported publications generated over 53,560 citation counts by April 2021. The average cites per year and the relative citation ratio (RCR) mean of these publications improved from 33 and 2.26 in 2017 to 48 and 2.58 in 2021. The number of involved UNC units in the most published authors' collaboration network increased from 7 (2017) to 10 (2021). NC TraCS-supported co-authorship involved 61 NC organizations. PlumX metrics identified articles with the highest altmetrics scores. About 96% NC TraCS-supported publications have above the average SciVal Topic Prominence Percentile; the average approximate potential to translate of the included publication was 54.2%; and 177 publications addressed health disparity issues. Bibliometric measures (e.g., citation counts, RCR) and PlumX metrics (i.e., Citations, Captures, and Social-Media) are positively correlated (p < .05).

Conclusion

Bibliometrics, SNA, and altmetrics offer distinctive but related perspectives to examine CTSA research performance and longitudinal growth, especially at the individual program hub level. These perspectives can help CTSAs build program foci.

Slow, slow, quick, quick, slow: five altmetric sources observed over a decade show evolving trends, by research age, attention source maturity and open access status

The study of temporal trends in altmetrics is under-developed, and this multi-year observation study addresses some of the deficits in our understanding of altmetric behaviour over time. The attention surrounding research outputs, as partially captured by altmetrics, or alternative metrics, constitutes many varied forms of data. Over the years 2008–2013, a set of 7739 papers were sampled on six occasions. Five altmetric data sources were recorded (Twitter, Mendeley, News, Blogs and Policy) and analysed for temporal trends, with particular attention being paid to their Open Access status and discipline. Twitter attention both starts and ends quickly. Mendeley readers accumulate quickly, and continue to grow over the following years. News and blog attention is quick to start, although news attention persists over a longer timeframe. Citations in policy documents are slow to start, and are observed to be growing over a decade after publication. Over time, growth in Twitter activity is confirmed, alongside an apparent decline in blogging attention. Mendeley usage is observed to grow, but shows signs of recent decline. Policy attention is identified as the slowest form of impact studied by altmetrics, and one that strongly favours the Humanities and Social Sciences. The Open Access Altmetrics Advantage is seen to emerge and evolve over time, with each attention source showing different trends. The existence of late-emergent attention in all attention sources is confirmed.

Evaluation of the i10-Index in Plastic Surgery Research and its Correlation with Altmetric Attention Scores and Traditional Author Bibliometrics: An Evaluation of a Single Journal

Background  Although the Hirsch index (H-index) has become one of the most accepted measures of scholarly output, its limitations have led to the proposition of newer alternative metrics. The i10-index, notable for being easy to calculate and free to access, has potential, given its association with the power and ubiquity of Google. This study aims to evaluate the utility of the i10-index for plastic surgery research by examining its relationship with author bibliometrics and article metrics, including the H-index and Altmetric Attention Score (AAS). Methods  Article metrics were extracted from articles published in the highest impact plastic surgery journal, Plastic and Reconstructive Surgery, over a 2-year period (2017-2019). Senior author bibliometrics, including i10-index and H5-index, were obtained from Web of Science. Correlation analysis was performed using Spearman's rank correlation coefficient (r s ). Results  A total of 1,668 articles were published and 971 included. Senior author i10-index measurements demonstrated moderate correlation with times emailed (r s  = 0.47), and weak correlations with H5-index, total publications, and sum of times cited with and without self-citations. The H5-index correlated very strongly with total publications (r s  = 0.91) and sum of times cited (both r s  = 0.97), moderately with average citations per item (r s  = 0.66) and times emailed (r s  = 0.41), and weakly with number of citations by posts, AAS, and times tweeted. Conclusions  Although the i10 strongly correlates with the H5-index, it fails to prove superior to the H5-index in predicting the impact of specific research studies in the field of plastic surgery.

Big splashes & ripple effects: a narrative review of the short- & long-term impact of publications supported by an NIH CTSA pediatrics program

BACKGROUND AND OBJECTIVE

This review examines a promising new framework for analyzing outputs of pediatric research in the context of translational advancement. We demonstrate a method for evaluating the impact of an NIH Clinical and Translational Science Award's (CTSA) Pediatrics Program through publications that have emerged from supported research. The Georgia CTSA Pediatrics Program provides training, funding, and infrastructure to ensure that researchers have the resources to advance pediatric health. Internal evaluations found that research supported by this program is exceptionally impactful within the academic community and commands high interest within the lay community. Therefore, we examined the impact of this research in both traditional academic and broader community spheres using bibliometrics-the study of supported publications. Bibliometrics describe a pivotal stage in the translational process of bringing scientific discoveries to clinical/community use and include both academic citations and 'altmetric' or non-academic attention. These complementary approaches combine to shed light on the short- and long-term impact of the research on segments of the translational pipeline, including academic literature, community discourse, technological advancement, and public health policy.

METHODS

The authors identified a portfolio of 250 articles supported by the Georgia CTSA Pediatrics Program from 2007-2020. We utilized various bibliometrics to analyze both short-term attention, or 'splash' made by articles, and long-term influence, or 'ripples' made across both academic and public spheres.

KEY CONTENT AND FINDINGS

The short-term splash of the portfolio was indicated through publication in high-impact factor journals, peer faculty recommendations, and Mendeley readership, as well as by early altmetric attention in news stories, blogs, and Twitter posts. The portfolio's long-term ripples were demonstrated by high absolute and relative rates of academic citation and by downstream altmetric influence in public-facing documents, including Wikipedia articles, patent applications, and policy documents.

CONCLUSIONS

This article reviews a useful bibliometric methodology for illustrating the waves of impact made by pediatric research. Whereas splash provides a picture of early interest in a publication, a preliminary indicator of eventual utility and impact, ripples provide a measure of the cumulative influence of an article over time. Both reflect opportunities for a line of research to advance along the translational spectrum.

Integrating digital scholarship in dental education: Why, what, and how?

The aim of this perspective paper is to highlight the potential role and value digital scholarship can have in dental education. The use of digital scholarship and alternative metrics for academic assessment, promotion, and tenure is growing rapidly among healthcare scholars and can complement traditional frameworks. The US dental academic institutions might consider expanding the scope of their promotion and tenure guidelines from reliance on publications and grants to a greater appreciation of the impact of the scholar's work in the digital sphere by integrating newer citation indices and altmetrics.

Information and Scientific Impact of Advanced Therapies in the Age of Mass Media: Altmetrics-Based Analysis of Tissue Engineering

BACKGROUND

Tissue engineering (TE) constitutes a multidisciplinary field aiming to construct artificial tissues to regenerate end-stage organs. Its development has taken place since the last decade of the 20th century, entailing a clinical revolution. TE research groups have worked and shared relevant information in the mass media era. Thus, it would be interesting to study the online dimension of TE research and to compare it with traditional measures of scientific impact.

OBJECTIVE

The objective of this study was to evaluate the online dimension of TE documents from 2012 to 2018 using metadata obtained from the Web of Science (WoS) and Altmetric and to develop a prediction equation for the impact of TE documents from altmetric scores.

METHODS

We analyzed 10,112 TE documents through descriptive and statistical methods. First, the TE temporal evolution was exposed for WoS and 15 online platforms (news, blogs, policy, Twitter, patents, peer review, Weibo, Facebook, Wikipedia, Google, Reddit, F1000, Q&A, video, and Mendeley Readers). The 10 most cited TE original articles were ranked according to the normalized WoS citations and the normalized Altmetric Attention Score. Second, to better comprehend the TE online framework, correlation and factor analyses were performed based on the suitable results previously obtained for the Bartlett sphericity and Kaiser-Meyer-Olkin tests. Finally, the linear regression model was applied to elucidate the relation between academics and online media and to construct a prediction equation for TE from altmetrics data.

RESULTS

TE dynamic shows an upward trend in WoS citations, Twitter, Mendeley Readers, and Altmetric Scores. However, WoS and Altmetric rankings for the most cited documents clearly differ. When compared, the best correlation results were obtained for Mendeley Readers and WoS (ρ=0.71). In addition, the factor analysis identified 6 factors that could explain the previously observed differences between academic institutions and the online platforms evaluated. At this point, the mathematical model constructed is able to predict and explain more than 40% of TE WoS citations from Altmetric scores.

CONCLUSIONS

Scientific information related to the construction of bioartificial tissues increasingly reaches society through different online media. Because the focus of TE research importantly differs when the academic institutions and online platforms are compared, basic and clinical research groups, academic institutions, and health politicians should make a coordinated effort toward the design and implementation of adequate strategies for information diffusion and population health education.

Association Between Twitter Mentions and Academic Citations in Otolaryngology Literature

OBJECTIVE

To evaluate the relationship between Twitter mentions and academic citations in otolaryngology literature.

STUDY DESIGN

Retrospective cross-sectional analysis.

SETTING

Altmetric Twitter mention and Google Scholar citation rosters.

METHODS

Original research articles from 10 leading otolaryngology journals were assessed 26 months after print publication. Article Twitter mentions were tracked through the Altmetric Bookmarklet, and article citation data were determined through the Google Scholar search engine. Twitter mentions and citation metrics of articles were compared through 2-tailed t test analysis (P < .05).

RESULTS

Of all articles, 50.7% (152/300) had at least 1 Twitter mention. Of all article Twitter mentions, 25% (432/1758) happened within the first week of online publication dates, while 64% (1130/1758) occurred between online and print publication dates. Articles mentioned on Twitter had 1.6-fold more Google Scholar citations (8.6 ± 0.7, mean ± SD) than articles with no Twitter mentions (5.4 ± 0.4, P < .01). A total of 8% (24/300) of publications were tweeted by their authors. Articles self-tweeted by authors were associated with an 8.4-citation increase (14.8 ± 3.1) for Google Scholar when compared with articles not shared by their authors on Twitter (6.4 ± 0.4; 2.3-fold increase, P < .01).

CONCLUSION

Most otolaryngology articles are disseminated over Twitter, with greatest Twitter activity occurring before print publication date of articles. Citations within 2 years of release are positively associated with the number of mentions on Twitter. Article Twitter mentions may augment the academic influence of otolaryngology publications.

Six Sigma in Health Literature, What Matters?

Six Sigma has been widely used in the health field for process or quality improvement, constituting a quite profusely investigated topic. This paper aims at exploring why some studies have more academic and societal impact, attracting more attention from academics and health professionals. Academic and societal impact was addressed using traditional academic metrics and alternative metrics, often known as altmetrics. We conducted a systematic search following the PRISMA statement through three well-known databases, and identified 212 papers published during 1998–2019. We conducted zero-inflated negative binomial regressions to explore the influence of bibliometric and content determinants on traditional academic and alternative metrics. We observe that the factors influencing alternative metrics are more varied and difficult to apprehend than those explaining traditional impact metrics. We also conclude that, independently of how the impact is measured, the paper’s content, rather than bibliometric characteristics, better explains its impact. In the specific case of research on Six Sigma applied to health, the papers with more impact address process improvement focusing on time and waste reduction. This study sheds light on the aspects that better explain publications’ impact in the field of Six Sigma application in health, either from an academic or a societal point of view.

Social Media as a Means to Disseminate and Advocate Cardiovascular Research: Why, How, and Best Practices

The use of Twitter or other social media platforms for research can transcend current limitations in the dissemination of research and present new opportunities for research networking by connecting researchers, clinicians, policymakers, the public, and other stakeholders on a global scale. As social media influence continues to expand, it becomes increasingly important for cardiovascular researchers to employ social media strategies to increase the impact of their research work, for the ultimate goal of improved outcomes for patients living with or at risk for cardiovascular diseases. Altmetrics are novel metrics that track the attention that scholarly outputs are receiving in non-traditional sources such as in the news, blogs, and social media posts. These alternative metrics record research dissemination beyond traditional journal citation indices, while also predicting and even promoting future citations. This review outlines various methods of how social media can be used to disseminate research, guidance on how to develop a social media portfolio for consideration of academic promotion, and some best practices for promoting one’s research work.

Is rapid scientific publication also high quality? Bibliometric analysis of highly disseminated COVID-19 research papers

The impact of COVID-19 has underlined the need for reliable information to guide clinical practice and policy. This urgency has to be balanced against disruption to journal handling capacity and the continued need to ensure scientific rigour. We examined the reporting quality of highly disseminated COVID-19 research papers using a bibliometric analysis examining reporting quality and risk of bias (RoB) amongst 250 top scoring Altmetric Attention Score (AAS) COVID-19 research papers between January and April 2020. Method-specific RoB tools were used to assess quality. After exclusions, 84 studies from 44 journals were included. Forty-three (51%) were case series/studies, and only one was an randomized controlled trial. Most authors were from institutions based in China (n = 44, 52%). The median AAS and impact factor was 2015 (interquartile range [IQR] 1,105-4,051.5) and 12.8 (IQR 5-44.2) respectively. Nine studies (11%) utilized a formal reporting framework, 62 (74%) included a funding statement, and 41 (49%) were at high RoB. This review of the most widely disseminated COVID-19 studies highlights a preponderance of low-quality case series with few research papers adhering to good standards of reporting. It emphasizes the need for cautious interpretation of research and the increasingly vital responsibility that journals have in ensuring high-quality publications.

Tracking trends of transgender health research online: are researchers and the public on the same page?

Altmetric Explorer was searched for the most popular online articles published in Pubmed-indexed journals. The 75 articles with the highest Altmetric Attention Score (AAS) were screened for article information (date, journal, access), authorship (number of authors, affiliation and origin of the corresponding author), and research (type, subject, funding). The reviewed articles displayed a mean AAS of 241.52, were broadcast 17.03 times by news agencies, posted on Twitter 101.47 times, downloaded by 67.21 Mendeley readers, and received 62.67 citations. There was intense online interest in the transgender health literature, mainly related to mental health and social well-being. Online visibility of transgender health articles was not significantly correlated with citation counts, implying that the public, likely including transgender persons and allies, may place emphasis on different health issues than scholars. Monitoring altmetrics and interactions on electronic media may help researchers conduct research that is more meaningful to transgender individuals, and to society in general.

Evaluating Scholars' Impact and Influence: Cross-sectional Study of the Correlation Between a Novel Social Media-Based Score and an Author-Level Citation Metric

BACKGROUND

The development of an author-level complementary metric could play a role in the process of academic promotion through objective evaluation of scholars' influence and impact.

OBJECTIVE

The objective of this study was to evaluate the correlation between the Healthcare Social Graph (HSG) score, a novel social media influence and impact metric, and the h-index, a traditional author-level metric.

METHODS

This was a cross-sectional study of health care stakeholders with a social media presence randomly sampled from the Symplur database in May 2020. We performed stratified random sampling to obtain a representative sample with all strata of HSG scores. We manually queried the h-index in two reference-based databases (Scopus and Google Scholar). Continuous features (HSG score and h-index) from the included profiles were summarized as the median and IQR. We calculated the Spearman correlation coefficients (ρ) to evaluate the correlation between the HSG scores and h-indexes obtained from Google Scholar and Scopus.

RESULTS

A total of 286 (31.2%) of the 917 stakeholders had a Google Scholar h-index available. The median HSG score for these profiles was 61.1 (IQR 48.2), and the median h-index was 14.5 (IQR 26.0). For the 286 subjects with the HSG score and Google Scholar h-index available, the Spearman correlation coefficient ρ was 0.1979 (P<.001), indicating a weak positive correlation between these two metrics. A total of 715 (78%) of 917 stakeholders had a Scopus h-index available. The median HSG score for these profiles was 57.6 (IQR 46.4), and the median h-index was 7 (IQR 16). For the 715 subjects with the HSG score and Scopus h-index available, ρ was 0.2173 (P<.001), also indicating a weak positive correlation.

CONCLUSIONS

We found a weak positive correlation between a novel author-level complementary metric and the h-index. More than a chiasm between traditional citation metrics and novel social media-based metrics, our findings point toward a bridge between the two domains.

Questioning the Impact of the Impact Factor. A Brief Review and Future Directions

Purpose: To provide a brief review of literature on the journal impact factors (JIF) and the newer research metrics being proposed or implemented.Methods: The authors performed a PubMed search of articles published in the English language on the journal impact factors. Data captured include historical perspectives, evolution, calculation, criticisms of JIF and their rebuttals, and organized efforts to address JIF issues, alternate research metrics, and future directions. Specific emphasis was laid on evaluating the criticisms, current lacunae, and the changing practice patterns.Results: One of the measures to assess the research impact of an article is the number of citations it receives. Hence, citation-based metrics are commonly used for such purposes. While editors and well-known scholars refrain from attributing article success to the journal's prominence, the same is not true for most authors. JIF is still one of the top factors when deciding on an article submission. JIF is today an acceptable objective and quantifiable measure of knowledge dissemination. However, JIF should not be used as a surrogate measure to assess an individual researcher or an individual article. The reverence to JIF in this regard needs to be questioned. While alternate metrics or altmetrics have their advantages and limitations, they nevertheless augur well an era where scientometrics are complementary to one another without undue reliance on a sole parameter.Conclusion: While there is no need to demonize the JIF, its role in the scholarly assessment should be scaled down. The over-reliance and undue hype surrounding it should be discouraged at multiple scientific levels.

The Most Widely Disseminated COVID-19-Related Scientific Publications in Online Media: A Bibliometric Analysis of the Top 100 Articles with the Highest Altmetric Attention Scores

The novel coronavirus disease 2019 (COVID-19) is a global pandemic. This study's aim was to identify and characterize the top 100 COVID-19-related scientific publications, which had received the highest Altmetric Attention Scores (AASs). Hence, we searched Altmetric Explorer using search terms such as "COVID" or "COVID-19" or "Coronavirus" or "SARS-CoV-2" or "nCoV" and then selected the top 100 articles with the highest AASs. For each article identified, we extracted the following information: the overall AAS, publishing journal, journal impact factor (IF), date of publication, language, country of origin, document type, main topic, and accessibility. The top 100 articles most frequently were published in journals with high (>10.0) IF (n = 67), were published between March and July 2020 (n = 67), were written in English (n = 100), originated in the United States (n = 45), were original articles (n = 59), dealt with treatment and clinical manifestations (n = 33), and had open access (n = 98). Our study provides important information pertaining to the dissemination of scientific knowledge about COVID-19 in online media.

Multidimensional indicators of scholarly impact in the skin oncology literature: is there a correlation between bibliometric and altmetric profiles?

Introduction: Bibliometric and altmetric analyses are used to identify landmark publications in their respective research field. We hypothesised that highly cited skin oncology articles correlate positively with the Oxford Evidence Based Medicine scoring level, altmetric score (AS) and rank within the top 100 manuscripts.Methods: Thomson Reuter's Web of Science citation indexing database was searched to identify all English-language skin oncology full-text articles in the last 75 years. The top 100 articles with the highest citation count were analysed by subject matter, publishing journal, author, year, institution, individual and five-year impact factor, AS and Oxford EBM level. Results: 180,132 articles were identified. The most cited article (Hodi et al.) demonstrated improved survival with ipilimumab in patients with metastatic melanoma (7894 citations). The article with the highest AS was Esteva et al. (AS = 576.7, 'dermatologist-level classification of skin cancer with deep neural networks'). No difference was found between evidence level and citation count (r = -0.1239, p = 0.2291), but a significant difference was seen for AS (r = -0.3024, p = 0.0028). AS scores increased over time, whereas bibliometrics did not. Conclusion: This work highlights the most influential work in the skin oncology field in the last 75 years. We have identified a differential relationship between commonly used metrics and evidence level in the field of skin oncology. As the digitalisation of research output and consumption increases, both bibliometric and altmetric analyses need to be considered when an article's impact is being assessed.

Characteristics of the Most Cited, Most Downloaded, and Most Mentioned Articles in General Medical Journals: A Comparative Bibliometric Analysis

We compared the characteristics of the most cited, most downloaded, and most mentioned (the highest Altmetric Attention Score) articles published in general medical journals. We identified the 640 most frequently cited, 662 most frequently downloaded, and 652 most mentioned articles from 48 general medical journals. A comparison was made of the following characteristics of articles in the most cited, most downloaded, and most mentioned articles: medical specialty, publication type, country of origin, year of publication, and accessibility. There was only a 2.5% overlap in these three groups. Original articles were the more frequent among the most mentioned articles, whereas reviews, case reports, and guidelines/consensus statements were more frequent among the most downloaded articles. The most cited articles were more frequently published in 2010 and before, whereas the most downloaded articles were published in 2017−2018. The most mentioned articles were more frequently open-access articles, compared to the most downloaded articles. The most cited were more frequently older, the most downloaded were more frequently recent and educational, and the most mentioned were more frequently original and open-access articles. The results of our study may provide insights into various measures of article impact.

Articles with impact: insights into 10 years of research with machine learning

Worldwide scientific output is growing faster and faster. Academics should not only publish much and fast, but also publish research with impact. The aim of this study is to use machine learning to investigate characteristics of articles that were published in the Journal of Applied Physiology between 2009 and 2018, and characterize high-impact articles. Article impact was assessed for 4,531 publications by three common impact metrics: the Altmetric Attention Scores, downloads, and citations. Additionally, a broad collection of (more than 200) characteristics was collected from the article's title, abstract, authors, keywords, publication, and article engagement. We constructed random forest (RF) regression models to predict article impact and articles with the highest impact (top-25% and top-10% for each impact metric), which were compared with a naive baseline method. RF models outperformed the baseline models when predicting the impact of unseen articles (P < 0.001 for each impact metric). Also, RF models predicted top-25% and top-10% high-impact articles with a high accuracy. Moreover, RF models revealed important article characteristics. Higher impact was observed for articles about exercise, training, performance and V̇o_2max, reviews, human studies, articles from large collaborations, longer articles with many references and high engagement by scientists, practitioners and public or via news outlets and videos. Lower impact was shown for articles about respiratory physiology or sleep apnea, editorials, animal studies, and titles with a question mark or a reference to places or individuals. In summary, research impact can be predicted and better understood using a combination of article characteristics and machine learning.NEW & NOTEWORTHY Common measures of article impact are the Altmetric Attention Scores, number of downloads, and number of citations. To our knowledge, this is the first study that applies machine learning on a comprehensive collection of article characteristics to predict article attention scores, downloads, and citations. Using 10 years of research articles, we obtained accurate predictions of high-impact articles and discovered important article characteristics related to article impact.

Top-cited articles versus top Altmetric articles in nuclear medicine: a comparative bibliometric analysis

BACKGROUND

The impact of any scientific article has traditionally been measured by the number of citations received. More recently, alternative metrics (altmetrics) reflect the digital dissemination of knowledge across the online mediasphere.

PURPOSE

To evaluate and compare the characteristics of top-cited articles with those of top Altmetric articles related to nuclear medicine (NM).

MATERIAL AND METHODS

We performed a search of the Web of Science and Altmetric databases using 114 search terms to identify the 50 top-cited and 50 top Altmetric articles, respectively, in the field of NM. We then compared the following characteristics of the selected articles: publication type; journal category; country of origin; year of publication; topic; imaging modality; and accessibility. Chi-square tests were performed for statistical analysis.

RESULTS

There were no overlaps between the 50 top-cited and 50 top Altmetric articles. In general, compared to the leading Altmetric articles in this field, the cited articles were: more frequently review works published in NM and radiology journals (76% vs. 13%, P = 0.000); published in or before 2005 (84% vs. 0%, P = 0.000); the majority were related to oncology (56% vs. 44%, P = 0.000); and originated from the Netherlands (12% vs. 0%, P = 0.000). Compared to the top-cited articles, the leading Altmetric articles were: more frequently original articles published in other clinical field journals (54% vs. 0%, P = 0.000); primarily published between 2016 and 2018 (70% vs. 0%, P = 0.000); focused on neurology (50% vs. 22%, P = 0.000); and originated from the UK (18% vs. 2%, P = 0.000).

CONCLUSION

Citation counts and Altmetric scores represent unique perspectives for evaluating the impact of NM research.

Tracing open data in emergencies: The case of the COVID-19 pandemic

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic constitutes an ongoing, burning Public Health Emergency of International Concern (PHEIC). In 2015, the World Health Organization adopted an open data policy recommendation in such situations.

OBJECTIVES

The present cross-sectional meta-research study aimed to assess the availability of open data and metrics of articles pertaining to the COVID-19 outbreak in five high-impact journals.

METHODS

All articles regarding the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), published in five high-impact journals (Ann Intern Med, BMJ, JAMA, NEJM and Lancet) until March 14, 2020 were retrieved. Metadata (namely the type of article, number of authors, number of patients, citations, errata, news and social media mentions) were extracted for each article in each journal in a systematic way. Google Scholar and Scopus were used for citations and author metrics respectively, and Altmetrics and PlumX were used for news and social media mentions retrieval. The degree of adherence to the PHEIC open data call was also evaluated.

RESULTS

A total of 140 articles were published until March 14, 2020, mostly opinion papers. Sixteen errata followed these publications. The number of authors in each article ranged from 1 to 63, whereas the number of patients with a laboratory-confirmed SARS-CoV-2 infection reached 2645. Extensive hyperauthorship was evident among case studies. The impact of these publications reached a total of 4210 cumulative crude citations and 342 790 news and social media mentions. Only one publication (0.7%) provided complete open data, while 32 (22.9%) included patient data.

CONCLUSIONS

Even though a large number of manuscripts was produced since the pandemic, availability of open data remains restricted.

The level of evidence, scientific impact and social impact of clinical studies in periodontology: A methodological study

AIMS

To analyse the level of evidence (LOE) of clinical studies in the field of periodontology, and to investigate whether LOE is a predictor of scientific impact and social impact.

MATERIALS AND METHODS

Clinical studies published in five leading periodontal journals during 2015-2019 were identified. The LOE of included studies were assessed with a modified LOE classification system based on Oxford 2009 LOE, Oxford 2011 LOE and GRADE guidelines. Citation counts were harvested from Web of Science and Scopus. Altmetric Attention Scores (AAS) were obtained from Altmetric Explorer. Multivariable generalized estimation equation (GEE) analyses were used to investigate association between LOE and citation count, as well as between LOE and AAS.

RESULTS

Among 768 studies included, the proportion of level-1, level-2, level-3 and level-4 was 10.4%, 44.8%, 13.7% and 31.1%, respectively. In the multivariable GEE analyses, high LOE was a significant predictor of higher average citation count (p = .010) and higher AAS (p < .001).

CONCLUSION

The LOE of clinical studies in the periodontal field is relatively high in general, although it varies significantly in different journals. Studies with high LOE tend to have greater scientific impact and social impact than low LOE studies.

Mapping social media attention in Microbiology: identifying main topics and actors

This paper aims to map and identify topics of interest within the field of Microbiology and identify the main sources driving such attention. We combine data from Web of Science and Altmetric.com, a platform which retrieves mentions to scientific literature from social media and other non-academic communication outlets. We focus on the dissemination of microbial publications in Twitter, news media and policy briefs. A two-mode network of social accounts shows distinctive areas of activity. We identify a cluster of papers mentioned solely by regional news media. A central area of the network is formed by papers discussed by the three outlets. A large portion of the network is driven by Twitter activity. When analyzing top actors contributing to such network, we observe that more than half of the Twitter accounts are bots, mentioning 32% of the documents in our dataset. Within news media outlets, there is a predominance of popular science outlets. With regard to policy briefs, both international and national bodies are represented. Finally, our topic analysis shows that the thematic focus of papers mentioned varies by outlet. While news media cover the wider range of topics, policy briefs are focused on translational medicine and bacterial outbreaks.

What Is the Predictive Ability and Academic Impact of the Altmetrics Score and Social Media Attention?

BACKGROUND

Citation rate and journal impact factor have traditionally been used to assess research impact; however, these may fail to represent impact beyond the sphere of academics. Given that social media is now used to disseminate research, alternative web-based metrics (altmetrics) were recently developed to better understand research impact on social media. However, the relationship between altmetrics and traditional bibliometrics in orthopaedic literature is poorly understood.

PURPOSE

To (1) assess the extent that altmetrics correlate with traditional bibliometrics and (2) identify publication characteristics that predict greater altmetrics scores.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published in The American Journal of Sports Medicine (AJSM), The Journal of Bone and Joint Surgery, Clinical Orthopaedics and Related Research, Acta Orthopaedica, and Knee Surgery, Sports Traumatology, Arthroscopy between January 2016 and December 2016 were analyzed. Among the extracted publication characteristics were journal, number of authors, geographic region of origin, highest degree of first author, study subject and design, sample size, conflicts of interest, and level of evidence; number of references, institutions, citations, tweets, Facebook mentions, and news mentions; and Altmetric Attention Score (AAS). Multivariate regressions were used to determine (1) publication characteristics predictive of AAS and social media attention (mentions on Twitter, Facebook, and the news) and (2) the relationship between AAS and citation rate.

RESULTS

A total of 496 published articles were included, with a mean AAS of 8.6 (SD, 31.7; range, 0-501) and a mean citation rate of 15.0 (SD, 16.1; range, 0-178). Articles in AJSM (β = 19.9; P < .001), publications from North America (β = 8.5; P = .033), and studies concerning measure validation/reliability (β = 25.5; P = .004) were independently associated with higher AAS. Greater AAS score significantly predicted a greater citation rate (β = 0.16; P < .0001). The citation rate was an independent predictor of greater social media attention on Twitter, Facebook, and the news (odds ratio range, 1.02-1.03; P < .05 all).

CONCLUSION

AAS had a significant positive association with citation rates of articles in 5 high-impact orthopaedic journals. Articles in AJSM, studies concerning measure validation and reliability, and publications from North America were positively associated with greater AAS. A greater number of citations was consistently associated with publication attention received on social media platforms.

Altmetric analysis of the contemporary scientific literature in Endodontology

AIM

To analyse and visualize the knowledge structure of scientific articles in the field of Endodontology with high altmetric attention scores to discover hot topics, active researchers and the journals involved.

METHODOLOGY

On 5 June 2019, the altmetric database (Altmetric LLP, London, UK) was searched using the titles of 11 endodontic journals. Bibliometric data from endodontic articles and journals with an altmetric score >5 (top 5%) were retrieved from PubMed and analysed using the VOSviewer. Science mapping of articles with an altmetric score >5 at two levels was created: author keywords co-occurrence and co-authorship network analysis.

RESULTS

Of the 2197 articles in the field of Endodontology identified with altmetrics, 192 had altmetric scores >5 (top 5%). Considering the total mentions amongst all altmetric resources, the Journal of Endodontics had the highest rank followed by the International Endodontic Journal and Australian Endodontic Journal. Twitter was the most popular altmetric data resource followed by patents and Facebook. Meta-analysis, systematic review and pulpitis were the hot topics. At the author level, Dummer P.M.H had the greatest influence on the network. There was no significant correlation between altmetric score and citations count (P > 0.05). Mendeley mentions correlated with citations (P < 0.05).

CONCLUSIONS

Overall, the altmetric scores of topics within Endodontology were low, possibly due to the specific and specialized nature of the specialty, as well as the difficulty members of the public probably have in understanding endodontic research. Journals and researchers with a focus on Endodontology would have more influence if they were to set-up their own social media profiles and thus enhance their visibility and social impact by immediately sharing research findings and communicating with their network and audience.

Research Impact Core: A Research Impact Initiative at the University of Michigan

Informationists at the Taubman Health Sciences Library, University of Michigan, formed a research impact consultation and education initiative in early 2017 to increase engagement with the health sciences community around the informed, responsible use of a range of citation and alternative metrics and associated tools. So far, the Research Impact Core has primarily entailed developing training content and cultivating partnerships related to publication metrics and associated best practices. This article reports on progress from the first two years of the Research Impact Core, including a snapshot of information session registrants, and a broader discussion of collaborative partnerships around research impact in the health sciences and library system.

Study Comparing Traditional Versus Alternative Metrics to Measure the Impact of the Critical Care Medicine Literature

Supplemental Digital Content is available in the text.

Our objective was to evaluate the association between traditional metrics such as Impact Factor and Eigenfactor with respect to alternative metrics. The Altmetric Attention Score for the top nine pulmonary and critical care journals was compared with Impact Factor, Eigenfactor, and citations over two time periods (2007–2011 and 2012–2016). There was a significant increase in the Altmetric Attention Score (52 from 2007 to 2011 vs 1,061 from 2012 to 2016; p < 0.001) but no significant differences in Total Citations, Impact Factor, or Eigenfactor. There was a strong positive correlation between citations and Altmetric Attention Score, negative correlations between Eigenfactor and Altmetric Attention Score for most journals, and no clear association between Impact Factor and Altmetric Attention Score. Over time, the digital reach of traditional publications has increased significantly, while no significant increase was noted for the traditional metrics. These findings likely reflect discussions of articles online that are not captured by traditional metrics and hence their impact on the community at large.

Measuring the social impact of nursing research: An insight into altmetrics

AIMS

The objectives of this study were to (a) identify nursing journal articles that provoked the most online activity and discussion and (b) assess the association between these articles' altmetric scores and publication characteristics, citation counts; and publishing journals metrics.

BACKGROUND

Altmetrics, or alternative metrics, have recently emerged as a complementary way of measuring the societal impact of research by assessing the public engagement with research output. To date, no studies have yet investigated the online attention about scientific papers published in the nursing field.

DESIGN

Integration of quantitative and qualitative synthesized evidence.

DATA SOURCES AND REVIEW METHODS

InCites Journal Citation Report was used to identify a list of nursing journals indexed in the Web of Science Core Collection. Altmetric Explorer was selected as an altmetrics harvesting tool. The search in Altmetric Explorer yielded 66,608 research outputs from 118 nursing journals. The articles with the top 100 altmetric attention score (AAS) were identified and a new search, limited to only those 100 articles, was run to produce aggregate metrics specific to those articles. The articles were also exported for thematic analysis.

RESULTS

The median AAS for the 100 articles was 248, ranging from 138 - 649. The articles were mostly discussed on Twitter, followed by news outlets and Mendeley. Articles indexed in the nursing journals category attracted low online attention compared with articles published in other health journal categories. Twitter remained the dominant source of attention over the years 2012-2018, followed distantly by news outlets. Most online attention came from the USA and the UK. Of the top 100 articles included in the study, the Journal of Advanced Nursing published the highest number of articles (N = 26; Median AAS = 179). The AAS was not significantly different between articles published in Q1 journals and those published in Q2 and Q3 journals. There was a significant relationship between articles' AASs and their citation counts on Scopus and Web of Science. Publication date was significantly related to citation counts on Scopus and Web of Science but not with AASs.

CONCLUSION

Altmetrics will likely continue to evolve alongside the rapidly expanding use of social media and online platforms. As nursing continues to strive to have our research and scholarship inform policy, translated into practice and recognized for its scientific merit, we have to remain vigilant about the best ways to disseminate the important work we are doing. Research, such as this study, will allow nursing scholars to benchmark our progress as we adapt to the changing environment for measuring impact and quality in the digital age.

Identifying diffusion patterns of research articles on Twitter: A case study of online engagement with open access articles

The growing presence of research shared on social media, coupled with the increase in freely available research, invites us to ask whether scientific articles shared on platforms like Twitter diffuse beyond the academic community. We explore a new method for answering this question by identifying 11 articles from two open access biology journals that were shared on Twitter at least 50 times and by analyzing the follower network of users who tweeted each article. We find that diffusion patterns of scientific articles can take very different forms, even when the number of times they are tweeted is similar. Our small case study suggests that most articles are shared within single-connected communities with limited diffusion to the public. The proposed approach and indicators can serve those interested in the public understanding of science, science communication, or research evaluation to identify when research diffuses beyond insular communities.

GitHub Statistics as a Measure of the Impact of Open-Source Bioinformatics Software

Modern research is increasingly data-driven and reliant on bioinformatics software. Publication is a common way of introducing new software, but not all bioinformatics tools get published. Giving there are competing tools, it is important not merely to find the appropriate software, but have a metric for judging its usefulness. Journal's impact factor has been shown to be a poor predictor of software popularity; consequently, focusing on publications in high-impact journals limits user's choices in finding useful bioinformatics tools. Free and open source software repositories on popular code sharing platforms such as GitHub provide another venue to follow the latest bioinformatics trends. The open source component of GitHub allows users to bookmark and copy repositories that are most useful to them. This Perspective aims to demonstrate the utility of GitHub "stars," "watchers," and "forks" (GitHub statistics) as a measure of software impact. We compiled lists of impactful bioinformatics software and analyzed commonly used impact metrics and GitHub statistics of 50 genomics-oriented bioinformatics tools. We present examples of community-selected best bioinformatics resources and show that GitHub statistics are distinct from the journal's impact factor (JIF), citation counts, and alternative metrics (Altmetrics, CiteScore) in capturing the level of community attention. We suggest the use of GitHub statistics as an unbiased measure of the usability of bioinformatics software complementing the traditional impact metrics.

Bibliometrics in a digital age: help or hindrance

Bibliometrics are a range of techniques and quantitative measures that provide an analysis of written publications such as books and articles, and which assess the impact of research outputs. They are commonly applied to individual authors in the form of citation metrics but can also be used to assess the influence of research groups or even entire institutions. With the increased importance of social media as a means of communicating and publicising research findings, additional alternative measures of impact (altmetrics) are now being used. In addition to analysing the reach of a research output, bibliometrics can also be used as search tools to identify related and updated research, author networks and connections between institutions. This review summarises the range of tools and services that are available, their advantages and disadvantages, and some of the challenges and issues presented by the existence of multiple digital versions of research outputs.

A Science Products Inventory for Citizen-Science Planning and Evaluation

Citizen science involves a range of practices involving public participation in scientific knowledge production, but outcomes evaluation is complicated by the diversity of the goals and forms of citizen science. Publications and citations are not adequate metrics to describe citizen-science productivity. We address this gap by contributing a science products inventory (SPI) tool, iteratively developed through an expert panel and case studies, intended to support general-purpose planning and evaluation of citizen-science projects with respect to science productivity. The SPI includes a collection of items for tracking the production of science outputs and data practices, which are described and illustrated with examples. Several opportunities for further development of the initial inventory are highlighted, as well as potential for using the inventory as a tool to guide project management, funding, and research on citizen science.

A Science Products Inventory for Citizen-Science Planning and Evaluation

Citizen science involves a range of practices involving public participation in scientific knowledge production, but outcomes evaluation is complicated by the diversity of the goals and forms of citizen science. Publications and citations are not adequate metrics to describe citizen-science productivity. We address this gap by contributing a science products inventory (SPI) tool, iteratively developed through an expert panel and case studies, intended to support general-purpose planning and evaluation of citizen-science projects with respect to science productivity. The SPI includes a collection of items for tracking the production of science outputs and data practices, which are described and illustrated with examples. Several opportunities for further development of the initial inventory are highlighted, as well as potential for using the inventory as a tool to guide project management, funding, and research on citizen science.

Measuring the impact of pharmacoepidemiologic research using altmetrics: A case study of a CNODES drug-safety article

Purpose

To provide an overview of altmetrics, including their potential benefits and limitations, how they may be obtained, and their role in assessing pharmacoepidemiologic research impact.

Methods

Our review was informed by compiling relevant literature identified through searching multiple health research databases (PubMed, Embase, and CIHNAHL) and grey literature sources (websites, blogs, and reports). We demonstrate how pharmacoepidemiologists, in particular, may use altmetrics to understand scholarly impact and knowledge translation by providing a case study of a drug‐safety study conducted by the Canadian Network of Observational Drug Effect Studies.

Results

A common approach to measuring research impact is the use of citation‐based metrics, such as an article's citation count or a journal's impact factor. “Alternative” metrics, or altmetrics, are increasingly supported as a complementary measure of research uptake in the age of social media. Altmetrics are nontraditional indicators that capture a diverse set of traceable, online research‐related artifacts including peer‐reviewed publications and other research outputs (software, datasets, blogs, videos, posters, policy documents, presentations, social media posts, wiki entries, etc).

Conclusion

Compared with traditional citation‐based metrics, altmetrics take a more holistic view of research impact, attempting to capture the activity and engagement of both scholarly and nonscholarly communities. Despite the limited theoretical underpinnings, possible commercial influence, potential for gaming and manipulation, and numerous data quality‐related issues, altmetrics are promising as a supplement to more traditional citation‐based metrics because they can ingest and process a larger set of data points related to the flow and reach of scholarly communication from an expanded pool of stakeholders. Unlike citation‐based metrics, altmetrics are not inherently rooted in the research publication process, which includes peer review; it is unclear to what extent they should be used for research evaluation.