Alternative Metrics ("Altmetrics") for Assessing Article Impact in Popular General Radiology Journals

Which Topics Drive Dissemination? Alternative Bibliometrics Analysis of the Highest-Ranking Articles in 3 Infectious Diseases Journals Before COVID-19

Dissemination of research is paramount to improving patient care. Historically, dissemination is reported in conventional bibliometrics. However, with the increased utilization of digital platforms for communication, alternative bibliometrics describe more real-time dissemination of information. This study documents dissemination of publication topics in infectious diseases journals prior to the COVID-19 pandemic.

Post or perish? Social media strategies for disseminating orthopedic research

Social media usage, particularly Twitter, among scientists in academia has increased in recent years. However, Twitter's use in scholarly post-publication dissemination of orthopaedic research and musculoskeletal advocacy remains low. To enhance usage of Twitter among musculoskeletal researchers, this article reviews data supporting the professional benefits of using the platform to disseminate scholarly works. Next, we provide a linear workflow for Tweet curation, discuss the importance of data-driven decision making behind tweet curation and posting, and propose new guidelines for professional Twitter usage. Since this workflow may not eliminate all the identified barriers and new institutionalized shifts in policies regarding curation and consumption of social media on Twitter, we also briefly introduce and explore using other social media platforms. We hope this information will be persuasive and compelling to those in the orthopedic research field and be broadly applicable to others in related scientific fields who wish to disseminate findings and engage a public audience on social media. In addition, we encourage the Orthopedic Research Society (ORS) and Journal of Orthopedic Research (JOR) communities to take advantage of the many tools curated by the Wiley editorial office and the ORS social media committee to increase dissemination of their scholarly works online. Twitter and social media can assist in accomplishing our mission of creating a world without musculoskeletal limitations via the timely dissemination of orthopedic information. However, this can only be accomplished if the orthopedic research community has a unified and strong online presence actively engaged in orthopaedic research findings and news.

Sparking attention on African swine fever research on social media platform: An altmetric evaluation of top 100 highly cited articles

African swine fever (ASF) is one of the highly contagious diseases of pigs that affect both domestic and wild pigs. The primary purpose of this research was to evaluate the online social attention on the ASF research to inform the research scientists and key stakeholders in the field by reporting the concise information of the most influential articles, social engagement, and impacts of the research. This study employed the altmetrics tool to evaluate the research papers. Bibliographic data of 100 articles were collected from Scopus; altmetric data was collected from the Altmetric.com database and analyzed using SPSS and Tableau. The articles were mainly mentioned on Twitter, followed by News Outlets and significant readers on Mendeley. Pearson correlation coefficients revealed a weak and insignificant correlation between Scopus Citation and Altmetric Attention Score (AAS). Mendeley Readership and Scopus Citation were moderately correlated. However, there was a significant positive correlation between the AAS and Mendeley readership. Using altmetric tools, the paper is the first research to shed light on the characteristics of ASF on social media.

The growing role of social media for research and education in radiology

While the adoption of social media platforms has become commonplace for personal life use, its practice has tremendously grown for professional usage in medicine, and particularly in the field of radiology. The use of alternative metrics, or altmetrics, scores have developed in an effort to quantify the impact of research beyond traditional metrics, such as citation rate and journal impact factor, particularly to recognize the impact of social media on dissemination and promotion of research. Social media usage in the realm of radiology has expectedly been adopted for the purposes of medical education, research, networking, and advocacy. However, some platforms have been used as a medium to discuss and share the day-to-day nature in the field of radiology, burnout in radiology, as well as radiology-themed humor. The purpose of this review article was discuss the role of altmetrics, as well as the specific uses of social media platforms including Twitter, Instagram, Facebook, Youtube, and more. The role of ethics in social media practice related to radiology are discussed.

The Altmetric Attention Score Is Associated With Citation Rates and May Reflect Academic Impact in the Total Joint Arthroplasty Literature

Background: Given the increasing interest and potential use of social media for the promotion of orthopedic literature, there is a need to better understand Altmetrics. Purposes: We sought to determine the relationship between the Altmetric Attention Score (AAS) and the number of citations for articles on total joint arthroplasty (TJA) published in orthopedics journals. We also sought to determine the predictors of greater social media attention for these articles. Methods: Articles on TJA published in Bone and Joint Journal (BJJ), Journal of Bone and Joint Surgery (JBJS), Clinical Orthopedics and Related Research (CORR), Journal of Arthroplasty, Journal of Knee Surgery, Hip International, and Acta Orthopaedica in 2016 were extracted (n = 498). One-way analysis of variance with Bonferroni corrections was used to compare AAS and citations across journals. Multivariate regressions were used to determine predictors of social media attention and number of citations. Results: The mean AAS and number of citations were 7.5 (range: 0-289) and 16.7 (range: 0-156), respectively. Significant between-group effects were observed according to journal for AAS and number of citations. Publishing an article in JBJS was the strongest predictor of higher number of citations. Publishing an article in BJJ was the only independent predictor of higher AAS, while publishing an article in JBJS or CORR trended toward statistical significance. A higher AAS was a significant predictor of a higher number of citations. Number of citations and number of study references were positive predictors of greater social media attention on Twitter and Facebook. Conclusions: In articles on TJA published in 7 journals in 2016, a higher AAS was a associated with a higher number of citations. Various bibliometric characteristics were found to be significantly associated with greater social media attention; the most common influences were number of citations and number of references. Researchers in orthopedics can use this information when considering how to assess the impact of their work.

Association of Non-traditional Indicators of Readers' Engagement With Traditional Dissemination Metrics of COVID-19-Related Research

INTRODUCTION

 Researchers are increasingly interested in appraising the impact of their research work, which eventually drives public perception. The overall impact of a study can only be gauged if we consider both traditional and non-traditional dissemination patterns. Hence, we preferred to study the association between the non-traditional reader engagement metrics and traditional dissemination metrics in relation to coronavirus disease 2019 (COVID-19)-related research published in five high-impact peer-reviewed medical journals.

METHOD

 This observational study was conducted using data sourced from Altmetric, including the Altmetric attention score (AAS), an aggregate score of an article's dissemination. New England Journal of Medicine (NEJM), Lancet Infectious Diseases, Clinical Infectious Diseases (CID), Chest Journal (CHEST), and Journal of the American Medical Association (JAMA) were included in the study based on the prevalence of COVID-19-related original research published in each of them. The number of citations was framed as the reference for traditional metrics. To avoid artificial variance, data were collected on the same day, November 13, 2022. Correlational analyses were performed using the Pearson correlation coefficient using Minitab 17 (Minitab Inc., State College, PA). The relationship between the variables was considered very weak if r<0.3, weak if r: 0.3 to 0.5, moderate if r: 0.5 to 0.7, and strong for r>0.7.

RESULTS

 We found a very weak correlation between citations and AAS for Clinical Infectious Diseases, Lancet Infectious Diseases, and CHEST, whereas the correlation was moderate for NEJM and JAMA. The correlation between citations and Twitter mentions was very weak for Clinical Infectious Disease, Lancet Infectious Disease, and CHEST, but it improved for NEJM and JAMA. There was a very weak correlation between citations and news mentions for Clinical Infectious Diseases, Lancet Infectious Diseases, and CHEST.

CONCLUSION

 Our study highlights that the traditional indicator, i.e., citation has a very weak to moderate correlation with the AAS and it doesn't capture the entire influence of a research publication. Also, the current method of determining a journal's impact factor doesn't take this disparity into consideration. Hence, there needs to have a more inclusive strategy to define the impact of scientific research on the general population in real-time.

Evaluation of Altmetric Analysis Scores of the Top 100 Articles on Polycystic Ovary Syndrome Published in the Last 10 Years

OBJECTIVE

In the current study, we analyzed the 100 most cited articles with the topic, title, and keywords of polycystic ovary syndrome (PCOS) published in all journals in terms of traditional metrics and the altmetric score (AS).

METHODS

The term "polycystic ovary syndrome (PCOS)" was searched in the Web of Science (WoS) database and filtered for articles published in all journals. Bibliographic data and AS were obtained for 100 highly cited papers from January 2012 to July 2022. Descriptive statistics were reported and correlation analysis between traditional bibliographies and the AS was performed.

RESULTS

The Journal of Clinical Endocrinology & Metabolism, with 14 articles, had the most publications on the Top 100 list. When the studies were classified according to subtypes, 56 papers were original scientific papers with mean AS of 32.5 (15.3-52.7), whereas 44 papers were reviews and meta-analyses with AS of 16.0 (8.6-43.2). The AS was positively correlated with H-index, total WoS citation number of article and Q category. There were no correlations with impact factor (IF), five-year IF, journal impact factor (JIF) percentile and journal citation indicator (JCI) value.

CONCLUSION

Our results suggest that the AS is related with article total WoS citation number, journal Q category, and journal H-index. Effective communication on social media can promote scientific productivity and have a positive impact on society.

Assessing Factors Associated With Altmetric Attention Score: A Preliminary Study of 3 Hand Surgery Journals

BACKGROUND

Traditional measures of evaluating scholarly output do not capture the impact social media can provide in disseminating and promoting research. We sought to better understand the level of online attention that high-quality hand research received.

METHODS

Scientific manuscripts published from 2017 in Journal of Hand Surgery (American Volume) (JHS-A), Journal of Hand Surgery (European Volume) (JHS-E), and HAND were recorded. Manuscript characteristics were determined, including the number of citations. Altmetric Attention Score (AAS), a measure of a manuscript's online attention and impact, was determined, as well as Twitter mentions, Facebook mentions, and news outlet mentions. Spearman rho (ρ) correlation coefficient was used to evaluate the relationship between the number of citations and AAS. Multivariable linear regression analysis was performed to determine which manuscript factors were associated with AAS.

RESULTS

A total of 323 manuscripts were identified. There was a weak positive correlation between the number of citations and AAS; however, this relationship did not exist for each individual journal. Publication in HAND and JHS-E were associated with lower average manuscript AAS when using JHS-A as the reference group. Two additional factors were also associated with increased manuscript AAS: (1) being a clinical study focused on a specific upper extremity anatomical location; and (2) increasing number of institutions on a study.

CONCLUSIONS

Publication in HAND and JHS-E were associated with lower manuscript AAS when using JHS-A as the reference group, suggesting that HAND and JHS-E have room for improvement in using social media to share their high-quality hand surgery scientific articles.

Positive Association Between Online Attention and the Bibliometric Impact of Shoulder Instability Publications

Purpose

To obtain a quantifiable measure of the frequency with which a shoulder instability article is discussed online and the association with its corresponding bibliometric impact, based on the Scopus Cite Score (SCS) or Web of Science Impact Factor (WSIF).

Methods

The top 100 most-mentioned articles on shoulder instability based on Altmetric Attention Score (AAS) were extracted from the Altmetric Database. Mentions within blogs, news articles and outlets, public policy, and social media platforms, such as Twitter and Facebook, were included. Study impact was assessed using SCS or WSIF. The degree of association between AAS and impact was determined using Spearman correlation, logarithmic regression, and multivariate regression.

Results

The most common study designs were “Clinical Trial,” with 52 articles (49.5%), “Systematic Review” with 16 articles (15.2%), and “Review” with 10 articles (9.5%). Twitter provided more online mentions than other platforms, with the average article being discussed 27.7 times (range 0-220 times). A significant positive effect (estimate = 2.616, P = .0075) was observed between the AAS and WSIF, based on the logarithmic regression. Multivariate regression revealed that blogs help raise both WSIF and SCS (estimate = 7.272, P < .05).

Conclusions

Social media and other online platforms are a strong way to disseminate information to patients. A positive association was observed between overall online attention and the bibliometric impact of an article related to shoulder instability. Clinical trials related to shoulder instability that receive online mentions, especially discussion in blogs, are more likely to be cited in the future than their counterparts.

Clinical Relevance

The results of our study can guide authors as they aim to disseminate their articles. Twitter may be used as a tool to reach patients who may not venture into academic journals with current peer-reviewed articles. Further, blogs may be used to reach academic audiences and raise bibliometric impact broadly.

Effect of Altmetric score on manuscript citations: A randomized-controlled trial

BACKGROUND

Alternative metrics to traditional, citation-based metrics are increasingly being used. These are complementary to traditional metrics, like downloads and citations, and give information on how often a given journal article is discussed and used in professional (reference managers) and social networks, such as mainstream media and Twitter. Altmetrics is used in most journals and is available in all indexed headache medicine journals. Whether Altmetrics have an input on traditional, citation-based metrics or whether it is a stand-alone metric system is not clear. Actively promoting a paper through media channels will probably increase the Altmetric score but the question arises whether this will also increase citations and downloads of this individual paper.

METHODS

Focusing on this point we performed a randomized study in order to test the hypothesis that a promotion intervention would improve citations and other science metric scores. We selected 48 papers published in Cephalalgia from July 2019 to January 2020 and randomized them to either receive an active promotion through social media channels or not. The primary outcome used was the difference between mean article citations with versus without intervention 12 months after the intervention period.

RESULTS

The results show that the alternative metrics significantly increased for those papers randomly selected to receive an intervention compared to those who did not. This effect was observed in the first 12 months, right after the boosting strategy was performed. The higher promoted paper diffusion in social media lead to a significantly higher number of citations and downloads.

CONCLUSION

Further promotion strategies should be studied in order to tailor the best cost-benefit intervention.

Influences of the Social Media Continuum Model in Radiology: Theory and Reality

Social media utilization has been growing exponentially worldwide and has created a thriving venue for radiologists and the profession of radiology to engage in on both the academic and social levels. The aim of this article is to conduct updated literature review and address a gap in the literature by introducing a simple classification for social media utilization and a new theoretical model to outline the role and potential value of social media in the realm of radiology. We propose classifying social media through usage-driven and access-driven indices. Furthermore, we discuss the interdependency of radiologists, other physicians and non-physician stakeholders, scientific journals, conferences/meetings and the general public in an integrated social media continuum model. With the ongoing sub-specialization of radiology, social media helps mitigate the physical barriers of making connections with peers and audiences which would have otherwise been unfeasible. The constant evolution and diversification of social media platforms necessitates a novel approach to better understand its role through a radiological lens. With the looming fear of 'ancillary service' labelling, social media could be the golden plate to halt the path towards commoditization of radiology.

A Metrics-Based Research Salary Award System and Its 9-Year Impact on Publication Productivity

RATIONALE AND OBJECTIVES

Although metrics-based systems may incentivize academic output, no prior studies have evaluated the impact on publication metrics in academic radiology. This study presents a metrics-based system of awarding research protected time, and retrospectively evaluates its 9-year impact on publication productivity and impact factor.

MATERIALS AND METHODS

Based on a metrics-based algorithm to award department funded Research Protected Time (RPT), metrics pre-RPT (2003-2009) and during the RPT period (2010-2018) from an academic radiology department were retrospectively analyzed to test the hypothesis that the RPT program resulted in higher publication productivity and journal impact factor at the departmental level and for faculty members receiving the award. Comparison was made between (1) pre-RPT and RPT periods and (2) during the RPT period, between RPT and non-RPT faculty members, for annual publication productivity normalized to faculty count (Student's t test) and median impact factor (Wilcoxon rank sum test).

RESULTS

For the evaluation period of 2003-2018, 724 unique publications were identified: 15% (107/724) pre-RPT period and 85% (617/724) RPT period. Normalized annual publication productivity was higher during the RPT period compared to the Pre-RPT period (1.2 vs. 0.3, p = 0.002), and within the RPT period, higher among faculty who received RPT vs. non-RPT faculty (3.5 vs. 0.4, p = 0.002). Median impact factor was higher during the RPT period compared to pre-RPT period (2.843 vs. 2.322, p = 0.044), and within the RPT period, higher in RPT vs. non-RPT faculty (3.016 vs. 2.346, p < 0.001).

CONCLUSION

The implementation of a metrics-based system of funded, research protected time, was associated with increased publication productivity and increased impact factor.

Altmetric and bibliometric analysis of obstetrics and gynecology research: influence of public engagement on citation potential

BACKGROUND

Whether research engagement on social media and other public platforms results in increased citations in obstetrics and gynecology remains uncertain. The Altmetric Attention Score is a metric of research influence based on mentions on social media and public platforms, such as newsfeeds and Wikipedia. The correlation between Altmetric Attention Scores, absolute citation rates, and the Relative Citation Ratio (a novel metric of research engagement also based on citation rates) in obstetrics and gynecology research is uncertain.

OBJECTIVE

To evaluate the correlation between Altmetric Attention Score, absolute citation rate, and Relative Citation Ratio for articles published in obstetrics and gynecology journals from 2004 to 2019. Our second objective was to identify, characterize, and compare the 100 articles with highest Altmetric Attention Scores, the 100 most-cited articles, and the 100 articles with highest Relative Citation Ratios.

STUDY DESIGN

We performed a cross-sectional altmetric and bibliometric study of all obstetrics and gynecology articles indexed in the National Institutes of Health Open Citation Collection from 2004 to 2019. Articles were included if they were published in obstetrics and gynecology journals according to InCites Journal Citation Reports indexing. Citation data, including citation numbers and Relative Citation Ratios, were downloaded on March 20, 2021 and merged with altmetric data from the Altmetric Explorer on the basis of each article's unique PubMed identification number. We assessed correlation between Altmetric Attention Scores and number of citations and Altmetric Attention Scores and Relative Citation Ratios by calculating the Pearson correlation coefficient. The 100 articles with highest Altmetric Attention Scores, the 100 most-cited articles, and the 100 articles with highest Relative Citation Ratios were characterized and compared using means (standard deviations) and mean differences (95% confidence intervals).

RESULTS

There were 156,592 articles published in 82 obstetrics and gynecology journals and indexed in the National Institutes of Health Open Citation Collection between 2004 and 2019. The correlation coefficient was 0.18 (95% confidence interval, 0.17-0.19) for Altmetric Attention Scores vs number of citations and 0.10 (95% confidence interval, 0.09-0.11) for Altmetric Attention Scores vs Relative Citation Ratios. There was no overlap among the 100 articles on the highest Altmetric Attention Score list and the 100 most-cited list, and there was minimal overlap among the 100 articles on the highest Altmetric Attention Score list and the 100 highest Relative Citation Ratio list (98 unique articles on each list). Articles with highest Altmetric Attention Scores generated substantially more engagement on social media and other public platforms than most-cited articles (mean Altmetric Attention Score, 763.1 [standard deviation, 520.8] vs 49.9 [standard deviation, 81.6]; mean difference, -713.2 [95% confidence interval, -819.9 to -606.6]) and highest Relative Citation Ratio articles (mean, 116.2 [standard deviation, 415.9]; mean difference, -661.5 [95% confidence interval, -746.2 to -576.9]). In contrast, the articles with highest Altmetric Attention Scores generated far fewer citations than most-cited articles (mean, 39.7 [standard deviation, 47.6] vs 541.8 [standard deviation, 312.8]; mean difference, 502.0 [95% confidence interval, 439.0-565.0]) and highest Relative Citation Ratio articles (mean, 458.9 [standard deviation, 363.5]; mean difference, 427.7 [95% confidence interval, 353.8-501.6]). Nearly half of articles with highest Altmetric Attention Scores were basic/translational studies, often about menopause and environmental factors impacting fertility, whereas most-cited articles and articles with highest Relative Citation Ratios were more likely to be reviews and consensus statements, respectively, often about placentation and polycystic ovary syndrome, respectively. Articles with highest Altmetric Attention Scores were more likely to be published as open-access.

CONCLUSION

There seems to be weak short-term correlation between Altmetric Attention Scores and citation rates. Further study is warranted to ascertain whether there may be long-term correlation between alternative metrics and citation rates in obstetrics and gynecology.

ALTMETRIC ANALYSIS OF THE MOST-CITED 100 ARTICLES ON THE RETINA PUBLISHED BETWEEN 2010 AND 2020

PURPOSE

Altmetric analysis is a way of assessing the social impact of scientific articles. In this study, we aimed to analyze the 100 most-cited articles on the topic of the retina published in ophthalmology journals in traditional metrics and altmetrics.

METHODS

The term "retina" was searched in the Web of Science database, and articles published in ophthalmology journals were filtered out. A total of 100 highly cited articles from 2010 to 2020 were evaluated for bibliographic data and altmetrics. First, descriptive statistics and then correlation analysis between traditional bibliographies and altmetrics were performed.

RESULTS

According to the Web of Science search, the number of citations of the articles listed in the top 100 list ranged from 809 to 137. The altmetric scores of the articles listed in the top 100 list ranged from 0 to 1,340. There was no statistically significant correlation between the altmetric scores and the number of citations, but there was a statistically weak correlation between the altmetric scores and the average citations per year, H index, impact factor, and number of years since publication.

CONCLUSION

Altmetrics is not sufficient to determine the scientific value of articles and can be affected by many factors, unlike traditional bibliometrics. However, being a good communicator in social media can support scientific productivity and create social impact.

Alternative publication metrics in the time of COVID-19

Alternative metrics are unique bibliometrics comprising social, news, and other sources of media outside of traditional academic citations. Some have suggested that these metrics can complement traditional metrics of research impact, including public engagement with research. The COVID-19 pandemic provides a unique opportunity to study alternative metrics and the dissemination of scientific research given the heightened academic and public interest. This study analyzed Altmetric Attention Scores for the top 25 publications on COVID-19 and the top 25 non-COVID-19 publications in 2020. There were significantly higher levels of social attention scores across multiple metrics for COVID-19 articles than for non-COVID-19 articles for that year. There was a slightly higher goodness of fit between Altmetric Attention Scores and academic citations for COVID-19 publications than for non-COVID-19 publications, although trendline differences were not significant. These results suggest that researchers should be aware that their studies can become highly visible on publicly available social and news media platforms, especially during events of high interest (such as a global pandemic).

Dissemination of Gastroenterology and Hepatology Research on Social Media Platforms Is Associated With Increased Citation Count

INTRODUCTION

The purpose of this study was to find out whether dissemination of gastroenterology and hepatology (GI) research on social media networks correlates with citation count at 5 years.

METHODS

We correlated the Altmetric Attention Score with Web of Science citation counts at 5 years for scholarly work published in the 10 highest impact factor GI journals in 2014.

RESULTS

In 4,026 analyzed items, the correlation (r) between Altmetric Attention Score and citations at 5 years was 0.62 (P < 0.001), representing strong correlation. Twitter was the platform with the strongest correlation with citations.

DISCUSSION

Social media attention garnered by GI scholarly work strongly correlates with the number of citations at 5 years.

The Use of Twitter by Medical Journals: Systematic Review of the Literature

BACKGROUND

Medical journals use Twitter to engage and disseminate their research articles and implement a range of strategies to maximize reach and impact.

OBJECTIVE

This study aims to systematically review the literature to synthesize and describe the different Twitter strategies used by medical journals and their effectiveness on journal impact and readership metrics.

METHODS

A systematic search of the literature before February 2020 in four electronic databases (PubMed, Web of Science, Scopus, and ScienceDirect) was conducted. Articles were reviewed using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines.

RESULTS

The search identified 44 original research studies that evaluated Twitter strategies implemented by medical journals and analyzed the relationship between Twitter metrics and alternative and citation-based metrics. The key findings suggest that promoting publications on Twitter improves citation-based and alternative metrics for academic medical journals. Moreover, implementing different Twitter strategies maximizes the amount of attention that publications and journals receive. The four key Twitter strategies implemented by many medical journals are tweeting the title and link of the article, infographics, podcasts, and hosting monthly internet-based journal clubs. Each strategy was successful in promoting the publications. However, different metrics were used to measure success.

CONCLUSIONS

Four key Twitter strategies are implemented by medical journals: tweeting the title and link of the article, infographics, podcasts, and hosting monthly internet-based journal clubs. In this review, each strategy was successful in promoting publications but used different metrics to measure success. Thus, it is difficult to conclude which strategy is most effective. In addition, the four strategies have different costs and effects on dissemination and readership. We recommend that journals and researchers incorporate a combination of Twitter strategies to maximize research impact and capture audiences with a variety of learning methods.

A critical review on altmetrics: can we measure the social impact factor?

Altmetrics measure the digital attention received by a research output. They allow us to gauge the immediate social impact of an article by taking real-time measurements of how it circulates in the Internet. While there are several companies offering attention scores, the most extensive are Altmetric.com (Altmetric Attention Score—AAS) and Plum X (Plum Print). As this is an emerging topic, many medical specialities have tried to establish if there is a relationship between an article’s altmetric data and the citations it subsequently receives. The results have varied depending on the research field. In radiology, the social network most used is Twitter and the subspeciality with the highest AAS is neuroimaging. This article will review the process involved from the start when an article is published through to finally obtaining its altmetric score. It will also address the relationship between altmetrics and more traditional approaches focusing on citations in radiology and will discuss the advantages and limitations of these new impact indicators.

What are the variables associated with Altmetric scores?

BACKGROUND

Social media has been used to disseminate the contents of scientific articles. To measure the impact of this, a new tool called Altmetric was created. Altmetric aims to quantify the impact of each article through online media. This systematic review aims to describe the associations between the publishing journal and published article variables and Altmetric scores.

METHODS

Searches on MEDLINE, EMBASE, CINAHL, CENTRAL, and Cochrane Library were conducted. We extracted data related to both the publishing article and the publishing journal associated with Altmetric scores. The methodological quality of included articles was analyzed by the Appraisal Tool for Cross-sectional Studies.

RESULTS

A total of 19 articles were considered eligible. These articles summarized a total of 573,842 studies. Citation counts, journal impact factor, access counts, papers published as open access, and press releases generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to strong.

CONCLUSION

Citation counts and journal impact factor are the most common variables associated with Altmetric scores. Other variables such as access counts, papers published in open access journals, and the use of press releases are also likely to be associated with online media attention.

SYSTEMATIC REVIEW REGISTRATION

This review does not contain health-related outcomes. Therefore, it is not eligible for registration.

Tracking trends of transgender health research online: are researchers and the public on the same page?

Altmetric Explorer was searched for the most popular online articles published in Pubmed-indexed journals. The 75 articles with the highest Altmetric Attention Score (AAS) were screened for article information (date, journal, access), authorship (number of authors, affiliation and origin of the corresponding author), and research (type, subject, funding). The reviewed articles displayed a mean AAS of 241.52, were broadcast 17.03 times by news agencies, posted on Twitter 101.47 times, downloaded by 67.21 Mendeley readers, and received 62.67 citations. There was intense online interest in the transgender health literature, mainly related to mental health and social well-being. Online visibility of transgender health articles was not significantly correlated with citation counts, implying that the public, likely including transgender persons and allies, may place emphasis on different health issues than scholars. Monitoring altmetrics and interactions on electronic media may help researchers conduct research that is more meaningful to transgender individuals, and to society in general.

The Online Attention to Cleft Lip and Palate Research: An Altmetric Analysis

OBJECTIVES

To identify research articles related to cleft lip and/or cleft palate (CL/P) that generated the highest online attention.

METHODS

Altmetric Explorer was used to identify the 100 articles with the highest Altmetric Attention Score (AAS). Descriptive and correlation statistics were performed to study the characteristics of these articles in relation to their publication data, research type and domain, number of Mendeley readers, and dimensions citations. Citation counts were extracted from Scopus and Google Scholar.

RESULTS

The median AAS for the top 100 outputs was 22 (range from 12 to 458). The outputs were mostly discussed on Twitter (median = 8; range = 0-131). Topics discussing treatment and care for patients with CL/P accounted for 38% of the articles with the highest AAS followed by etiology and risk factors (32%). The majority of articles originated from the USA (46%) followed by Europe (16%) and the United Kingdom (15%). No significant differences were observed in AAS among different study designs, topic domains, journals' ranking and impact factor, and the number of citations in Scopus and Google Scholar.

CONCLUSIONS

Researchers should consider use of social platforms to disseminate their work among scholars and nonscholars. Altmetrics can be combined with traditional metrics for a more comprehensive assessment of research impact.

Meta-Analysis of Correlations between Altmetric Attention Score and Citations in Health Sciences

INTRODUCTION

In recent years, several controversial reports of the correlation between altmetric score and citations have been published (range: -0.2 to 0.8). We conducted a meta-analysis to provide an in-depth statistical analysis of the correlation between altmetric score and number of citations in the field of health sciences.

METHODS

Three online databases (Web of Science, Scopus, and PubMed) were systematically searched, without language restrictions, from the earliest publication date available through February 29, 2020, using the keywords "altmetric," "citation," and "correlation." Grey literature was also searched via WorldCat, Open Grey, and Google Scholar (first 100 hits only). All studies in the field of health sciences that reported on this correlation were included. Effect sizes were calculated using Fisher's z transformation of correlations. Subgroup analyses based on citation source and sampling methods were performed.

RESULTS

From 27 included articles, 8 articles comprise several independent studies. The total sample size was 9,943 articles comprised of 35 studies. The overall pooled effect size was 0.19 (95% confidence interval 0.13 to 0.26). Bivariate partial prediction of interaction between effect size, citation source, and sampling method showed a greater effect size with Web of Science compared with Scopus and Dimensions. Egger's regression showed a marginally nonsignificant publication bias (p = 0.055), and trim-and-fill analysis estimated one missing study in this meta-analysis.

CONCLUSION

In health sciences, currently altmetric score has a positive but weak correlation with number of citations (pooled correlation = 0.19, 95% C.I 0.12 to 0.25). We emphasize on future examinations to assess changes of correlation pattern between altmetric score and citations over time.

Orthopaedic Randomized Controlled Trials Published in General Medical Journals Are Associated With Higher Altmetric Attention Scores and Social Media Attention Than Nonorthopaedic Randomized Controlled Trials

PURPOSE

To (1) compare the Altmetric Attention Score (AAS) and citation rates between orthopaedic and nonorthopaedic randomized controlled trials (RCTs) from 5 high-impact medical journals and (2) identify general characteristics of these articles associated with greater exposure on social media platforms.

METHODS

Articles published in The New England Journal of Medicine (NEJM), Lancet, The Journal of the American Medical Association (JAMA), Annals of Internal Medicine, and Archives of Internal Medicine between January 2011 and December 2016 were analyzed. These journals were selected based on retaining high impact factors with rigorous publication standards and availability of the AAS for their publications. The queried time frame was chosen to balance the inception of the AAS with an optimal period for citation accrual. A total of 14 article characteristics, in addition to number of Tweets, Facebook shares, news mentions, and the AAS, were extracted. Inclusion criteria were orthopaedic RCTs reporting on outcomes after surgical intervention. Linear regression was used to assess the relationship between publication characteristics and the AAS and social media attention.

RESULTS

A total of 9 orthopaedic and 59 nonorthopaedic RCTs were included. The mean AASs were significantly different (574 ± 565.7 versus 256.9 ± 222.3, P = .003), whereas citation rate was not (192.2 ± 117.1 versus 382.3 ± 560.3, P = .317). Orthopaedic RCTs had a significantly greater number of mentions on Twitter and Facebook (P < .001). A higher AAS significantly associated with a greater number of citations (β = 0.75, P = .019) for orthopaedic RCTs. The mean AAS of orthopaedic RCTs favoring nonoperative management (809.6 ± 676.3) was greater than those favoring operative treatment (292.0 ± 248.9) but was not statistically significant (P = .361).

CONCLUSION

Orthopaedic RCTs published in 5 high-impact general medical journals had a significantly greater mean AAS relative to nonorthopaedic RCTs, with no differences in citation rates. Additionally, there was a strong association between the AAS and citation rate of orthopaedic RCTs. Orthopaedic RCTs had greater social media exposure on both Twitter and Facebook.

CLINICAL RELEVANCE

Orthopaedic surgeons, researchers, and providers who publish RCTs in high-impact medical journals can anticipate extensive social media attention for their articles relative to other nonorthopaedic RCTs in the same journals. Social media attention may be related to operative versus nonoperative management topics. This study provides further evidence for the increasing use of the AAS and its association with citation accrual.

Correlation Between Altmetric Score and Traditional Bibliometrics in Total Joint Arthroplasty Research

Background

Citation analysis is commonly used to evaluate the impact of academic publications within an area of study. The purpose of this study is to review the publications with the highest Altmetric scores related to total hip arthroplasty (THA) and total knee arthroplasty (TKA) and assess the correlation with traditional bibliometrics.

Methods

Altmetric scores were obtained for the top articles relating to THA and TKA and organized from highest to lowest scores. The Clarivate Analytics Web of Knowledge database was used to search the top articles by Altmetric score. Articles meeting inclusion criteria were reviewed for various metrics, including number of citations, journal impact factor at the time of publication, and study type and design.

Results

The top THA and TKA publications were cited 3042 times and 7523 times, respectively. The former were published in journals with an average impact factor of 17.861, and the latter in journals with an average impact factor of 15.564. For THA, Altmetric score demonstrated a significant, weakly positive correlation with the number of citations (P = .008). For TKA, Altmetric score was found to have a significant, weakly positive correlation with impact factor of journal (P = .04); however, it was nonsignificant for citation count (P = .11). When one outlier is removed, the correlation with citation count is statistically significant (P = .009).

Conclusion

While alternative metrics cannot replace traditional bibliometrics, they may serve a complimentary role in describing the influence of research. Thus, the Altmetric score represents an additional tool to identify the most influential articles to guide learning and evidence-based practice.

The impact of low back pain systematic reviews and clinical practice guidelines measured by the Altmetric score: Cross-Sectional study

BACKGROUND

Although Altmetric has been widely used by researchers to monitor the audience of their articles, there are no studies that have analysed factors associated with Altmetric score for systematic reviews and clinical practice guidelines.

OBJECTIVES

1) To analyse factors that could be associated with Altmetric scores for low back pain systematic reviews and clinical practice guidelines. 2) To describe the characteristics of these articles and their Altmetric scores.

METHODS

We searched for all low back pain systematic reviews and guidelines indexed on the Physiotherapy Evidence Database published between 2015 and 2017. We extracted data related to the published paper, the publishing journal, and Altmetric scores.

RESULTS

A total of 66 systematic reviews and 5 guidelines were included. The variable impact factor (independent variable) was associated with Altmetric mentioned score (dependent variable) with a β coefficient of 15.4 (95% CI: 0.97, 29.7) ajusted to all remaining variables. The variable number of citations normalized by year of publication (independent variable) was associated with Altmetric reader score (dependent variable) with a β coefficient of 6.4 (95% CI: 4.03, 8.72) ajusted to all remaining variables. We also found that the majority of the systematic reviews and guidelines were published in English, had a descriptive title, were published as open access, included multicenter studies, and had media release generated by the publishing journal.

CONCLUSION

Metrics related to the number of citations, such as the impact factor are associated with Altmetric scores.

The Most Influential Publications on Endoscopic Submucosal Dissection: A Bibliometric Analysis

Background/Aims: Endoscopic submucosal dissection (ESD) is the first-line treatment for superficial gastrointestinal neoplasms with negligible lymph node metastasis. It has evolved through improvements in expertise and equipment, increased understanding of indications and short- and long-term outcomes, and better management of complications. This study aimed to assess and characterize the most influential publications in ESD research.Materials and Methods: We searched the top 50 most cited articles using Web of Science Core Collection (WoSCC) and Google Scholar (GS) from the inception of these services to January 2019. The top 50 Altmetric Attention Score (AAS) articles based on online media mentions were also searched. Each article was evaluated for the number of citations, title, journal, and publication year.Results: The number of citations for the top 50 WoSCC articles on ESD ranged from 37 to 199; Endoscopy published the most articles (20%). Among the top 50 GS articles, Gastrointestinal Endoscopy published the most ESD articles (34%) and the most shared AAS articles (42.6%). PubMed Central article citations in WoSCC or GS showed significant correlation with those from each metric, unlike AAS. The words with the highest relevance scores were “submucosal tunnel dissection,” “guideline,” “novel submucosal gel,” “adhesive material,” “cell sheet,” “esophageal ulcer,” “hemospray,” and “endoscopic closure,” while the following words were influential: “meta-analysis,” “esophageal stricture,” “perforation,” “bleeding,” “fibrin glue,” “artificial ulcer,” “porcine model” and “esophageal squamous cell neoplasia,” excluding “ESD.”Conclusions: This study presents a detailed list of influential articles, journals, and topic words.

How to Use YouTube for Radiology Education

YouTube, the most commonly used free video-sharing platform globally, is increasingly being used as an educational tool in Radiology. Trainees worldwide now have the opportunity to learn about medical imaging at their own pace in the comfort of their homes, without geographical and financial constraints. Unfortunately, because YouTube is an easily accessible platform, it also incurs the risk of disseminating erroneous medical information or low-quality educational content. This article outlines the primary considerations when creating educational content on YouTube, including technical aspects, best practices, and measures to maximize effectiveness and success. Additionally, we discuss the current usage of the platform for Radiology education and its advantages and disadvantages and list some of the most popular Radiology YouTube channels.

Characteristics of the Most Cited, Most Downloaded, and Most Mentioned Articles in General Medical Journals: A Comparative Bibliometric Analysis

We compared the characteristics of the most cited, most downloaded, and most mentioned (the highest Altmetric Attention Score) articles published in general medical journals. We identified the 640 most frequently cited, 662 most frequently downloaded, and 652 most mentioned articles from 48 general medical journals. A comparison was made of the following characteristics of articles in the most cited, most downloaded, and most mentioned articles: medical specialty, publication type, country of origin, year of publication, and accessibility. There was only a 2.5% overlap in these three groups. Original articles were the more frequent among the most mentioned articles, whereas reviews, case reports, and guidelines/consensus statements were more frequent among the most downloaded articles. The most cited articles were more frequently published in 2010 and before, whereas the most downloaded articles were published in 2017−2018. The most mentioned articles were more frequently open-access articles, compared to the most downloaded articles. The most cited were more frequently older, the most downloaded were more frequently recent and educational, and the most mentioned were more frequently original and open-access articles. The results of our study may provide insights into various measures of article impact.

If I tweet will you cite later? Follow-up on the effect of social media exposure on article downloads and citations

OBJECTIVES

We previously reported that random assignment of scientific articles to a social media exposure intervention did not have an effect on article downloads and citations. In this paper, we investigate whether longer observation time after exposure to a social media intervention has altered the previously reported results.

METHODS

For articles published in the International Journal of Public Health between December 2012 and December 2014, we updated article download and citation data for a minimum of 24-month follow-up. We re-analysed the effect of social media exposure on article downloads and citations.

RESULTS

There was no difference between intervention and control group in terms of downloads (p = 0.72) and citations (p= 0.30) for all papers and when we stratified by open access status.

CONCLUSIONS

Longer observation time did not increase the relative differences in the numbers of downloads and citations between papers in the social media intervention group and papers in the control group. Traditional impact metrics based on citations, such as impact factor, may not capture the added value of social media for scientific publications.

The Top 100 Spine Surgery Articles on Social Media: An Altmetric Study

STUDY DESIGN

Literature review.

OBJECTIVE

To discern which social media outlets contribute most to spine surgery literature dissemination and to investigate how popular articles compare to articles with most citations.

SUMMARY OF BACKGROUND DATA

Scientific literature is increasingly disseminated through social media. The Altmetric Attention Score (AAS) is defined as an automated, weighted score calculation from counts of all online attention received by various research outputs. Increasing AAS values indicate more online attention. For example, the overall top 100 Altmetric spine surgery peer-reviewed articles since 2010 had an AAS range from 78 to 1537. Among all spine surgery literature reviewed since 2010, the mean AAS was 5.3 with a median of 1.0.

METHODS

We performed an Altmetric database search of nine spine surgery journals from January 2010 to October 2019. Mean AAS was summarized alongside metrics including citation count and impact factor. We assessed correlations between AAS and online sources, readers, and citations. Journals were grouped by impact factor, and analysis-of-variance compared mean AAS. The 100 highest AAS articles were compared to the most cited.

RESULTS

13,601 articles were included. The mean AAS was 5.3, with Twitter contributing the most. The three highest associations were news (P < 0.001), Twitter (P < 0.001), and Facebook (P < 0.001). There was no significant association between impact factor and AAS. Compared with the most cited articles, the top 100 AAS articles had significantly more article types, more prospective studies, fewer retrospective studies, fewer reviews, and fewer systematic reviews (P < 0.001 for all). Spine contributed the most articles in both top 100 sets.

CONCLUSION

Our evaluation revealed Twitter, newsfeeds, and Facebook were the most significant social media outlets. Compared to articles with the most citations, the most popular articles are prospective and encompass broader study designs. Social media plays an integral role in dissemination, both within spine literature and the public sphere.

LEVEL OF EVIDENCE

3.

The Use of Social Media to Increase the Impact of Health Research: Systematic Review

BACKGROUND

Academics in all disciplines increasingly use social media to share their publications on the internet, reaching out to different audiences. In the last few years, specific indicators of social media impact have been developed (eg, Altmetrics), to complement traditional bibliometric indicators (eg, citation count and h-index). In health research, it is unclear whether social media impact also translates into research impact.

OBJECTIVE

The primary aim of this study was to systematically review the literature on the impact of using social media on the dissemination of health research. The secondary aim was to assess the correlation between Altmetrics and traditional citation-based metrics.

METHODS

We conducted a systematic review to identify studies that evaluated the use of social media to disseminate research published in health-related journals. We specifically looked at studies that described experimental or correlational studies linking the use of social media with outcomes related to bibliometrics. We searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases using a predefined search strategy (International Prospective Register of Systematic Reviews: CRD42017057709). We conducted independent and duplicate study selection and data extraction. Given the heterogeneity of the included studies, we summarized the findings through a narrative synthesis.

RESULTS

Of a total of 18,624 retrieved citations, we included 51 studies: 7 (14%) impact studies (answering the primary aim) and 44 (86%) correlational studies (answering the secondary aim). Impact studies reported mixed results with several limitations, including the use of interventions of inappropriately low intensity and short duration. The majority of correlational studies suggested a positive association between traditional bibliometrics and social media metrics (eg, number of mentions) in health research.

CONCLUSIONS

We have identified suggestive yet inconclusive evidence on the impact of using social media to increase the number of citations in health research. Further studies with better design are needed to assess the causal link between social media impact and bibliometrics.

Association Between Citations, Altmetrics, and Article Views in Pediatric Research

This cross-sectional study examines the association between article citations, Altmetric attention scores, and cumulative page views of pediatric research articles from 4 high-impact medical journals.

What Is the Predictive Ability and Academic Impact of the Altmetrics Score and Social Media Attention?

BACKGROUND

Citation rate and journal impact factor have traditionally been used to assess research impact; however, these may fail to represent impact beyond the sphere of academics. Given that social media is now used to disseminate research, alternative web-based metrics (altmetrics) were recently developed to better understand research impact on social media. However, the relationship between altmetrics and traditional bibliometrics in orthopaedic literature is poorly understood.

PURPOSE

To (1) assess the extent that altmetrics correlate with traditional bibliometrics and (2) identify publication characteristics that predict greater altmetrics scores.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published in The American Journal of Sports Medicine (AJSM), The Journal of Bone and Joint Surgery, Clinical Orthopaedics and Related Research, Acta Orthopaedica, and Knee Surgery, Sports Traumatology, Arthroscopy between January 2016 and December 2016 were analyzed. Among the extracted publication characteristics were journal, number of authors, geographic region of origin, highest degree of first author, study subject and design, sample size, conflicts of interest, and level of evidence; number of references, institutions, citations, tweets, Facebook mentions, and news mentions; and Altmetric Attention Score (AAS). Multivariate regressions were used to determine (1) publication characteristics predictive of AAS and social media attention (mentions on Twitter, Facebook, and the news) and (2) the relationship between AAS and citation rate.

RESULTS

A total of 496 published articles were included, with a mean AAS of 8.6 (SD, 31.7; range, 0-501) and a mean citation rate of 15.0 (SD, 16.1; range, 0-178). Articles in AJSM (β = 19.9; P < .001), publications from North America (β = 8.5; P = .033), and studies concerning measure validation/reliability (β = 25.5; P = .004) were independently associated with higher AAS. Greater AAS score significantly predicted a greater citation rate (β = 0.16; P < .0001). The citation rate was an independent predictor of greater social media attention on Twitter, Facebook, and the news (odds ratio range, 1.02-1.03; P < .05 all).

CONCLUSION

AAS had a significant positive association with citation rates of articles in 5 high-impact orthopaedic journals. Articles in AJSM, studies concerning measure validation and reliability, and publications from North America were positively associated with greater AAS. A greater number of citations was consistently associated with publication attention received on social media platforms.

Analysis of the altmetric top 100 articles with the highest altmetric attention scores in medical imaging journals

PURPOSE

Alternative metrics (altmetrics) provides a complementary measure of the impact of scientific articles. The aim of this study was to identify and characterize the top 100 articles with the highest Altmetric attention score (AAS) in medical imaging journals.

MATERIALS AND METHODS

We searched for AASs articles published in 116 journals in the subject category "Radiology, Nuclear Medicine and Medical Imaging" using the Web of Science. We determined the top 100 articles with the highest AAS using the Altmetric.com database, and then analyzed them with regard to the publishing journal, year, country of origin, type of article, subspecialty, main topic, and imaging technique.

RESULTS

The top 100 articles were published in 18 imaging journals, led by Radiology (47%). These articles were published from 2016 to 2018 (94%). Most of the articles were from the United States (62%) and were original clinical studies (59%). The most common subspecialties and imaging technique of study were neuroimaging (30%) and MRI (35%). The main topics of articles were brain disease and function (30%).

CONCLUSION

Our study presents a detailed list and analysis of the top 100 articles with the highest AAS in imaging literature, which provides unique characteristics representing the public's attention in this field.

Altmetric analysis of the contemporary scientific literature in Endodontology

AIM

To analyse and visualize the knowledge structure of scientific articles in the field of Endodontology with high altmetric attention scores to discover hot topics, active researchers and the journals involved.

METHODOLOGY

On 5 June 2019, the altmetric database (Altmetric LLP, London, UK) was searched using the titles of 11 endodontic journals. Bibliometric data from endodontic articles and journals with an altmetric score >5 (top 5%) were retrieved from PubMed and analysed using the VOSviewer. Science mapping of articles with an altmetric score >5 at two levels was created: author keywords co-occurrence and co-authorship network analysis.

RESULTS

Of the 2197 articles in the field of Endodontology identified with altmetrics, 192 had altmetric scores >5 (top 5%). Considering the total mentions amongst all altmetric resources, the Journal of Endodontics had the highest rank followed by the International Endodontic Journal and Australian Endodontic Journal. Twitter was the most popular altmetric data resource followed by patents and Facebook. Meta-analysis, systematic review and pulpitis were the hot topics. At the author level, Dummer P.M.H had the greatest influence on the network. There was no significant correlation between altmetric score and citations count (P > 0.05). Mendeley mentions correlated with citations (P < 0.05).

CONCLUSIONS

Overall, the altmetric scores of topics within Endodontology were low, possibly due to the specific and specialized nature of the specialty, as well as the difficulty members of the public probably have in understanding endodontic research. Journals and researchers with a focus on Endodontology would have more influence if they were to set-up their own social media profiles and thus enhance their visibility and social impact by immediately sharing research findings and communicating with their network and audience.

Immunology and social networks: an approach towards impact assessment

Scientific journals have changed the mechanisms they use for distribution and dissemination of information. Different approaches towards determining impact have emerged and among these, metrics derived from activity on social media are an emerging trend. This article aims to assess whether a correlation exists between the traditional impact factor and activity on social media. We assessed journals categorized within the area of "immunology" on the SCImago Journal and Country Rank website. Variables reflecting traditional and alternative measures of impact were collected. Differences between journals with and without social networks were assessed using non-parametric Mann-Whitney U tests. Correlation was assessed through Spearman tests. 156 journals were analyzed, 17% had at least one social network. 48.2% of journals with social networks were classified within SJR's quartile 1. An almost perfect correlation was found between the SJR and the number of followers on Twitter, this correlation remained statistically significant after adjusting for time since creation of the account [Spearman's correlation (r_s) = 0.83]. We propose the use of Twitter as a mechanism for dissemination of information by immunology journals, as well as other social networks for their potential to increase their audience, as well as the dissemination and impact of their publications.