Newsworthiness vs scientific impact: are the most highly cited urology papers the most widely disseminated in the media?

The bibliometric and altmetric analysis of chronic traumatic encephalopathy research: how great is the impact?

Background

The study of chronic traumatic encephalopathy (CTE) has received great attention from academia and the general public. This study aims to analyze the research productivity on CTE and investigate the most discussed articles in academia and the general public by conducting bibliometric and altmetric analyses.

Methods

Data of articles were obtained from the Web of Science Core Databases and Altmetric Explore. VOSviewer and CiteSpace software were used to analyze and visualize the articles. The correlation between Altmetric attention scores (AAS) and citation counts were assessed by Spearman correlation coefficient.

Results

788 publications of CTE were eventually gathered and analyzed, and 100 articles with highest citation counts (Top-cited) and 100 articles with highest AASs (Top-AAS) were then identified. The keywords density map showed both the general public and the scientists were particularly interested in the risk factors and pathology of CTE, and scientists were interested in the causes and characteristics of neurodegenerative diseases while the public became increasingly concerned about the detection and prevention of CTE. By examining the shared characteristics of the 44 articles (High-High articles) that overlapped between Top-cited and Top-AAS articles, we identified certain traits that may potentially contribute to their high citation rates and high AASs. Besides, significant positive correlations with varied strength between AAS and citation were observed in the 788 articles, Top-cited, Top-AAS and High-High datasets.

Conclusion

This study is the first to link bibliometric and altmetric analyses for CTE publications, which may provide deeper understanding of the attention of the scientists and the general public pay to the study of CTE, and offer some guidance and inspiration for future CTE in the selection of research topics and directions.

Twitter Gynecologic Surgery Journal Club: Impact on Citation Scores and Social Media Attention

STUDY OBJECTIVE

To study the impact of a Twitter-based gynecologic surgery journal club of articles published in the Journal of Minimally Invasive Gynecology (JMIG) on their social media attention and citation scores.

DESIGN

A cross-sectional study.

SETTING

N/A.

PATIENTS

N/A.

INTERVENTIONS

Comparison of citation and social media attention scores was conducted for all articles presented in the JMIG Twitter Journal Club (#JMIGjc), a monthly scientific discussion on Twitter of JMIG selected articles, between March 2018 and September 2021 (group A), with 2 matched control groups of other JMIG articles: group B, articles mentioned on social media but not promoted in any JMIG social media account, and group C, articles with no social media mentions and not presented in #JMIGjc. Matching was performed for publication year, design, and topic in a 1:1:1 ratio. Citation metrics included number of citations per year (CPY) and relative citation ratio (RCR). Altmetric Attention Score (AAS) was used to measure social media attention. This score tracks research articles' online activity from different sources such as social media platforms, blogs, and websites. We further compared group A with all JMIG articles published during the same period (group D).

MEASUREMENTS AND MAIN RESULTS

Thirty-nine articles were presented in the #JMIGjc (group A) and were matched to 39 articles in groups B and C. Median AAS was higher in group A than groups B and C (10.00 vs 3.00 vs 0, respectively, p <.001). CPY and RCR were similar among groups. Median AAS was higher in group A than group D (10.00 vs 1.00, p <.001), as were median CPY and RCR (3.00 vs 1.67, p = .001; 1.37 vs 0.89, p = .001, respectively).

CONCLUSION

Although citation metrics were similar among groups, #JMIGjc articles had higher social media attention metrics than matched controls. Compared with all publications within the same journal, #JMIGjc articles resulted in higher citation metrics.

Measuring the impact of COVID-19 papers on the social web: an altmetric study

Purpose

The rapid spread and severity of the coronavirus (COVID-19) virus have prompted a spate of scholarly research that deals with the pandemic. The purpose of this study is to measure and assess the coverage of COVID-19 research on social media and the engagement of readers with COVID-19 research on social media outlets.

Design/methodology/approach

An altmetric analysis was carried out in three phases. The first focused on retrieving all papers related to COVID-19. Phase two of the research aimed to measure the presence of the retrieved papers on social media using altmetric application programming interface (API). The third phase aimed to measure Mendeley readership categories using Mendeley API to extract data of readership from Mendeley for each paper.

Findings

The study suggests that while social media platforms do not give accurate measures of the impact as given by citations, they can be used to portray the social impact of the scholarly outputs and indicate the effectiveness of COVID-19 research. The results confirm a positive correlation between the number of citations to articles in databases such as Scopus and the number of views on social media sites such as Mendeley and Twitter. The results of the current study indicated that social media could serve as an indicator of the number of citations of scientific articles.

Research limitations/implications

This study’s limitation is that the studied articles’ altmetrics performance was examined using only one of the altmetrics data service providers (altmetrics database). Hence, future research should explore altmetrics on the topic using more than one platform. Another limitation of the current research is that it did not explore the academic social media role in spreading fake information as the scope was limited to scholarly outputs on social media. The practical contribution of the current research is that it informs scholars about the impact of social media platforms on the spread and visibility of COVID-19 research. Also, it can help researchers better understand the importance of published COVID-19 research using social media.

Originality/value

This paper provides insight into the impact of COVID-19 research on social media. The paper helps to provide an understanding of how people engage with health research using altmetrics scores, which can be used as indicators of research performance.

A critical review on altmetrics: can we measure the social impact factor?

Altmetrics measure the digital attention received by a research output. They allow us to gauge the immediate social impact of an article by taking real-time measurements of how it circulates in the Internet. While there are several companies offering attention scores, the most extensive are Altmetric.com (Altmetric Attention Score—AAS) and Plum X (Plum Print). As this is an emerging topic, many medical specialities have tried to establish if there is a relationship between an article’s altmetric data and the citations it subsequently receives. The results have varied depending on the research field. In radiology, the social network most used is Twitter and the subspeciality with the highest AAS is neuroimaging. This article will review the process involved from the start when an article is published through to finally obtaining its altmetric score. It will also address the relationship between altmetrics and more traditional approaches focusing on citations in radiology and will discuss the advantages and limitations of these new impact indicators.

Examining the Relationship between Altmetric Score and Traditional Bibliometrics in the Ophthalmology Literature for 2013 and 2016 Cohorts

Background In this study, we reviewed a select sample of ophthalmology literature to determine if there was a correlation between Altimetric and traditional citation-based and impact factor metrics. We hypothesized that Altmetric score would more closely correlate with impact factor and citations in 2016.

Methods Journal Citation Reports for the year 2013 was used to find the 15 highest impact factor ophthalmology journals in 2013. Then Elsevier's Scopus was used to identify the 10 most cited articles from each journal for the years 2013 and 2016. Metrics for all identified articles were collected using the Altmetric Bookmarklet, and date of Twitter account creation was noted for journals with such an account. Altmetric scores, impact factor, and citation counts were tabulated for each article. Pearson's correlation coefficient (r) determined correlation of independent variables (number of citations or impact factor) with dependent variable (Altmetric score). For our Twitter analysis, account age was the independent variable and calculated correlation coefficients (r) were the dependent variable. Proportion of variance was determined with a coefficient of determination (R 2).

Results This study included 300 articles, evenly split between 2013 and 2016. Within the 2013 cohort, three journals had significant positive correlations between citation count and Altmetric score. For the 2016 cohort, both Altmetric score and citation count (r = 0.583, p < 0.001) and Altmetric score and impact factor (r = 0.183, p = 0.025) revealed significant positive correlations. In 2016, two journals were found to have significant correlations between Altmetric score and citation number. Neither year revealed a significant correlation between the age of a journal's Twitter profile and the relationship between Altmetric score and citation count. In each year, Twitter accounted for the highest number of mentions.

Conclusion The findings suggest that correlation between Altmetric score and traditional quality metric scores may be increasing. Altmetric score was correlated with impact factor and number of citations in 2016 but not 2013. At this time, Altmetrics are best used as an adjunct that is complementary but not an alternative to traditional bibliometrics for assessing academic productivity and impact.

Meta-Analysis of Correlations between Altmetric Attention Score and Citations in Health Sciences

INTRODUCTION

In recent years, several controversial reports of the correlation between altmetric score and citations have been published (range: -0.2 to 0.8). We conducted a meta-analysis to provide an in-depth statistical analysis of the correlation between altmetric score and number of citations in the field of health sciences.

METHODS

Three online databases (Web of Science, Scopus, and PubMed) were systematically searched, without language restrictions, from the earliest publication date available through February 29, 2020, using the keywords "altmetric," "citation," and "correlation." Grey literature was also searched via WorldCat, Open Grey, and Google Scholar (first 100 hits only). All studies in the field of health sciences that reported on this correlation were included. Effect sizes were calculated using Fisher's z transformation of correlations. Subgroup analyses based on citation source and sampling methods were performed.

RESULTS

From 27 included articles, 8 articles comprise several independent studies. The total sample size was 9,943 articles comprised of 35 studies. The overall pooled effect size was 0.19 (95% confidence interval 0.13 to 0.26). Bivariate partial prediction of interaction between effect size, citation source, and sampling method showed a greater effect size with Web of Science compared with Scopus and Dimensions. Egger's regression showed a marginally nonsignificant publication bias (p = 0.055), and trim-and-fill analysis estimated one missing study in this meta-analysis.

CONCLUSION

In health sciences, currently altmetric score has a positive but weak correlation with number of citations (pooled correlation = 0.19, 95% C.I 0.12 to 0.25). We emphasize on future examinations to assess changes of correlation pattern between altmetric score and citations over time.

Assessing number and quality of urology open access journals: 2011 to 2018

Background/Aims:

There is clear evidence that publishing research in an open access (OA) journal or as an OA model is associated with higher impact, in terms of number of reads and citation rates. The development of OA journals and their quality are poorly studied in the field of urology. In this study, we aim to assess the number of OA journals, their quality in terms of CiteScore, percent cited and quartiles, and their scholarly production during the period from 2011 to 2018.

Methods:

We obtained data about journals from www.scopus.com, and we filtered the list for urology journals. We obtained data for all Scopus indexed journals during the period from 2011 to 2018. For each journal, we extracted the following indices: CiteScore, Citations, scholarly output, and SCImago quartiles. We analyzed the difference in quality indices between OA and non-OA urology journals.

Results:

Urology journals have increased from 66 journals in 2011 to 99 journals in 2018. The number of OA urology journals has increased from only 10 (15.2%) journals in 2011 to 33 (33.3%) journals in 2018. The number of quartile 1 (the top 25%) journals has increased from only 1 journal in 2011 to 5 journals in 2018. Non-OA urology journals had significantly higher CiteScore compared with OA journals till the year 2015, after which the mean difference in CiteScore became smaller with insignificant p-value.

Conclusion:

Number and quality of OA journals in the field of urology have increased throughout the last few years. Despite this increase, non-OA urology journals still have higher quality and output.

YouTube as a resource for surgical education with a focus on plastic surgery - a systematic review

Surgery trainees use videos as a means to learn about surgical procedures. YouTube is the biggest online video platform and used for educational content as well but the medical information provided does not undergo peer review or other forms of scientific screening and can thus be of poorer quality. We performed a systematic review that examined the quality of educational videos about surgery and plastic surgery in particular on YouTube. The focus was towards studies on the benefit of YouTube videos for surgical trainees. A literature review was performed to determine the educational quality of plastic surgery videos found on YouTube. Articles reviewing the educational quality of videos about surgical procedures, their accuracy, and their utility for surgical trainees were included. An additional review was performed evaluating the literature about the quality of educational plastic surgery videos. Eleven articles were selected reviewing the educational quality of videos about surgical procedures. Six studies were fully assessed and evaluated concerning the quality of educational plastic surgery videos. There currently seems to be a lack of comprehensive educational surgery and in particular plastic surgery-related information on YouTube. The popularity of YouTube among surgical trainees is high. The quality of available educational surgical video content varies widely. It is in the interest of plastic surgery teaching institutions to provide trainees with high-quality educational video material.

Does quality of research in otolaryngology correlate with academic impact?

OBJECTIVE

To determine if the quality of otolaryngology-related journal articles correlates with traditional measures of article impact.

METHODS

All articles published by Laryngoscope in 2011 were categorized according to level of evidence (LOE) according to the Oxford Center for Evidence Based Medicine rubric. Articles without a level of evidence assigned were alphabetically subcategorized type with letters A-D corresponding to Contemporary Reports, Case Reports, Basic Science or Animal Studies, and Other respectively. Citations per article were then recorded per article each year from 2012 to 2018.

RESULTS

A total of 494 articles were included for analysis, 315 had numerical LOE and 179 had alphabetical LOE. There was a strong negative correlation between numerical LOE and median and interquartile number of citations (R = -.9014, P = .037). Overall, numerical LOE had a significantly higher median number of citations per article compared with the non-number/alphabetical group (14 vs 6, P < .001).

CONCLUSIONS

Higher quality research as determined by level of evidence is in fact being cited more frequently than lower quality articles. Although the scope of this study was relatively limited, these data suggest that better designed studies may exhibit greater impact by traditional measures. Such findings should serve as an impetus for (and validation of) continued pursuit of high LOE research.

LEVEL OF EVIDENCE

NA.

The Top 100 Spine Surgery Articles on Social Media: An Altmetric Study

STUDY DESIGN

Literature review.

OBJECTIVE

To discern which social media outlets contribute most to spine surgery literature dissemination and to investigate how popular articles compare to articles with most citations.

SUMMARY OF BACKGROUND DATA

Scientific literature is increasingly disseminated through social media. The Altmetric Attention Score (AAS) is defined as an automated, weighted score calculation from counts of all online attention received by various research outputs. Increasing AAS values indicate more online attention. For example, the overall top 100 Altmetric spine surgery peer-reviewed articles since 2010 had an AAS range from 78 to 1537. Among all spine surgery literature reviewed since 2010, the mean AAS was 5.3 with a median of 1.0.

METHODS

We performed an Altmetric database search of nine spine surgery journals from January 2010 to October 2019. Mean AAS was summarized alongside metrics including citation count and impact factor. We assessed correlations between AAS and online sources, readers, and citations. Journals were grouped by impact factor, and analysis-of-variance compared mean AAS. The 100 highest AAS articles were compared to the most cited.

RESULTS

13,601 articles were included. The mean AAS was 5.3, with Twitter contributing the most. The three highest associations were news (P < 0.001), Twitter (P < 0.001), and Facebook (P < 0.001). There was no significant association between impact factor and AAS. Compared with the most cited articles, the top 100 AAS articles had significantly more article types, more prospective studies, fewer retrospective studies, fewer reviews, and fewer systematic reviews (P < 0.001 for all). Spine contributed the most articles in both top 100 sets.

CONCLUSION

Our evaluation revealed Twitter, newsfeeds, and Facebook were the most significant social media outlets. Compared to articles with the most citations, the most popular articles are prospective and encompass broader study designs. Social media plays an integral role in dissemination, both within spine literature and the public sphere.

LEVEL OF EVIDENCE

3.

Trends and relevance in the bladder and bowel dysfunction literature: PlumX metrics contrasted with fragility indicators

INTRODUCTION

The concepts of fragility index (FI) and fragility quotient (FQ) have been previously described. PlumX metrics encompass online "footprints" of research in addition to traditional citations. Herein we explore PlumX metrics against the quality of BBD literature.

OBJECTIVE

To explore altmetrics against the quality of bladder and bowel dysfunction (BBD) literature.

STUDY DESIGN

A literature search was conducted using Pubmed, Medline, Embase for BBD and related terms. A total of 54,045 abstracts were screened, followed by 693 full text reviews and data extraction from 126. Studies were included if they reported on 2 groups being compared, had dichotomous outcomes, and had significant results.

RESULTS

The median FI score was 4 (0-500) and there were 20 studies which had a FI of 0. The FQ had a median of 0.04 (0-0.32). PlumX usage was 263 ± 540, captures were 45 ± 60 and social media attention was 2 ± 2. Overall, 42% of papers were clinical trials (RCTs). When compared to other study designs, we noted a significant difference in PlumX captures (57 ± 72 RCT vs. 35 ± 47 other; p = 0.03). RCTs had higher usage, social media engagement and citations however, the differences were not significant. H-Index had a significant correlation with FI (p = 0.036), however correlations for PlumX usage and captures, while modestly positive (0.04-0.10) for the FI and FQ, were not significant. A comparison of FI and FQ by topic can be reviewed in the Summary Table.

DISCUSSION

When considering the FI and FQ robustness indicators of the BBD literature, we found similarities when compared to other studies. It was reported that overall, the hydronephrosis literature was fragile with many studies requiring only a few events to nullify significance, regardless of the study design. Similarly, in a review of pediatric vesicoureteral reflux (VUR) clinical trials, results were also fragile. When comparing fragility measures to altmetric variables we noted that despite the growing popularity of altmetrics, citation counts, and h-indices remain the traditional measures to monitor research consumption. There has been a reported correlation between manuscript citation counts, author h-index, altmetrics measures in several specialties and across many domains of research including medical sciences, arts, and the humanities, however in the present study only weak correlations were noted.

CONCLUSION

The body of BBD comparative studies is fragile in keeping with other pediatric urology literature populations. Despite fragile results, RCTs generate slightly moreattention as measured by select PlumX metrics. These results suggest the need for including fragility measures in our literature, aiming to focus attention towards more robust articles.

The Use of Social Media to Increase the Impact of Health Research: Systematic Review

BACKGROUND

Academics in all disciplines increasingly use social media to share their publications on the internet, reaching out to different audiences. In the last few years, specific indicators of social media impact have been developed (eg, Altmetrics), to complement traditional bibliometric indicators (eg, citation count and h-index). In health research, it is unclear whether social media impact also translates into research impact.

OBJECTIVE

The primary aim of this study was to systematically review the literature on the impact of using social media on the dissemination of health research. The secondary aim was to assess the correlation between Altmetrics and traditional citation-based metrics.

METHODS

We conducted a systematic review to identify studies that evaluated the use of social media to disseminate research published in health-related journals. We specifically looked at studies that described experimental or correlational studies linking the use of social media with outcomes related to bibliometrics. We searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases using a predefined search strategy (International Prospective Register of Systematic Reviews: CRD42017057709). We conducted independent and duplicate study selection and data extraction. Given the heterogeneity of the included studies, we summarized the findings through a narrative synthesis.

RESULTS

Of a total of 18,624 retrieved citations, we included 51 studies: 7 (14%) impact studies (answering the primary aim) and 44 (86%) correlational studies (answering the secondary aim). Impact studies reported mixed results with several limitations, including the use of interventions of inappropriately low intensity and short duration. The majority of correlational studies suggested a positive association between traditional bibliometrics and social media metrics (eg, number of mentions) in health research.

CONCLUSIONS

We have identified suggestive yet inconclusive evidence on the impact of using social media to increase the number of citations in health research. Further studies with better design are needed to assess the causal link between social media impact and bibliometrics.

A case study exploring associations between popular media attention of scientific research and scientific citations

The association between mention of scientific research in popular media (e.g., the mainstream media or social media platforms) and scientific impact (e.g., citations) has yet to be fully explored. The purpose of this study was to clarify this relationship, while accounting for some other factors that likely influence scientific impact (e.g., the reputations of the scientists conducting the research and academic journal in which the research was published). To accomplish this purpose, approximately 800 peer-reviewed articles describing original research were evaluated for scientific impact, popular media attention, and reputations of the scientists/authors and publication venue. A structural equation model was produced describing the relationship between non-scientific impact (popular media) and scientific impact (citations), while accounting for author/scientist and journal reputation. The resulting model revealed a strong association between the amount of popular media attention given to a scientific research project and corresponding publication and the number of times that publication is cited in peer-reviewed scientific literature. These results indicate that (1) peer-reviewed scientific publications receiving more attention in non-scientific media are more likely to be cited than scientific publications receiving less popular media attention, and (2) the non-scientific media is associated with the scientific agenda. These results may inform scientists who increasingly use popular media to inform the general public and scientists concerning their scientific work. These results might also inform administrators of higher education and research funding mechanisms, who base decisions partly on scientific impact.

Predictors of High-Impact Articles in The Annals of Thoracic Surgery

BACKGROUND

Altmetric scores are increasingly used as nontraditional metrics of scholarly impact that capture article social media attention. This study aims to characterize articles from The Annals of Thoracic Surgery that achieved the greatest online reach over a longitudinal period.

METHODS

The 50 articles with the highest Altmetric scores published in The Annals of Thoracic Surgery for 2013, 2015, and 2017 were identified. Spearman rank correlation coefficient was used to assess the correlation of Altmetric scores with citations. Independent predictors of Altmetric scores (25 or greater) were identified through univariable and multivariable logistic regression analysis.

RESULTS

Over time, article Altmetric scores increased significantly (2013, 11.9; 2015, 24.8; and 2017, 75.3; P < .001); with more authors on Twitter in recent years to disseminate scholarly work (2013, 10%; 2015, 20%; and 2017, 42%; P < .001). Recent articles attracted greater attention from news outlets (2013, 1.02; 2015, 2.36; and 2017, 7.48; P < .001) and tweets (2013, 1.84; 2015, 6.68; and 2017, 27.8; P < .001), reaching a larger readership through Twitter (2013, 4210; 2015, 19,300; and 2017, 66,800; P < .001). Log-transformed Altmetric scores were correlated with log-transformed citations for articles published in 2017 (r_s = 0.40; P = .02). On multivariable analysis, tweets (odds ratio = 1.27; 95% confidence interval, 1.01-1.61; P = .044) and mentions by news outlets (odds ratio = 30.49; 95% confidence interval, 4.03-230.16; P = .001) were predictive of high Altmetric scores.

CONCLUSIONS

This longitudinal analysis demonstrates that social media attention related to top performing articles has increased in recent years. Social media is an effective tool to increase article reach and knowledge translation, with Altmetric scores that correlated with citations.

Analysis of the altmetric top 100 articles with the highest altmetric attention scores in medical imaging journals

PURPOSE

Alternative metrics (altmetrics) provides a complementary measure of the impact of scientific articles. The aim of this study was to identify and characterize the top 100 articles with the highest Altmetric attention score (AAS) in medical imaging journals.

MATERIALS AND METHODS

We searched for AASs articles published in 116 journals in the subject category "Radiology, Nuclear Medicine and Medical Imaging" using the Web of Science. We determined the top 100 articles with the highest AAS using the Altmetric.com database, and then analyzed them with regard to the publishing journal, year, country of origin, type of article, subspecialty, main topic, and imaging technique.

RESULTS

The top 100 articles were published in 18 imaging journals, led by Radiology (47%). These articles were published from 2016 to 2018 (94%). Most of the articles were from the United States (62%) and were original clinical studies (59%). The most common subspecialties and imaging technique of study were neuroimaging (30%) and MRI (35%). The main topics of articles were brain disease and function (30%).

CONCLUSION

Our study presents a detailed list and analysis of the top 100 articles with the highest AAS in imaging literature, which provides unique characteristics representing the public's attention in this field.

Comparison of Traditional Citation Metrics and Altmetrics Among Dermatology Journals: Content and Correlational Analysis Study

Background

Research impact has traditionally been measured using citation count and impact factor (IF). Academics have long relied heavily on this form of metric system to measure a publication’s impact. A higher number of citations is viewed as an indicator of the importance of the research and a marker for the impact of the publishing journal. Recently, social media and online news sources have become important avenues for dissemination of research, resulting in the emergence of an alternative metric system known as altmetrics.

Objective

We assessed the correlation between altmetric attention score (AAS) and traditional scientific impact markers, namely journal IF and article citation count, for all the dermatology journal and published articles of 2017.

Methods

We identified dermatology journals and their associated IFs available in 2017 using InCites Journal Citation Reports. We entered all 64 official dermatology journals into Altmetric Explorer, a Web-based platform that enables users to browse and report on all attention data for every piece of scholarly content for which Altmetric Explorer has found attention.

Results

For the 64 dermatology journals, there was a moderate positive correlation between journal IF and journal AAS (rs=.513, P<.001). In 2017, 6323 articles were published in the 64 dermatology journals. Our data show that there was a weak positive correlation between the traditional article citation count and AAS (rs=.257, P<.001).

Conclusions

Our data show a weak correlation between article citation count and AAS. Temporal factors may explain this weak association. Newer articles may receive increased online attention after publication, while it may take longer for scientific citation counts to accumulate. Stories that are at times deemed newsworthy and then disseminated across the media and social media platforms border on sensationalism and may not be truly academic in nature. The opposite can also be true.

Can altmetrics predict future citation counts in critical care medicine publications?

Introduction

Social media is increasingly used in the dissemination of medical research. Traditional measures of the impact of a paper do not account for this. Altmetrics are a measure of the dissemination of a publication via social media websites. The purpose of this study is to ascertain if the altmetric attention score of an article is a reliable measure of the impact it has in the field of critical care medicine. To this end, we investigated if a correlation exists between future citation count and altmetric attention score.

Methods

The top nine journals by impact factor in the field of critical care medicine were identified for 2014 and 2015. The 100 most cited articles from these journals were recorded to form the Scientific Impact Group, i.e. those with the greatest impact on the scientific community. The altmetric attention score was recorded for each article. The top 100 articles by altmetric attention score were also identified to form the Media Impact Group, i.e. those that generated the most online attention. Their citation counts' were recorded. Statistical analysis was performed on each group to identify a correlation between altmetric attention score and citation count.

Results

There was a moderately positive correlation in the Scientific Impact Group, with a Spearman r score of 0.4336 (P = 0.0001). A weakly positive correlation was found in the Media Impact Group, with a Spearman r score of 0.3033 (P = 0.002).

Conclusions

There is a positive correlation between traditional bibliographic metrics and altmetrics in the field of critical care medicine. Highly cited papers are more likely to generate online attention. However, papers that generate a lot of online attention are less likely to have a high citation count. Therefore, altmetric attention score is not a reliable predictor of future citation count in critical care medicine.

Examining the Correlation Between Altmetric Score and Citations in the General Surgery Literature

BACKGROUND

With the emergence of social media platforms, new bibliometric profiles measuring impact and exposure of scientific research online have been introduced as an alternative to traditional bibliometric outcomes. The objective of this article is to evaluate relationships between Altmetric scores, journal impact factor, and citation counts among the surgical literature.

METHODS

We analyzed the top 10 highest cited articles for the 10 general surgery journals with the highest impact factors for 2013 and 2016 by noting citation counts and Altmetric scores for each article. We also identified the journal impact factor and age of journal associated Twitter accounts. Variables were assessed for correlation using Pearson's correlation testing via Microsoft Excel.

RESULTS

A total of 240 articles were analyzed. For 2013, Altmetrics score analysis demonstrated a significant, positive correlation with citation number (r = 0.462, P < 0.0001) and journal impact factor (r = 0.439, P < 0.0001). The 2016 cohort also demonstrated significant, positive correlations between Altmetric scores and citation count after the removal of one outlier (r = 0.182, P = 0.047) and journal impact factor when considering all articles (r = 0.425, P < 0.0001). From 2013 to 2016, the total number of citations for all articles decreased from 11,027 to 7661, but cumulative Altmetric scores increased from 1078 to 4782. Age of creation for a journal's Twitter account did not significantly affect Altmetric score or traditional bibliometric measures in either 2013 (r = 0.370, P = 0.293) or 2016 (r = 0.441, P = 0.202).

CONCLUSIONS

Altmetric scores, while significantly associated with citation count in the surgical literature, should not necessarily be used as a surrogate marker for evaluating research performance, impact, or exposure. It is possible, however, that as the use of social media for distributing and sharing scientific research continues to expand, that exposure on such platforms could impact future interest or studies.

Top 100 Publications as Measured by Altmetrics in the Field of Central Nervous System Inflammatory Demyelinating Disease

Background

Altmetrics analyze the visibility of articles in social media and estimate their impact on the general population. We performed an altmetric analysis of articles on central nervous system inflammatory demyelinating disease (CIDD) and investigated its correlation with citation analysis.

Methods

Articles in the 91 journals comprising the “clinical neurology,” “neuroscience,” and “medicine, general, and internal” Web of Science categories were searched for their relevance to the CIDD topic. The Altmetric Explorer database was used to determine the Altmetric.com Attention Score (AAS) values of the selected articles. The papers with the top 100 AAS values were characterized.

Results

Articles most frequently mentioned online were primarily published after 2014 and were published in journals with high impact factors. All articles except one were dealt with the issue of multiple sclerosis. Most were original articles, but editorials were also common. Novel treatments and risk factors are the most frequent topics. The AAS was weakly correlated with journal impact factors; however, no link was found between the AAS and the number of citations.

Conclusions

We present the top 100 most frequently mentioned CIDD articles in online media using an altmetric approach. Altmetrics can rapidly offer alternative information on the impact of research based on a broader audience and can complement traditional metrics.

The impact of social media on citation rates in coloproctology

AIM

This study aimed to investigate the association between Twitter exposure and the number of citations for coloproctology articles.

METHOD

Original articles from journals using Twitter between June 2015 and May 2016 were evaluated for the following characteristics: publishing journal; article subject; study design; nationality, speciality and affiliation of the author(s); and reference on Twitter. Citation data for these articles were retrieved from Google Scholar (https://scholar.google.com) in January 2018. We performed a univariate analysis using these data followed by a multivariate, logistic regression analysis to search for factors associated with a high citation level, which was defined as accrual of more than five citations.

RESULTS

Out of six coloproctology journals listed on the InCites JCR database, three (Diseases of the Colon & Rectum, Colorectal Disease and Techniques in Coloproctology) used Twitter, where 200 (49.5%) out of a total of 404 articles had been featured. Citation rates of articles that featured on Twitter were significantly higher than those that did not (11.4 ± 9.2 vs 4.1 ± 3.1, P < 0.001). In multivariate analysis, Twitter exposure (OR 8.6, P = 0.001), European Union nationality (OR 2.4, P = 0.004), Colorectal Disease journal (OR 3.3, P = 0.005) and systematic review articles (OR 3.4, P = 0.009) were associated with higher citation levels.

CONCLUSION

Article exposure on Twitter was strongly associated with a high citation level. Medical communities should encourage journals as well as physicians to actively utilize social media to expedite the spread of new ideas and ultimately benefit medical society as a whole.

Examining the Correlation Between Altmetric Score and Citations in the Urology Literature

OBJECTIVE

To determine if article Altmetric scores correlate with journal impact factor and citation count in the urologic literature.

METHODS

We identified the top 10 most-cited articles for the 15 urology journals with the highest impact factor in 2013 and 2016. Citation count and Altmetric scores were recorded for each of the articles. The journal impact factor and date of Twitter account development were recorded for each of the journals. The variables were analyzed in Microsoft excel using Pearson's correlation testing.

RESULTS

A total of 300 articles were analyzed. In 2013, Altmetric scores and citation number showed a significant positive correlation (r = 0.164, P = .045), although Altmetric scores did not correlate with journal impact factor (r = 0.005, P = .957). In 2016, there was significant positive correlation between Altmetric scores and citation number (r = 0.268, P = .0009), as well as between Altmetric scores and journal impact factor (r = 0.201, P = .014). The total citation count decreased from 15,235 in 2013 to 8622 in 2016 while the total Altmetric score increased from 1135 in 2013 to 2563 in 2016. Older Twitter accounts were not associated with increasing correlations between Altmetric score and bibliometrics in either 2013 (r = 0.221, P = .54) or 2016 (r = 0.083, P = .819).

CONCLUSION

At this point in time, Altmetric score is only weakly correlated with citation counts in the urology literature. Altmetrics and traditional bibliometrics should be viewed as complements to one another rather than surrogates when determining research dissemination and impact.

Participation of the hospitals in the Republic of Ireland in international research over more than a decade: a bibliometric analysis

BACKGROUND

This study provides an accurate awareness of the present situation of health research in Irish hospitals both public and private. We aimed to analyze factors that may influence it and provide recommendations for active steps to improve the current situation of Irish health research based on our findings.

METHODS

We performed a bibliometric analysis to assess qualitatively and quantitatively the publications from Ireland over a period between 2007 and 2018. We also investigated the associated variables with the quality of research. Furthermore, we conducted a Joinpoint analysis to see the trends in Irish research over these years.

RESULTS

From 12,828 included peer-reviewed articles, the average citation count per article was 19.98. Furthermore, we showed that a higher impact factor (IF) and institutions number, present per article, were significantly associated with more citations. Also, the publication count and the mean IF showed an increase over the years according to the Joinpoint analysis. Moreover, the oncology research had the highest output, followed by pediatrics, then neurology while the specialties with least publications were ear, nose, and throat (ENT), urology, plastic surgery, and dentistry. Additionally, cardiovascular, obstetrics and gynecology, oncology, pediatrics, pulmonology, dermatology, ophthalmology, dentistry, and radiology research showed an increased publication count trend in recent years. While anesthesiology, ENT, general surgery, gastroenterology and hepatology, infection and tropical medicine, nephrology, neurology, orthopedics, plastic surgery, and urology showed a decrease in the publications trend.

CONCLUSIONS

Our findings may serve as a useful approach to benchmark scientific output from hospitals and guide the future allocation of research spending.

Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy

BACKGROUND AND AIMS

Social media activity in the area of scientific journal publications has increased significantly over the last decade. Several studies have suggested that journal article citations can be positively impacted by an increase in social media activity. Preliminary data have also suggested that published articles in the field of gastroenterology and hepatology that were discussed on Twitter were independently associated with higher citation rates compared with those that were not. Gastrointestinal Endoscopy (GIE) is the most widely cited endoscopy-focused journal in the world. We aimed to assess the association of social media exposure of published articles in GIE and its impact on article citations.

METHODS

Data regarding journal article publication status, number of citations per article, and social media exposure per article using Altmetric data were collected from the publisher. All original articles published in GIE from 2000 to 2016 were reviewed. Editorials, case reports, and other nonresearch correspondence were excluded. Multivariable linear regression was used to assess for independent Altmetric predictors of higher citation rates over time. Logistic regression was used to assess the impact of an article mention on various social media platforms and whether it was cited during the study period.

RESULTS

A total of 2361 original research articles were evaluated; 2050 articles (86.8%) were cited at least once during the follow-up period. Mean number of citations per article was 16.8 ± 23. The average Altmetric Attention score was 2.0 ± 13. The most profound independent predictor of article citation was whether an article was tweeted (odds ratio [OR], 14.2; 95% confidence interval [CI], 8.93-22.45). Other predictors were Facebook posts (OR, 1.08; 95% CI, 1.03-1.51) and number of Mendeley readers (OR, 1.359; 95% CI, 1.28-1.45). On multivariable linear regression, number of tweeters (β = 2.3, P = .022), F1000 reviews (β = 5.87, P < .001), policy documents (β = 7.6, P < .001), and number of Mendeley readers (β = 14.21, P < .001) were significantly associated with higher citation rates of published articles.

CONCLUSIONS

For original articles published in GIE, there was a strong association between social media exposure on Twitter and rates of journal article citations. To a lesser degree, Facebook posts and Mendeley readers also were associated with a higher rate of article citations. This could represent both cause and effect and may be representative of higher quality articles being more commonly mentioned on social media by third-party users. A randomized controlled study evaluating different degrees of social media exposure on individual articles could be considered to further assess for causality.

The most mentioned neurointervention articles in online media: a bibliometric analysis of the top 101 articles with the highest altmetric attention scores

Background and purpose

Alternative metrics (altmetrics), based on mentions in online media, is a new tool that can help to identify the most influential articles to diverse public audiences. This article aimed to determine the 100 most mentioned articles in the field of neurointervention and to analyze their characteristics.

Materials and methods

We selected the 808 journals that were considered journals potentially publishing articles on neurointervention. We also selected articles using keywords. Using the Altmetric.com search tool, we identified the 101 most mentioned neurointervention articles based on the highest altmetric attention scores (AASs) within selected journals and articles. Each article was evaluated for several characteristics including AAS, number of citations, journal title, journal category, impact factor of the journal, year of publication, authorship, country, type of document, and topic.

Results

The AASs for the top 101 articles ranged from 1586 to 39. Stroke published the largest number of articles (19.8%) followed by the New England Journal of Medicine (17.8%). The majority of articles were published in multidisciplinary journals (38.6%), were published in 2017–2018 (43.6%), originated from the USA (54.5%), were original articles (66.3%), and dealt with intra-arterial thrombolysis or thrombectomy for acute ischemic stroke (58.4%). Tudor G Jovin was the most prolific author, authoring 18 of the most mentioned neurointervention articles

Conclusions

This study presents a detailed list of the 101 most mentioned neurointervention articles in online media, thus providing useful information on the dissemination of neurointervention research to the general public.

Measuring Scholarly Productivity: A Primer for Junior Faculty. Part III: Understanding Publication Metrics

There are approximately 78 indexed journals in the specialty of emergency medicine (EM), making it challenging to determine which is the best option for junior faculty. This paper is the final component of a three-part series focused on guiding junior faculty to enhance their scholarly productivity. As an EM junior faculty's research career advances, the bibliometric tools and resources detailed in this paper should be considered when developing a publication submission strategy. The tenure and promotion decision process in many universities relies at least in part on these types of bibliometrics. This paper provides an understanding of new, alternative metrics that can be used to promote scientific progress in a transparent and timely manner.

Increasing Interest of Mass Communication Media and the General Public in the Distribution of Tweets About Mental Disorders: Observational Study

Background

The contents of traditional communication media and new internet social media reflect the interests of society. However, certain barriers and a lack of attention towards mental disorders have been previously observed.

Objective

The objective of this study is to measure the relevance of influential American mainstream media outlets for the distribution of psychiatric information and the interest generated in these topics among their Twitter followers.

Methods

We investigated tweets generated about mental health conditions and diseases among 15 mainstream general communication media outlets in the United States of America between January 2007 and December 2016. Our study strategy focused on identifying several psychiatric terms of primary interest. The number of retweets generated from the selected tweets was also investigated. As a control, we examined tweets generated about the main causes of death in the United States of America, the main chronic neurological degenerative diseases, and HIV.

Results

In total, 13,119 tweets about mental health disorders sent by the American mainstream media outlets were analyzed. The results showed a heterogeneous distribution but preferential accumulation for a select number of conditions. Suicide and gender dysphoria accounted for half of the number of tweets sent. Variability in the number of tweets related to each control disease was also found (5998). The number of tweets sent regarding each different psychiatric or organic disease analyzed was significantly correlated with the number of retweets generated by followers (1,030,974 and 424,813 responses to mental health disorders and organic diseases, respectively). However, the probability of a tweet being retweeted differed significantly among the conditions and diseases analyzed. Furthermore, the retweeted to tweet ratio was significantly higher for psychiatric diseases than for the control diseases (odds ratio 1.11, CI 1.07-1.14; P<.001).

Conclusions

American mainstream media outlets and the general public demonstrate a preferential interest for psychiatric diseases on Twitter. The heterogeneous weights given by the media outlets analyzed to the different mental health disorders and conditions are reflected in the responses of Twitter followers.

An examination of the effect of open versus paywalled access publication on the disseminative impact and citation count of publications in intensive care medicine and anesthesia

PURPOSE

We aimed to assess the impact of open access (OA) versus paywalled access (PA) publication on Altmetric Attention Scores (AAS) and whether AAS correlates with future citation count access in the context of intensive care medicine (ICM) and anesthesia.

METHODS

1854 and 2935 publications, in the year 2015, were identified in ICM and anesthesia respectively, using a Pubmed search. The mean AAS was measured for each article.

RESULTS

More ICM articles were OA, compared to of anesthesia articles (38.9% v 35.0% p = 0.02). The mean AAS for OA ICM publications was significantly higher than that of PA ICM publications (17.34 vs 8.45, p < 0.01), however, this was not observed when examined in a fixed follow up time frame. AAS appear to correlate with future citation counts.

CONCLUSIONS

ICM publications that are available as OA in the medium term result in higher AAS when compared to PA publications, this phenomenon was not observed in anesthesia. AAS correlate with future citation counts, however, a larger study is required to confirm this.

Communicating your research (part 2): to the wider community

Dissemination of research findings via digital tools and research engagement activities is rapidly becoming accepted practice for reaching a wider audience. In addition, they offer the opportunity for finding collaborative research partners, networking with peers, and informing funders, clinical practitioners and policy makers. However, exposure should extend beyond a scientific audience and should incorporate the general public, patients and their families, enabling them to be involved with research, facilitating the potential impacts of research to be realised.

Level of evidence: Not applicable for an opinion piece.

The Use of Twitter by the Trauma and Orthopaedic Surgery Journals: Twitter Activity, Impact Factor, and Alternative Metrics

Aim Social media (SoMe) platforms have become leading methods of communication and dissemination of scientific information in the medical community. They allow for immediate discussion and widespread engagement around important topics. It has been hypothesized that the activity on Twitter positively correlates with highly cited articles. The purpose of this study was to analyze the prevalence and activity of Trauma and Orthopaedic Surgery journals on Twitter, with the hypothesis that the impact factor is positively associated with the Twitter usage. Methods The top 50 Trauma and Orthopaedic Surgery journals, ranked by 2016 Impact Factor were analyzed. The Twitter profiles of each journal or affiliated society were identified. Other SoMe platforms used were also recorded. The Twitonomy software (Digonomy Pty Ltd, New South Wales, Australia) was used to analyze the Twitter profiles over a one-year period. The Twitter Klout scores were recorded for each journal to approximate the SoMe influence. The Altmetric scores (the total number of mentions via alternative metrics) were also recorded. The statistical analysis was carried out to identify correlations between journal Impact Factors, SoMe activity, Twitter Klout scores and Altmetric scores.  Results Twenty-two journals (44%) changed to 'had dedicated Twitter profiles [corrected]. Fourteen journals (28%) were associated with societies that had profiles and 14 journals (28%) had no Twitter presence. The mean Impact Factor overall was 2.16 +/- 0.14 (range, 1.07-5.16). The journals with dedicated Twitter profiles had higher Impact Factors than those without (mean 2.41 vs. 1.61; P=0.005). A greater number of Twitter followers were associated with higher Impact Factors (R2 0.317, P=0.03). The journals with higher Twitter Klout scores had higher Impact Factors (R2 0.357, P=0.016). The Altmetric score was positively associated with an Impact Factor (R2 0.310, P=0.015). The journals with higher numbers of retweets (virtual citations in the Twittersphere) had higher Altmetric scores (R2 0.463, P=0.015). Conclusion The Trauma and Orthopaedic Surgery journals with dedicated Twitter profiles have higher Impact Factors than those without. The Altmetrics is likely to play a significant role in the literature evaluation going forward along with the traditional metrics. The engagement with the Twitter by Trauma and Orthopaedic surgeons should be encouraged.