Technique Articles Are More Effective at Increasing Social Media Attention in Comparison With Original Research Articles: An Altmetrics-Based Analysis

Artificial Intelligence Research Receives Similar Online Attention but Increased Citation Rates Compared With Control Articles

PURPOSE

To compare the social media attention and citation rates between artificial intelligence (AI) and non-AI research in the orthopaedic surgery literature.

METHODS

Research articles using AI for the purpose of statistical analysis or evaluating the use of AI for specific clinical use cases were included. Articles selected from 6 high-impact orthopaedic surgery journals between January 2018 and December 2023 were selected by month of publication in a 1:4 ratio within the same journal issue. The primary outcomes were the Altmetric Attention Score (AAS), an aggregate score comprising the degree of social media attention received on multiple online platforms (with higher scores indicating more attention), and cumulative citation rates. Multivariate linear regression analyses were conducted to assess the relationship (1) between article type and the AAS and (2) between article type and the citation rate. A subanalysis comparing the AAS and citation rates between articles using AI for the purpose of statistical analysis versus evaluating the use of AI for a specific clinical use case was performed.

RESULTS

A total of 540 articles (110 AI-related articles [20.0%] and 440 control articles [80.0%]) were included. The median AAS was 2.0 (interquartile range [IQR], 1-6) for AI-related articles and 1.0 (IQR, 0-5) for control articles (P = .006). However, when controlling for covariates in the multivariate regression, article type (AI vs control) was not significantly associated with the AAS (β = -1.1, P = .602). The median number of citations was significantly greater among AI-related articles (median, 18 [IQR, 9-35]) compared with control articles (median, 9 [IQR, 4-16]) (P < .001). Multivariate linear regression showed an additional average increase of 13.4 citations (95% confidence interval, 9.4-17.5; P < .001) per article for AI articles compared with control articles.

CONCLUSIONS

AI research published in 6 orthopaedic surgery journals received approximately 13 additional citations on average compared with control articles published during a similar period. AI research did not receive greater social media attention on average after controlling for confounding variables.

CLINICAL RELEVANCE

AI research in orthopaedic surgery did not receive greater online attention but was cited more frequently relative to control articles. Given the increasing academic impact of these articles, orthopaedic surgeons should become familiarized with AI research as clinical applications and AI solutions derived from such research become increasingly relevant and implemented within clinical practice workflows.

Evaluating Funding for Authors Achieving High Citations vs. Online Popularity

INTRODUCTION

The Altmetric score is a validated tool that measures online attention of scientific studies. A relationship between government/industry funding for authors and their study's citations have been previously demonstrated. It is not known whether funding is related to greater online attention. We hypothesize authors publishing studies achieving greater online attention (higher Altmetric scores) receive greater monetary support from industry while authors publishing studies achieving critical acclaim (more citations) receive greater monetary support from the National Institute of Health (NIH).

METHODS

Top spine surgery studies between 2010 and 2021 were selected based on Altmetric scores and citation number. The Open Payments Database was accessed to evaluate industry financial relationships while the NIH Research Portfolio Online Reporting Tool was accessed to evaluate NIH funding. Payments were compared between groups and analyzed with the Student t-test, analysis of variance, and chi square analysis. Alpha <0.05.

RESULTS

There were 60 and 51 authors with payment data in the top 50 Altmetric and top 50 citation studies, respectively, with eight authors having studies in both groups. Total industry payments between groups were not markedly different. The eight authors with studies in both groups received markedly more industry payments for consulting, travel/lodging, and faculty/speaking fees. Authors with articles in both groups (50%) were significantly more likely to receive NIH support, compared with authors of the top Altmetric articles (5%; P < 0.001) and top citation articles (12%; P < 0.001). Authors receiving NIH support received significantly less industry payments compared with authors not receiving NIH support ($148,544 versus $2,159,526; P < 0.001).

DISCUSSION

These findings reject our hypothesis: no notable differences for industry payments and NIH funding between authors for top Altmetric and citation studies. Authors receiving funding from industry versus the NIH are generally two distinct groups, but there is a small group supported by both. These studies achieve both critical acclaim (citations) and online popularity (Altmetric scores).

DATA AVAILABILITY

Data can be available on reasonable request.

Post or perish? Social media strategies for disseminating orthopedic research

Social media usage, particularly Twitter, among scientists in academia has increased in recent years. However, Twitter's use in scholarly post-publication dissemination of orthopaedic research and musculoskeletal advocacy remains low. To enhance usage of Twitter among musculoskeletal researchers, this article reviews data supporting the professional benefits of using the platform to disseminate scholarly works. Next, we provide a linear workflow for Tweet curation, discuss the importance of data-driven decision making behind tweet curation and posting, and propose new guidelines for professional Twitter usage. Since this workflow may not eliminate all the identified barriers and new institutionalized shifts in policies regarding curation and consumption of social media on Twitter, we also briefly introduce and explore using other social media platforms. We hope this information will be persuasive and compelling to those in the orthopedic research field and be broadly applicable to others in related scientific fields who wish to disseminate findings and engage a public audience on social media. In addition, we encourage the Orthopedic Research Society (ORS) and Journal of Orthopedic Research (JOR) communities to take advantage of the many tools curated by the Wiley editorial office and the ORS social media committee to increase dissemination of their scholarly works online. Twitter and social media can assist in accomplishing our mission of creating a world without musculoskeletal limitations via the timely dissemination of orthopedic information. However, this can only be accomplished if the orthopedic research community has a unified and strong online presence actively engaged in orthopaedic research findings and news.

Open Access Articles Garner Increased Social Media Attention and Citation Rates Compared With Subscription Access Research Articles: An Altmetrics-Based Analysis

BACKGROUND

To better understand the research impact on social media, alternative web-based metrics (Altmetrics) were developed. Open access (OA) publishing, which allows for widespread distribution of scientific content, has become increasingly common in the medical literature. However, the relationship between OA publishing and social media impact remains unclear.

PURPOSE

To compare social media attention and citation rates between OA and subscription access (SA) research articles within the orthopaedic and sports medicine literature.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published as either OA or SA in 5 high-impact hybrid orthopaedic journals between January 2019 and December 2019 were analyzed. The primary outcome was the Altmetric Attention Score (AAS), a validated measure of social media attention. Secondary outcomes included citation rates, article characteristics, and the number of shares on social media. Independent t tests and chi-square analyses were used to compare outcomes between OA and SA articles. A multivariable linear regression analysis was performed to determine the association between article type and AAS while controlling for bibliometric characteristics.

RESULTS

A total of 2143 articles (246 OA articles, 11.5%; 1897 SA articles, 88.5%) were included. The mean AAS among all OA articles was 62.4 ± 184.6 (range, 0-2032), whereas the mean AAS among all SA articles was 18.4 ± 109.8 (range, 0-3425), representing a statistically significant difference (P < .001). The mean citation rate among OA articles was significantly higher (17.0 ± 22.5; range, 0-139) than that of SA articles (8.6 ± 13.4; range, 0-169) (P < .001). Multivariable linear regression analysis demonstrated that OA status (β = 15.15; P = .044), number of institutions (β = 2.13; P = .023), studies classified as epidemiological investigations (β = 107.40; P < .001), and disclosure of a conflict of interest (β = -11.18; P = .032) were significantly associated with a higher AAS.

CONCLUSION

OA articles resulted in significantly greater AAS and citations in comparison with SA articles. Articles published through the OA option in hybrid journals as well as those with a higher number of institutions, those that disclosed a conflict of interest, and those classified as epidemiological investigations were positively associated with greater AAS in addition to a greater number of citations. The potential for more extensive research dissemination inherent in the OA option may therefore translate into greater reach and social media attention.