The relative citation ratio and the h-index among academic ophthalmologists: A retrospective cross-sectional analysis

Descriptive Analysis of Vitreoretinal Surgery Fellowship Program Directors

Purpose: To evaluate the demographic, educational, and scholarly characteristics of Association of University Professors of Ophthalmology-accredited vitreoretinal surgery fellowship program directors in the United States and Canada. Methods: Demographic, educational, and scholarly profiles of identified program directors were collated from online public resources. Characteristics were compared by sex, program size, ranking, and affiliation. Results: Eighty-one program directors (mean age [±SD] 54.7 ± 11.0 years) from 78 fellowship programs were identified. The minority were women (14.8%), who were on average 6 years younger than their male counterparts (P = .07). The majority of program directors had an academic affiliation (90.1%), most commonly professor (54.8%). The mean h-index, 5-year h-index, and m-quotient were 20.9 ± 14.9, 5.9 ± 4.4, and 0.82 ± 0.42, respectively. Compared with their counterparts, program directors of both "top 10" and large programs published more manuscripts (P < .05), accrued more citations (P < .05), and had a higher h-index (P < .05). Fellowship programs with female program directors had a significantly larger proportion of female retina faculty (P = .002). Conclusions: The backgrounds of vitreoretinal surgery program directors are diverse. However, women remain underrepresented in this position, highlighting an area with the potential for greater equity in ophthalmology.

Analysis of Research Activity Among Ophthalmology Fellows

PURPOSE

The academic success of ophthalmology fellows may be significantly impacted by their research engagement. Evaluating the research activity of fellows by subspecialty may provide insight into trends useful for prospective applicants. The objective of this study was to assess the research activity of ophthalmology fellows in the year 2023.

METHODS

Ophthalmology fellowship programs were compiled using the Association of University Professors of Ophthalmology and the American Society of Ophthalmic Plastic and Reconstructive Surgery websites. Program subspecialties were categorized as: Cornea and External Disease [CED]; Cornea, External Disease, and Refractive Surgery [CEDRS]; Glaucoma; Medical Retina [MR]; Neuro-Ophthalmology [NO]; Oculoplastics; Pediatric Ophthalmology [PO]; Surgical Retina [SR]; Uveitis; and Other. "Other" consisted of Oncology, Pathology, combined Oncology and Pathology fellowships. We identified current fellows using publicly available online information. Programs and fellows that did not have information available were excluded. Total publications, Hirsch index (h-index), and weighted-relative citation ration (w-RCR) were utilized as measures of research output, while total citations and mean-RCR (m-RCR) served as proxies for research impact. Duration of publishing was calculated using the years of the oldest and most recent publications. Statistical analysis was performed using Kruskal-Wallis H tests with an alpha value of .05.

RESULTS

A total of 373 fellows from 229 programs met our inclusion criteria. More than half of fellows were men (54.4%), and the most common degree type was MD (93.0%). Across all subspecialties, the median h-index was 3.0 (IQR = 4), m-RCR was .9 (IQR = 1.2) and w-RCR was 3.2 (IQR = 9.8). The median number of publications was six (IQR = 10, with 34 citations (IQR = 110) and 4 years of publishing (IQR = 6). We observed significant differences in h-index (p = .038), total publications (p < .001), and w-RCR (p = .028) by subspecialty.

CONCLUSION

We observed significant differences in research output, but no differences in research impact by subspecialty. Overall, Uveitis and Oculoplastics fellows had higher research activity, while Medical Retina and Other fellows had the lowest. This data is pertinent to better understand the landscape of ophthalmology fellowship applications.

Evaluation of Research Productivity Among Academic Cornea, External Diseases, and Refractive Surgery Ophthalmologists Using the Relative Citation Ratio

PURPOSE

The purpose of this study was to provide relative citation ratio (RCR) benchmark data for cornea and external diseases specialists.

DESIGN

This is a cross-sectional bibliometric analysis.

SUBJECTS

Subjects included were fellowship-trained cornea and external diseases faculty at Accreditation Council for Graduate Medical Education-accredited institutions in the United States.

METHODS

Academic specialists were indexed using the National Institutes of Health iCite Web site. Publication count, mean RCR score, and weighted RCR score were obtained between October 2022 and January 2023 by examining PubMed-listed publications from 1980 to 2022. Data were compared by sex, career duration, academic rank, and acquisition of a Doctor of Philosophy.

MAIN OUTCOME MEASURES

The main outcome measures were publication count, mean RCR value, and weighted RCR value.

RESULTS

The cohort included 602 specialists from 112 Accreditation Council for Graduate Medical Education-accredited institutions. These clinician-scientists produced highly impactful research with a median publication count of 15 (interquartile ranges 4-41), median RCR of 1.4 (interquartile ranges 0.91-1.88), and median-weighted RCR of 20.28 (5.3-66.69). Both academic rank and career length were associated with greater publication count and RCR values. Male sex was also associated with greater publications counts and RCR scores compared with female faculty. Acquisition of a Doctor of Philosophy was associated with greater publication counts and weighted RCR scores but no difference in mean RCR scores.

CONCLUSIONS

Academic cornea and external diseases specialists conduct high-impact research, with a median RCR of 1.4, exceeding the NIH standard value of 1. These data provide RCR benchmark data for the field to inform self, institutional, and departmental evaluations. These results also highlight a significant gender disparity in the field necessitating efforts to increase female representation and ensure equal opportunities.

The Relative Citation Ratio: An Impartial Assessment of Productivity in Academic Otolaryngologists

OBJECTIVES

Quantification of academic productivity relies on bibliometric measurements, such as the Hirsch index (h-index). The National Institutes of Health (NIH) recently developed the relative citation ratio (RCR), an article-level, citation-driven metric that compares researchers with others within their respective fields. Our study is the first to compare the usage of RCR in academic otolaryngology.

STUDY DESIGN

Retrospective Database Review.

METHODS

Academic otolaryngology residency programs were identified using the 2022 Fellowship and Residency Electronic Interactive Database. Demographic and training data were collected for surgeons using institutional websites. RCR was calculated using the NIH iCite tool, and h-index was calculated using Scopus. Mean RCR (m-RCR) is the average score of the author's articles. Weighted RCR (w-RCR) is the sum of all article scores. These derivatives are a measure of impact and output, respectively. The career duration of a physician was categorized into the following cohorts: 0-10, 11-20, 21-30, and 31+ years.

RESULTS

A total of 1949 academic otolaryngologists were identified. Men had higher h-indices and w-RCRs than women (both p less than 0.001). m-RCR was not different between genders (p = 0.083). There was a difference in h-index and w-RCR (both p less than 0.001) among the career duration cohorts, but there was no difference in m-RCR among the cohorts (p = 0.416). The faculty rank professor was the greatest for all metrics (p < 0.001).

CONCLUSION

Critics of the h-index argue that it is reflective of the time a researcher has spent in the field, instead of impact. The RCR may reduce historic bias against women and younger otolaryngologists.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:592-599, 2024.

Research Productivity and Impact in Foot and Ankle Surgery: Insights From Relative Citation Ratio Analysis of Recent Fellowship Graduates

INTRODUCTION

The relative citation ratio (RCR), a novel bibliometric tool supported by the National Institute of Health, provides a standardized approach to evaluate research productivity and impact across different fields. This study aims to evaluate RCR of fellowship-trained foot and ankle orthopaedic surgeons to analyze the influence of various surgeon demographics.

METHODS

Fellow names listed on the American Orthopaedic Foot and Ankle Society website were extracted from the year 2008 to 2009 to the year 2022 to 2023. Demographic information for each fellow was collected including sex, degree type, and academic title. The iCite database developed by the National Institute of Health was used to obtain total publications, mean RCR, weighted RCR, and change in RCR after fellowship graduation for each fellow. Univariate and multivariate analysis was conducted to predict these four parameters based on sex, degree type, academic position, and career longevity.

RESULTS

Of the 820 fellows, 674 (82%) were male. Most fellows (n = 587, 71%) did not go on to hold academic positions. Multivariate analysis revealed that male sex (β = 2.32, P < 0.001), holding an academic position (β = 6.44, P < 0.001), holding a PhD (β = 22.96, P < 0.001), and a shorter length time since graduation (β = -0.50, P < 0.001) were independent predictors of number of total publications. Holding a DO degree was an independent predictor of decreased mean RCR (β = 0.39, P = 0.039). Finally, multivariate analysis revealed that male sex (β = 4.05, P = 0.003), a career in academics (β = 4.61, P < 0.001), and a shorter time since graduation (β = -0.45, P = 0.001) were associated with a larger weighted RCR.

DISCUSSION

The findings highlight the importance of addressing gender disparities and promoting research opportunities across different programs. Moreover, academic institutions should provide adequate support and mentorship to early-career foot and ankle-trained orthopaedic surgeons to foster sustained research productivity.

Retainment of U.S. Orthopaedic Surgeons in Academia from 2016 to 2022

Background

Academic medical centers greatly benefit from retaining their physicians; that ensures continuity in patient care, enhances resident education, and maintains a pool of experienced clinicians and researchers. Despite its importance, little research has been published on the retainment of academic faculty in orthopaedics. To address this gap, this study investigates the demographic trends of academic orthopaedic surgeons from 2016 to 2022. By analyzing data pertaining to gender distribution, years of practice, research productivity, and institutional rankings, we aimed to gain insights into the factors influencing faculty retainment, institution changes, and new entrants into academic orthopaedics.

Methods

A retrospective cross-sectional analysis of U.S. academic orthopaedic surgeons affiliated with programs under the Accreditation Council for Graduate Medical Education (ACGME) in 2016 and 2022 was performed. Faculty present in both the 2016 and the 2022 data were classified as being "retained" in academia; those present only in 2016, as having "left" academia; and those present only in 2022, as being "new" to academia. The retained group was then divided into movers (those who moved to other institutions) and non-movers.

Results

Retained orthopaedists had fewer years of practice, a higher h-index (Hirsch index), and more publications. Non-fellowship-trained orthopaedists had less retainment in academia, and orthopaedists with fellowships in oncology had more retainment in academia. Additionally, movers also had fewer years in practice but an equal level of scholarly productivity when compared with non-movers. Lastly, higher-ranked academic programs retained a greater proportion of orthopaedic surgeons.

Conclusions

Over the study period, a majority of orthopaedists (56.99%) chose to remain in academia. Those retained tended to be in the earlier stages of their careers, yet demonstrated higher research output. Notably, the representation of female orthopaedists in academic orthopaedics is on the rise. Conversely, lower-ranked programs faced higher turnover rates, highlighting the challenges that they encounter in retaining faculty members.

Clinical Relevance

Academic medical centers benefit from retaining orthopaedic surgeons by maintaining patient relationships, having consistency in resident education, and building on clinical and research expertise. Likewise, orthopaedists benefit from understanding the trends in current academic employment, in order to optimize career planning decisions.

Academic Rank and Productivity Among United States Cardiothoracic Surgeons

BACKGROUND

Despite a significant growth of women trainees in cardiothoracic surgery recently, women remain a minority of cardiothoracic surgeons and hold a minority of leadership positions. This study evaluates differences in cardiothoracic surgeon subspecialty choices, academic rank, and academic productivity between men and women.

METHODS

The Accreditation Council for Graduate Medical Education database was used to identify 78 cardiothoracic surgery academic programs in the United States, including integrated, 4+3, and traditional fellowships, as of June 2020. A total of 1179 faculty members were identified within these programs, 585 adult cardiac surgeons (50%), 386 thoracic surgeons (33%), and 168 congenital surgeons (14%), and other, 40 (3%). Data were collected using institutional websites, ctsnet.org, doximity.com, linkedin.com, and Scopus.

RESULTS

Of the 1179 surgeons, only 9.6% were women. Overall, women composed 6.7% of adult cardiac, 15% of thoracic, and 7.7% of congenital surgeons. Among subspecialties, women represent 4.5% (17 of 376) of full professors and 5% (11 of 195) of division chiefs in cardiothoracic surgery in the United States, have shorter career durations, and lower h-indices compared with men. However, women had similar m-indices, which factors in career length, compared with men in adult cardiac (0.63 vs 0.73), thoracic (0.77 vs 0.90), and congenital (0.67 vs 0.78) surgeons.

CONCLUSIONS

Career duration, including cumulative research productivity, appears to be the most important factors predicting full professor rank, potentially contributing to persistent sex-based disparities in academic cardiothoracic surgery.

Examination of the Novel National Institutes of Health-Supported Relative Citation Ratio, a Measure of Research Productivity, Among Academic Plastic Surgeons

BACKGROUND

Considerable interest has been devoted to quantifying research productivity for the purposes of academic appointment and promotion in plastic surgery. A novel bibliometric, the relative citation ratio (RCR), integrates features unavailable in prior metrics, such as the h-index, including the ability to compare researchers in distinct fields. This investigation examines the RCR in relation to established measures of academic productivity and provides the benchmark data in plastic surgery.

METHODS

Online sources were queried to identify the characteristics of 955 academic plastic surgeons from 94 programs, ie, academic rank, gender, degrees, and fellowships. Bibliometric data were acquired using the iCite and Scopus databases. Comparative and correlational analyses of variables were performed.

RESULTS

Academic plastic surgeons were exceptionally productive, with a mean RCR of 1.20 (interquartile range, 0.79-1.67) and a weighted RCR of 17.68 (interquartile range, 5.14-52.48). Increased mean RCR was significantly associated with advanced academic rank and fellowship training. Increased weighted RCR was significantly associated with advanced academic rank, male gender, PhD acquisition, publication experience, and fellowship training. The h-index was weakly correlated with mean RCR but strongly correlated with weighted RCR and publication experience.

CONCLUSIONS

The RCR was associated with established markers of academic productivity, indicating its validity as a reliable field-normalized measure for the evaluation of plastic surgery faculty. Because scholarly output is a potential factor with respect to decisions of hiring, promotion, and allocation of funding, this modality of standardized comparison is paramount for plastic surgeons who exist within a larger general surgery department.

Use of the Relative Citation Ratio in Conjunction With H-Index to Promote Equity in Academic Orthopaedics

INTRODUCTION

Quantification of a researcher's productivity relies on objective bibliometric measurements, such as the Hirsch index (h-index). However, h-index is not field and time-normalized and possesses bias against newer researchers. Our study is the first to compare the relative citation ratio (RCR), a new article-level metric developed by the National Institutes of Health, with h-index in academic orthopaedics.

METHODS

Academic orthopaedic programs in the United States were identified using the 2022 Fellowship and Residency Electronic Interactive Database. Available demographic and training data for surgeons were collected. RCR was calculated using the National Institutes of Health iCite tool, and h-index was calculated using Scopus.

RESULTS

Two thousand eight hundred twelve academic orthopaedic surgeons were identified from 131 residency programs. H-index, weighted RCR (w-RCR), and mean RCR (m-RCR) all significantly differed by faculty rank and career duration. However, while h-index and w-RCR varied between sexes (P < 0.001), m-RCR did not (P = 0.066), despite men having a longer career duration (P < 0.001).

DISCUSSION

We propose that m-RCR be used in conjunction with w-RCR or h-index to promote a fairer, comprehensive depiction of an orthopaedic surgeon's academic effect and productivity. Use of m-RCR may reduce the historic bias against women and younger surgeons in orthopaedics, which has implications in employment, promotion, and tenure.

Publication trends in the field of the cornea in the last 4 decades: a bibliometric study

PURPOSE

The objective of this bibliometric study was to identify the top 100 most-cited articles on the cornea published in the English language between 1980 and 2021 using multidimensional citation analysis.

METHODS

The data were obtained from the Thomson Reuters Web of Science Core Collection and the PubMed databases. The top 100 articles in terms of citation number were identified and analyzed.

RESULTS

A total of 40,792 articles related to the cornea were retrieved. The 100 most-cited articles were published between 1995 and 2000. The average time since publication was 19.64 ± 5.75 years. The mean impact factor of the journals was 10.27 ± 17.14 and the Q category of most journals was Q1. Ophthalmology was the journal with the most published articles (n = 10), which represented level 3 evidence. The three most common topics among the top 100 articles were treatment modality, histopathology, and diagnostic imaging. The most frequently mentioned treatments were related to limbal stem cell failure, crosslinking, and lamellar keratoplasty. We observed a negative correlation between the average number of citations per year and the time passed since publication (r = - 0.629; p = 0.001).

CONCLUSION

Our analysis of the top 100 most-cited articles on the cornea revealed scientific contributions, vital current data related to clinical implementations, and valuable insights into the current developments in ophthalmology. To our knowledge, this is the first study to evaluate the most influential papers on the cornea, and our findings highlight the research quality and latest discoveries and trends in the management cornea diseases.

The Relative Citation Ratio: Evaluating a New Measure of Scientific Influence Among Academic Sports Medicine Surgeons

Background

Objective measures of research influence are being increasingly utilized to evaluate and compare academic faculty. However, traditional bibliometrics, such as the Hirsch index and article citation count, are biased by time-dependent factors and are limited by a lack of field normalization. The relative citation ratio (RCR) is a new field- and time-normalized article-level metric developed by the National Institutes of Health (NIH).

Purpose/Hypothesis

The purpose of this study was to evaluate the RCR among fellowship-trained academic sports medicine surgeons and to analyze physician factors associated with RCR values. We hypothesized that the mean RCR score for fellowship-trained academic sports medicine surgery faculty will fall above the NIH standard.

Study Design

Cross-sectional study.

Methods

A retrospective data analysis was performed using the iCite database for all fellowship-trained sports medicine surgery faculty associated with Accreditation Council for Graduate Medical Education-accredited orthopaedic surgery residency programs in December 2021. In eligible faculty, the mean RCR, weighted RCR, and total publication count were compared by sex, career duration, academic rank, and presence of additional degrees. A mean RCR value of 1.0 is the NIH-funded field-normalized standard. The data herein are presented as the median and interquartile range, in addition to the mean and standard deviation, to account for outliers of the mean and weighted RCR scores.

Results

A total of 624 fellowship-trained sports medicine surgery faculty members from 160 orthopaedic surgery residency programs were included in the analysis. Overall, faculty produced impactful research, with a median RCR of 1.6 (interquartile range, 1.0-2.2) and a median weighted RCR of 19.3 (interquartile range, 5.1-69.3). Advanced academic rank and career longevity were associated with increased weighted RCR and total publication count. All subgroups analyzed had an RCR value >1.0.

Conclusion

Study findings indicate that fellowship-trained academic sports medicine surgery faculty are highly productive and produce impactful research, as evidenced by the high median RCR value relative to the benchmark NIH RCR value of 1.0.

The Relative Citation Ratio: A Modern Approach to Assessing Academic Productivity within Plastic Surgery

The accurate assessment of physician academic productivity is paramount and is frequently included in decisions for promotion and tenure. Current metrics such as h-index have been criticized for being biased toward older researchers and misleading. The relative citation ratio (RCR) is a newer metric that has been demonstrated within other surgical subspecialties to be a superior means of measuring academic productivity. We sought to demonstrate that RCR is a valid means of assessing academic productivity among plastic surgeons, and to determine demographic factors that are associated with higher RCR values.

METHODS

All Accreditation Council for Graduate Medical Education-accredited plastic and reconstructive surgery residency programs and faculty throughout the United States were compiled from the American Council of Academic Plastic Surgeons website. Demographic information was obtained for each surgeon via the program's website, and RCR data were obtained utilizing iCite, a bibliometrics tool provided by the National Institutes of Health. Surgeons were excluded if any demographic or RCR data were unavailable.

RESULTS

A total of 785 academic plastic surgeons were included in this analysis. Surgeons who belonged to departments with more than six members had a higher median RCR (1.23). Increasing academic rank (assistant: 12.27, associate: 24.16, professor: 47.58), chief/chairperson status (47.58), male gender (25.59) and integrated model of residency training program (24.04) were all associated with higher median weighted RCR.

CONCLUSIONS

RCR is a valid metric for assessing plastic surgeon academic productivity. Further research is warranted in assessing disparities among different demographics within academic plastic surgery.

Evaluation of the National Institutes of Health–supported relative citation ratio among American orthopedic spine surgery faculty: A new bibliometric measure of scientific influence

•

Analyze new NIH-Supported bibliometric tool, the relative citation ratio (RCR).

•

Evaluate research productivity among fellowship trained Orthopedic Spine Surgeons.

•

Conduct demographic analysis of research productivity among fellowship trained Orthopedic Spine Surgeons.

Background

Publication metrics have been traditionally used to compare research productivity amongst academic faculty. However, traditional bibliometrics lack field-normalization and are often biased towards time-dependent publication factors. The National Institutes of Health (NIH) has developed a new, field-normalized, article-level metric, known as the “relative citation ratio” (RCR), that can be used to make accurate self, departmental, and cross-specialty comparisons of research productivity. This study evaluates the use of the RCR amongst academic orthopedic spine surgery faculty and analyzes physician factors associated with RCR values.

Methods

A retrospective data analysis was performed using the iCite database for all fellowship trained orthopedic spine surgery (OSS) faculty associated with Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopedic surgery residency program. Mean RCR, weighted RCR, and total publication count were compared by sex, career duration, academic rank, and presence of additional degrees. A value of 1.0 is the NIH-funded field-normalized standard. Student t-tests were used for two-group analyses whereas the analysis of variance tests (ANOVA) was used for between-group comparisons of three or more subgroups. Statistical significance was achieved at P < 0.05.

Results

A total of 502 academic OSS faculty members from 159 institutions were included in the analysis. Overall, OSS faculty were highly productive, with a median RCR of 1.62 (IQR 1.38-2.32) and a median weighted RCR of 68.98 (IQR 21.06-212.70). Advancing academic rank was associated with weighted RCR, career longevity was associated with mean RCR score, and male sex was associated with having increased mean and weighted RCR scores. All subgroups analyzed had an RCR value above 1.0.

Conclusions

Academic orthopedic spine surgery faculty produce impactful research as evidenced by the high median RCR relative to the standard value set by the NIH of 1.0. Our data can be used to evaluate research productivity in the orthopedic spine community.

Evaluation of the Relative Citation Ratio Among Academic Orthopedic Hand Surgeons: A Novel Measure of Research Impact

Background: Publication metrics such as article citation count and the Hirsch index (h-index) are used to evaluate research productivity among academic faculty. However, these bibliometric indices are not field-normalized and yield inaccurate cross-specialty comparisons. We evaluate the use of the relative citation ratio (RCR), a new field-normalized article-level metric developed by the National Institutes of Health (NIH), among academic orthopedic hand surgeons and analyze physician factors associated with RCR values.

Methods: A retrospective analysis was performed using the iCite database. Fellowship-trained orthopedic hand surgeons affiliated with accredited orthopedic surgery residency programs were included. Mean RCR, weighted RCR, and publication count were compared by sex, career duration, academic rank, and presence of additional degrees. Mean RCR represents the total number of citations per year of a publication divided by the average number of citations per year received by NIH-funded papers in the same field. Mean RCR serves as a measure of overall research impact. A value of 1.0 is the NIH-funded field-normalized standard. Weighted RCR is the sum of all article-level RCR scores and represents overall research productivity.

Results: A total of 620 academic orthopedic hand surgeons from 164 programs were included. These physicians produced highly impactful research with a median RCR of 1.27 (interquartile range [IQR] 0.86-1.66). Weighted RCR was associated with advanced degree, advanced academic rank, and longer career duration.

Conclusions: Fellowship-trained academic orthopedic hand surgeons produce highly impactful research. Our benchmark data can be used to assess grant outcomes, promotion, and continued evaluation of research productivity within the hand surgery community.