Analysis of Orthopedic Resident Ability to Apply Levels of Evidence Criteria to Scientific Articles

Performance of Orthopaedic Shoulder and Elbow Surgeons on a Biostatistical Knowledge Examination

Background

The objective of this study is to evaluate the biostatistical interpretation abilities of fellowship trained orthopaedic surgeons.

Methods

A cross-sectional survey was administered to orthopaedic surgeon members of the American Shoulder and Elbow Surgeons (ASES), assessing orthopaedic surgeon attitudes towards biostatistics, confidence in understanding biostatistics, and ability to interpret biostatistical measures on a multiple-choice test.

Results

A 4.5% response rate was achieved with 55 complete survey responses. The mean percent correct was 55.2%. Higher knowledge test scores were associated with younger age and fewer years since board exam completion (p ≤ 0.001). Greater average number of publications per year correlated with superior statistical interpretation (p=0.009). Respondents with higher self-reported confidence were more likely to accurately interpret results (p ≤ 0.017). Of the respondents, 93% reported frequently using statistics to form medical opinions, 98% answered that statistical competency is important in the practice of orthopaedic surgery, and 80% were eager to continue learning biostatistics.

Conclusions

It is concerning that fellowship-trained shoulder and elbow surgeons, many of whom frequently publish or are reviewing scientific literature for publication, are scoring 55.2% correctly on average on this biostatistical knowledge examination. Surgeons that are further from formal statistical knowledge training are more likely to have lower biostatistical knowledge test scores. Respondents who published at the highest rate were associated with higher scores. Continuing medical education in biostatistics may be beneficial for maintaining statistical knowledge utilised in the current literature.

Protected Time for Research During Orthopaedic Residency Correlates with an Increased Number of Resident Publications

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) requires orthopaedic residency programs to promote scholarship and research, which manifest differently among programs. We assess the impact of protected research time during orthopaedic residency on the number of resident publications.

METHODS

Rotation schedules and resident names were collected from 125 ACGME-accredited U.S. orthopaedic residency programs. Protected research time was classified as 1 of 3 types: (1) block time, (2) longitudinal time, or (3) no dedicated time. In April 2016, we searched residents in postgraduate year (PGY)-3 to PGY-5 on pubmed.gov to generate all orthopaedic publications with a PubMed identifier published during residency. Each publication's 2015 Thomson Reuters Journal Citation Reports 5-Year Journal Impact Factor and resident first authorship were noted. The number of PubMed identifiers for each program was summed and was divided by the number of residents in PGY-3 to PGY-5, giving a mean number of publications per resident. The relationship between output and program research time was compared using t tests and analysis of variance (ANOVA).

RESULTS

A total of 1,690 residents were included, with an overall mean number (and standard deviation) of 1.2 ± 2.4 publications per resident. Eighty-seven programs reported block time, 14 programs reported longitudinal time, and 24 programs reported no time. There was a significant difference (p = 0.02) in the mean number of publications per resident when compared between programs with protected time (1.1 ± 1.2 publications) and programs with no protected time (0.6 ± 0.5 publication). One-way ANOVA demonstrated a significant mean difference across the 3 groups (p < 0.001), with longitudinal time correlating with significantly greater output at 1.9 ± 1.8 publications than block time at 1.0 ± 1.0 publication or no time at 0.6 ± 0.5 publication, a difference that persisted when adjusted to include only impact factors of >0 and exclude case reports (p = 0.0015).

CONCLUSIONS

Both the presence of and the type of dedicated research time correlate with residents' research productivity; further consideration of protected research time during residency is warranted.

CLINICAL RELEVANCE

This article provides objective data with regard to research strategies in training orthopaedic surgeons.