Prevalence of answers to orthopaedic in-training examination questions in 3 commonly used orthopedic review sources

Orthopaedic In-Training Examination: History, Perspective, and Tips for Residents

Introduced in 1963, the orthopaedic in-training examination (OITE) is a standardized, national test administered annually to orthopaedic residents by the American Academy of Orthopaedic Surgeons. The examination consists of 275 multiple-choice questions that cover 11 domains of orthopaedic knowledge, including basic science, foot and ankle, hand, hip and knee, oncology, pediatrics, shoulder and elbow, spine, sports medicine, trauma, and practice management. The OITE has been validated and is considered predictive of success in both orthopaedic surgery residency and on the American Board of Orthopaedic Surgery part I examination. This article provides a historical overview of the OITE, details its current structure and scoring system, and reviews currently available study materials. For examination preparation, the residents are encouraged to (1) start the examination preparation early, (2) practice on old OITE or self-assessment examination questions, (3) focus on the questions answered incorrectly, (4) focus on comprehension over memorization, and (5) recognize and avoid burnout. Finally, the residents should have a systemic way of approaching each multiple-choice question, both during practice and on the actual examination.

Orthopaedic In-Training Examination Resources and Residency Training for the Foot and Ankle Domain

BACKGROUND

Residency programs use the annual Orthopaedic In-Training Examination (OITE) prepared by the American Academy of Orthopaedic Surgeons (AAOS) to monitor resident progress and prepare them for the part 1 of the American Board of Orthopaedic Surgeons (ABOS) Certifying Examination. The purpose of this study was to determine resources residents currently use to prepare for the OITE and also to learn about their perception of training they receive in the foot and ankle subspecialty in their program and their interest in foot and ankle fellowship after residency.

METHODS

An anonymous survey was sent to both allopathic programs and osteopathic residents to learn what resources residents used to study for the OITE, preparatory question sets, on-call resources, their perception on training received in foot and ankle surgery, and their intent to pursue fellowship training.

RESULTS

A total of 130 residents participated in the survey. The majority of residents in allopathic and osteopathic residencies used Orthobullets (OB) to prepare for the OITE and use this resource while on-call. Most residents also used OB question sets to study along with the AAOS self-assessment examinations. In total, 43.2% of osteopathic residents felt they did not get enough exposure to foot and ankle subspecialty while in training, in contrast to 31.2% of allopathic residents. A total of 35% of all orthopaedic surgery residents felt they lacked enough exposure to foot and ankle orthopaedic surgery. Only 7 residents (6%, 6 allopathic, 1 osteopathic) intended to pursue a foot and ankle fellowship following graduation.

CONCLUSION

Online resources such as OB continue to be frequently used by residents for preparation for the OITE. Greater than one-third of orthopaedic residents feel they do not get enough exposure to foot and ankle orthopaedic surgery. Improvement in this area could be helped by continued endeavors from the American Orthopaedic Foot and Ankle Society such as the Visiting Professor Program and Resident Scholarship Program.

LEVELS OF EVIDENCE

Level V: Single Cross-Sectional Study.

Orthopedic In-Training Examination: An Analysis of the Sports Medicine Section-An Update

OBJECTIVE

Previously published studies have evaluated Orthopedic In-Training Examination sports medicine questions, but none have evaluated whether question difficulty has changed over time.

DESIGN

Sports medicine subsection questions between 2012 and 2016 were evaluated and compared with previously published data on Orthopedic In-Training Examination from 2004 to 2009. Question categories were classified into 1 of 3 taxonomy levels-basic recall, diagnosis, and advanced problem solving.

SETTING

Medical University of South Carolina; Charleston, SC, 29425; Institution.

PARTICIPANTS

Two residents evaluated the Sports Subsection questions separately. Then an attending physician evaluated the questions to resolve discrepancies. A statistician was used for analysis.

RESULTS

Utilization of imaging modalities averaged 37%, with 28% (11/39) of the questions containing 2 or more imaging modalities. There were increases in utilization of advanced problem-solving questions (45% vs. 27%, p = 0.002) and decreases in basic recall questions (49% vs. 67%, p = 0.008) compared with previously published data.

CONCLUSIONS

While the percentage of the Orthopedic In-Training Examination represented by sports medicine has remained relatively unchanged, there were fewer questions requiring residents to demonstrate simple recall and diagnosis, and increased demand to perform advanced problem solving while utilizing multiple imaging modalities.

Does a Weekly Didactic Conference Improve Resident Performance on the Pediatric Domain of the Orthopaedic In-Training Examination?

BACKGROUND

Performance on the Orthopaedic In-training Examination (OITE) has been correlated with performance on the written portion of the American Board of Orthopaedic Surgery examination. Herein we sought to discover whether adding a regular pediatric didactic lecture improved residents' performance on the OITE's pediatric domain.

METHODS

In 2012, a didactic lecture series was started in the University of Pittsburgh Medical Center (UPMC) Hamot Orthopaedic Residency Program (Hamot). This includes all topics in pediatric orthopaedic surgery and has teaching faculty present, and occurs weekly with all residents attending. A neighboring program [UMPC Pittsburgh (Pitt)] shares in these conferences, but only during their pediatric rotation. We sought to determine the effectiveness of the conference by comparing the historic scores from each program on the pediatric domain of the OITE examination to scores after the institution of the conference, and by comparing the 2 programs' scores.

RESULTS

Both programs demonstrated improvement in OITE scores. In 2008, the mean examination score was 19.6±4.3 (11.0 to 30.0), and the mean percentile was 57.7±12.6 (32.0 to 88.0); in 2014, the mean examination score was 23.5±4.2 (14.0 to 33.0) and the mean percentile was 67.1±12.1 (40.0 to 94.0). OITE scores and percentiles improved with post graduate year (P<0.0001). Compared with the preconference years, Hamot residents answered 3.99 more questions correctly (P<0.0001) and Pitt residents answered 2.93 more questions correctly (P<0.0001). Before the conference, site was not a predictor of OITE score (P=0.06) or percentile (P=0.08); there was no significant difference found between the mean scores per program. However, in the postconference years, site did predict OITE scores. Controlling for year in training, Hamot residents scored higher on the OITE (2.3 points higher, P=0.003) and had higher percentiles (0.07 higher, P=0.004) than Pitt residents during the postconference years.

CONCLUSIONS

This study suggests that adding a didactic pediatric lecture improved residents' scores on the OITE and indirectly suggests that more frequent attendance is associated with better scores.

LEVEL OF EVIDENCE

Level III-retrospective case-control study.

Analysis of the Korean Orthopedic In-Training Examination: The Hip and Pelvis Section

Purpose

The purpose of this study was to analyze the questions in the hip and pelvis section of the Korean Orthopaedic In-Training Examination (KOITE).

Materials and Methods

We analyzed all KOITE questions pertaining to hip and pelvis surgery between 2010 and 2014. A thorough analysis of the contents was performed after categorizing as tested topics, imaging modalities used, taxonomic classification, and recommended references. We also analyzed the scores of the hip and pelvis section of the KOITE.

Results

Seventy-five of five-hundred questions (weight, 15.0%) were related to the hip and pelvis. Trauma including fracture and dislocation (26/75, 34.7%) was asked more commonly than disease and basics. The description-only questions (65/75, 86.7%) were the most frequently asked. According to taxonomic classification, taxonomy 3 (decision; 39/75, 52.0%) was most frequently asked. Campbell's Operative Orthopedics (52/75, 69.3%) was the reference that covered most of the questions.

Conclusion

This analysis of the hip and pelvis section of KOITE could be used for resident training programs in teaching hospitals.

An analysis of references used for the Orthopaedic In-Training Examination: what are their levels of evidence and journal impact factors?

BACKGROUND

Although the references recommended for the Orthopaedic In-Training Examination (OITE) have been evaluated in certain subspecialty domains, suggested reference level of evidence (LOE), impact factor, and citation age have not been evaluated comprehensively to our knowledge.

QUESTIONS/PURPOSES

We present an analysis of all references cited in the OITE recommended readings for each test question including the duration of time between their initial publication and their use in the OITE, which we defined as citation age, LOE, and the impact factor of the journals referenced.

METHODS

We evaluated all references for the 2010 to 2012 OITE administrations (three examinations; 825 questions total). Publication characteristics, including citation age, were noted. The LOE for each journal article and the impact factor of each journal were determined; differences in LOE and impact factor were compared between test sections. A total of 1817 references were cited in the 825 questions we evaluated; this denominator was used in all calculations that follow.

RESULTS

The recommended reading references included 1337 journal article references (74%), 469 text references (26%), and 11 multimedia sources (0.6%; eg, websites, instructional DVDs). The three most commonly recommended journals were general orthopaedic journals, The Journal of Bone and Joint Surgery (American Volume), Journal of American Academy of Orthopaedic Surgeons, and Clinical Orthopaedics and Related Research. The majority (72.2%) of the cited journal references were published within 10 years of the test date, with a mean ± SD citation age of 8.3 ± 7.4 years. The majority of the cited journal articles were Levels IV and V evidence (mean, 4.16 ± 1.1). The Spine section had higher LOE (3.74; p < 0.001), although the practical relevance of such a difference is questionable, as all but two sections' LOE rounded to Level IV evidence. The Spine and Basic Science sections were published in journals with a larger mean impact factor (Basic Science, 7.16 ± 12.67; Spine, 5.73 ± 12.08; p < 0.001).

CONCLUSIONS

Our data show that the majority of the recommended readings for the OITE stem from higher impact general orthopaedic and major subspecialty journals. Furthermore the observed mean LOE of the recommended readings shows a preponderance of Levels IV and V research. These data may suggest that test-takers may find benefit in the review of high-level general orthopaedic journals, and review articles in particular while preparing for the OITE, although further study is necessary to determine optimal test preparation strategies. Finally, our study provides a baseline analysis of the study designs of OITE recommended references, and may provide insight for educators designing resident educational curricula.

Developing an electronic teaching and training portfolio

Technological advances, in particular the rise of the internet, have led to dramatic changes in medical education. The recent global financial crisis and issues with medical staffing have meant that training programs and universities are increasingly exploring electronic means to provide efficient and cost effective education techniques. In this article, we explore methods by which orthopedic trainees can develop their educational portfolio through electronic resources and similarly, how training or residency programs can utilize these advances in technology to both increase efficiency and enhance their teaching reputation. Finally, we explore the merits of trainees keeping track of their careers through electronic portfolios.

Is there an association between study materials and scores on the American Board of Orthopaedic Surgeons Part 1 examination?

BACKGROUND

Previous studies have shown that certain orthopaedic in-training examination scores can be used to identify which residents may be at risk for failing the American Board of Orthopaedic Surgeons (ABOS) Part 1 examination. However, no studies have examined how study resources may affect residents' ABOS Part 1 scores. The goal of this study is to determine which review sources or review courses, if any, are associated with improved ABOS Part 1 scores.

METHODS

A survey was sent to 221 of the 865 examinees who took the ABOS Part 1 examination in 2012. The questions inquired the respondents how well they performed on previous orthopaedic in-training examinations and ABOS Part 1, along with the study sources they most commonly used, review courses they attended, and resources they would recommended if they were to retake ABOS Part 1 examination.

RESULTS

Overall, 118 of the 221 (53%) survey recipients completed the survey. Six (5%) of the respondents failed ABOS Part 1 examination. Orthobullets and the American Academy of Orthopaedic Surgeons self-assessment examinations were recommended as the primary study source significantly more (p < 0.01) than most other resources, but there was no significant association between study source and passing ABOS Part 1 or scoring in a certain percentile on ABOS Part 1. Similarly, there were no associations between attending a review course and either passing or scoring in a certain percentile for ABOS Part 1. Half of the respondents who failed ABOS Part 1 attended multiple review courses.

CONCLUSIONS

There does not appear to be an association between improved ABOS Part 1 scores and orthopedic study materials or review courses. Further research into the value of certain educational modalities should be conducted to determine the best ways to educate orthopedic residents and determine the value of some of these commonly used orthopedic review modalities.