Examination of Skill Acquisition and Grader Bias in a Distal Radius Fracture Fixation Model

OBJECTIVES

Primary: Assess the ability of faculty graders to predict the objectively measured strength of distal radius fracture fixation. Secondary: Compare resident skill variation and retention related to other knowable training data.

DESIGN

Residents were allowed 60 minutes to stabilize a standardized distal radius fracture using an assigned fixed-angle volar plate. Faculty observed and subjectively graded the residents without providing real-time feedback. Objective biomechanical evaluation (construct strength and stiffness) was compared to subjective grades. Resident-specific characteristics (sex, PGY, and ACGME case log) were also used to compare the objective data.

SETTING

A simulated operating room in our laboratory.

PARTICIPANTS

Post-graduate year 2, 3, 4, and 5 orthopedic residents.

RESULTS

Primary: Faculty were not successful at predicting objectively measured fixation, and their subjective scoring suggests confirmation bias as PGY increased. Secondary: Resident year-in-training alone did not predict objective measures (p = 0.53), but was predictive of subjective scores (p < 0.001). Skills learned were not always retained, as 29% of residents objectively failed subsequent to passing. Notably, resident-reported case-specific experience alone was inversely correlated with objective fixation strength.

CONCLUSIONS

This testing model enabled the collection of objective and subjective resident skill scores. Faculty graders did not routinely predict objective measures, and their subjective assessment appears biased related to PGY. Also, in vivo case volume alone does not predict objective results. Familiar faculty teaching consistency, and resident grading by external faculty unfamiliar with tested residents, might alter these results.

Quality: the Mayo Clinic approach

Developing highly reliable care for patients requires changes in some traditional beliefs of medical practice, an evolution toward a "system" of health care, the disciplined application of scientific principles, modifications in the way all future providers are trained, and a fundamental understanding by leadership that quality must become a business strategy and core work, not an expense or regulatory requirement. Quality at Mayo is defined as a composite of outcomes, safety, and service. A 4-part strategic construct focusing on Culture, Infrastructure, Engineering, and Execution has been developed to guide improvement activities and to ensure a comprehensive approach to better patient care. The Mayo Clinic experience has led to a greater understanding of the leadership commitment, organizational challenges, and the breadth of initiatives necessary to achieve highly reliable care.