A Multifaceted Organizational Physician Assessment Program: Validity Evidence and Implications for the Use of Performance Data

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Using Peer Feedback to Promote Clinical Excellence in Hospital Medicine

Hospitalists provide a significant amount of direct clinical care in both academic and community hospitals. Peer feedback is a potentially underutilized and low resource method for improving clinical performance, which lends itself well to the frequent patient care handoffs that occur in the practice of hospital medicine. We review current literature on peer feedback to provide an overview of this performance improvement tool, briefly describe its incorporation into multi-source clinical performance appraisals across disciplines, highlight how peer feedback is currently used in hospital medicine, and present practical steps for hospital medicine programs to implement peer feedback to foster clinical excellence among their clinicians.

Experiential knowledge of risk and support factors for physician performance in Canada: a qualitative study

OBJECTIVE

To identify, understand and explain potential risk and protective factors that may influence individual and physician group performance, by accessing the experiential knowledge of physician-assessors at three medical regulatory authorities (MRAs) in Canada.

DESIGN

Qualitative analysis of physician-assessors' interview transcripts. Telephone or in-person interviews were audio-recorded on consent, and transcribed verbatim. Interview questions related to four topics: Definition/discussion of what makes a 'high-quality physician;' factors for individual physician performance; factors for group physician performance; and recommendations on how to support high-quality medical practice. A grounded-theory approach was used to analyse the data.

SETTING

Three provinces (Alberta, Manitoba, Ontario) in Canada.

PARTICIPANTS

Twenty-three (11 female, 12 male) physician-assessors from three MRAs in Canada (the College of Physicians & Surgeons of Alberta, the College of Physicians and Surgeons of Manitoba and the College of Physicians and Surgeons of Ontario).

RESULTS

Participants outlined various protective factors for individual physician performance, including: being engaged in continuous quality improvement; having a support network of colleagues; working in a defined scope of practice; maintaining engagement in medicine; receiving regular feedback; and maintaining work-life balance. Individual risk factors included being money-oriented; having a high-volume practice; and practising in isolation. Group protective factors incorporated having regular communication among the group; effective collaboration; a shared philosophy of care; a diversity of physician perspectives; and appropriate practice management procedures. Group risk factors included: a lack of or ineffective communication/collaboration among the group; a group that doesn't empower change; or having one disruptive or 'risky' physician in the group.

CONCLUSIONS

This is the first qualitative inquiry to explore the experiential knowledge of physician-assessors related to physician performance. By understanding the risk and support factors for both individual physicians and groups, MRAs will be better-equipped to tailor physician assessments and limited resources to support competence and enhance physician performance.