The Ottawa Resident Observation Form for Nurses (O-RON): Assessment of Resident Performance through the Eyes of the Nurses

The Ottawa resident observation form for nurses (O-RON): evaluation of an assessment tool's psychometric properties in different specialties

BACKGROUND

Workplace-based assessment (WBA) used in post-graduate medical education relies on physician supervisors' feedback. However, in a training environment where supervisors are unavailable to assess certain aspects of a resident's performance, nurses are well-positioned to do so. The Ottawa Resident Observation Form for Nurses (O-RON) was developed to capture nurses' assessment of trainee performance and results have demonstrated strong evidence for validity in Orthopedic Surgery. However, different clinical settings may impact a tool's performance. This project studied the use of the O-RON in three different specialties at the University of Ottawa.

METHODS

O-RON forms were distributed on Internal Medicine, General Surgery, and Obstetrical wards at the University of Ottawa over nine months. Validity evidence related to quantitative data was collected. Exit interviews with nurse managers were performed and content was thematically analyzed.

RESULTS

179 O-RONs were completed on 30 residents. With four forms per resident, the ORON's reliability was 0.82. Global judgement response and frequency of concerns was correlated (r = 0.627, P < 0.001).

CONCLUSIONS

Consistent with the original study, the findings demonstrated strong evidence for validity. However, the number of forms collected was less than expected. Exit interviews identified factors impacting form completion, which included clinical workloads and interprofessional dynamics.

Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Politics or markets: The dual role of the motivation to achieve organizational legitimacy in the development of knowledge management capabilities and business model innovation

Despite business model innovation being the object of much interest, limited attention has paid on how and when knowledge management capabilities enhance business model innovation in the literature. Build upon institutional theory and knowledge-based view, we seek to investigate how knowledge management capabilities affect the business model innovation by exploring the dual role of different types of legitimation motivations in triggering knowledge management capabilities, and moderating the relationship between knowledge management capabilities, and business model innovation. The data collected from the 236 Chinese new ventures running their businesses across a variety of sectors. The results indicate the both political and market legitimation motivation positively affect knowledge management capabilities. The relationship between knowledge management capabilities and business model innovation are more strongly in high motivation to achieve market legitimacy. However, the positive effect of knowledge management capabilities stimulate business model innovation is more strongly in moderately motivation to achieve political legitimacy than in low or highly political legitimation motivation. The paper has significantly contributed to advancing the body of knowledge of institutional and business model innovation theory and providing deeper insights on the correlation between firm's motivation to achieve legitimacy and knowledge management capabilities for business model innovations.

A Unique Lens: Understanding What Nurses Are Best Positioned to Assess About Residents

BACKGROUND

Resident feedback is generally elicited from attending physicians, although nurses can also provide feedback on distinct domains. Physicians may be hesitant to accept feedback from nurses if they perceive that nurses are being asked about areas outside their expertise. Understanding specific resident behaviors that nurses are best suited to assess is critical to successful implementation of feedback from nurses to residents.

OBJECTIVE

To understand specific resident behaviors nurses are uniquely positioned to assess from the perspectives of both nurses and residents.

METHODS

We performed a qualitative study using thematic analysis of 5 focus groups with 20 residents and 5 focus groups with 17 nurses at a large free-standing children's hospital in 2020. Two reviewers developed a codebook and subsequently analyzed all transcripts. Codes were organized into themes and subthemes. Thematic saturation was achieved prior to analyzing the final transcript.

RESULTS

We identified 4 major themes. Nurses are positioned to provide feedback: (1) on residents' interprofessional collaborative practice; (2) on residents' communication with patients and their families; and (3) on behalf of patients and their families. Within each of these, we identified subthemes noting specific behaviors on which nurses can provide feedback. The fourth theme encompassed topics that may not be best suited for nursing feedback-medical decision-making and resident responsiveness.

CONCLUSIONS

Nurses and residents described specific resident behaviors that nurses were best positioned to assess.

Unobserved Observers: Nurses' Perspectives About Sharing Feedback on the Performance of Resident Physicians

PURPOSE

Postgraduate training programs are incorporating feedback from registered nurses (RNs) to facilitate holistic assessments of resident performance. RNs are a potentially rich source of feedback because they often observe trainees during clinical encounters when physician supervisors are not present. However, RN perspectives about sharing feedback have not been deeply explored. This study investigated RN perspectives about providing feedback and explored the facilitators and barriers influencing their engagement.

METHOD

Constructivist grounded theory methodology was used in interviewing 11 emergency medicine and 8 internal medicine RNs at 2 campuses of a tertiary care academic medical center in Ontario, Canada, between July 2019 and March 2020. Interviews explored RN experiences working with and observing residents in clinical practice. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis.

RESULTS

RNs felt they could observe authentic day-to-day behaviors of residents often unwitnessed by supervising physicians and offer unique feedback related to patient advocacy, communication, leadership, collaboration, and professionalism. Despite a strong desire to contribute to resident education, RNs were apprehensive about sharing feedback and reported barriers related to hierarchy, power differentials, and a fear of overstepping professional boundaries. Although infrequent, a key stimulus that enabled RNs to feel safe in sharing feedback was an invitation from the supervising physician to provide input.

CONCLUSIONS

Perceived hierarchy in academic medicine is a critical barrier to engaging RNs in feedback for residents. Accessing RN feedback on authentic resident behaviors requires dismantling the negative effects of hierarchy and fostering a collaborative interprofessional working environment. A critical step toward this goal may require supervising physicians to model feedback-seeking behavior by inviting RNs to share feedback. Until a workplace culture is established that validates nurses' input and creates safe opportunities for them to contribute to resident education, the voices of nurses will remain unheard.