The Challenges of Multisource Feedback: Feasibility and Acceptability of Gathering Patient Feedback for Pediatric Residents

Real Patient Participation in Workplace-Based Assessment of Health Professional Trainees: A Scoping Review

The aim of this scoping review is to outline the existing landscape of how real patients participate in the workplace-based assessment of trainees across diverse healthcare professions. In 2019-2020, the authors searched MEDLINE, EMBASE, CINAHL, PsycINFO, ERIC, and Web of Science databases for studies that included descriptions of experiences whereby patients received care from a health professional trainee and participated in workplace-based assessments of that trainee. Full-text articles published in English from 2009 to 2020 were included in the search. Of the 8770 studies screened; 77 full-text articles were included. Analysis showed that strategies for patient participation in workplace-based assessment varied widely. Aspects studied ranged from validation of an assessment tool to evaluation of the impact of an educational intervention on trainees' performance. Assessment of patient satisfaction was the most common approach to patient involvement. The majority of studies were conducted in North America and in the context of physician training. Formal patient participation in the assessment of health professional trainees appears heterogeneous across health professions. Gaps in the literature are evident; therefore, this review points to an inclusive approach to workplace-based assessment to ensure patient feedback of the trainees who care for them is represented.

Exploring the concepts of consumer feedback systems for occupational therapy student learning during practice placements: A scoping review

INTRODUCTION

Consumer contribution to occupational therapy student learning is mandated for Australian educational programs. However, there is limited research about how consumers contribute to student learning during practice placements. A scoping review was completed to explore the concepts of existing feedback systems for consumers to authentically contribute to student learning during practice placements.

METHODS

Five databases were searched for all articles up to and including July 2023. All publications were included if they described and investigated authentic contribution to student learning during a practice placement experience. Data from the results and discussion sections of the papers were transformed into qualitative data and thematically analysed to develop a conceptual understanding of consumer feedback systems.

RESULTS

Forty-six papers met the eligibility criteria. Most articles originated from Medicine (30%) and Nursing (45%) education. Feedback systems were primarily designed to critique student communication and professional behaviours using a variety of standardised and non-standardised methods. Five interconnected themes were created that addressed the concepts of consumer feedback systems. The themes were (1) creating value specific for each stakeholder is essential, (2) preparation is required and nuanced; (3) consumers do engage and provide critical feedback when the system is supportive of all stakeholders; (4) gathering approaches need to be responsive to the diversity of practice and learning; (5) processing feedback with a trusted mentor is critical for learning.

CONCLUSIONS

Consumers, academics, practice educators, and students recognised that consumers have an important role in contributing to student learning during practice placements. Future consumer feedback systems need to be co-produced to create an optimal mode for consumers to authentically contribute to student learning constructively and safely.

CONSUMER AND COMMUNITY INVOLVEMENT

A consumer consultant was a member of the advisory panel for the larger research project that this review is part of and provided advice to the research team at all stages of the project.

PLAIN LANGUAGE SUMMARY

In Australia, occupational therapy training programs require input from consumers to help students learn. However, there is not much research on how consumers help occupational therapy students during their practical training. We did a review to see how existing feedback systems let consumers genuinely help students during their training. We searched five databases for research up to July 2023. We included research that talked about how consumers help all health students learn during practical training. We found 46 articles, mostly from Medicine and Nursing education. We analysed them to understand how consumer feedback systems work and found five main themes: (1) making sure everyone involved gets value out of it, (2) being prepared is important, (3) consumers give useful feedback when the system supports everyone involved, (4) different approaches are needed because practices and learning vary, and (5) discussing feedback with other people helps students learn. Current feedback systems mostly focused on how students communicate and behave professionally. Consumers, educators, and students all agree that consumers play a vital role in helping students learn during practical training. In the future, we need to work together to make feedback systems that let consumers help students in the best way possible, making sure it is constructive and safe for everyone.

A Unique Lens: Understanding What Nurses Are Best Positioned to Assess About Residents

BACKGROUND

Resident feedback is generally elicited from attending physicians, although nurses can also provide feedback on distinct domains. Physicians may be hesitant to accept feedback from nurses if they perceive that nurses are being asked about areas outside their expertise. Understanding specific resident behaviors that nurses are best suited to assess is critical to successful implementation of feedback from nurses to residents.

OBJECTIVE

To understand specific resident behaviors nurses are uniquely positioned to assess from the perspectives of both nurses and residents.

METHODS

We performed a qualitative study using thematic analysis of 5 focus groups with 20 residents and 5 focus groups with 17 nurses at a large free-standing children's hospital in 2020. Two reviewers developed a codebook and subsequently analyzed all transcripts. Codes were organized into themes and subthemes. Thematic saturation was achieved prior to analyzing the final transcript.

RESULTS

We identified 4 major themes. Nurses are positioned to provide feedback: (1) on residents' interprofessional collaborative practice; (2) on residents' communication with patients and their families; and (3) on behalf of patients and their families. Within each of these, we identified subthemes noting specific behaviors on which nurses can provide feedback. The fourth theme encompassed topics that may not be best suited for nursing feedback-medical decision-making and resident responsiveness.

CONCLUSIONS

Nurses and residents described specific resident behaviors that nurses were best positioned to assess.

Australian occupational therapists' perspectives of consumers authentically contributing to student learning during practice placements: ‘It just makes sense!’ but ‘we need a process’

INTRODUCTION

Collaborating with consumers in designing, delivering, and evaluating curricula is an ongoing initiative within occupational therapy tertiary courses in Australia. Within the Australian educational context, consumers are involved in on-campus educational activities. Student occupational therapists must complete 1000 hours of practice placements as part of their education. To date, no research has explored how consumers could contribute to student occupational therapists' learning during practice placements. This study aimed to explore Australian occupational therapists' perceptions of consumers providing feedback to students during practice placements.

METHODS

A qualitative descriptive approach was adopted to engage with the diversity of practice contexts and gain a rich dataset from the occupational therapy profession. A qualitative questionnaire was developed and distributed using snowballing techniques. The questionnaire asked recipients to reflect on the risks, challenges, and benefits of consumers providing feedback to student occupational therapists from all stakeholders' perspectives. Demographic data were collated, and reflexive thematic analysis was used to construct themes.

FINDINGS

Responses were received from 81 participants. Most respondents identified as experienced occupational therapists from metropolitan locations across Australia. Reflective thematic analysis was used to construct three themes: Personal capability of consumers and students will enable, inhibit, and be developed by engaging in a feedback process; an educator-controlled process to ensure safety for all stakeholders is required for time-poor practice contexts; and us versus them: Shifting control to consumers can disempower practice educators.

CONCLUSION

Engaging with consumers throughout all aspects of student occupational therapists' educational programme is required, including practice placements. New educational initiatives need to consider all stakeholders' concerns to ensure that authentic contribution from consumers is made within the various practice contexts. A co-design approach that involves all stakeholders to develop a feedback process may result in high-quality learning experiences that assist students to become safer, consumer-centred health professionals.

Patient involvement in assessment of postgraduate medical learners: A scoping review

CONTEXT

Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements towards greater patient partnership in health care generally, inclusion of patients in postgraduate medical learners' assessment is largely absent.

METHODS

We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched two databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment.

RESULTS

We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however choose not to engage when, for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity or medical condition seems to remain consistent. Patients may participate in assessment as a stand-alone group or part of a multi-source feedback process. Patients generally provided high ratings but commented on the observed professional behaviours and communication skills in comparison with physicians who focused on medical expertise.

CONCLUSION

Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programmes to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness and, in particular, physician readiness to partner with patients in this way.

Assessing anaesthesiology and intensive care specialty physicians: An Italian language multisource feedback system

BACKGROUND

Physician professionalism, including anaesthesiologists and intensive care doctors, should be continuously assessed during training and subsequent clinical practice. Multi-source feedback (MSF) is an assessment system in which healthcare professionals are assessed on several constructs (e.g., communication, professionalism, etc.) by multiple people (medical colleagues, coworkers, patients, self) in their sphere of influence. MSF has gained widespread acceptance for both formative and summative assessment of professionalism for reflecting on how to improve clinical practice.

METHODS

Instrument development and psychometric analysis (feasibility, reliability, construct validity via exploratory factor analysis) for MSF questionnaires in a postgraduate specialty training in Anaesthesiology and intensive care in Italy. Sixty-four residents at the Università del Piemonte Orientale (Italy) Anesthesiology Residency Program. Main outcomes assessed were: development and psychometric testing of 4 questionnaires: self, medical colleague, coworker and patient assessment.

RESULTS

Overall 605 medical colleague questionnaires (mean of 9.3 ±1.9) and 543 coworker surveys (mean 8.4 ±1.4) were collected providing high mean ratings for all items (> 4.0 /5.0). The self-assessment item mean score ranged from 3.1 to 4.3. Patient questionnaires (n = 308) were returned from 31 residents (40%; mean 9.9 ± 6.2). Three items had high percentages of "unable to assess" (> 15%) in coworker questionnaires. Factor analyses resulted in a two-factor solution: clinical management with leadership and accountability accounting for at least 75% of the total variance for the medical colleague and coworker's survey with high internal consistency reliability (Cronbach's α > 0.9). Patient's questionnaires had a low return rate, a limited exploratory analysis was performed.

CONCLUSIONS

We provide a feasible and reliable Italian language MSF instrument with evidence of construct validity for the self, coworkers and medical colleague. Patient feedback was difficult to collect in our setting.

Evaluating the effectiveness of undergraduate clinical education programs

Medical schools should use a variety of measures to evaluate the effectiveness of their clinical curricula. Both outcome measures and process measures should be included, and these can be organized according to the four-level training evaluation model developed by Donald Kirkpatrick. Managing evaluation data requires the institution to employ deliberate strategies to monitor signals in real-time and aggregate data so that informed decisions can be made. Future steps in program evaluation includes increased emphasis on patient outcomes and multi-source feedback, as well as better integration of existing data sources.