Improved detection of patient centeredness in objective structured clinical examinations through authentic scenario design

Implementation of virtual OSCE in health professions education: A systematic review

INTRODUCTION

The Objective Structured Clinical Examination (OSCE) has been widely used in health professions education since the 1970s. The global disruption caused by the COVID-19 pandemic restricted in-person assessments and medical educators globally sought alternative means to assess and certify students and trainees to meet the acute demand for health-care workers. One such solution was through virtual OSCE (vOSCE), which modified traditional in-person OSCE using videoconference platforms. This meta-ethnography sought to synthesise qualitative literature on candidates' and assessors' experiences of vOSCE to evaluate whether it may have a role in future assessment practices.

METHODS

In June 2022, we systematically searched PsycINFO, Medline and ERIC for peer-reviewed qualitative and mixed-methods articles that described candidates' and assessors' experiences of virtual OSCE in health professions education. Of 1069 articles identified, 17 were synthesised using meta-ethnography.

RESULTS

The final synthesis represented 1190 candidates and assessors from faculties of medicine, dentistry, nursing, pharmacy and osteopathy. We developed our findings into four key concepts. 'Strengthening confidence in a virtual environment' highlighted attempts to overcome and mitigate concerns associated with transitioning from in-person to virtual assessment. 'Understanding the scope of use as an assessment' reflected on the suitability of vOSCE in assessing various skills. 'Refining operational processes' emphasised the technical challenges of implementing vOSCE and impacts on accessibility and resources. 'Envisioning its future role' considered the applicability of vOSCE in the climate of rapid development in telehealth.

CONCLUSION

This meta-ethnography highlighted that although vOSCE was primarily considered a temporary and crisis response, candidates and assessors recognised positive, as well as negative, consequences of the transition towards them. Moving forward, medical education policymakers should carefully consider the extent to which elements of vOSCE could be incorporated into assessment systems, particularly in light of the rise of telehealth in clinical practice.

What is the value of simulated patient assessment in structured clinical examinations of undergraduate students, and how should it be used?

INTRODUCTION

The aim of this study was to explore correlations between clinical assessor and simulated patient (SP) scores drawn from summative Integrated Structured Clinical Examination (ISCE) and inform the best use of SP scores in future assessments.

MATERIALS AND METHODS

This retrospective study explores summative clinical assessor and formative SP numeric scores drawn from summative ISCE assessments spanning three academic years (2017-18 to 2019-20). Analyses were carried out using R 3.5.1 (R Core Team, 2018), with the stats package.

RESULTS

The sample consisted of 169 final-year BDS students across the three cohorts and included 95 females (56.2%) and 74 males (43.8%). Data from eight substations where SPs were included, were explored. Kendall's Tau, a non-parametric correlation, was used to investigate the relationships between the assessor and SP scores. Clinical assessor scores were out of a total of 20 points across various assessed domains within each substation. The formative SP assessment was out of 10 points with the same five affective domains related to communication included in each substation. Overall, the assessor and patient substation scores were not correlated (τ = 0.04, p = .272) indicating that communication skills alone, as assessed by patients, do not correlate with more holistic performance across other domains. There was significant positive correlation for two of the eight substations with the other substations showing very little correlation.

CONCLUSION

This study shows that assessment of student performance by SPs does not show a correlation with examiner scores and may provide additional information relating to affective skills of students. Notwithstanding the limitations of this study, the findings underscore the need to investigate further the value of involvement of SPs in clinical assessments to explore if scores by SPs can be used to enhance the validity of assessments if used summatively.

Has the OSCE Met Its Final Demise? Rebalancing Clinical Assessment Approaches in the Peri-Pandemic World

The Objective Structured Clinical Examination (OSCE) has been traditionally viewed as a highly valued tool for assessing clinical competence in health professions education. However, as the OSCE typically consists of a large-scale, face-to-face assessment activity, it has been variably criticized over recent years due to the extensive resourcing and relative expense required for delivery. Importantly, due to COVID-pandemic conditions and necessary health guidelines in 2020 and 2021, logistical issues inherent with OSCE delivery were exacerbated for many institutions across the globe. As a result, alternative clinical assessment strategies were employed to gather assessment datapoints to guide decision-making regarding student progression. Now, as communities learn to “live with COVID”, health professions educators have the opportunity to consider what weight should be placed on the OSCE as a tool for clinical assessment in the peri-pandemic world. In order to elucidate this timely clinical assessment issue, this qualitative study utilized focus group discussions to explore the perceptions of 23 clinical assessment stakeholders (examiners, students, simulated patients and administrators) in relation to the future role of the traditional OSCE. Thematic analysis of the FG transcripts revealed four major themes in relation to participants' views on the future of the OSCE vis-a-vis other clinical assessments in this peri-pandemic climate. The identified themes are (a) enduring value of the OSCE; (b) OSCE tensions; (c) educational impact; and (d) the importance of programs of assessment. It is clear that the OSCE continues to play a role in clinical assessments due to its perceived fairness, standardization and ability to yield robust results. However, recent experiences have resulted in a diminishing and refining of its role alongside workplace-based assessments in the new, peri-pandemic programs of assessment. Future programs of assessment should consider the strategic positioning of the OSCE within the context of utilizing a range of tools when determining students' clinical competence.

Awareness and agreement with neurofibromatosis care guidelines among U.S. neurofibromatosis specialists

Introduction

The neurofibromatoses (NF) are a group of rare, genetic diseases sharing a predisposition to develop multiple benign nervous system tumors. Given the wide range of NF symptoms and medical specialties involved in NF care, we sought to evaluate the level of awareness of, and agreement with, published NF clinical guidelines among NF specialists in the United States.

Methods

An anonymous, cross-sectional, online survey was distributed to U.S.-based NF clinicians. Respondents self-reported demographics, practice characteristics, awareness of seven NF guideline publications, and level of agreement with up to 40 individual recommendations using a 5-point Likert scale. We calculated the proportion of recommendations that each clinician rated “strongly agree”, and assessed for differences in guideline awareness and agreement by respondent characteristics.

Results

Sixty-three clinicians (49% female; 80% academic practice) across > 8 medical specialties completed the survey. Awareness of each guideline publication ranged from 53%-79% of respondents; specialists had higher awareness of publications endorsed by their medical professional organization (p < 0.05). The proportion of respondents who “strongly agree” with individual recommendations ranged from 17%-83%; for 16 guidelines, less than 50% of respondents “strongly agree”. There were no significant differences in overall agreement with recommendations based on clinicians’ gender, race, specialty, years in practice, practice type (academic/private practice/other), practice location (urban/suburban/rural), or involvement in NF research (p > 0.05 for all).

Conclusions

We identified wide variability in both awareness of, and agreement with, published NF care guidelines among NF experts. Future quality improvement efforts should focus on evidence-based, consensus-driven methods to update and disseminate guidelines across this multi-specialty group of providers. Patients and caregivers should also be consulted to proactively anticipate barriers to accessing and implementing guideline-driven care. These recommendations for improving guideline knowledge and adoption may also be useful for other rare diseases requiring multi-specialty care coordination.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13023-022-02196-x.