The OSCE has landed: One small step for British psychiatry?

The Psychiatry OSCE: a 20-year retrospective

OBJECTIVE

Twenty years ago researchers at the University of Toronto launched the Psychiatry Skills Assessment Project (PSAP), a research program exploring Objective Structured Clinical Examinations (OSCEs) in psychiatry. Between 1994 and 2005 PSAP produced publications on the feasibility, reliability, validity, ethics, and practical concerns of OSCEs in psychiatry. The current review has two parts: a review of the state of the art of OSCEs in psychiatry 20 years after they were introduced and documentation of the impact of the PSAP research program.

METHODS

A literature search identified all publications on OSCEs and psychiatry. Articles were coded thematically, and locations of agreement and controversies were identified. Bibliometric analysis identified citations of PSAP research papers, which were analyzed thematically.

RESULTS

As of May 2013, there were 250 publications related to OSCEs in psychiatry (not including 10 PSAP papers), published in 29 different countries and ten languages. Prominent topics were the validity and acceptability of OSCEs and SPs, systems issues in adopting OSCEs in psychiatry, and the effects on learning. Eighty-eight percent of all publications cited PSAP work (300 citations). Citations were employed for four purposes: as evidence/justification (54 %); to frame replication research (14 %); to support adaptation of OSCEs in other countries and professions (15 %); and for debate (18 %).

CONCLUSIONS

Over the past 20 years, use of OSCEs has grown steadily in psychiatry, and several national certification organizations have adopted OSCEs. PSAP work, introduced two decades ago, continues to provide a scholarly foundation for psychometric, practical, and ethical issues of interest to this field.

Learner perception of oral and written examinations in an international medical training program

Background

There are an increasing number of training programs in emergency medicine involving different countries or cultures. Many examination types, both oral and written, have been validated as useful assessment tools around the world; but learner perception of their use in the setting of cross-cultural training programs has not been described.

Aims

The goal of this study was to evaluate learner perception of four common examination methods in an international educational curriculum in emergency medicine.

Methods

Twenty-four physicians in a cross-cultural training program were surveyed to determine learner perception of four different examination methods: structured oral case simulations, multiple-choice tests, semi-structured oral examinations, and essay tests. We also describe techniques used and barriers faced.

Results

There was a 100% response rate. Learners reported that all testing methods were useful in measuring knowledge and clinical ability and should be used for accreditation and future training programs. They rated oral examinations as significantly more useful than written in measuring clinical abilities (p < 0.01). Compared to the other three types of examinations, learners ranked oral case simulations as the most useful examination method for assessing learners’ fund of knowledge and clinical ability (p < 0.01).

Conclusions

Physician learners in a cross-cultural, international training program perceive all four written and oral examination methods as useful, but rate structured oral case simulations as the most useful method for assessing fund of knowledge and clinical ability.

Electronic supplementary material

The online version of this article (doi:10.1007/s12245-009-0147-2) contains supplementary material, which is available to authorized users.

Psychiatric education and simulation: a review of the literature

OBJECTIVES

Simulation methodologies are integral to health professional education at all levels of training and across all disciplines. This article reviews the literature on simulation in psychiatric education and explores recent innovations and emerging ethical considerations related to teaching and evaluation.

METHOD

The authors searched the MEDLINE, ERIC, and PsycINFO databases from 1986 to 2006 using multiple search terms. A detailed manual search was conducted of Academic Psychiatry, Academic Medicine, and Medical Education. Literature indirectly relevant to the search parameter was also included.

RESULTS

Of the more than 5000 articles retrieved from the literature on simulation and health professional education, 72 articles and books used the terms simulation and standardized patients or role play and psychiatry education. Of the more than 900 articles on objective structured clinical examinations (OSCE), 24 articles related specifically to psychiatry OSCEs.

CONCLUSIONS

Live simulation is used in teaching, assessment, and research at all levels of training in psychiatric education. Simulated and standardized patients are useful and appropriate for teaching and assessment and are well accepted at both undergraduate and post-graduate level. There is also an important place for role play. Further research is needed regarding the implications of different simulation technologies in psychiatry.

"You can do it late at night or in the morning. You can do it at home, I did it with my flatmate." The educational impact of an OSCE

BACKGROUND

The use of an objective structured clinical examination (OSCE) has been a powerful influence on doctor training but assessments do not always drive study behaviour in predictable ways.

AIMS

To investigate the impact an OSCE has on study behaviours by exploring how 5th year medical students identify what to learn for a summative OSCE and the role of the clinical environment in their preparation.

METHODS

A semi-structured questionnaire survey asked about strategies used by students to prepare for the OSCE. Focus group interviews explored successful methods of preparation for the OSCE. Themes were identified and classified.

RESULTS

The questionnaire response rate was 84%. Topic identification was usually from the list of examinable problems, past OSCE papers and a booklet prepared by a previous student containing a series of OSCE checklists. The study behaviours of students preparing for the OSCE exam were predominantly to practise on each other, and to rehearse routines. Strategic and efficient study habits were favoured over conscious utilization of the clinical environment.

CONCLUSION

The expectation that an OSCE drives learning into the clinical workplace was not supported by this study. This suggests the role of clinical experience in helping students prepare for the exam may be more subliminal, or that an OSCE is more as a test of psychomotor skills than a marker of clinical experience. An unexpected benefit may be to drive more collaborative learning.