The Oral and Maxillofacial Objective Structured Assessment of Technical Skills (OMOSATS) examination: a pilot study

Kern's six-step approach allows for meaningful implementation of an orthognathic surgery training course

OBJECTIVES

This study investigated the effectiveness of simulation training in improving the confidence and competency of oral and maxillofacial surgery (OMS) residents in performing orthognathic surgery (OGS).

METHODS

Kern's six-step approach was applied when designing the simulation training for OMS residents. The difficulties encountered by the residents when learning OGS were considered when designing the training program. A training course consisting of didactic sessions, hands-on training on three-dimensional training models, and an assessment tool was implemented for OMS residents. Improvement in the confidence and competence of OMS residents in performing OGS, fidelity of the three-dimensional models, and satisfaction with the course was evaluated.

RESULTS

All OMS residents (10/10) completed the course. The perceived difficulty in learning OGS was mainly related to the manipulation of the jawbones. While there were improvements in the median confidence and competence scores (3/5 to 4/5), only the differences in competence were found to be statistically significant (p < 0.01, Wilcoxon signed-rank test). Improvements in confidence and competence did not correlate. The mean fidelity scores of both the maxillary and mandibular models were adequate at 3.2 out of 5. Overall, satisfaction with the course was high (5/5).

CONCLUSIONS

The six-step approach provides a guided process for educators to formulate a training course directed toward the perceived needs of students. Targeted training can significantly enhance the students' competence. Greater efforts should also be put in place to allow simultaneous developments in the students' confidence along with their competence.

Assessment of suturing and scaling skills of periodontology and oral medicine residents by OSATS method: a pilot study

INTRODUCTION

Updating the method for evaluating suturing and scaling skills in dental education has attracted relatively little attention and there is no consensus to what should be assessed and how. The purpose of this study was to investigate the applicability of the Objective Structured Assessment of Technical Skill (OSATS) method for these two basic skills, the possible association between the scores and demographic factors, and the level of satisfaction of residents with this method.

METHODS

All six periodontics and three oral medicine residents were recruited by census method and video-recorded while performing a simple interrupted suture, a figure eight suture and scaling on a model. Then, the videos were evaluated independently via a checklist and a global rating scale (GRS) by two expert raters. Agreement between raters and residents' satisfaction were evaluated. Correlation between demographic factors of participants and scores was also assessed. T-test and linear regression analysis were used.

RESULTS

There was no significant difference between the scores based on the views of the two raters for each of the checklist (ICC = 0.99, CI = 0.96-0.99, P < 0.001) and GRS (ICC = 0.97, CI = 0.86-0.99, P < 0.001). Linear regression showed no correlation between gender and scores but periodontics major and higher year of education showed correlation with higher scores.

CONCLUSION

Considering the excellent agreement between raters in using both the checklist and GRS components of OSATS, and satisfaction of 88% the residents with this method, it seems to be able to provide a reliable assessment.