The Structured Clinical Operative Test (SCOT) in dental competency assessment

Student perceptions of exodontia competency assessment in a UK dental school

INTRODUCTION

Modern medical and dental training has migrated from assessing only the quantity of procedures performed to a combined assessment of both competency and quantity. This study explores student perceptions of competency assessment in exodontia at a UK dental school.

MATERIALS AND METHODS

Anonymous questionnaires were distributed to dental students in years three, four and five at the School of Dentistry, Cardiff University (n=149). Responses consisted of dichotomous tick boxes and 5-point Likert scales, with thematic analysis of free-text responses. Discrete variables were analysed using simple descriptive statistics. Recurring themes were identified from the responses.

RESULTS

A total of 129 questionnaires were returned (response rate 87%). Feedback from students indicated that they felt well prepared to undertake the competency assessment, agreeing that year three is the most appropriate year to assess competency (69%; n=86). In 50% of cases (n=65), the clinical supervisor was not present for the duration of the assessment. The undergraduate student body would like further teaching in the use of elevators (89%; n=114).

CONCLUSION

The competency assessment was deemed fit for purpose by the undergraduate student body. Further developments in the areas of clinical supervision and teaching on the use of elevators were considered and recommendations made to the School of Dentistry, Cardiff University. The current oral surgery course incorporates some of the recommendations.

Competence, competency-based education, and undergraduate dental education: a discussion paper

INTRODUCTION

The aim of undergraduate dental education is to provide competent dentists to serve societal needs and improve population oral healthcare. Competency-based education has influenced the development of dental education for decades but this term is problematic. This article explores components of competency-based undergraduate health professional education in order to help the dental profession have a better understanding of the context and purposes of undergraduate dental education.

METHODS

This is a discussion paper based on a wide reading of the literature on the education of health professionals with a specific focus on competency-based undergraduate education.

RESULTS AND DISCUSSION

Competence comprises an integration of knowledge, skills and attitudes indicating a capability to perform professional tasks safely and ethically. The process of becoming a competent practitioner is complex. Four characteristics of competency-based education are: curriculum components and content shaped by societal needs; focused on student-centred learning; learning achievement; and limited attention to time-based training and numerical targets. Alongside a competency-based approach, undergraduate dental education can be influenced by institutional features and external factors but these receive little consideration in the literature.

CONCLUSION

Understanding competence, competency-based education, and institutional and external factors will help to improve educational quality, define roles and professional development for the dental educator, and inform further research.

Current Products and Practices

This paper aims to encourage a debate on the learning outcomes that have been developed for orthodontic specialist education. In outcome-based education the learning outcomes are clearly defined. They determine curriculum content and its organization, the teaching and learning approaches, the assessment techniques and hope to focus the minds of the students on ensuring all the learning outcomes are met. In Orthodontic Specialist Registrar training, whether constructive alignment can be achieved depends on the relationship between these aspects of the education process and the various bodies responsible for their delivery in the UK.

A generic consensus assessment of undergraduate competence in forceps exodontia in the United Kingdom

The assessment of competence in clinical skills has become more frequent in published healthcare curricula and syllabuses recently. There are agreed mechanisms for the assessment of competence in the post-graduate environment, but no consensus within the undergraduate curriculum. This paper seeks to develop an agreed generic checklist for the assessment of competence in forceps exodontia.

MATERIALS AND METHODS

A modified Delphi process was undertaken with representatives from all UK dental schools (n = 13) to develop a generic checklist for the assessment of competence in forceps exodontia. A content analysis of the assessments employed by each school was used to help discussion and inform the Delphi process.

RESULTS

Seven schools currently employ a summative assessment of competence in forceps exodontia, with the majority employing a structured clinical objective test (n = 6). From the seven assessments, there were a total of 29 putative items and 10 putative domains identified for a generic checklist. These were reduced to five domains and 19 items through the content analysis and Delphi process, and a generic overarching checklist was created.

CONCLUSION

Using this generic checklist, it may now be possible to pool data inter-institution to perform more powerful analyses on how our students obtain, or fail to obtain competence in forceps exodontia.

Developing a primary dental care outreach (PDCO) course--part 1: practical issues and evaluation of clinical activity

BACKGROUND

The primary dental care outreach (PDCO) course in Newcastle, UK commenced in September 2004 with dental undergraduates attending outreach clinics on a fortnightly rotation over a 2 year continuous placement.

OBJECTIVES

To evaluate the PDCO with respect to practical issues and clinical activity.

METHODS

Clinical activity data were collected using data sheets and Access software together with data on patient attendances and Structured Clinical Operative Tests (SCOTs). Comparative clinical data were collected from the same group of students in Child Dental Health (CDH) in the School of Dental Sciences.

RESULTS

In 2004/2005, 1683 clinical procedures were undertaken in PDCO and 1362 in CDH. Of the treatment undertaken in PDCO, 37.1% was examination and treatment planning, 17.1% basic intracoronal restorations and 13.1% fissure sealing, the activity representative of day to day in general practice. Completion rates for the five piloted SCOTs in cross infection control, writing a prescription, writing a referral letter, taking a valid consent and taking a radiograph ranged from 74% to 97% in 2004/2006. The practical issues and challenges of delivering a new clinical course broadly related to induction of new PDCO staff, support of staff, establishing effective communication, timetabling logistics, delivery of clinical teaching and quality assurance.

CONCLUSION

Once the practical issues and challenges of setting up a new clinical course have been overcome PDCO has a valuable role to play in preparing undergraduates for their future practising careers.

Assessment of knowledge and competencies related to implant dentistry in undergraduate and postgraduate university education

Learning in academic settings is strongly related to the way the students are tested or examined. Assessment therefore must be integrated in the curriculum design, coordinated and should reflect the learning outcomes of the education. Assessment within the field of implant dentistry must fulfil four major objectives: complete and direct the learning process with feedback (formative), ensure that students are adequately prepared (summative), assess attitudes and skills such as critical thinking, reflection and self-assessment ability, and supply continuous feedback to teachers on curricular content and impact. Different assessment methods should be used to assess different levels of competencies throughout the curriculum. Various forms of written or oral assessment methodologies are applicable at earlier stages in the curriculum. At intermediate levels, interactive assessment methods, such as patient simulations (paper based or virtual) and more could encourage the necessary synthesis of several disciplines and aspects of the theoretical knowledge. At higher levels of competence, documentation of clinical proficiency by means of reflective portfolios and diaries is an appropriate assessment method with both formative and summative potential. The highest level of competence requires performance assessment using structured, objective, clinical criteria. The group strongly encourages the use of reflective forms of assessment methods which engage the students in a process of self-appraisal, identification of individual learning needs and self-directed learning. The ultimate goal of this would be to allow the student to develop a lifelong learning attitude.

Nursing competence 10 years on: fit for practice and purpose yet?

AIMS AND OBJECTIVES

This paper examines how nurses are prepared to be clinically competent and safe at registration, so that they are fit for practice and purpose. It follows up two papers on competence published in 1997 and 1998 and investigates subsequent developments.

BACKGROUND

In 1979, major changes in nursing affected nurse education and preparation for competence. In the following two decades, it became clear that nurses lacked clinical skills. This paper examines subsequent changes and asks the question whether this crucial shortcoming has now been remedied. This paper considers the background and context of change in nursing and nurse education in the 1980s. It looks at the new ideology, to prepare the 'knowledgeable doer' and examines the consequences of the change on nursing competency from the 1990s to the present day.

METHODS

This is a position paper. Professional policy documents from the English National Board for Nursing, Midwifery and Health Visiting, United Kingdom Central Council for Nursing, Midwifery and Health Visiting and Nursing and Midwifery Council, government reports and legislation on nursing and relevant nursing literature are examined and critically analysed and conclusions drawn.

CONCLUSIONS

From 1923-1977, mandatory nursing syllabuses set by the General Nursing Council of England and Wales required the registered nurse to have acquired certain specific clinical skills. These were rigorously tested to an explicit standard set by the General Nursing Council before a nurse was awarded state registration. Twenty-five years later, the loss of this system for ensuring this competence and the implications of this loss, have been widely recognised. As a result, many nurse training institutions have introduced clinical skills laboratories, simulation of practice and the Objective Structured Clinical Examination. However, to the authors' surprise and contrary to their initial expectations, the Nursing and Midwifery Council has not made these systems uniform or mandatory and so still has no way of ensuring all nurse training is producing safe nurses in the United Kingdom. The authors conclude that the untested educational ideology that brought root and branch change to nurse training in 1983 and which failed to produce nurses 'fit for practice and purpose' may still prevail.

RELEVANCE TO CLINICAL PRACTICE

The present paper demonstrates that United Kingdom nurse training still has no uniform and mandatory system in place to ensure, as far as is possible, that all registered nurses are clinically competent and safe to practice.

Objective structured clinical examination use in advanced orthodontic dental education

INTRODUCTION

Continuous evaluation of residents and the advanced education curriculum is required to maintain the quality of a program and ensure appropriate accreditation status. The purpose of this project was to describe the process of introducing the objective structured clinical examination (OSCE) into an advanced orthodontic education program and to evaluate its impact on the curriculum.

METHODS

To determine the content of the examination, 60 orthodontic practitioners were consulted to identify the most important clinical skills that an entry-level practitioner must be able to perform proficiently. From 18 critical clinical skills, 13 were evaluated by the OSCE in the areas of diagnosis, clinical evaluation and synthesis, and orthodontic technique. The OSCE was given to 3 classes of residents each year for 3 years (n = 46).

RESULTS

The classes were tracked longitudinally, and the students' levels were evaluated cross-sectionally. Curriculum changes were evaluated for impact in areas of deficiency. There were no significant correlations between student confidence and performance or faculty prediction of proficiency and performance beyond the first year. Great gains in performance were observed between the first and second years of the 3-year program. All skills did not reach proficiency levels, and peak performance did not always occur during the third year. Some curriculum changes were fruitful, and others were not sustained by improved performance.

CONCLUSIONS

An OSCE can provide important information regarding students and the curriculum in a clinically meaningful way that might be otherwise unattainable.

Correlation between a patient-centred outcome score and surgical skill in oral surgery

Our aim was to find out whether scores derived from a patient-centred outcome questionnaire correlated with the skill of the surgeon as assessed by a trainer. We found a relatively poor correlation after removal of a third mandibular molar, probably because outcome scores incorporate other measures, including patients' perceptions, anaesthesia, clarity of explanations and instructions, apart from surgical skill. The experience of the surgeon and the length of operation also influenced the results. The correlation was closest (i.e. higher levels of skill produced milder symptoms) when the operation was under local anaesthesia, probably because the surgeon's skill, behaviour, and attitude were visible to the patient throughout, and other team members had a less prominent role. In future, it may be more appropriate to assess the performance of the whole surgical team, rather than the surgeon alone.

The influence of self-deception and impression management upon self-assessment in oral surgery

OBJECTIVE

To see if poor self-assessment of surgical performance during removal of mandibular third molars is influenced by self-deception (lack of insight) and impression management (trying to convey a favourable impression).

DESIGN

A prospective study of 50 surgeons, surgically removing a lower third molar tooth.

SETTING

One UK dental school over a two year period.

METHODS

The surgeons' surgical skills were assessed (by two assessors) and self-assessed using check-list and global rating scales. Post-operatively, surgeons completed validated deception questionnaires which measured both self-deception enhancement (lack of insight), and impression management (the tendency to deliberately convey a favourable impression).

MAIN OUTCOME MEASURES

Reliability between assessors, and between assessors' and surgeons' self-assessments were calculated. Discrepancies between assessors' and surgeons' scores were correlated with surgeons' deception scores.

RESULTS

Reliability between assessors was excellent for checklist (0.96) and global rating scales (0.89) and better than the reliability between assessors and surgeons (0.51 and 0.49). There was a statistically significant correlation (r=0.45 p=0.001 checklist, r= 0.48 p<0.001 global) between over/ under-rating of their surgical performance by surgeons and their impression management scores. No statistically significant correlation was found between this inaccuracy in self-assessment and surgeons' individual self-deception scores.

CONCLUSION

The majority of surgeons scored themselves higher than their assessors did for surgical skill in removing a single mandibular third molar tooth. Impression management (the tendency to deliberately convey a favourable impression) may contribute to a surgeon's inaccurate self-reporting of performance. Lack of insight appears to be much less important as a contributing factor. The authors speculate that pressure to provide evidence of good performance may be encouraging surgeons to manage their image and over-score themselves.

Undergraduate orthodontic teaching in the new millennium--the Newcastle model

This paper identifies the main factors which have influenced the nature of the undergraduate orthodontic curriculum presently taught at Newcastle University. It also outlines the philosophy that underpins the teaching, states the professional aims of the tutors and spells out the clinical objectives set for the students.

Development of a structured clinical operative test (SCOT) in the assessment of practical ability in the oral surgery undergraduate curriculum

Competence in undergraduate oral surgery involves assessment of the students' knowledge, practical skill and attitude. The assessment of practical skills can be achieved using a formative assessment method such as the structured clinical operative test (SCOT) which uses a checklist for the assessment of a clinical task. The aim of this study was primarily to determine whether SCOTs could be integrated into the oral surgery undergraduate course, and secondarily to collect feedback from both the students and the staff on this assessment. A validated checklist was used by trained examiners to assess the performance of 49 students in their second clinical year undertaking simple exodontia. Feedback was collected from the students by questionnaire and from the staff by interview. The SCOT was well received by both students and staff. The main problem highlighted by the students was the perceived inter-examiner variability and by the staff, the disruptive effect on the clinics. The checklist must be objective enough to prevent misinterpretation. Despite these limitations, the SCOT provides a more objective assessment and serves to highlight weaker students, allowing them to be targeted for closer supervision and instruction as well as providing the students with a tool to measure their progress in oral surgery. The use of peer assessment by students may alleviate the burden on the staff.

Orthodontic undergraduate education: assessment in a modern curriculum

This paper describes the problems of assessment for students on an undergraduate orthodontic course which is using problem-based learning to encourage active self-learning, deep learning and give students the skills to become effective lifelong learners in the future. Traditional assessment does little to encourage a range of behaviours and skills which are encouraged through problem-based learning but which must be rewarded, in the minds of the students, by the chosen method of assessment. This is because learner behaviour is driven by assessment. Details of the range of assessment methods available and those chosen on the orthodontic undergraduate course at the University of Manchester Dental School are reported and discussed.