Development of a structured clinical operative test (SCOT) in the assessment of practical ability in the oral surgery undergraduate curriculum

Assessing competency in exodontia: A UK perspective

INTRODUCTION

Assessment ensures that undergraduate students and postgraduate trainees are providing optimal patient care. Numerous assessment methods are currently being used within the United Kingdom. Therefore, the purpose of this review was to provide an overview of the methods used to assess competency in exodontia and compare methods against criteria defined for good assessment.

MATERIALS AND METHODS

A systematic narrative review was carried out. Databases were electronically searched for published studies assessing the competence of undergraduate students and postgraduate trainees in exodontia up to October 2023.The validity, reproducibility, equivalence, feasibility, educational impact, catalytic effect and acceptability of each assessment method was considered.

RESULTS

In this review, 591 undergraduate students (n = 529) and postgraduate dental trainees (n = 62) had their competence in exodontia assessed. Undergraduate students were assessed on simple exodontia with forceps and elevators, whereas postgraduate trainees were assessed on the surgical removal of mandibular third molars. Only one study took place in a simulated dental environment, the remaining (n = 7) took place in a clinical environment. A range of formative and summative assessment methods were identified including structured clinical objective tests (SCOT), self-assessment, peer assessment and logbooks. Common marking criteria such as global rating and checklist scales were also highlighted.

CONCLUSION

This review highlights the lack of evidence to support one form of assessment and the lack of consensus on assessment within the United Kingdom. Further research related to this topic, alongside the opinion of trainees and trainers is required to improve assessments. This will maximise learning opportunities for trainees and translate to better patient care.

Exodontia skills acquisition: Focusing on clinical teaching and training

OBJECTIVES

The aim of the research was to evaluate the traditional exodontia block course. The objectives were to explore the experiences and views of students, clinical teachers and dental practitioners of the various elements of the course curriculum.

METHODS

The study was a qualitative, participatory action research study using descriptive analysis. The study was conducted at a Dental Faculty in South Africa. A purposive sample of students, clinical teachers and dental practitioners were invited to participate. Focus group discussions were used to collect data which was analysed by an external coder.

RESULTS

The study population consisted of 15 undergraduate dentistry students, 10 clinical teachers and seven dental practitioners. Four broad themes with sub-themes emerged from the study. The main themes identified strengths and deficiencies of the traditional course and made recommendations for its improvement. The themes identified were i) Integration of knowledge and skills, ii) Block course structure, iii) Challenges associated, and iv) Recommendations for improvement. Overall, the participants were satisfied that the course met its objectives. The results pertaining to clinical skills acquisition identified that teaching the use of elevators and luxators in the course, and standardisation of terminology among all clinical teachers as areas requiring attention. Teaching and learning strategies such as community-based learning, peer learning, case reviews, feedback and visual technology were viewed by the student, as well as clinical teacher samples, as strategies most beneficial to clinical learning.

CONCLUSIONS

The review of the curriculum for exodontia skills acquisition and development, provided several benefits. Firstly, this research served as a quality assurance indicator. It further highlighted many teaching and learning strategies that would improve clinical skills development, reduce stress and anxiety, and support student learning. To a large extent, pertinent information was obtained that served to inform the subsequent redesigning of the course. The findings of the study augment the literature currently available on the best practice for exodontia skills acquisition and development and provide baseline information for the planning and redesign of related courses.

An investigation into the oral surgery curriculum and undergraduate experience in a sample of DFT's in the UK

AIM

There is conflicting evidence concerning the ability of recent graduates to meet the standards required in exodontia to be independent practitioners. In particular, there is a lack of confidence in surgical extractions. This study evaluated Foundation Trainee's confidence levels in their oral surgery experience and the teaching of oral surgery in UK dental schools.

MATERIALS AND METHODS

Two separate studies were carried out, and the first was a survey to all foundation trainees in the Yorkshire and Humber Deanary in 2019 which analysed their oral surgery experience and confidence levels. The second study surveyed all Dental Schools in the UK with regard to their contact hours, assessment methods, instruments taught and number of extractions in oral surgery for their undergraduate programmes.

RESULTS

There was a response rate of 52% for the foundation trainees which showed that they were confident in performing non-surgical extractions but lacked confidence with surgical extractions. There was a positive correlation between the amount of undergraduate experience and confidence levels in exodontia. The second survey which had a response rate of 71% showed variations in the numbers of hours taught for oral surgery, instruments taught and assessment methods, with the structured clinical operative test used more commonly. On average, 41 non-surgical extractions were performed by students before graduation and two surgical extractions.

CONCLUSION

There is a need to identify how oral surgery teaching can be improved in UK dental schools to ensure that foundation trainees are more confident and competent in both surgical and non-surgical exodontia. In addition, teaching and assessment methods need to be developed to reflect such requirements.

Self-confidence in oral and maxillofacial surgery: a cross-sectional study of undergraduate dental students at Kuwait University

BACKGROUND

To evaluate the self-confidence of undergraduate dental students in relation to oral and maxillofacial surgery (OMFS) to assess the teaching curriculum at Kuwait University using a validated questionnaire originally developed by the Association of British Academic Oral Maxillofacial Surgeons (ABAOMS).

METHODS

A cross-sectional survey of sixth year (n = 20) and seventh year (n = 19) dentistry students was conducted by Kuwait University Faculty of Dentistry between the 1st and 15th of May 2020. The ABAOMS questionnaire is composed of 17 questions assessing various areas of the undergraduate OMFS curriculum. The response options to the questionnaire utilised a Likert scale. Independent sample t-tests were performed to assess the difference in responses between the 2 year groups. Spearman's rho correlations were calculated to measure the strength of association between confidence in all aspects of surgical and forceps exodontia.

RESULTS

A total of 39 questionnaires were completed by the students. The majority of students expressed feelings of confidence that they have enough knowledge to undertake independent practice (61%). General aspects of the questionnaire were answered favourably except for surgical extraction of teeth, in which both classes reported a lower level of self-confidence.

CONCLUSIONS

The ABAOMS survey revealed the students' self-confidence in undertaking independent practice and preforming basic oral surgery procedures. Students felt comfortable with exodontia using forceps and elevators, root removal, managing acute pericoronitis, managing haemorrhage from a socket, assessing impacted teeth, and recognising the clinical features of potentially malignant and malignant lesions of the oral cavity. They reported a lower level of confidence in performing surgical procedures.

Knowledge, attitude, and practice of senior dental students toward management of complications in exodontia

Introduction

Dental graduates have a professional responsibility of being competent in managing the complications in exodontia, a very commonly performed procedure.

Aim

The aim of this paper was to assess the knowledge, attitude, and practice of senior dental students toward the management of complications in exodontia.

Materials and Methods

Four hundred dental students participated in the study voluntarily. Verbal consent was obtained after elaborating the purpose of the study. A short validated questionnaire consisting of 12 close-ended multiple choice questions was distributed to all the students. The questionnaires were collected back and results from fully filled questionnaires were tabulated in Microsoft Excel 2007.

Results

Nearly 93.4% of the students perform extractions in their practice on a routine basis. Among those 32.5% of the students have encountered complications, out of which only 20% of the students are confident in managing them. The remaining 64.8% of students who have not encountered complications are not confident in managing the same.

Conclusion

This study revealed the need for increase in cases which will in turn help the students to transfer their theoretical knowledge into professional skills.

Evaluating the effectiveness of a preclinical practice of tooth preparation using digital training system: A randomised controlled trial

PURPOSE

This study aimed to evaluate the effectiveness of preclinical training on ceramic crown preparation using digital training system compared with traditional training method.

MATERIALS AND METHODS

A sample of 66 participating fourth-year undergraduate dental students were randomly assigned to one of two groups for ceramic crown preparation: experimental group as trained using digital method with Online Peer-Review System (OPRS) and Real-time Dental Training and Evaluation System (RDTES); control group as trained using traditional method with instructor demonstration and evaluation. At the completion of training periods, both groups performed the preparation of ceramic crown of upper left central incisor, which were blindly scored by the experienced instructors under the pre-defined assessment criteria. The results of both were compared using Student's t-test or Wilcoxon signed rank test. The level of significance was P < .05. The questionnaires regarding the benefits or drawbacks of digital training system were answered by the students of experimental group.

RESULTS

Five of 15 items in the assessment outcome of the digital group were significantly better than the traditional group. The questionnaire results from the students of the experimental group indicated 96.97% of the students agreed or strongly agreed that using digital training system could better improve the practical ability than traditional method. The total scores of practical results were significantly positively correlated with the points of the questionnaires.

CONCLUSION

The digital training system with OPRS and RDTES might be a good alternative to the traditional training method in the preclinical course of dental practice.

Student perceptions of exodontia competency assessment in a UK dental school

INTRODUCTION

Modern medical and dental training has migrated from assessing only the quantity of procedures performed to a combined assessment of both competency and quantity. This study explores student perceptions of competency assessment in exodontia at a UK dental school.

MATERIALS AND METHODS

Anonymous questionnaires were distributed to dental students in years three, four and five at the School of Dentistry, Cardiff University (n=149). Responses consisted of dichotomous tick boxes and 5-point Likert scales, with thematic analysis of free-text responses. Discrete variables were analysed using simple descriptive statistics. Recurring themes were identified from the responses.

RESULTS

A total of 129 questionnaires were returned (response rate 87%). Feedback from students indicated that they felt well prepared to undertake the competency assessment, agreeing that year three is the most appropriate year to assess competency (69%; n=86). In 50% of cases (n=65), the clinical supervisor was not present for the duration of the assessment. The undergraduate student body would like further teaching in the use of elevators (89%; n=114).

CONCLUSION

The competency assessment was deemed fit for purpose by the undergraduate student body. Further developments in the areas of clinical supervision and teaching on the use of elevators were considered and recommendations made to the School of Dentistry, Cardiff University. The current oral surgery course incorporates some of the recommendations.

Dental student suturing skills: a multicentre trial of a checklist-based assessment

BACKGROUND

The assessment of clinical skills is essential to determine whether an undergraduate is competent to perform the tasks outlined in the curriculum. Such assessments in dentistry have historically not been subjected to large scale validity and reliability testing due the relatively small student numbers at each institute. The aims of this study were to test the validity and reliability of a standardised, checklist-based, suturing objective structured clinical examination (OSCE) and then to perform a multicentre trial to determine its performance over a large cohort of students.

MATERIALS AND METHODS

A total of seven UK schools agreed to take part in the trial. To test the validity and reliability of the checklist, the examiner at each institution reviewed and scored video footage of 10 students performing the assessment. Each institution then carried out the assessment providing a checklist score and a global score for each of their own students.

RESULTS

The assessment was well received by the staff, with acceptable inter-examiner variability. In total, 496 students completed the suturing OSCE with a success rate of 81% with a variation between schools of between 66% and 96%. A significant correlation was found between the checklist score and the global score (r = 0.361, P = 0.000). No one item on the checklist was found to be a determinant factor in the outcome of the OSCE.

CONCLUSIONS

This checklist-based assessment of suturing skills was found to have face and content validity. Its reliability was promising, but merits further investigation. There may be an argument for the standardisation of the assessment of this core surgical skill throughout several UK-based dental schools.

Implementation and validation of an extraction course using mannequin models for undergraduate dental students

OBJECTIVE

There is a gap in the education of dental undergraduate students between theoretical knowledge and the practical procedure of tooth extraction. It was the aim of this study to implement and validate an undergraduate course for oral extraction using mannequin models, in order to enhance practical competence in this field of dentistry.

MATERIAL AND METHODS

A one-term oral extraction course was implemented as part of the oral surgery curriculum for undergraduate dental students, including 4 h of theoretical education, followed by 4 h of practical training in tooth extraction on mannequin models, with the students divided into small groups. Forty-nine students attending this extraction course were asked to validate their training in dental extraction using a questionnaire (TRIL-mod; University of Trier, Trier, Germany). The students' practical competence in oral extraction was assessed by a senior supervisor at the end of the course, using a standardized checklist.

RESULTS

All questionnaires were returned. Overall, the course was rated with an average score of 4.7 (rating range: 1-6; 1 = worst/6 = best). The assessment of practical skills yielded a successful and complete performance of tooth extraction by 94% of the students.

CONCLUSIONS

A tooth extraction course using a mannequin model is appreciated by dental students and can achieve a sufficient level of competence in tooth extraction with reasonable educational efforts.

Invited commentary: Issues with assessing competence in undergraduate dental education

Competence of practising dentists is an issue that is being addressed in both the undergraduate and postgraduate environment in dental education. The Association of Dental Education in Europe has opened the debate further with guidelines for qualifying dentists in Europe. This puts further pressure on dental schools to produce graduates who are transparently competent. This invited commentary examines the differing concepts of competence and how they might be assessed in the undergraduate dental curriculum.

The retention of suturing skills in dental undergraduates

BACKGROUND

The ability to place sutures is an essential skill in dentistry. In our institution, the introduction of a video teaching aid and a checklist-based assessment has improved the standardisation of teaching as well as the transparency and objectivity of the assessment of suture skills. The aim of this study was to determine whether this skill once acquired is retained in the short term.

MATERIALS AND METHODS

The same cohort of 67 fourth-year students performed the same assessment at two different time points 2 months apart. The first was a formative assessment and the second was a summative assessment. Feedback was collected on both occasions.

RESULTS

On the first sitting of the assessment, 93% of the students were successful compared with 79% on the second occasion. Student feedback was similar on both occasions with 77% and 75% reporting that they thought it was an easy exam. On the first occasion, 27% thought that it was a stressful examination compared with 43% on the second occasion, which may have affected their performance.

CONCLUSIONS

These findings suggest that regular reinforcement is required to avoid deterioration of suturing skills even in the short term. This might involve exposing the students to surgery earlier in the course and introducing regular formative assessments which the students appear to associate with less stress.

A generic consensus assessment of undergraduate competence in forceps exodontia in the United Kingdom

The assessment of competence in clinical skills has become more frequent in published healthcare curricula and syllabuses recently. There are agreed mechanisms for the assessment of competence in the post-graduate environment, but no consensus within the undergraduate curriculum. This paper seeks to develop an agreed generic checklist for the assessment of competence in forceps exodontia.

MATERIALS AND METHODS

A modified Delphi process was undertaken with representatives from all UK dental schools (n = 13) to develop a generic checklist for the assessment of competence in forceps exodontia. A content analysis of the assessments employed by each school was used to help discussion and inform the Delphi process.

RESULTS

Seven schools currently employ a summative assessment of competence in forceps exodontia, with the majority employing a structured clinical objective test (n = 6). From the seven assessments, there were a total of 29 putative items and 10 putative domains identified for a generic checklist. These were reduced to five domains and 19 items through the content analysis and Delphi process, and a generic overarching checklist was created.

CONCLUSION

Using this generic checklist, it may now be possible to pool data inter-institution to perform more powerful analyses on how our students obtain, or fail to obtain competence in forceps exodontia.

The assessment of suturing skills of dental undergraduates

INTRODUCTION

To improve the assessment of suturing skills, we firstly developed a checklist based assessment. As an aid to the teaching of suturing skills a video was subsequently developed.

MATERIALS AND METHODS

To validate the checklist a cohort of 57 final year students were videoed performing the formative suturing assessment which was assessed by two examiners independently. Two cohorts of third year students took part in the formal assessment but only the second cohort of students would have access to the video.

RESULTS

The first cohort of 58 third year students' success rate was 93% and 1 year later 94% of 53 third year students were successful. One hundred per cent of the first cohort and 98% of the second cohort thought that the exam was a fair assessment of suturing skills. The majority of students thought that the checklist was helpful (94% and 93%). However, 62% and 55% thought that the assessment was a stressful experience. However, only 80% of the first cohort compared with 98% of the second cohort thought that they could now place sutures in intra-oral wounds.

CONCLUSION

Our findings suggest that checklist based assessments of suturing skills were well received by the students and improved the objectivity and transparency of the assessment process. Further work is required to determine if teaching initially on a tabletop model can be transferred to the dental clinical situation.

Oral surgery undergraduate teaching and experience in the United Kingdom: a national survey

The aim of this work was to determine the structure, mode of delivery, mode of assessment and staffing of the oral surgery undergraduate curriculum within UK dental schools. A questionnaire was distributed by e-mail in January 2006 to each of the 15 dental schools with undergraduate dental degree programmes in Ireland and the UK. Those providing feedback then met to clarify any areas as required. Thirteen completed questionnaires were returned. There were a total of 55 academics involved in the teaching of oral surgery at these 13 institutions. Over the three clinical years the mean number of clinical sessions was 51. The mean staff student ratio for supervision of forceps exodontia was 1:5. On average 51 teeth were extracted by each student in the clinical years. The mean staff student ratio for surgical extractions was 1:2. The mean number of surgical extractions for each student was 6. All schools formatively assessed competency in forceps exodontia and 9 of 13 assessed surgical extractions. Summative assessment of exodontia was done in six schools and surgical extractions in 4 of 13 schools. All 13 schools deliver teaching programmes designed to meet the requirements of the frameworks governing the central curriculum. There were, however, variations between individual schools in the content and delivery of the oral surgery clinical teaching programmes. There were dramatic variations in the numbers of academic staff involved and some institutions relied on their NHS colleagues to deliver the clinical teaching.

The influence of staffing and timetabling on achieving competence in surgical extractions

INTRODUCTION

Competence based education is becoming more important in dentistry and medicine. In dentistry clinical skills are assessed using longitudinal assessments or structured objective clinical tests. We have previously presented the assessment of competence in surgical extractions however the success rate for this was poor. The opportunity to alter staffing levels and timetabling arose and we present the influence of this on the achievement of competence.

METHODS

The competence assessments and portfolios of two consecutive years of dental undergraduates were examined after completing their surgical extraction course. The first cohort received 9 sessions of teaching spread over 2 years with one staff supervisor per session. The second cohort received 10 sessions with varying numbers of staff supervisors.

RESULTS

The first cohort required 210 staff sessions and performed 275 surgical extractions (mean 4), and 23% achieved competence. The second cohort required 240 staff sessions and performed 403 surgical extractions (mean 6), and 66% achieved competence. Thirty six extra sessions were provided for students in the second cohort who failed to complete their competence during the allocated blocks and following this 99% of the second cohort achieved competence. These differences are significant (P < 0.01).

CONCLUSION

It is possible to demonstrate competence in large numbers of undergraduates in surgical extraction. The process can be influenced by staffing and timetabling changes which focus student experience and learning.

The relationship between formative and summative assessment of undergraduates in oral surgery

INTRODUCTION

Regular, on-going formative assessments or a more comprehensive end of program summative assessment can be used to assess student's progress. The literature is scarce regarding the use of both types of assessment coincidentally and if the outcomes of formative assessments can predict the grades achieved summatively. This paper investigates whether there is any correlation between the formative assessments used in routine clinical contacts in the Newcastle course and the final summative assessment grade.

MATERIALS AND METHODS

The logbooks of 72 third year dental students were examined to collect data on the gradings achieved for formative and summative assessments. The categories of history and presentation, local anaesthesia, extraction and overall grades for each were compared using Spearman's Rho.

RESULTS

Six logbooks were incomplete so were omitted from analysis (n = 66). In the formative assessment, the total number of teeth extracted was 1913, giving a mean per student of 29 (SD = 6.88). In the summative assessment, 15 students failed on their first attempt but all subsequently passed on resit.

DISCUSSION

Positive correlation was found within each of the two types of assessment but no correlation was found between the overall grades for the formative and summative assessment. The overall grade achieved by a student in the formative assessment does not appear to predict their grade in the summative. The use of both assessments is beneficial, however, their use together must be considered carefully to avoid confusing students.

Checklists for assessment and certification of clinical procedural skills omit essential competencies: a systematic review

OBJECTIVE

To develop generic criteria for the global assessment of clinical procedural competence and to quantify the extent to which existing checklists allow for holistic assessment of procedural competencies.

METHODS

We carried out a systematic review and qualitative analysis of published clinical procedural skills assessment checklists and enumerated the contents of each. Source materials included all English-language papers published from 1990 to June 2005, identified from 18 databases, which described or referred to an assessment document for any clinical procedural skill. A pair of reviewers identified key generic themes and sub-themes through in-depth analysis of a subset of 20 checklists with iterative agreement and independent retesting of a coding framework. The resulting framework was independently applied to all checklists by pairs of reviewers checking for the emergence of new themes and sub-themes. Main outcome measures were identification of generic clinical procedural skills and the frequency of occurrence of each in the identified checklists.

RESULTS

We identified 7 themes ('Procedural competence', represented in 85 [97%] checklists; 'Preparation', 65 [74%]; 'Safety', 45 [51%]; 'Communication and working with the patient', 32 [36%]; 'Infection control', 28 [32%]; 'Post-procedural care', 24 [27%]; 'Team working', 13 [15%]) and 37 sub-themes, which encapsulated all identified checklists. Of the sub-themes, 2 were identified after the initial coding framework had been finalised.

CONCLUSIONS

It is possible to develop generic criteria for the global assessment of clinical procedural skills. A third and a half of checklists, respectively, do not enable explicit assessment of the key competencies 'Infection control' and 'Safety'. Their assessment may be inconsistent in assessments which use such checklists.

UK national curriculum for undergraduate oral surgery subgroup for teaching of the Association of British Academic Oral and Maxillofacial Surgeons

This article describes a curriculum in oral surgery for undergraduate dental education in the United Kingdom prepared by the Education Subgroup of The British Academic Oral and Maxillofacial Surgeons. This group is made up of representatives from each of the 13 UK Dental Schools, one Irish Dental School and one Post-graduate Institute. The document represents a group consensus of an undergraduate UK oral surgery curriculum that is founded on the frameworks outlined by the General Dental Council, the Quality Assurance Agency for Higher Education and the Association for Dental Education in Europe. Our curriculum document is more prescriptive than the information available in the aforementioned documents. It is based on UK undergraduate oral surgery experience and thus attempts to set achievable core competencies and, in a few areas, challenges the available curriculum and related documentation.

Assessing competency in dentoalveolar surgery: a 3-year study of cumulative experience in the undergraduate curriculum

AIM

To assess and observe the development of competence in oral surgical skills during a 3-year undergraduate programme.

METHOD

Over a 3-year period 75 students were followed through from the beginning of their clinical course to their Bachelor of Dental Surgery graduation and their surgical experience monitored by the use of logbooks. Their development of competence was assessed objectively through structured assessments and subjectively by a single tutor responsible for each year. Assessments were made of their ability in exodontia, pre-surgical assessment and the surgical extraction of teeth/roots.

RESULTS

Seventy-three students completed the course (97%). Successful completion rates for the objective testing were 100% for both exodontia and pre-surgical assessment. The surgical assessment, (surgical extraction of a tooth or root) had a successful completion rate of only 23% and the caseload for students was low with a mean of four teeth removed surgically upon graduation. Relationships were examined between total numbers of teeth extracted, total number of minor oral surgical procedures completed and the successful completion of the surgical competence assessment, but no significant relationships were found.

CONCLUSIONS

This study demonstrates that it is possible to achieve objectively measurable levels of competence in undergraduates undertaking oral surgery procedures. It is however, a labour and time intensive process and appropriate clinical and teaching resources are required. National co-operation towards agreed standardised competencies should be encouraged to allow data to be pooled and more powerful analyses to occur.