Clinical guidelines and self-assessment in dental education

Study on self-assessment regarding knowledge of temporomandibular disorders in children/adolescents by Swedish and Saudi Arabian dentists

OBJECTIVE

To estimate the degree of self-assessed knowledge among dentists in Sweden and Saudi Arabia regarding temporomandibular disorders (TMD) in children and adolescents using a summative form of assessment and further to investigate the possible factors that may influence the self-assessed knowledge.

MATERIALS AND METHODS

A questionnaire survey covering four domains (Etiology; Diagnosis and classification; Chronic pain and pain behavior; Treatment and prognosis) regarding TMD knowledge was used. Out of 250 questionnaires (125 in each country) a total of 65 (52%) were returned in Sweden and 104 (83%) in Saudi Arabia.

RESULTS

Self-assessed individual knowledge was significantly associated to the level of actual knowledge among the Swedish groups in the domains Etiology; Diagnosis and classification and Treatment and prognosis (p < 0.05). However, in the Saudi Arabian groups a corresponding significant association was only found in the domain Diagnosis and classification (p < 0.05).

CONCLUSIONS

This study showed that there is a difference in the accuracy of self-assessment of own knowledge between the dentists in Sweden and Saudi Arabia. The Swedish dentists have a better ability to assess their level of knowledge compared to Saudi Arabian dentists regarding TMD in children and adolescents. This difference could be related to several factors such as motivation, positive feedback, reflection, psychomotor, and interpersonal skills, which all are more dominant in the Swedish educational tradition.

Interventions for providers to promote a patient-centred approach in clinical consultations

BACKGROUND

Communication problems in health care may arise as a result of healthcare providers focusing on diseases and their management, rather than people, their lives and their health problems. Patient-centred approaches to care delivery in the patient encounter are increasingly advocated by consumers and clinicians and incorporated into training for healthcare providers. However, the impact of these interventions directly on clinical encounters and indirectly on patient satisfaction, healthcare behaviour and health status has not been adequately evaluated.

OBJECTIVES

To assess the effects of interventions for healthcare providers that aim to promote patient-centred care (PCC) approaches in clinical consultations.

SEARCH METHODS

For this update, we searched: MEDLINE (OvidSP), EMBASE (OvidSP), PsycINFO (OvidSP), and CINAHL (EbscoHOST) from January 2000 to June 2010. The earlier version of this review searched MEDLINE (1966 to December 1999), EMBASE (1985 to December 1999), PsycLIT (1987 to December 1999), CINAHL (1982 to December 1999) and HEALTH STAR (1975 to December 1999). We searched the bibliographies of studies assessed for inclusion and contacted study authors to identify other relevant studies. Any study authors who were contacted for further information on their studies were also asked if they were aware of any other published or ongoing studies that would meet our inclusion criteria.

SELECTION CRITERIA

In the original review, study designs included randomized controlled trials, controlled clinical trials, controlled before and after studies, and interrupted time series studies of interventions for healthcare providers that promote patient-centred care in clinical consultations. In the present update, we were able to limit the studies to randomized controlled trials, thus limiting the likelihood of sampling error. This is especially important because the providers who volunteer for studies of PCC methods are likely to be different from the general population of providers. Patient-centred care was defined as a philosophy of care that encourages: (a) shared control of the consultation, decisions about interventions or management of the health problems with the patient, and/or (b) a focus in the consultation on the patient as a whole person who has individual preferences situated within social contexts (in contrast to a focus in the consultation on a body part or disease). Within our definition, shared treatment decision-making was a sufficient indicator of PCC. The participants were healthcare providers, including those in training.

DATA COLLECTION AND ANALYSIS

We classified interventions by whether they focused only on training providers or on training providers and patients, with and without condition-specific educational materials. We grouped outcome data from the studies to evaluate both direct effects on patient encounters (consultation process variables) and effects on patient outcomes (satisfaction, healthcare behaviour change, health status). We pooled results of RCTs using standardized mean difference (SMD) and relative risks (RR) applying a fixed-effect model.

MAIN RESULTS

Forty-three randomized trials met the inclusion criteria, of which 29 are new in this update. In most of the studies, training interventions were directed at primary care physicians (general practitioners, internists, paediatricians or family doctors) or nurses practising in community or hospital outpatient settings. Some studies trained specialists. Patients were predominantly adults with general medical problems, though two studies included children with asthma. Descriptive and pooled analyses showed generally positive effects on consultation processes on a range of measures relating to clarifying patients' concerns and beliefs; communicating about treatment options; levels of empathy; and patients' perception of providers' attentiveness to them and their concerns as well as their diseases. A new finding for this update is that short-term training (less than 10 hours) is as successful as longer training.The analyses showed mixed results on satisfaction, behaviour and health status. Studies using complex interventions that focused on providers and patients with condition-specific materials generally showed benefit in health behaviour and satisfaction, as well as consultation processes, with mixed effects on health status. Pooled analysis of the fewer than half of included studies with adequate data suggests moderate beneficial effects from interventions on the consultation process; and mixed effects on behaviour and patient satisfaction, with small positive effects on health status. Risk of bias varied across studies. Studies that focused only on provider behaviour frequently did not collect data on patient outcomes, limiting the conclusions that can be drawn about the relative effect of intervention focus on providers compared with providers and patients.

AUTHORS' CONCLUSIONS

Interventions to promote patient-centred care within clinical consultations are effective across studies in transferring patient-centred skills to providers. However the effects on patient satisfaction, health behaviour and health status are mixed. There is some indication that complex interventions directed at providers and patients that include condition-specific educational materials have beneficial effects on health behaviour and health status, outcomes not assessed in studies reviewed previously. The latter conclusion is tentative at this time and requires more data. The heterogeneity of outcomes, and the use of single item consultation and health behaviour measures limit the strength of the conclusions.

Students' perception of the prognosis for a single-surface amalgam restoration

The aim of the study was to assess the self-rated skill, clinical judgement and competency of final-year pre-doctoral dental students. The study was also designed to determine whether the students' estimated durability of their single-surface restorations was determined by their confidence. Students were given a questionnaire and completed it anonymously. One hundred and ninety-six forms were returned from the 238 students (82% return rate). Complete analysis was possible on 169 questionnaires. The students' self-assessed skill and judgement were a significant factor in predicting how long they would tell a patient that their restoration would last. Those ranking themselves below average were more likely to state that their amalgam restorations would have a shorter life (odds increased by 2.82 for those ranking themselves lower than most others). There was a significant association between the students' self-assessed skill and judgement and their confidence in dealing with both the initial management of patients' medical emergencies (ρ = 0.244, P = 0.001, n = 191) and their general confidence in dealing with medical emergencies (ρ = 0.187, P = 0.01, n = 188). Given a conversational scenario with a patient, the treatment prognosis given by final-year pre-doctoral students for a single-surface amalgam restoration was dependent on their self-assessed competence, skill and judgement.

Improving clinical assessment: evaluating students' ability to identify and apply clinical criteria

AIM

There is ongoing concern by health educators over the inability of professionals to accurately self-assess their clinical behaviour and standards, resulting in doubts over a key expectation of effective self-regulation in the health professions. Participation by students in the assessment process has been shown to increase the understanding of assessment criteria in written assessment tasks. How this might transfer to the clinical setting is the focus of this study. This paper is part of an ongoing investigation of the impact on learning of a series of activities that provides students with opportunities to discuss and apply criteria and standards associated with self-assessment in clinical dentistry. Our aim was to evaluate whether participation in these assessment activities improved the ability of first-year dental students to recognise behaviours demonstrated by 'peers' in videos of clinical scenarios and to relate these to the assessment criteria.

MATERIALS AND METHODS

A series of three workshops in conjunction with weekly clinical assessment activities in Semesters 1 and 2 were use to support first-year students' learning of clinical assessment criteria. The design of the workshops was based on the principles of social constructivist theories of learning and the concept of tacit knowledge. Accordingly workshop activities were planned around videos that were specifically constructed to illustrate procedures and behaviours typical of those observed by staff and tutors in the first year of the dental course at The University of Adelaide, Australia. First-year students viewed the videos prior to and after the workshops and recorded observed behaviours that related to the assessment criteria that were used in their clinical practice course. Student learning outcomes were assessed 10-14 weeks after the initial workshop and again up to 42 weeks later. To check whether learning resulted from repeated viewing of the videos without formal discussion, a reference group of third-year students who did not attend the workshops also viewed the videos two times, separated by 12 weeks, and recorded observations in the same way.

RESULTS

There was no consistent evidence that repeat viewing of the videos in isolation resulted in improved recognition of 'peer' behaviours by third-year dental students. Results for the first-year students indicated that the workshops and clinical assessment activities had a significantly positive effect on the ability of students to identify 'peer' behaviours related to the criteria used for clinical assessment. In particular, students' recognition in others of knowledge and professional behaviours improved significantly. This improvement was retained over the year and students were able to recognise these behaviours in other scenarios relevant to their year level.

CONCLUSIONS

This early exposure to the process of clinical assessment, coupled with ongoing self-assessment and tutor feedback throughout first year, improved the ability of first-year students to identify and apply some key assessment criteria to observed 'peer' behaviour, and this ability was retained over time.

The introduction and perception of an OSCE with an element of self- and peer-assessment

The purpose of the present study was to encourage reflective dental students by performing an educational Objective Structured Clinical Examination (OSCE) with an element of self- and peer-assessment. An interdisciplinary OSCE comprising cariology, endodontics and microbiology was set up for all third-year students. A blueprint secured representation of the skills to be tested, i.e. knowledge, interdisciplinary knowledge, communication, clinical reasoning and practical procedures. At each station positive and constructive feedback was given to the students based on predefined criteria. Further, the students received written marks after completion of the OSCE. At one station the feedback and marks were replaced by self- and peer-assessment performed by the students in groups after the OSCE. Afterwards, the 68 students and 8 teachers participating in the OSCE answered a questionnaire on their opinion and perception of the examination. The results showed good correlation between the marks given and the students' perception of task difficulty. Generally, there were no systematic variations in the marks given during the week or by individual assessors at the same station, except for one, as well as agreement with marks of the ordinary clinical assessment. The marks given during self- and peer-assessment differed widely, indicating a need for training in this aspect. The questionnaires revealed a very positive perception of the OSCE from both students and teachers. Thus, the majority found the examination relevant and of educational benefit, capable of improving the learning of the students and useful for assessment purposes. Also, the self- and peer-assessment was found useful by the students. In conclusion, this interdisciplinary OSCE stressing constructive feedback to the students was perceived very positively by students and teachers and recognised for its beneficial possibilities in education and assessment.

The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10

BACKGROUND

Health professionals are increasingly expected to identify their own learning needs through a process of ongoing self-assessment. Self-assessment is integral to many appraisal systems and has been espoused as an important aspect of personal professional behaviour by several regulatory bodies and those developing learning outcomes for clinical students. In this review we considered the evidence base on self-assessment since Gordon's comprehensive review in 1991. The overall aim of the present review was to determine whether specific methods of self-assessment lead to change in learning behaviour or clinical practice. Specific objectives sought evidence for effectiveness of self-assessment interventions to: a. improve perception of learning needs; b. promote change in learning activity; c. improve clinical practice; d. improve patient outcomes.

METHODS

The methods for this review were developed and refined in a series of workshops with input from an expert BEME systematic reviewer, and followed BEME guidance. Databases searched included Medline, CINAHL, BNI, Embase, EBM Collection, Psychlit, HMIC, ERIC, BEI, TIMElit and RDRB. Papers addressing self-assessment in all professions in clinical practice were included, covering under- and post-graduate education, with outcomes classified using an extended version of Kirkpatrick's hierarchy. In addition we included outcome measures of accuracy of self-assessment and factors influencing it. 5,798 papers were retrieved, 194 abstracts were identified as potentially relevant and 103 papers coded independently by pairs using an electronic coding sheet adapted from the standard BEME form. This total included 12 papers identified by hand-searches, grey literature, cited references and updating. The identification of a further 12 papers during the writing-up process resulted in a total of 77 papers for final analysis.

RESULTS

Although a large number of papers resulted from our original search only a small proportion of these were of sufficient academic rigour to be included in our review. The majority of these focused on judging the accuracy of self-assessment against some external standard, which raises questions about assumed reliability and validity of this 'gold standard'. No papers were found which satisfied Kirkpatrick's hierarchy above level 2, or which looked at the association between self-assessment and resulting changes in either clinical practice or patient outcomes. Thus our review was largely unable to answer the specific research questions and provide a solid evidence base for effective self-assessment. Despite this, there was some evidence that the accuracy of self-assessment can be enhanced by feedback, particularly video and verbal, and by providing explicit assessment criteria and benchmarking guidance. There was also some evidence that the least competent are also the least able to self-assess accurately. Our review recommends that these areas merit future systematic research to further our understanding of self-assessment.

CONCLUSION

As in other BEME reviews, the methodological issues emerging from this review indicate a need for more rigorous study designs. In addition, it highlights the need to consider the potential for combining qualitative and quantitative data to further our understanding of how self-assessment can improve learning and professional clinical practice.

Evaluation of student and faculty perceptions of the PBL curriculum at two dental schools from a student perspective: a cross-sectional survey

Problem-based learning (PBL) research has primarily highlighted issues related to medical education and less evaluation has been reported from the field of dental education. Furthermore, literature reports tend to focus mainly on PBL from a pedagogic and curriculum constructional view and up to this date, studies from a student perspective are lacking. The aim of this study was to approach the evaluation of student and staff perceptions of PBL curricula from a student perspective at two separate schools: the Faculty of Odontology at the University of Malmö, Sweden and the dental school of the University of Southern California, School of Dentistry (USCSD), Los Angeles, CA, USA. The study was initiated and conducted by two of the authors, at the time senior students at the Faculty of Odontology in Malmö, Sweden. The study was comprised of a literature search, a 2 week field trip to USCSD, USA, survey distribution to students and faculty in both schools, analysis of the data and a written report for oral defence. The results from the survey were intended to provide feedback on student and faculty perceptions regarding the PBL curriculum. The results indicate a general student and faculty satisfaction with the PBL curriculum. Perhaps, surprisingly their perceptions did not differ significantly despite differences in geography, culture and implementation of PBL pedagogy.

Implementing an Objective Structured Clinical Examination (OSCE) in dental education: effects on students' learning strategies

OBJECTIVES

To investigate the effect of an Objective Structured Clinical Examination (OSCE) on dental students' learning strategies and competence to manage periodontal diseases in patients. The implemented OSCE was expected to be superior to the existing Written Exam in fostering the acquisition of clinical competencies in terms of study strategies that are more oriented towards clinical practice, longer study time, greater clinical proficiency, and more realistic self-assessment.

MATERIAL AND METHODS

After a clinical course in periodontology, 72 third year dental students were assessed summatively, either using a Written Exam or an OSCE (P-OSCE). The students were informed beforehand about the assessment formats. The self-assessed clinical competence, study time and strategies (i.e. practice with a manikin, peers and patient case) were evaluated by means of a questionnaire. After a comprehensive dental care course, all 72 students were assessed by an overall end-of-year OSCE, in which three periodontal stations were included 'measuring pockets', 'educating patients' and 'tracing an X-ray with bone-loss'. The competence of the previous Written Exam group and the P-OSCE group was investigated by determining the mean scores and pass-fail scores of three periodontal test-stations as well as the total score of the end-of-year OSCE. The degree of realistic self-assessment was studied by correlating the self-assessed competencies as evaluated by means of the questionnaire with the total score of the end-of-year OSCE.

RESULTS

Self-assessed clinical competence, study time and study strategies showed no differences between the P-OSCE and the Written Exam-group. The clinical competence determined in the test-station 'measuring pockets' in the end of year overall OSCE was higher for the P-OSCE group (P = 0.05) when compared with the Written Exam group; the two groups performed equally well in the test station 'educating patients', whereas the performance in 'tracing an X-ray with bone-loss' was better in the Written Exam group. This group also had a higher total score in the end-of-year OSCE (P = 0.05). The degree of realistic self-assessment was higher in the P-OSCE group than in the Written Exam group: in the P-OSCE group the self-assessed clinical competencies correlated significantly with the total score of the overall end-of-year OSCE (P < or = 0.05).

CONCLUSIONS

No effects of the implementation of an OSCE in undergraduate periodontal education were observed in study strategies, but the implementation of an OSCE in undergraduate periodontal education appears to stimulate learning, resulting in greater achievement of specific clinical competence and a greater level of realistic self-assessment.

The interactive examination: assessing students' self-assessment ability

BACKGROUND

The ability to self-assess one's competence is a crucial skill for all health professionals. The interactive examination is an assessment model aiming to evaluate not only students' clinical skills and competence, but also their ability to self-assess their proficiency.

METHODS

The methodology utilised students' own self-assessment, an answer to a written essay question and a group discussion. Students' self-assessment was matched to the judgement of their instructors. As a final task, students compared their own essay to one written by an "expert". The differences pointed by students in their comparison documents and the accompanying arguments were analysed and categorised. Students received individual feedback on their performance and learning needs. The model was tested on 1 cohort of undergraduate dental students (year 2001, n = 52) in their third semester of studies, replacing an older form of examination in the discipline of clinical periodontology.

RESULTS

Students' acceptance of the methodology was very positive. Students tended to overestimate their competence in relation to the judgement of their instructors in diagnostic skills, but not in skills relevant to treatment. No gender differences were observed, although females performed better than males in the examination. Three categories of differences were observed in the students' comparison documents. The accompanying arguments may reveal students' understanding and methods of prioritising.

CONCLUSIONS

Students tended to overestimate their competence in diagnostic rather than treatment skills. The interactive examination appeared to be a convenient tool for providing deeper insight into students' ability to prioritise, self-assess and steer their own learning.

Interventions for providers to promote a patient-centred approach in clinical consultations

BACKGROUND

Communication problems in health care may arise as a result of health care providers focusing on diseases and their management, rather than people, their lives and their health problems. Patient-centred approaches to care are increasingly advocated by consumers and clinicians and incorporated into training for health care providers. The effects of interventions that aim to promote patient-centred care need to be evaluated.

OBJECTIVES

To assess the effects of interventions for health care providers that aim to promote patient-centred approaches in clinical consultations.

SEARCH STRATEGY

We searched Medline (1966 - Dec 1999); Health Star (1975 - Dec 1999); PsycLit (1887- Dec 1999); Cinahl (1982 - Dec 1999); Embase (1985-Dec 1999) and the bibliographies of studies assessed for inclusion.

SELECTION CRITERIA

Randomised controlled trials, controlled clinical trials, controlled before and after studies, and interrupted time series studies of interventions for health care providers that promote patient-centred care in clinical consultations. Patient-centred care was defined as a philosophy of care that encourages: (a) shared control of the consultation, decisions about interventions or management of the health problems with the patient, and/or (b) a focus in the consultation on the patient as a whole person who has individual preferences situated within social contexts (in contrast to a focus in the consultation on a body part or disease). The participants were health care providers, including those in training.

DATA COLLECTION AND ANALYSIS

Two reviewers independently extracted data onto a standard form and assessed study quality for each study. We extracted all outcomes other than health care providers' knowledge, attitudes and intentions.

MAIN RESULTS

17 studies met the inclusion criteria. These studies display considerable heterogeneity in terms of the interventions themselves, the health problems or health concerns on which the interventions focused, the comparisons made and the outcomes assessed. All included studies used training for health care providers as an element of the intervention. Ten studies evaluated training for providers only, while the remaining studies utilised multi-faceted interventions where training for providers was one of several components. The health care providers were mainly primary care physicians (general practitioners or family doctors) practising in community or hospital outpatient settings. In two studies, the providers also included nurses. There is fairly strong evidence to suggest that some interventions to promote patient-centred care in clinical consultations may lead to significant increases in the patient centredness of consultation processes. 12 of the 14 studies that assessed consultation processes showed improvements in some of these outcomes. There is also some evidence that training health care providers in patient-centred approaches may impact positively on patient satisfaction with care. Of the eleven studies that assessed patient satisfaction, six demonstrated significant differences in favour of the intervention group on one or more measures. Few studies examined health care behaviour or health status outcomes.

REVIEWER'S CONCLUSIONS

Interventions to promote patient-centred care within clinical consultations may significantly increase the patient centredness of care. However, there is limited and mixed evidence on the effects of such interventions on patient health care behaviours or health status; or on whether these interventions might be applicable to providers other than physicians. Further research is needed in these areas.

Improving student learning in root canal treatment using self-assessment

AIM

This paper reports the first attempts to implement a new course in endodontology for undergraduate dental students based upon independent, reflective learning and self-assessment.

METHODOLOGY

The responses were analysed of two successive cohorts of third-year students to laboratory courses in (i) the restoration of teeth with crowns and (ii) endodontology and root canal treatment. The major changes introduced to the endodontology course were a substantial reduction in the number of formal lectures, the introduction of a series of structured tutorials, time for private study and the use of self-assessment in practical classes.

RESULTS

The evidence indicates that the new approach increased student confidence in practical skills, although the time available for operative practice was actually reduced; there was a perceived increase in problem-solving ability; and that students felt encouraged to pursue greater understanding. Reports from tutors suggested that students' self-assessments became more accurate as the courses proceeded. Seven experimental learning points were derived from the study. These included the importance of careful planning, well-designed assessment procedures and the creation of a climate of trust and openness.

CONCLUSIONS

The task of developing fully the students' skills of accurate self-assessment remains a challenge.

The Malmö model: a problem-based learning curriculum in undergraduate dental education

A new undergraduate dental curriculum was introduced in Malmö in 1990 which is based on problem-based learning (PBL). The principles of PBL are strongly influenced by evidence from cognitive psychology and they form the educational strategy throughout the whole curriculum. Two further essential principles underpin the curriculum: a holistic attitude to patient care, and the promotion of oral health. Basic sciences and clinical dentistry are integrated within a structure based on oral conditions prevalent in the community. Students are encouraged to build their new knowledge, understanding and skills into the context of what they have already learned. This approach is facilitated by the presentation of conceptual models, one of which, The Oral Ecosystem is described in detail. The educational programme also gives students opportunities to learn in the clinical context from an early stage, and we endeavour to promote a scientific attitude from the very beginning of the programme. This paper describes the curriculum at three organisational levels (whole curriculum, single course, individual week).