Progress testing in undergraduate dental education: the Peninsula experience and future opportunities

Case-based learning in undergraduate orthodontic education: A cross sectional study

Introduction

Student centric learning approaches have been reported to be effective in introducing higher order cognitive skills required by the health professionals. However, learners' perceptions must be constructively aligned with new learning interventions to achieve a positive impact on their learning. The aim of this study was to explore the learning experiences of undergraduate dental students with case-based learning in orthodontics.

Methods

A case-based learning model was introduced on orthodontic diagnosis and treatment planning for final year students on a Bachelor of Dentistry programme toward the end of their academic year. A survey was conducted to explore the perceptions and experiences of the participants. The research instrument was based on a previously validated questionnaire and included information on demographics and consisted of 12 items aimed at evaluating the benefits and challenges of cased based learning.

Results

All 67 students in the final-year cohort participated in study, yielding a response rate of 100 percent. Participants across the board perceived CBL to be an effective strategy to learn the subject content and helpful in improving the students' skills in orthodontic diagnosis, treatment planning and team-working. CBL did not pose any significant challenges or barriers to student learning.

Conclusion

Participants reported high acceptance of CBL in orthodontic teaching and learning and a positive impact on their educational experiences. CBL was perceived to be an appropriate strategy to enhance the diagnostic, treatment planning and team-working skills of dental students.

Ten tips for effective use and quality assurance of multiple-choice questions in knowledge-based assessments

Multiple-choice questions (MCQs) are the most popular type of items used in knowledge-based assessments in undergraduate and postgraduate healthcare education. MCQs allow assessment of candidates' knowledge on a broad range of knowledge-based learning outcomes in a single assessment. Single-best-answer (SBA) MCQs are the most versatile and commonly used format. Although writing MCQs may seem straight-forward, producing decent-quality MCQs is challenging and warrants a range of quality checks before an item is deemed suitable for inclusion in an assessment. Like all assessments, MCQ-based examinations must be aligned with the learning outcomes and learning opportunities provided to the students. This paper provides evidence-based guidance on the effective use of MCQs in student assessments, not only to make decisions regarding student progression but also to build an academic environment that promotes assessment as a driver for learning. Practical tips are provided to the readers to produce authentic MCQ items, along with appropriate pre- and post-assessment reviews, the use of standard setting and psychometric evaluation of assessments based on MCQs. Institutions need to develop an academic culture that fosters transparency, openness, equality and inclusivity. In line with contemporary educational principles, teamwork amongst teaching faculty, administrators and students is essential to establish effective learning and assessment practices.

ChatGPT-A double-edged sword for healthcare education? Implications for assessments of dental students

INTRODUCTION

Open-source generative artificial intelligence (AI) applications are fast-transforming access to information and allow students to prepare assignments and offer quite accurate responses to a wide range of exam questions which are routinely used in assessments of students across the board including undergraduate dental students. This study aims to evaluate the performance of Chat Generative Pre-trained Transformer (ChatGPT), a generative AI-based application, on a wide range of assessments used in contemporary healthcare education and discusses the implications for undergraduate dental education.

MATERIALS AND METHODS

This was an exploratory study investigating the accuracy of ChatGPT to attempt a range of recognised assessments in healthcare education curricula. A total of 50 independent items encompassing 50 different learning outcomes (n = 10 per item) were developed by the research team. These included 10 separate items based on each of the five commonly used question formats including multiple-choice questions (MCQs); short-answer questions (SAQs); short essay questions (SEQs); single true/false questions; and fill in the blanks items. Chat GPT was used to attempt each of these 50 questions. In addition, ChatGPT was used to generate reflective reports based on multisource feedback; research methodology; and critical appraisal of the literature.

RESULTS

ChatGPT application provided accurate responses to majority of knowledge-based assessments based on MCQs, SAQs, SEQs, true/false and fill in the blanks items. However, it was only able to answer text-based questions and did not allow processing of questions based on images. Responses generated to written assignments were also satisfactory apart from those for critical appraisal of literature. Word count was the key limitation observed in outputs generated by the free version of ChatGPT.

CONCLUSION

Notwithstanding their current limitations, generative AI-based applications have the potential to revolutionise virtual learning. Instead of treating it as a threat, healthcare educators need to adapt teaching and assessments in medical and dental education to the benefits of the learners while mitigating against dishonest use of AI-based technology.

Progress testing: An educational perspective exploring the rationale for progress testing and its introduction into a Diagnostic Radiography curriculum

INTRODUCTION

In March 2020, the first diagnostic radiography degree apprenticeship programme in England was launched at the authors' institution. As part of the programme development and design, the programme development team explored and then implemented progress testing into a strand of the programme. The objective of this educational perspective is to scrutinise the literature around the use of progress testing in higher education programmes, namely medicine, to explain how and why this decision was reached.

METHODS

The initial search strategy was developed using the electronic databases CINHAL Complete and SCOPUS. Key words included 'progress test' and 'medicine' or 'health' or 'education' or 'higher education'. Eliminating articles that were not relevant, and also identifying and adding additional articles by key authors and experts resulted in thirty-three key articles being considered for review.

RESULTS

The thirty-three articles were a mixture of review articles, empirical research, case studies and conference presentations. Five key themes were identified which are discussed in this article; the evolution of progress testing; advantages of progress testing, disadvantages of progress testing, developing a test framework and academic progression and student feedback.

DISCUSSION

Progress testing is now well established in pre-registration medical programmes globally. The advantages of progress testing and the use of frequent look rapid remediation appear to be undisputed. Key disadvantages with progress testing were identified as it being an administrative heavy assessment process as well as a perceived bias towards male students undertaking this type of assessment.

CONCLUSION

Now this assessment practice is established within medicine, it seems reasonable to explore its use in other areas of healthcare, such as radiography.

Building a student learning-focused assessment and grading system in dental school: One school's experience

PURPOSE/OBJECTIVES

As health professions education moves toward competency-based education, there has been increased focus on the structure of assessment systems that support student competency development and learning. This has been buoyed by a growing body of research supporting assessment for learning processes to promote student growth and learning rather than relying on assessment systems primarily to measure performance. This paper presents the rationale and evidence for moving to an assessment for learning system and the results of a quasi-experimental interrupted time series study using data from 2015 to 2022 to evaluate the impacts of these changes.

METHODS

Columbia University College of Dental Medicine faculty voted to implement assessment for learning system changes beginning in 2017 with the graduating class of 2021. These changes included moving from using a grading system for didactic courses with Honors, Pass, and Fail as available grades to a grading system with only Pass and Fail as available grades, as well as creating synthesis and assessment weeks, weekly problem sets, post-exam review sessions, exam remediation opportunities, and formative progress exams throughout the curriculum. The revised assessment and grading system changes were communicated to residency program directors, and programmatic competency data about student performance across the curriculum were shared with programs in Dean's Letters.

RESULTS

Once assessment system changes were implemented, it was found that student exam failure rates were lower, course exam scores were the same or higher, and performance on board exams improved compared to the national average. Students reported positive perceptions with regard to well-being and learning climate that they associated with the adoption of Pass/Fail grading. Match outcomes, including student satisfaction and program director ratings, have remained consistently positive.

CONCLUSION

As dental educators, our goal is to nurture students to become life-long learners. Adopting a grading structure that is Pass/Fail and an assessment system that fosters learning allows students to shape learning practices that favor long-term retention and application of information, also enhancing the learning environment and student well-being. These system changes may also facilitate the inclusion and support of students whose backgrounds are underrepresented in dentistry.

Dental education in primary care: 14 years of Peninsula Dental School

Peninsula Dental School, established in 2006, was the UK's first new dental school in 40 years. It had the freedom to develop a completely new dental education curriculum planned on pedagogic thinking, designed to equip the dental care professionals of the twenty-first century. This was based on three distinct pillars: professionalism (developing a student's trust in their own autonomy); dental skills of the highest order (not just technical skills but also communication skills); and social engagement. As such, a truly innovative approach to dental education was created that has strong roots in evidence.

This paper describes the University of Plymouth Peninsula Dental School's achievements against these initial objectives under the following areas: training in primary care; a novel spiralling integrated curriculum and assessments; facilities reaching out to deliver patient care; bringing meaningful patient contact to students from the earliest months of their course; embedding community engagement within the curriculum; development of Peninsula Dental Social Enterprise; and team working, training a variety of dental care profession students side by side.

The University of Plymouth Peninsula Dental School, working with all its partners, has successfully pioneered and delivered significant changes in the field of education and continues to strive to further develop these and more for the future.

An insight into the development of a new UK dental school.

A focus on dental education based in primary care, reaching out to deliver patient care, embedding community engagement in a curriculum and team working of dentists and dental care professionals side by side.

An overview of the benefits of a social enterprise clinical placement model.

Written-Based Progress Testing: A Scoping Review

PURPOSE

Progress testing is an increasingly popular form of assessment in which a comprehensive test is administered to learners repeatedly over time. To inform potential users, this scoping review aimed to document barriers, facilitators, and potential outcomes of the use of written progress tests in higher education.

METHOD

The authors followed Arksey and O'Malley's scoping review methodology to identify and summarize the literature on progress testing. They searched 6 databases (Academic Search Complete, CINAHL, ERIC, Education Source, MEDLINE, and PsycINFO) on 2 occasions (May 22, 2018, and April 21, 2020) and included articles written in English or French and pertaining to written progress tests in higher education. Two authors screened articles for the inclusion criteria (90% agreement), then data extraction was performed by pairs of authors. Using a snowball approach, the authors also screened additional articles identified from the included reference lists. They completed a thematic analysis through an iterative process.

RESULTS

A total of 104 articles were included. The majority of progress tests used a multiple-choice and/or true-or-false question format (95, 91.3%) and were administered 4 times a year (38, 36.5%). The most documented source of validity evidence was internal consistency (38, 36.5%). Four major themes were identified: (1) barriers and challenges to the implementation of progress testing (e.g., need for additional resources); (2) established collaboration as a facilitator of progress testing implementation; (3) factors that increase the acceptance of progress testing (e.g., formative use); and (4) outcomes and consequences of progress test use (e.g., progress testing contributes to an increase in knowledge).

CONCLUSIONS

Progress testing appears to have a positive impact on learning, and there is significant validity evidence to support its use. Although progress testing is resource- and time-intensive, strategies such as collaboration with other institutions may facilitate its use.

Transfer of basic science knowledge in a problem-based learning curriculum

AIM

The aim of this study was to evaluate transfer of basic science knowledge for clinical application in our BDS programme by exploring the correlations between student performance in integrated dental science (IDS) examinations and applied dental knowledge (ADK) tests.

METHODS

Numeric test scores were drawn from summative IDS and ADK assessments undertaken by BDS students spanning six academic years (2013-14 to 2018-19) for two cohorts (2013 and 2014). The data included a total of 13 test scores for each cohort with four IDS tests, taken in Years 1 and 2, and nine ADK tests taken in Years 3, 4 and 5.

RESULTS

The sample included 120 students across both cohorts with 65 females (54%) and 55 males (46%). The correlation coefficients between the successive tests and the combinations of IDS and ADK scores were positive, all being greater than 0.48, and all being significantly correlated (P < .001). Regarding correlation between standardised averages across all IDS tests and all ADK tests, performance remained significantly correlated: (2013 cohort: r (53) = .667, P < .001; 2014 cohort: r (50) = .700, P < .001).

CONCLUSIONS

The results of this study show that the students' knowledge of basic sciences correlates with their applied dental knowledge and may offer a predictive value. These findings may be attributed to a PBL curriculum and student-led learning at our school.

A core curriculum in the biological and biomedical sciences for dentistry

INTRODUCTION

The biomedical sciences (BMS) are a central part of the dental curriculum that underpins teaching and clinical practice in all areas of dentistry. Although some specialist groups have proposed curricula in their particular topic areas, there is currently no overarching view of what should be included in a BMS curriculum for undergraduate dental programmes. To address this, the Association for Dental Education in Europe (ADEE) convened a Special Interest Group (SIG) with representatives from across Europe to develop a consensus BMS curriculum for dental programmes.

CURRICULUM

This paper summarises the outcome of the deliberations of this SIG and details a consensus view from the SIG of what a BMS curriculum should include.

CONCLUSIONS

Given the broad nature of BMS applied to dentistry, this curriculum framework is advisory and seeks to provide programme planners with an indicative list of topics which can be mapped to specific learning objectives within their own curricula. As dentistry becomes increasingly specialised, these will change, or some elements of the undergraduate curriculum may move to the post-graduate setting. So, this document should be seen as a beginning and it will need regular review as BMS curricula in dentistry evolve.

Introducing Summative Progress Testing in Radiology Residency: Little Change in Residents' Test Results After Transitioning from Formative Progress Testing

INTRODUCTION

Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results.

METHODS

We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015-2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents' relative test scores and percentage of residents that reached pass standards.

RESULTS

Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4-5, nor residents in the formative condition in year 0.5-2. Across the 7 tests, relative test scores in postgraduate year 1-3 of the summative resident group and year 3.5-4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups.

DISCUSSION

Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.

How does dyslexia impact on the educational experiences of healthcare students? A qualitative study

AIMS

To explore the impact of dyslexia on the educational experiences of undergraduate students in medicine, dentistry, dental therapy and biomedical sciences.

METHODS

It was a qualitative study based on semi-structured interviews. The study was conducted at the Faculty of Medicine and Dentistry at a University in the South West region of the United Kingdom. Purposive sampling was used, and the participants included undergraduate students from the schools of medicine, dentistry and biomedical sciences.

RESULTS

The sample consisted of fifteen undergraduate students including five from medicine; four from dentistry; three from dental therapy; and three from biomedical sciences. All students had a formal diagnosis of Dyslexia. The students shared their views and experiences regarding disclosure, transition into the university, learning environments, assessments and challenges after graduation.

CONCLUSIONS

This is the first study to explore the impact of dyslexia on the educational experiences of healthcare students from multiple programmes in a variety of educational settings. The findings show that the students were confident regarding disclosure of their dyslexia and had a formal diagnosis of dyslexia established during their school years. Although the participants experienced typical academic difficulties associated with dyslexia, problem-based learning (PBL) was perceived to be enjoyable and less stressful than traditional lectures and no specific challenges were reported in acquisition of clinical, communication and team-working skills.

To accept or decline academic remediation: What difference does it make?

Introduction: Academic remediation offered after failure in a knowledge-based progress-test assessment is voluntary and involves student-centered individualized support that helps students to learn most effectively for themselves. This paper explores whether accepting or declining the offer of academic remediation given to struggling students impacts their outcomes both short-term and longitudinally. Method: Data was collated from 2015-16, 2016-17, and 2017-18 and included all students offered academic remediation in the third, fourth, and fifth years of a five-year Dentistry program. Z-scores for each stage and test were calculated and centered on a triggering point; the point at which the offer of remediation was made. These students' average performance post-trigger test and longitudinal performance were analyzed. Results: While performance for both groups significantly improved for the immediate post-trigger test after academic remediation, those that accepted remediation sustained longitudinal improvements across subsequent tests compared to those that declined remediation. Discussion: Through the academic remediation support process students appear to increase their mastery of "learning to learn" and are able to implement sustainable effective learning strategies to carry with them throughout their program. Conclusion: Students who accept academic remediation maintain a more successful academic profile compared to those that do not take advantage of this.

Impact of Progress testing on the learning experiences of students in medicine, dentistry and dental therapy

AIMS

To investigate the impact of progress testing on the learning experiences of undergraduate students in three programs namely, medicine, dentistry and dental therapy.

METHODS

Participants were invited to respond to an online questionnaire to share their perceptions and experiences of progress testing. Responses were recorded anonymously, but data on their program, year of study, age, gender, and ethnicity were also captured on a voluntary basis.

RESULTS

A total of 167 participants completed the questionnaire yielding a response rate of 27.2% (n = 167). These included 96 BMBS students (27.4%), 56 BDS students (24.7%), and 15 BScDTH students (39.5%). A 3 -Program (BMBS, BDS, BScDTH) by 8-Topic (A-H) mixed analysis of variance (ANOVA) was conducted on the questionnaire responses. This revealed statistically significant main effects of Program and Topic, as well as a statistically significant interaction between the two (i.e. the pattern of topic differences was different across programs).

CONCLUSIONS

Undergraduate students in medicine, dentistry, and dental therapy and hygiene regarded PT as a useful assessment to support their learning needs. However, in comparison to students in dentistry and dental therapy and hygiene, the perceptions of medical students were less positive in several aspects of PT.

First reported implementation of a German-language progress test in an undergraduate dental curriculum: A prospective study

INTRODUCTION

Progress testing is a special form of longitudinal and feedback-oriented assessment. Even though well established in human medical curricula, this is not the case in dental education. The aim was the prospective development and implementation of the first reported German-language Dental Progress Test (DPT) for the undergraduate dental curriculum at the Medical University of Graz, Austria.

MATERIAL AND METHODS

Participation in DPT was compulsory for all dental students in terms 7-12 (years 4-6). Three tests, each consisting of 100 items out of a pool of 375, were administered within 3 consecutive terms in 2016 and 2017. Rasch analyses were used to evaluate the questionnaire and identify misfitting items.

RESULTS

In the item responses, 59.7% were "correct," 27.0% were "false" and 13.3% were answered with "don't know," with similar results at all 3 time points. The assumption of parallel ICC was met (T1: χ²  = 51.071, df = 74, P = .981; T2: χ²  = 57.044, df = 67, P = .802; T3: χ²  = 58.443, df = 72, P = .876) and item difficulties for the thematic fields were similarly distributed across the latent dimensions.

CONCLUSION

The newly introduced DPT is appropriate for testing dental students and is well balanced for the tested target group.

A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach

Background: Progress testing is a longitudinal assessment that aims at tracking students' development of knowledge. This approach is used in many medical schools internationally. Although progress tests are longitudinal in nature, and their focus and use of developmental aspects is a key advantage, individual students' learning trajectories themselves play, to date, only a minor role in the use of the information obtained through progress testing. Methods: We investigate in how far between-person differences in initial levels of performance and within-person rate of growth can be regarded as distinct components of students' development and analyze the extent to which these two components are related to performances on national licensing examinations using a latent growth curve model. Results: Both, higher initial levels of performances and steepness of growth are positively related to long-term outcomes as measured by performance on national licensing examinations. We interpret these findings as evidence for progress tests' suitability to monitor students' growth of knowledge across the course of medical training. Conclusions: This study indicates that individual development as obtained by formative progress tests is related to performance in high-stakes assessments. Future studies may put more focus on the use of between-persons differences in growth of knowledge.

Use of Progress Testing in a UK Dental Therapy and Hygiene Educational Program

Progress testing provides a longitudinal assessment of the development and sustainability of students' knowledge at regular intervals over the duration of an educational program. Comparisons of performance on successive tests are used to monitor growth in knowledge. The aim of this study was to evaluate the effects of progress testing in an undergraduate program for dental therapy and hygiene (DTH) students in the United Kingdom as the main tool for academic assessment. Data were collected for progress tests of all 38 DTH students from 2015 to 2017. Each test consisted of 100 single best answer multiple-choice items with accompanying vignette. The students chose their answer from five options. A score of 1 was awarded for a correct answer, -0.25 for an incorrect answer, and 0 for don't know (DK). Three cohorts of DTH students were included in the study, and seven progress tests were conducted over a period of three years. Analysis of performance showed growth in knowledge across successive years, with the largest increase in knowledge in the transition from Year 1 to Year 2 and concomitant reduction in incorrect and DK responses. This was a pioneering study to report the establishment and use of progress testing among undergraduate DTH students. Notwithstanding the challenges involved, the study found merit in further exploring the use of progress testing for students in the DTH program.

Impact of early clinical exposure on the learning experience of undergraduate dental students

AIMS

The aim of this research was to evaluate the impact of early clinical exposure on the learning experiences of undergraduate dental students.

METHODS

This study was based on mixed methods. The first phase involved administering a purposely designed questionnaire consisting of 16 items, grouped into three subscales. The second phase of the research was conducted using qualitative semi-structured interviews to explore the perceptions and experiences of stakeholders regarding early clinical exposure.

RESULTS

In total, 134 undergraduate dental students and eight clinical supervisors responded to the questionnaire and reported positive perceptions regarding the learning experiences, professional relationship and learning environment. Qualitative interviews were conducted with 12 participants, and early clinical exposure was perceived to be useful in providing a context to theoretical learning and development of interpersonal skills. Curriculum overload and further need for consolidation were highlighted as the main challenges.

CONCLUSIONS

This study provided insights into the clinical training model in an undergraduate dental programme and highlights the benefits and challenges of early clinical exposure in the study population. The study served as a vehicle for engagement with a range of stakeholders using a mixed methods approach to inform further development of the training model.

Comparison of graduate-entry and direct school leaver student performance on an applied dental knowledge test

AIMS

To compare the academic performance of graduate-entry and direct school leavers in an undergraduate dental programme.

METHODS

This study examined the results of students in applied dental knowledge (ADK) progress tests conducted during two academic years. A mixed model analysis of variance (ANOVA) was conducted to compare the performance of graduate-entry and direct school leavers. ADK was treated as a repeated measures variable, and the outcome variable of interest was percentage score on the ADK.

RESULTS

The results show statistically significant main effects for ADK [F (1,113) = 61.58, P < 0.001, η²_p = 0.35], Cohort [F (1,113) = 88.57, P < 0.001, η²_p = 0.44] and Entry [F (1,113) = 11.31, P = 0.001, η²_p = 0.09]. That is, students do better on each subsequent test (main effect of ADK), students in later years of the programme perform better than those in earlier years (main effect of cohort), and graduate-entry students outperform direct school leavers.

CONCLUSIONS

This is the first study to explore the differences in the academic performance of graduate-entry and direct school leavers in an undergraduate dental programme. The results show that the academic performance of graduate students was better than the direct school leavers in years 2 and 3. Further research is required to compare the performance of students longitudinally across the entire duration of undergraduate dental programmes and evaluate whether this difference persists throughout.

Academic performance of undergraduate dental students with learning disabilities

Aims To compare the academic performance of undergraduate dental students with known learning disabilities (LDs) to their peers.Methods This study analysed the results of students in applied dental knowledge (ADK) progress tests across four cohorts of dental students. A mixed model analysis of variance (ANOVA) was conducted to compare the performance of students with known disability to their peers. ADK test sitting was treated as a repeated measures variable, and the outcome variable of interest was percentage score on the ADK.Results Students' performance data on five ADK test sittings (ADK15, ADK16, ADK17, ADK18, and ADK19) by disability showed a significant main effect of test but no significant effect of disability or any interaction between disability and test.Conclusions This is the first study that explores the academic performance of dental students with a diagnosis of disability. The findings give reassurance to all stakeholders that, within the study population, students with LDs are not disadvantaged in knowledge-based assessments, demonstrating compliance with the legal obligations. Further research is required to explore how generalisable these findings are, as well as assess academic, clinical, and behavioural attributes of students with learning disabilities.