AMEE Guide Supplements: Peer-assisted learning: A planning and implementation framework. Guide supplement 30.6--practical application

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study

INTRODUCTION

Progress tests provide students feedback on their level of proficiency over the course of their medical studies. Peer-assisted learning and competency-based education have become increasingly important in medical education. Although progress tests have been proven to be useful as a longitudinal feedback instrument, there are currently no progress tests that have been created in cooperation with students or that focus on competency in medical education. In this study, we investigated the extent to which students can be included in the development of a progress test and demonstrated that aspects of knowledge related to competency can be represented on a competency-based progress test.

METHODS

A two-dimensional blueprint for 144 multiple-choice questions (MCQs) covering groups of medical subjects and groups of competency areas was generated by three expert groups for developing the competency-based progress test. A total of 31 students from seven medical schools in Germany actively participated in this exercise. After completing an intensive and comprehensive training programme, the students generated and reviewed the test questions for the competency-based progress test using a separate platform of the ItemManagementSystem (IMS). This test was administered as a formative test to 469 students in a pilot study in November 2013 at eight medical schools in Germany. The scores were analysed for the overall test and differentiated according to the subject groups and competency areas.

RESULTS

A pool of more than 200 MCQs was compiled by the students for pilot use, of which 118 student-generated MCQs were used in the progress test. University instructors supplemented this pool with 26 MCQs, which primarily addressed the area of scientific skills. The post-review showed that student-generated MCQs were of high quality with regard to test statistic criteria and content. Overall, the progress test displayed a very high reliability. When the academic years were compared, the progress test mapped out over the course of study not only by the overall test but also in terms of the subject groups and competency areas.

OUTLOOK

Further development in cooperation with students will be continued. Focus will be on compiling additional questions and test formats that can represent competency at a higher skill level, such as key feature questions, situational judgement test questions and OSCE. In addition, the feedback formats will be successively expanded. The intention is also to offer the formative competency-based progress test online.

International medical students--a survey of perceived challenges and established support services at medical faculties

INTRODUCTION

Medical students with a non-German background face several challenges during their studies. Besides support given by foreign student offices further specific projects for international students have been developed and are offered by medical faculties. However, so far, neither a systematic survey of the faculties' perceived problems nor of the offered support exists.

METHOD

All study deaneries of medical faculties in Germany were contacted between April and October 2013 and asked for their participation in a telephone interview. Interview partners were asked about 1.) The percentage of non-German students at the medical faculty; 2.) The perceived difficulties and problems of foreign students; 3.) The offers for non-German students; and 4.) The specification of further possibilities of support. Given information was noted, frequencies counted and results interpreted via frequency analysis.

RESULTS

Only 39% of the medical faculties could give detailed information about the percentage of non-German students. They reported an average share of 3.9% of students with an EU migration background and 4.9% with a non-EU background. Most frequently cited offers are student conducted tutorials, language courses and tandem-programs. The most frequently reported problem by far is the perceived lack of language skills of foreign students at the beginning of their studies. Suggested solutions are mainly the development of tutorials and the improvement of German medical terminology.

DISCUSSION

Offers of support provided by medical faculties for foreign students vary greatly in type and extent. Support offered is seen to be insufficient in coping with the needs of the international students in many cases. Hence, a better coverage of international students as well as further research efforts to the specific needs and the effectiveness of applied interventions seem to be essential.

Peer-assisted learning--beyond teaching: How can medical students contribute to the undergraduate curriculum?

INTRODUCTION

Peer-assisted learning (PAL) has become increasingly popular over recent years with many medical schools now formally incorporating peer-teaching programs into the curriculum. PAL has a sound evidence base with benefit to both peer-teacher and peer-learner. Aside from in teaching delivery, empowering students to develop education in its broadest sense has been much less extensively documented.

CASE STUDIES

Five case studies with supportive evaluation evidence illustrate the success of a broad range of peer-led projects in the undergraduate medical curriculum, particularly where these have been embedded into formal teaching practices.

DISCUSSION

These case studies identify five domains of teaching and support of learning where PAL works well: teaching and learning, resource development, peer-assessment, education research and evaluation and mentoring and support. Each case offers ways of engaging students in each domain.

CONCLUSIONS

Medical students can contribute significantly to the design and delivery of the undergraduate medical program above and beyond the simple delivery of peer-assisted "teaching". In particular, they are in a prime position to develop resources and conduct research and evaluation within the program. Their participation in all stages enables them to feel involved in course development and education of their peers and ultimately leads to an increase in student satisfaction.

Introducing a curricular program culminating in a certificate for training peer tutors in medical education

AIM

Student tutorials are now firmly anchored in medical education. However, to date there have only been isolated efforts to establish structured teacher training for peer tutors in medicine. To close this gap, a centralized tutor training program for students, culminating in an academic certificate, was implemented at Heidelberg University Medical School. The program also counts within the scope of the post-graduate Baden-Württemberg Certificate in Academic Teaching (Baden-Württemberg Zertifikat für Hochschuldidaktik).

METHOD

Based on a needs assessment, a modular program comprised of four modules and a total of 200 curricular units was developed in cooperation with the Department for Key Competencies and Higher Education at Heidelberg University and implemented during the 2010 summer semester. This program covers not only topic-specific training sessions, but also independent teaching and an integrated evaluation of the learning process that is communicated to the graduates in the form of structured feedback. In addition, to evaluate the overall concept, semi-structured interviews (N=18) were conducted with the program graduates.

RESULTS

To date, 495 tutors have been trained in the basic module on teaching medicine, which is rated with a mean overall grade of 1.7 (SW: 0.6) and has served as Module I of the program since 2010. A total of 17% (N=83) of these tutors have gone on to enroll in the subsequent training modules of the program; 27 of them (m=12, f=15) have already successfully completed them. Based on qualitative analyses, it is evident that the training program certificate and its applicability toward the advanced teacher training for university instructors pose a major incentive for the graduates. For successful program realization, central coordination, extensive coordination within the medical school, and the evaluation of the attained skills have proven to be of particular importance.

CONCLUSION

The training program contributes sustainably to both quality assurance and professionalism, as well as to solving the issue of resources in medical education. The introduction and continued development of similar programs is desirable.

Student-teacher education programme (STEP) by step: transforming medical students into competent, confident teachers

BACKGROUND

While most medical schools have students teach other students, few offer formal education in teaching skills, and fewer provide teaching theory together with experiential teaching practice. Furthermore, curriculum evaluation of teaching education is lacking.

AIM

This study aimed to examine effects of a novel didactic teaching curriculum for students embedded in a practical teaching experience.

METHODS

A longitudinal 12-week curriculum with complementary didactic and practical components for final-year students learning how to teach was developed, implemented and evaluated using a multi-level evaluation based on the Kirkpatrick approach with qualitative and quantitative methods.

RESULTS

Thirteen student-teachers acquired measureable knowledge, skills and attitudes necessary for teaching excellence. Confidence in teaching increased (p < 0.001), particularly in four key areas: oral feedback, written feedback, mentoring, and the difficult learner. Student-teachers demonstrated teaching competence as determined by self-assessment, student feedback, and faculty observation. Top teachers impacted their first-year students' performance in patient interviewing as measured by Objective Structured Clinical Examination (OSCE).

CONCLUSIONS

Reinforcing educational theory with practical teaching experience under direct faculty supervision promotes teaching competency for graduating medical students. The intertwined didactic plus practical model can be applied to various teaching contexts to fulfil the mandate that medical schools train graduates in core teaching knowledge, skills and attitudes in preparation for their future roles as clinical teachers.