Using video-cases to assess student reflection: development and validation of an instrument

Reflect to interact - fostering medical students' communication through reflection-focused e-learning

BACKGROUND

Previous research indicates that reflection can foster medical communication competence. However, best practices for embedding reflection in online medical education are missing. This study examined how reflection processes can be promoted and embedded in an e-learning course on physician-patient communication to foster learning.

METHODS

We investigated three differently designed e-learning conditions featuring different proportions of reflection triggers and compared their effects on students' reflections. We had 114 medical students in their first clinical year complete one of the variants: video modelling (VM, n = 39), video reflection (VR, n = 39), or a variant merging both approaches (VMR, n = 36). Each student wrote a total of nine reflections based on the same guiding questions at three time points embedded in the course. The students' levels of reflection were measured using an adapted version of the REFLECT rubric (scale 0-18).

RESULTS

Students of all course variants achieved good levels of reflection beyond the descriptive level at all three time points, with no significant differences between the variants. The mean reflection scores at the end of the course were M = 14.22 for VM (SD = 2.23), M = 13.56 for VR (SD = 2.48), and M = 13.24 for VMR (SD = 2.21). Students who completed VM showed significantly improved levels of reflection over the course, whereas we found no statistically significant development for those in VR or VMR. The reflection scores correlated significantly positively with each other, as did the text lengths of the written reflections. Reflection scores also correlated significantly positively with text length.

CONCLUSIONS

Our study offers a teaching strategy that can foster good levels of reflection, as demonstrated in the three e-learning variants. The developed reflection prompts can be easily embedded in various e-learning environments and enable reflections of good quality, even in settings with limited available teaching time.

Training in medical communication competence through video-based e-learning: How effective are video modeling and video reflection?

OBJECTIVE

The present study investigated the efficacy of the didactic approaches of video modeling (VM, best-practice examples), video reflection (VR, problem-based approach), and the combination of both (VMR) in fostering medical communication competence in a video-based digital learning environment.

METHODS

N = 126 third-year medical students who participated in the pre-post study were assigned to either the intervention groups (VM, VR, and VMR) or the wait-list control group. The efficacy of the three approaches was assessed by means of a situational judgment test (SJT) of medical communication competence. To investigate the differences between the wait-list control and the intervention groups (H1), between the single-mode and combined conditions (H2), and between VM and VR (H3), we applied planned contrast analyses.

RESULTS

The planned contrasts showed that the VR condition significantly improved learning outcomes in comparison to the VM condition (H3). The decreased mean scores of the VM condition offset the increased mean scores of VR, and thus no significant differences could be found in H1 and H2.

CONCLUSIONS

Our study provides promising evidence that VR fosters medical communication competence more effectively than VM.

PRACTICAL IMPLICATIONS

Medical students' learning in video-based digital environments can be facilitated by the use of interactive VR.

Enhancing Medical Students' Reflective Capacity: Utilizing Reflective Practice Questionnaire as an Action Research Diagnostic Tool

Introduction Reflection is the intentional evaluation of one's beliefs or understanding in consideration of the available evidence. Reflection has been noted to enhance profound learning and critical thinking and is an essential foundation of experiential learning. The Reflective Practice Questionnaire (RPQ) is a valid and reliable tool for assessing reflective capacity among medical students. It assesses not only reflective capacity but also other psychological constructs relevant to reflective practice, such as job satisfaction, confidence while interacting with patients, stress during patient interaction, desire for improvement, and feelings of uncertainty. The use of RPQ as a diagnostic tool for improving reflective capacity among medical students is scarcely available in the literature. Thus, the present study aimed to utilize the RPQ for identifying discrete action points for training and future improvement in reflective capacity. Materials and methods The present cross-sectional descriptive study was conducted among 300 medical students of a medical college. To identify the clusters or meaningful subgroups within the study population, cluster analysis was done. Inter-correlation between subscales of RPQ was performed by forming Pearson's correlation matrix to understand the dynamics between various subscales of RPQ. Results Five groups were identified in the cluster analysis: typical (25.6%), reflective (27%), non-reflective (16.7%), unconfident (14%), and overconfident (16.7%). This sets the template for specific action points for each group identified above. Reflective capacity was positively correlated with Desire for Improvement (DfI), General Confidence (CG), Communication Confidence (CC), and Job Satisfaction (JS). It was also positively correlated with negative outcomes such as Uncertainty (Unc) and Stress when Interacting with Patients (SiC). Conclusion RPQ can be used as a diagnostic tool in terms of reflective capacity for action research.

Grading reflective essays: the construct validity and reliability of a newly developed Tool- GRE-9

BACKGROUND

The main objective of this study is to assess the construct validity and reliability of the Grading Reflective Essays-9 (GRE-9).

METHODS

This study took place in a major tertiary academic medical center in Beirut, Lebanon. 104 reflective essays written by years 1-3 residents in the department of Family Medicine at the American University of Beirut Medical Center (AUBMC) were graded by 2 trained raters who independently scored the essays using GRE-9. GRE-9 scores were then correlated with scores on communication skills OSCE stations and in-training examinations to investigate, respectively, convergent and divergent validity. One of the 2 raters scored the essays twice one month apart to assess the reliability of the GRE-9 using intra rater reliability and internal consistency.

RESULTS

There was a weak, non-significant correlation between GRE-9 score and In training examination (ITE) score (r = - .213, p = .395). There was a moderate, non-significant correlation between GRE-9 scores and the Objective structured clinical examination (OSCE) communication station scores (r = - .412 p = .162). The correlation coefficient between trails 1 and 2 was significant (r = .832, p = .000). Intra class correlation coefficient (ICC) analysis demonstrated almost perfect intra-rater agreement (0.819; 95% CI: 0.741-0.875) of the test ratings over time.

CONCLUSIONS

GRE-9, is a short, concise, easy-to-use reliable grading tool for reflective essays that has demonstrated moderate to substantial intra-rater reliability and evidence of divergent validity. The study found non-significant correlations between reflective writing scores OSCE communication scores demonstrating a lack of relationship between reflective writing and this measure of performance.

Correlates, motivating factors, and barriers of engaging in regular self-reflection among public health students in higher education-A mixed methods approach

Despite the wide implementation of self-reflection in higher education, the body of literature has predominantly focused on students from the clinical health sciences rather than public health. The objective of this study was to evaluate the correlates as well as to explore the motivating factors and barriers of engaging in regular self-reflection among public health students in higher education. We used a mixed methods approach (explanatory sequential design), comprising a cross-sectional survey (quantitative phase) followed by in-depth interviews (qualitative phase). We evaluated the association between reflection frequency as well as the perceptions and facilitators in reflection using the modified Breslow-Cox proportional hazards regression model. Prevalence ratio (PR) was generated. Thematic data analysis was carried out to analyse the qualitative data. Quantitative findings revealed being a regular reflector was positively associated with being more motivated to learn when one applied self-reflection (adjusted PR 1.60, 95% CI 1.17-2.20), the perception of being more prepared for a public health career in the future (adjusted PR 1.28, 95% CI 1.02-1.60), as well as being given enough opportunities to carry out self-reflection in the public health modules (adjusted PR 1.24, 95% CI 1.05-1.45). Qualitative findings revealed most students started their self-reflection mainly due to extrinsic factors such as institutional support, social support, teacher influence and environmental influence. Of these, the most prominent was teacher influence, indicating that they are key agents in promoting self-reflection. Students expressed that it would be important to cultivate intrinsic motivation to sustain their practice of self-reflection along the learning journey such as for the development of career-related professional skills. Other than intrinsic motivation, environmental influences were also important to promote continual reflection among students such as the availability of ample opportunities. Prominent barriers to address included external student factors such as the imbalanced power relationship between teacher and student, and internal student factors such as the perception that self-reflection was too cumbersome and time consuming.

Assessment tools for reflection in healthcare learners: A scoping review

Reflection has been integrated in many healthcare educations programs to achieve deeper learning and improve professional practice. A variety of evaluation tools are used to assess reflection, but few guides are available to inform educators in their choice of a relevant evaluation tool. The aim of this paper is to identify all existing evaluation tools published along with their strengths and weaknesses. A review strategy retrieved tools available in Medline, PsychInfo, CINALH and Eric databases. The procedures outlined by Munn and colleagues were used to synthetize the information. Additionally, the reflection dimensions assessed in each tool (when sufficient information was available) were analyzed deductively, using thematic analysis according to the Killion and Todnem framework. Subthemes were identified inductively. Forty-five papers were identified, reporting on 34 different tools. The tools were based on a variety of theoretical models. Some had evidence of adequate validity and fidelity. Eleven components of reflection were identified across tools. No tool encompassed all components, but most tools included between three and five components. Context surrounding evaluation should be carefully considered when choosing an evaluation tool for reflection. There is a need for further research to validate the psychometric properties of reflection evaluation tools.

“I know it when I see it”: employing reflective practice for assessment and feedback of reflective writing in a makerspace classroom

Purpose

In reflective writing, students are encouraged to examine their own setbacks and progress. With a shortage of guidance in how to provide feedback to students on this type of writing, teachers are often left to figure it out on the job. The central hypothesis in this paper is that the lens of reflective practice can help focus teacher efforts and ultimately improve both feedback and instruction. The purpose of this paper is not to produce a universal prescription for assessing reflective writing but rather a protocol for teacher reflective practice that can apply to challenging grading and feedback-giving situations.

Design/methodology/approach

Student assessment is a chance for teachers to learn about their students’ abilities and challenges and to provide feedback for improvement. Assessment and grading sessions can also become opportunities for teachers to examine their own instructional and assessment practices. This self-examination process, a cornerstone of reflective practice (Schön, 1984), is challenging, but it may be especially valuable when guidelines for feedback and assessment are hard to come by. Such may be said to be the case in student-centered learning environments such as school Fablabs and makerspaces, where stated goals commonly include cultivating learner self-regulation and resilience. These hard-to-measure constructs are typically assessed through analysis of student reflective journals. This in-depth case study uses mixed-methods to examine how a semester-long intervention affected the grading, feedback and instructional practices of a teacher in a hands-on design classroom. The intervention involved 10 grade-aloud sessions using a computer-based rubric tool (Gradescope) and a culminating card-sorting task. The lens of reflective practice was applied to understanding the teacher’s development of their own reflective capabilities.

Findings

During the intervention, the participating teacher grappled with grading and feedback-giving dilemmas which led to clarifications of assessment objectives; changes to instruction; and improved feedback-giving practices, many of which persisted after the intervention. The teacher perceived the intervention as adding both rigor and productive “soul-searching” to their professional practice. Lasting changes in feedback behaviors included a comprehensive rubric and an increase in the frequency, specificity and depth of feedback given to student written work.

Originality/value

Significant prior efforts have been directed separately at the use of reflective practice for teachers, in general, and on the feedback and grading of student process journals. This work combines these lines of inquiry in the reflective classroom assessment protocol, a novel on-the-job professional development opportunity that fosters reflective practice in times of assessment to improve instructional and feedback practices.

Asynchronous Environment Assessment: A Pertinent Option for Medical and Allied Health Profession Education During the COVID-19 Pandemic

The emergence and global spread of COVID-19 has disrupted the traditional mechanisms of education throughout the world. Institutions of learning were caught unprepared and this jeopardised the face-to-face method of curriculum delivery and assessment. Teaching institutions have shifted to an asynchronous mode whilst attempting to preserve the principles of integrity, equity, inclusiveness, fairness, ethics, and safety. A framework of assessment that enables educators to utilise appropriate methods in measuring a student’s progress is crucial for the success of teaching and learning, especially in health education that demands high standards and comprises consistent scientific content. Within such a framework, this paper aims to present a narrative review of the currently utilised methods of assessment in health education and recommend selected modalities that could be administered in an asynchronous mode during the COVID-19 pandemic. Assessment methods such as open-ended short answer questions, problem-based questions, oral exams, and recorded objective structured clinical exams (OSCE) would be appropriate for use in an asynchronous environment to assess the knowledge and competence of health professional students during COVID-19. Fairness and integrity can be ensured by using technological tools such as video and audio recording surveillance.

Grading reflective essays: the reliability of a newly developed tool- GRE-9

BACKGROUND

The main objective of this study is the development of a short reliable easy-to-use assessment tool in the aim of providing feedback to the reflective writings of medical students and residents.

METHODS

This study took place in a major tertiary academic medical center in Beirut, Lebanon. Seventy-seven reflective essays written by 18 residents in the department of Family Medicine at the American University of Beirut Medical Center (AUBMC) were graded by 3 raters using the newly developed scale to assess the scale reliability. Following a comprehensive search and analysis of the literature, and based on their experience in reflective grading, the authors developed a concise 9-item scale to grade reflective essays through repeated cycles of development and analysis as well as the determination of the inter-rater reliability (IRR) using intra-class correlation coefficients (ICC) and Krippendorff's Alpha.

RESULTS

The inter-rater reliability of the new scale ranges from moderate to substantial with ICC of 0.78, 95% CI 0.64-0.86, p < 0.01 and Krippendorff's Alpha was 0.49.

CONCLUSIONS

The newly developed scale, GRE-9, is a short, concise, easy-to-use reliable grading tool for reflective essays that has demonstrated moderate to substantial inter-rater reliability. This will enable raters to objectively grade reflective essays and provide informed feedback to residents and students.

How can communicative competence instruction in medical studies be improved through digitalization?

The teaching of communicative competence plays an increasingly important role in medical education. In addition to traditional teaching formats, such as role-plays with simulated patients, technology-based approaches become more important in medical education. Teaching materials are increasingly augmented by videos of simulated doctor-patient conversations. This combination allows the content of teaching materials to be demonstrated with video or for videos to create a basis for reflection activities. In addition, conversation videos can illustrate different qualities of clinical communication and serve as illustrative material for describing particular issues in more detail. In addition to teaching clinical communicative competence, the assessment of this competence also plays an important role in medical educational research. So far, this has mainly been conducted through direct observation using checklists or rating scales. Relatively little is known about the assessment of communicative competence using standardized online-based tests. Situational Judgement Tests (SJTs) offer a promising approach in this respect. The BMBF-funded (BMBF = Bundesministerium für Bildung und Forschung - Federal Ministry of Education and Research) joint project voLeA (Entwicklung videobasierter Lehr- und Assessmentmodule zur Gesprächskompetenz im Medizinstudium = Development of video-based teaching and assessment modules for communicative competence in medical studies) addresses these two issues. Specifically, the project is engaged in developing e-learning modules to promote communicative competence and an assessment of this competence using an SJT. The present paper focuses on the benefits of technology-based learning and assessment units for clinical communicative competence in medical studies, using the voLeA project as an example.

Instruments to measure the ability to self-reflect: A systematic review of evidence from workplace and educational settings including health care

INTRODUCTION

Self-reflection has become recognised as a core skill in dental education, although the ability to self-reflect is valued and measured within several professions. This review appraises the evidence for instruments available to measure the self-reflective ability of adults studying or working within any setting, not just health care.

MATERIALS AND METHODS

A systematic review was conducted of 20 electronic databases (including Medline, ERIC, CINAHL and Business Source Complete) from 1975 to 2017, supplemented by citation searches. Data were extracted from each study and the studies graded against quality indicators by at least two independent reviewers, using a coding sheet. Reviewers completed a utility analysis of the assessment instruments described within included studies, appraising their reported reliability, validity, educational impact, acceptability and cost.

RESULTS

A total of 131 studies met the inclusion criteria. Eighteen were judged to provide higher quality evidence for the review and three broad types of instrument were identified, namely: rubrics (or scoring guides), self-reported scales and observed behaviour.

CONCLUSIONS

Three types of instrument were identified to assess the ability to self-reflect. It was not possible to recommend a single most effective instrument due to under reporting of the criteria necessary for a full utility analysis of each. The use of more than one instrument may therefore be appropriate dependent on the acceptability to the faculty, assessor, student and cost. Future research should report on the utility of assessment instruments and provide guidance on what constitutes thresholds of acceptable or unacceptable ability to self-reflect, and how this should be managed.

Design and Validation of an Instrument to Evaluate the Learning Acquired by Nursing Students from a Brief Tobacco Intervention (BTI-St©)

The aim of this study was to design and validate an instrument, based on the WHO 5As+5Rs model, to test the acquisition by nursing students of a brief tobacco intervention (BTI) learning. A validation design of an instrument following the criterion referenced tests model using videos of simulated BTIs in the primary care setting was carried out. The study included 11 experts in smoking prevention/care and 260 second-year nursing students. The study was in two stages: (1) selection and recording of clinical simulations (settings), and (2) test construction. Content was validated by applying the Delphi consensus technique and calculating the Content Validity Ratio (CVR) and Content Validity Index (CVI). A pilot test was conducted for item analysis. Reliability was evaluated as internal consistency (Kuder-Richardson [KR-20]) and test-retest temporal stability (intraclass correlation coefficient [ICC]). Three simulation settings were recorded. An instrument (BTI-St^®) was developed with 23 items for dichotomous (yes/no) response. CVR was >70% for all items, KR-20 of 0.81-0.88, and ICC between 0.68 and0.73 (p < 0.0001). The BTI-St^® is a robust and reliable instrument that is easily and rapidly applied. It follows the WHO 5As+5Rs model and offers objective criterion-referenced evaluation of BTI learning in nursing students.

Can a reflective rubric be applied consistently with raters globally? A study across three countries

INTRODUCTION

Reflection is a powerful tool for assisting students to develop the skills to make better informed decisions. As a pharmacy competency standard, reliable and fair assessment strategies are required to measure reflective skills and support students in developing their reflective capacity. The aim of this research was to explore whether we can extend the applicability of a previously tested rubric to a range of educational settings, to account for diversity of pharmacy educators and curricula internationally.

METHODS

Four raters from three countries applied a reflective rubric to assess a sample (n = 43) of reflective accounts, representing 41% of a cohort of 105 second-year undergraduate pharmacy students. The interrater reliability (IRR) was measured utilizing the intra-class correlation coefficient (ICC), using a two-way random effects model with absolute agreement, to determine the level of agreement between the raters' absolute scores. Generalizability Theory analysis was used to estimate generalizability of raters and stages.

RESULTS

Results indicated agreement of raters for (i) each of the seven stages of reflection and (ii) overall score for the reflective account, with moderate to substantial agreement (ICC = 0.55-0.69, p < 0.001); and high agreement for all raters for the overall score (ICC = 0.96, p < 0.001), respectively. The G-Study estimated a relative error coefficient of 0.78.

CONCLUSION

This additional analysis further confirms the reliability and applicability of the rubric to a range of rater academic backgrounds.

Using videos in a smartphone app logbook to aid obstetricians and gynaecologists with reflection following critical events

Introduction

Reflection is essential for continuous professional development. Many opportunities to reflect during training in obstetrics and gynaecology are not utilised. A Smartphone App (Healthcare Supervision Logbook), allows doctors training in obstetrics and gynaecology to view videos to aid reflection. The App incorporates a comprehensive logbook of practical skills. A self-facilitated reflective process, prompted by offering the viewing of a video when logging an event such as a shoulder dystocia, has been integrated into the logbook. The objective of this study was to evaluate how this function aids reflection.

Methods

Forty doctors training in obstetrics and gynaecology undertook a shoulder dystocia drill as part of a mock examination. After completing the drill, participants scored their performance on a proforma, before watching a video of simulated standard management for shoulder dystocia on a Smartphone. Participants then re-scored themselves before completing a survey. This assessed their agreement with three statements on a five-point Likert scale.

Results

50% of participants marked themselves lower after watching the video. 100% agreed or strongly agreed that watching the video helped them reflect on the shoulder dystocia drill. 80% agreed or strongly agreed that they would feel more prepared to deal with shoulder dystocia as a result of using the video to reflect. 93% agreed or strongly agreed that they would be willing to use a Smartphone App with videos to aid reflection following critical events. Student’s t test showed that viewing the standardised video clip on the App resulted in a significant down-marking (P<0.0005).

Conclusions

Many emergency scenarios in obstetrics and gynaecology unfold quickly and are rapidly resolved, doctors-in-training need to use opportunities for self-directed reflection. Using the video self-reflection function integrated into Healthcare Supervision Logbook Smartphone App could help to prompt this process, which could be utilised in other specialities and disciplines.

Applications of the reflective practice questionnaire in medical education

BACKGROUND

We sought to determine whether the Reflective Practice Questionnaire (RPQ) is a reliable measure of reflective capacity and related characteristics in medical students. We also planned to learn how the RPQ could be used in medical education.

METHODS

The RPQ is a 40 item self-report questionnaire that includes a multi-faceted approach to measuring reflective capacity. It also includes sub-scales on several other theoretically relevant constructs such as desire for improvement, confidence, stress, and job satisfaction. The reliabilities of reflective capacity and other sub-scales were determined by calculating their Cronbach alpha reliability values. In the present study, the RPQ was answered by 98 graduating fourth-year medical students from an American University, and these RPQ scores were compared with general public and mental health practitioner samples from a prior study using ANOVA and Bonferroni adjusted comparisons.

RESULTS

Medical students reported a higher reflective capacity than the general public sample, but students were statistically indistinguishable from the mental health practitioner sample. For medical students, reflective capacity was associated with features of confidence, stress, and desire for improvement. Job satisfaction was positively associated with confidence in communication with patients, and negatively associated with stress when interacting with patients. A cluster analysis revealed that around 19% of the medical students exhibited a relatively high level of anxiety interacting with patients, 23% were less engaged, 5% were dissatisfied, and 7% expressed a level of over-confidence in their knowledge and skills that was concerning.

CONCLUSIONS

The RPQ is a reliable measure of reflective capacity (Chronbach's alpha value = 0.84) and related characteristics (Cronbach's alpha values from 0.75 to 0.83) in medical students. The RPQ can be used as part of pre-post evaluations of medical education initiatives, to complement student self-reflection activities in the curriculum, and to identify students who might benefit from targeted intervention.

Evaluating Knowledge and Assessment-Centered Reflective-Based Learning Approaches

This paper addresses the development of knowledge and assessment-centered learning approaches within a reflective learning framework in a first-year physics class in a university faculty. The quality of students’ reflections was scored using a Self-reporting Reflective Learning Appraisal Questionnaire at the end of each learning approach. The results showed the differences between the approaches based on reflections on the learning control through self-knowledge, by connecting experience and knowledge, as well as through self-reflection and self-regulation. Assessment-centered activities fundamentally help students identify aspects of their attitudes towards, as well as regulate, their sustainability learning education.

Coaching the Coach: A Program for Development of Faculty Portfolio Coaches

PROBLEM

Faculty coaching is recognized as an essential element for effective use of portfolios in undergraduate medical education, yet best practices for training these coaches are uncertain.

INTERVENTION

New portfolio coaches participated in a multifaceted training program that included orienting modules, a 7.5-hr training workshop featuring analysis of reflective writing, an Observed Structured Teaching Exercise (OSTE), and subsequent longitudinal coaches' meetings for timely task training. Four desired coaching skills were emphasized in the initial training: creating a safe environment, explicitly using performance data, asking questions that elicit reflection, and guiding the student to develop future goals and plans. We collected and analyzed several outcomes: (a) coaches' self-assessment at key intervals, (b) open-ended written responses to three coaching vignettes, (c) video recordings of the OSTE, and (d) subsequent student evaluation of the coach. In an attempt to capture learning from the workshop, both the responses to written vignettes and the video-recorded encounters were coded for presence or absence of the four desired skills.

CONTEXT

Our portfolio and coaching program was instituted as part of a major undergraduate medical education reform. A new cohort of 25 coaches is enrolled with each matriculating student class, and each coach is assigned to work individually with 8-10 students, forming a coaching relationship that continues over 4 years. Coaches are compensated at 5% full-time equivalent.

OUTCOME

On coach self-assessment, the majority of coaches reported significant improvement in their perceived ability to assess a student's level of reflection, enhance reflection, use performance data, and guide a student to develop goals and plans. After two semesters, coach perception of improved abilities persisted. Students rated coaches as excellent (82%), reporting that coaches created safe environments (99%), promoted insight (92%), and aided in goal setting (97%). Written responses to vignettes before the OSTE found that several coaches omitted desired behaviors; however, posttraining responses showed no discernable pattern of learning. Coding of the OSTE, in contrast, documented that all coaches demonstrated all four of the desired skills.

LESSONS LEARNED

Although coaches reported learning related to key skills, learning was not apparent when responses to written vignettes were examined. In contrast, skills were demonstrated in the OSTE, perhaps due to the added structured tasks as well as anticipation of feedback. In conclusion, this portfolio coach training program achieved its desired aim of providing students with portfolio coaches who demonstrated the desired skills, as reported by both coaches and students.

A Retrospective Study on Students' and Teachers' Perceptions of the Reflective Ability Clinical Assessment

Objective. To evaluate student and teacher perceptions of the utility of the Reflective Ability Clinical Assessment (RACA) in an undergraduate pharmacy curriculum at an Australian university. Methods. A mixed-method study comprising the administration of a 7-item student survey on a 6-point Likert-type scale and a 45-minute focus group/phone interview with teachers. Results. Student (n=199) and teaching staff respondents (n=3) provided their perceptions of the implementation of the new educational tool. Student responses showed significant positive correlations between self-directed learning, counseling skills, relevance to future practice, and performance in an oral examination. Seven key themes emerged from the teacher interviews. Conclusion. The study revealed both students and teachers perceive the RACA as an effective educational tool that may enhance skill development for future clinical practice.

The Effect of Reflective Activities on Reflective Thinking Ability in an Undergraduate Pharmacy Curriculum

Objective. To determine the effectiveness of integrating reflective practice activities into a second-year undergraduate pharmacy curriculum and their impact on reflective thinking ability. Design. A cross-over design with repeated measures was employed. Newly developed reflective modules based on real hospital and community pharmacy cases were integrated into the second-year pharmacy practice curriculum. A novel strategy, the Reflective Ability Clinical Assessment (RACA), was introduced to enhance self- and peer reflection. Assessment. Student responses (n=214) to the adapted Kember et al(1) Reflective Thinking Questionnaire (RTQ) were compared before and after reflective activities were undertaken. Significant improvement in three indicators of reflective thinking was shown after students engaged in reflective activities. Conclusion. Integration of reflective activities into a pharmacy curriculum increased the reflective thinking capacity of students. Enhancing reflective thinking ability may help students make better informed decisions and clinical judgments, thus improving future practice.

A Model for Assessing Reflective Practices in Pharmacy Education

OBJECTIVE. To research the literature and examine assessment strategies used in health education that measure reflection levels and to identify assessment strategies for use in pharmacy education. METHODS. A simple systematic review using a 5-step approach was employed to locate peer-reviewed articles addressing assessment strategies in health education from the last 20 years. RESULTS. The literature search identified assessment strategies and rubrics used in health education for assessing levels of reflection. There is a significant gap in the literature regarding reflective rubric use in pharmacy education. CONCLUSION. Two assessment strategies to assess levels of reflection, including a reflective rubric tailored for pharmacy education, are proposed.

Refining a definition of reflection for the being as well as doing the work of a physician

BACKGROUND

Reflection is core to professional competency and supports the active, constructive process of professional identity formation.

AIMS

Medical educators thus grapple with operationalizing and effectively integrating reflection as a foundational construct within health care professions education and practice.

METHODS

Core elements of reflection including role of emotions and awareness of self, other and situation, do not appear within various working definitions of reflection.

RESULTS

This observation as well as noted recent shift in medical education toward emphasis on the "being" as well as "doing the work" of a physician led to the author's proposed refining of Sandars' reflection definition and expansion of Nguyen et al.'s reflection model.

CONCLUSIONS

A refined reflection definition is offered for a more inclusionary approach. A caveat regarding potential for expected reflective learning outcomes (given reflection as a process) is provided and the integral role of mentor-enhanced reflection is discussed. Reflection as a continuum is highlighted and exemplified within Wald et al.'s REFLECT rubric and Nguyen et al.'s reflection model.

Reclaiming a theoretical orientation to reflection in medical education research: a critical narrative review

CONTEXT

Reflection and reflective practice have become popular topics of scholarly dialogue in medical education. This popularity has given rise to checklists, portfolios and other tools to inspire and document reflection. We argue that some of the common ways in which reflection has been applied are influenced by broader discourses of assessment and evidence, and divorced from original theories of reflection and reflective practice.

METHODS

This paper was developed using a critical narrative approach. First we present two theoretical lenses provided by theories of reflection. Next we present a summary of relevant literature, indexed in PubMed from 2004 to 2014, relating to the application of reflection or reflective practice to undergraduate and postgraduate medical education. We categorise these articles broadly by trends and problematise the trends relative to the two theoretical lenses of reflection.

RESULTS

Two relevant theoretical orientations of reflection for medical education are: (i) reflection as epistemology of practice, and (ii) reflection as critical social inquiry. Three prevalent trends in the application of reflection to medical education are: (i) utilitarian applications of reflection; (ii) a focus on the self as the object of reflection, and (iii) reflection and assessment. These trends align with dominant epistemological positions in medicine, but not with those that underpin reflection.

CONCLUSIONS

We argue for continued theorising of and theoretically informed applications of reflection, drawing upon epistemologies of practice and critical reflection as critical social inquiry. These directions offer medical education research broad and deep potential in theories of reflection, particularly in relation to knowledge creation within uncertain and complex situations, and challenging of dominant discourses and structures. Future work could explore how dominant epistemological positions and discourses in medicine influence theories from other disciplines when these theories are deployed in medical education.

Medical students' assessment of pediatric patients - teaching and evaluation using video cases

BACKGROUND

We introduced video-based teaching in pediatrics. We evaluated the impact of a pediatric video program on student performance in assessing pediatric patients presented as video cases. The program consisted of a library of pediatric videos, and inclusion of these in the teaching and examination for pediatric medicine.

METHODS

Medical students on a pediatric clerkship at the University of Copenhagen assessed eight short pediatric video cases during autumn 2011 and spring 2012. Two independent observers evaluated a subset of records in a pilot study. A blind evaluation was made of the written records of 37 students before, and 58 students after, the introduction of the program using a Rubric score with four domains.

RESULTS

The intraobserver interclass correlation coefficient was 0.94 and the interobserver interclass correlation was 0.71(n=25). The students' mean total Rubric score in spring 2012 (7.0) was significantly higher (p<0.001, 95% CI 1.34-3.20) than autumn 2011 (4.7). Cohen's d was 1.1 (95% CI 0.6-1.7). Single domains scores increased significantly for general assessment (1.30 versus 0.57) (p<0.002, 95% CI 0.45-1.18), recognition of principal symptoms (1.38 versus 0.81) (p<0.008, 95% CI 0.22-0.91), appropriate diagnosis (2.28 versus 1.78) (p<0.002, 95% CI 0.16-0.82) and consistency between observed symptoms and diagnosis (1.94 versus 1.57) (p=0.0482, 95% CI 0.00-0.79).

CONCLUSIONS

Students improved in evaluating pediatric patients presented as video cases after the introduction of the program. The impact on real-life situations remains to be established.

A validation study of the psychometric properties of the Groningen Reflection Ability Scale

BACKGROUND

Reflection, the ability to examine critically one's own learning and functioning, is considered important for 'the good doctor'. The Groningen Reflection Ability Scale (GRAS) is an instrument measuring student reflection, which has not yet been validated beyond the original Dutch study. The aim of this study was to adapt GRAS for use in a Danish setting and to investigate the psychometric properties of GRAS-DK.

METHODS

We performed a cross-cultural adaptation of GRAS from Dutch to Danish. Next, we collected primary data online, performed a retest, analysed data descriptively, estimated measurement error, performed an exploratory and a confirmatory factor analysis to test the proposed three-factor structure.

RESULTS

361 (69%) of 523 invited students completed GRAS-DK. Their mean score was 88 (SD = 11.42; scale maximum 115). Scores were approximately normally distributed. Measurement error and test-retest score differences were acceptable, apart from a few extreme outliers. However, the confirmatory factor analysis did not replicate the original three-factor model and neither could a one-dimensional structure be confirmed.

CONCLUSIONS

GRAS is already in use, however we advise that use of GRAS-DK for effect measurements and group comparison awaits further review and validation studies. Our negative finding might be explained by a weak conceptualisation of personal reflection.

Does reflection have an effect upon case-solving abilities of undergraduate medical students?

BACKGROUND

Reflection on professional experience is increasingly accepted as a critical attribute for health care practice; however, evidence that it has a positive impact on performance remains scarce. This study investigated whether, after allowing for the effects of knowledge and consultation skills, reflection had an independent effect on students' ability to solve problem cases.

METHODS

Data was collected from 362 undergraduate medical students at Ghent University solving video cases and reflected on the experience of doing so. For knowledge and consultation skills results on a progress test and a course teaching consultation skills were used respectively. Stepwise multiple linear regression analysis was used to test the relationship between the quality of case-solving (dependent variable) and reflection skills, knowledge, and consultation skills (dependent variables).

RESULTS

Only students with data on all variables available (n = 270) were included for analysis. The model was significant (Anova F(3,269) = 11.00, p < 0.001, adjusted R square 0.10) with all variables significantly contributing.

CONCLUSION

Medical students' reflection had a small but significant effect on case-solving, which supports reflection as an attribute for performance. These findings suggest that it would be worthwhile testing the effect of reflection skills training on clinical competence.