Clerkship-based reflective writing: a rubric for feedback

Translation and psychometric evaluation of the reflective capacity scale in Iranian medical education

BACKGROUND

Examining the reflective capacity in medical students is a principal step for the development of effective educational strategies to improve it. Reflection scales available in Iran are inadequate due to the lack of focus on students' willingness and tendency to participate in reflection. This study aimed at translation and psychometric evaluation of "Reflective Capacity Scale" in Iranian medical education.

METHODS

This methodological research was completed in two parts: translation and psychometric evaluation. After getting permission from the main developer of the tool, the translation process was done based on Polit and Yang model in Persian language. Then, face validity and content validity of the tool were established using a qualitative method. Construct validity was surveyed with exploratory and confirmatory factor analysis via completing the questionnaire by 320 medical students, who were selected using convenience sampling. The reliability of the tool was also checked with two methods of internal consistency and stability. The gleaned data were analyzed with SPSS20 and AMOS.

RESULTS

"Reflective Capacity Scale" includes 16 items that were retained after cross-cultural translation. Face validity and content validity were acceptable. By performing exploratory factor analysis, four factors were identified that accounted for 63.79% of the total variance. In the confirmatory factor analysis, the values of the fit indices confirmed the appropriate fit of the model. The internal consistency reliability of the whole tool was equal to 0.83 and the intra-class correlation coefficient was equal to 0.98.

CONCLUSION

The translated and validated "Reflective Capacity Scale" provides a robust tool for assessing reflective capacity among Iranian medical students. Its validity and reliability underscore its potential for measuring the reflective capacity of medical students.

Reading the Self: Medical Students' Experience of Reflecting on Their Writing Over Time

PURPOSE

To investigate students' experience (over time) with meta-reflection writing exercises, called Signature Reflections. These exercises were used to strengthen reflective capacity, as part of a 4-year reflective writing portfolio curriculum that builds on a recognized strategy for reflection (narrative medicine) and employs longitudinal faculty-mentors.

METHOD

In 2018, the authors conducted 5 focus groups with 18 third-year students from the Columbia University Vagelos College of Physicians and Surgeons class of 2019 to examine students' experience with Signature Reflections. Using an iterative, thematic approach, they developed codes to reflect common patterns in the transcripts, distilled conceptually similar codes, and assembled the code categories into themes.

RESULTS

Three core themes (safe space, narrative experience, mirror of self) and 1 overarching theme (moving through time) were identified. Students frequently experienced relief at having a safe reflective space that promoted grappling with their fears or vulnerabilities and highlighted contextual factors (e.g., trusted faculty-mentors, protected time) that fostered a safe space for reflection and exploration. They often emphasized the value of tangible documentation of their medical school journey (narrative experience) and reported using Signature Reflections to examine their emerging identity (mirror of self). Overlapping with the core themes was a deep appreciation for the temporal perspective facilitated by the Signature Reflections (moving through time).

CONCLUSIONS

A longitudinal narrative medicine-based portfolio curriculum with pauses for meta-reflection allowed students, with faculty support, to observe their trajectory through medical school, explore fears and vulnerabilities, and narrate their own growth. Findings suggest that narrative medicine curricula should be required and sufficiently longitudinal to facilitate opportunities to practice the skill of writing for insight, foster relationships with faculty, and strengthen students' temporal perspectives of their development.

Writing exceptional (specific, student and criterion-focused) rubrics for nursing studies

Effective assessment of nurses in higher education is dependent on the interpretation and demonstration of learning outcomes by students and educators/academics. Rubrics can effectively communicate unit learning outcomes to students and may contribute towards the academic rigour of nursing courses, when assessment criteria are articulated clearly and consistently to both students and educators/academics. This paper aims to describe the different types and uses of rubrics, as well as outline the steps required to develop exceptional rubrics whilst using the literature as a basis for suggestions. Well written rubrics can facilitate consistency of marking across a student cohort, which may result in decreased student anxiety regarding assessment expectations, uniformity of assessment style and layout and may ensure that educators/academics across a teaching team are interpreting assessment criteria, consistently. Exceptional rubrics can empower students to success in academic assessments.

Grading reflective essays: the reliability of a newly developed tool- GRE-9

BACKGROUND

The main objective of this study is the development of a short reliable easy-to-use assessment tool in the aim of providing feedback to the reflective writings of medical students and residents.

METHODS

This study took place in a major tertiary academic medical center in Beirut, Lebanon. Seventy-seven reflective essays written by 18 residents in the department of Family Medicine at the American University of Beirut Medical Center (AUBMC) were graded by 3 raters using the newly developed scale to assess the scale reliability. Following a comprehensive search and analysis of the literature, and based on their experience in reflective grading, the authors developed a concise 9-item scale to grade reflective essays through repeated cycles of development and analysis as well as the determination of the inter-rater reliability (IRR) using intra-class correlation coefficients (ICC) and Krippendorff's Alpha.

RESULTS

The inter-rater reliability of the new scale ranges from moderate to substantial with ICC of 0.78, 95% CI 0.64-0.86, p < 0.01 and Krippendorff's Alpha was 0.49.

CONCLUSIONS

The newly developed scale, GRE-9, is a short, concise, easy-to-use reliable grading tool for reflective essays that has demonstrated moderate to substantial inter-rater reliability. This will enable raters to objectively grade reflective essays and provide informed feedback to residents and students.

Using text mining to analyze reflective essays from Japanese medical students after rural community placement

BACKGROUND

Following community clinical placements, medical students use reflective writing to discover the story of their journey to becoming medical professionals. However, because of assessor bias analyzing these writings qualitatively to generalize learner experiences may be problematic. This study uses a process-oriented text mining approach to better understand meanings of learner experiences by connecting key concepts in extended student reflective essays.

METHODS

Text mining quantitative analysis is used on self-evaluative essays (n = 47, unique word count range 43-575) by fifth-year students at a regional quota-system university in Japan that specializes in training general practitioners for underserved communities. First, six highly-occurring key words were identified: patient, systemic treatment, locale, hospital, care, and training. Then, standardized keyword frequency analysis robust to overall essay length and keyword volume used individual keywords as "nodes" to calculate per-keyword values for each essay. Finally, Principle Components Analysis and regression were used to analyze key word relationships.

RESULTS

Component loadings were strongest for the keyword area, indicating most shared variance. Multiply regressing three of the remaining keywords hospital, systemic treatment, and training yielded R² = 0.45, considered high for this exploratory study. In contrast, direct patient experience for students was difficult to generalize.

CONCLUSIONS

Impressions of the practicing area environment were strongest in students, and these impressions were influenced by hospital workplace, treatment provision, and training. Text mining can extract information from larger samples of student essays in an efficient and objective manner, as well as identify patterns between learning situations to create models of the learning experience. Possible implications for community-based clinical learning may be greater understanding of student experiences for on-site precepts benefitting their roles as mentors.

Does the Medium Matter? Evaluating the Depth of Reflective Writing by Medical Students on Social Media Compared to the Traditional Private Essay Using the REFLECT Rubric

Introduction

Social media is a novel medium to host reflective writing (RW) essays, yet its impact on depth of students’ reflection is unknown. Shifting reflection on to social platforms offers opportunities for students to engage with their community, yet may leave them feeling vulnerable and less willing to reflect deeply. Using sociomateriality as a conceptual framework, we aimed to compare the depth of reflection in RW samples submitted by medical students in a traditional private essay format to those posted on a secure social media platform.

Methods

Fourth-year medical students submitted a RW essay as part of their emergency medicine clerkship, either in a private essay format (academic year [AY] 2015) or onto a closed, password-protected social media website (AY 2016). Five raters used the Reflection Evaluation for Learners’ Enhanced Competencies Tool (REFLECT) to score 122 de-identified RW samples (55 private, 67 social media). Average scores on two platforms were compared. Students were also surveyed regarding their comfort with the social media experience.

Results

There were no differences in average composite REFLECT scores between the private essay (14.1, 95% confidence interval [CI], 12.0–16.2) and social media (13.7 95% CI, 11.4–16.0) submission formats (t [1,120] = 0.94, p = 0.35). Of the 73% of students who responded to the survey, 72% reported feeling comfortable sharing their personal reflections with peers, and 84% felt comfortable commenting on peers’ writing.

Conclusion

Students generally felt comfortable using social media for shared reflection. The depth of reflection in RW essays was similar between the private and social media submission formats.

Transforming Preprofessional Health Education Through Relationship-Centered Care and Narrative Medicine

ISSUE

The Institute of Medicine identified health care education reform as a key to improving the error prone, costly, and unsatisfying U.S. health care system. It called for health care education that no longer focuses exclusively on the mastery of technical skills but teaches students the human dimensions of care and develops their ability to collaborate with patients and colleagues to alleviate suffering and improve health. When should this educational reform begin, by what frameworks should it be guided, and which methods should it employ are important questions to explore.

EVIDENCE

There is increasing evidence that practitioners' relational skills, such as empathy and reflection, improve patients' health outcomes. Efforts to shift education toward patient-centered care in interprofessional teams have been made at the professional level, most notably in medical schools. However, reform must begin at the preprofessional level, to start cultivation of the habits that support humane care as early as possible and protect against empathic decline and the development of counterproductive attitudes to collaboration. The conceptual basis for reform is offered by relationship-centered care (RCC), a framework that goes beyond patient-centered care and interprofessional teamwork to focus on the reciprocal human interactions at the micro, mezzo, and macro levels of care. RCC identifies practitioners' relationships with patients, colleagues, community, and self as the critical interpersonal dimensions of healthcare and describes a foundation of values, knowledge, and skills required for teaching each dimension. The teaching of these foundations can be facilitated with techniques from narrative medicine, a compatible care model that conceptualizes health care as a context in which humans exchange stories and thus require narrative competence.

IMPLICATIONS

We suggest beginning the educational reform at the preprofessional level with the implementation of a formal curriculum based on the 4 RCC dimensions with students expected to gain beginner levels of competency on these dimensions in addition to evidence-based principles of health sciences. This requires interprofessional collaboration among health professions, social science, and liberal arts faculty and training of health professions faculty in narrative medicine. Next, we suggest engaging in incremental change in the organizational culture with professional development and team-building activities. Although we need systematic research on the efficacy of the components of the transformation, their impact on students' learning, and their costs, it is important to engage in efforts to prepare professionals who are able to respond to the complex health needs of individuals and society in the 21st century.

A review of creative and expressive writing as a pedagogical tool in medical education

CONTEXT

The act of writing offers an opportunity to foster self-expression and organisational abilities, along with observation and descriptive skills. These soft skills are relevant to clinical thinking and medical practice. Medical school curricula employ pedagogical approaches suitable for assessing medical and clinical knowledge, but teaching methods for soft skills in critical thinking, listening and verbal expression, which are important in patient communication and engagement, may be less formal. Creative and expressive writing that is incorporated into medical school courses or clerkships offers a vehicle for medical students to develop soft skills. The aim of this review was to explore creative and expressive writing as a pedagogical tool in medical schools in relation to outcomes of medical education.

METHODS

This project employed a scoping review approach to gather, evaluate and synthesise reports on the use of creative and expressive writing in US medical education. Ten databases were searched for scholarly articles reporting on creative or expressive writing during medical school. Limitation of the results to activities associated with US medical schools, produced 91 articles. A thematic analysis of the articles was conducted to identify how writing was incorporated into the curriculum.

RESULTS

Enthusiasm for writing as a pedagogical tool was identified in 28 editorials and overviews. Quasi-experimental, mixed methods and qualitative studies, primarily writing activities, were aimed at helping students cognitively or emotionally process difficult challenges in medical education, develop a personal identity or reflect on interpersonal skills. The programmes and interventions using creative or expressive writing were largely associated with elective courses or clerkships, and not required courses.

CONCLUSIONS

Writing was identified as a potentially relevant pedagogical tool, but not included as an essential component of medical school curricula.

Close Reading and Creative Writing in Clinical Education: Teaching Attention, Representation, and Affiliation

Medical educators increasingly have embraced literary and narrative means of pedagogy, such as the use of learning portfolios, reading works of literature, reflective writing, and creative writing, to teach interpersonal and reflective aspects of medicine. Outcomes studies of such pedagogies support the hypotheses that narrative training can deepen the clinician's attention to a patient and can help to establish the clinician's affiliation with patients, colleagues, teachers, and the self. In this article, the authors propose that creative writing in particular is useful in the making of the physician. Of the conceptual frameworks that explain why narrative training is helpful for clinicians, the authors focus on aesthetic theories to articulate the mechanisms through which creative and reflective writing may have dividends in medical training. These theories propose that accurate perception requires representation and that representation requires reception, providing a rationale for teaching clinicians and trainees how to represent what they perceive in their clinical work and how to read one another's writings. The authors then describe the narrative pedagogy used at the College of Physicians and Surgeons of Columbia University. Because faculty must read what their students write, they receive robust training in close reading. From this training emerged the Reading Guide for Reflective Writing, which has been useful to clinicians as they develop their skills as close readers. This institution-wide effort to teach close reading and creative writing aims to equip students and faculty with the prerequisites to provide attentive, empathic clinical care.

"Where Does the Circle End?": Representation as a Critical Aspect of Reflection in Teaching Social and Behavioral Sciences in Medicine

OBJECTIVE

This paper describes a reflective learning program within a larger curriculum on behavioral and social science that makes use of close reading, written representation of experience, discussion, and textual response. This response may in turn lead to further reflection, representation, and response in a circular pattern. A unique feature of this program is that it pays attention to the representation itself as the pivotal activity within reflective learning. Using the narrative methods that are the hallmark of this program, faculty writings were analyzed to characterize the essential benefits that derive from these practices.

METHODS

In the context of a faculty development seminar on the teaching of behavioral and social sciences in medical curricula, a group of 15 faculty members wrote brief narratives of reflective learning experiences in which they had made use of the methods described above. Their responses were submitted to iterative close reading and discussion, and potential themes were identified.

RESULTS

Four themes emerged: writing as attention to self, writing as attention to other, writing as reader/writer contract, and writing as discovery. In each instance, writing provides a new or deepened perspective, and in each case, the dividends for the writer are amplified by the narrative skills of those who read, listen, and respond.

CONCLUSIONS

The narrative pedagogy described and modeled herein provides a potentially promising approach to teaching the social, cultural, behavioral, and interpersonal aspects of medical education and practice. Future research will deepen our understanding of the benefits and limitations of this pedagogy and expand our appreciation of its applications.

An online evidence based medicine exercise prompts reflection in third year medical students

BACKGROUND

Reflective practice is a desirable trait in physicians, yet there is little information about how it is taught to or learned by medical students. The purpose of this study was to determine whether an online Evidence Based Medicine (EBM) exercise with a face-to-face debriefing session would prompt third year medical students to reflect on their current skills and lead them to further reflection on clinical decision making in the future.

METHODS

All third year medical students at the University Of Maryland School Of Medicine who completed their pediatrics clerkship between 7/1/09 and 2/11/11 were required to complete the EBM exercise. Following completion each student received a personal report (Learning Profile) of their responses and attended a one hour large group debriefing session. Student responses to a survey following the debriefing sessions were analyzed using a post-test survey design with a single experimental cohort.

RESULTS

Ninety-five percent of students completing the debriefing survey indicated that the debriefing session helped them better understand their learning profiles; 68% stated that their profiles allowed them to evaluate themselves and their decisions. Sixty-three percent noted that participating in the exercise and the debrief would lead them to either learn more about EBM and use EBM more in the future or reflect more on their own decision making.

CONCLUSIONS

The EBM exercise was a successful way to introduce the concept of reflective practice to third year medical students, and the graphic Learning Profiles were effective instigators of discussion and reflection.

Using video-cases to assess student reflection: development and validation of an instrument

BACKGROUND

Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners' verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection.

METHODS

Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students' reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS®). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets.

RESULTS

Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff's alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and ≥ 6 cases from 5th year students.

CONCLUSION

Use of StARS® to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.

Reflective writing in the internal medicine clerkship: a national survey of clerkship directors in internal medicine

BACKGROUND AND PURPOSE

Reflective writing programs have been implemented at many medical schools, but it is unclear to what extent and how they are structured.

METHODS

We surveyed the 107 Clerkship Directors of Internal Medicine member institutions on use of reflective writing assignments during the internal medicine clerkship.

RESULTS

Eighty-six of 107 (80%) institutional members completed the survey. Thirty-five percent reported having a reflective writing assignment, 48% did not, and 6% did not but were considering starting one within the next 2 years. Of the 30 assignments, most were partially structured (60%), involved small-group discussion (57%), and provided individual student feedback (73%). A minority (30%) contributed to the students' grade. Respondents believed assignments contributed to students' learning in multiple domains, most often Professionalism (97%) and Communication (77%).

CONCLUSIONS

Although reflective writing programs were common, variability existed in their structure. Further research is needed to determine how best to implement them.

Fostering and evaluating reflective capacity in medical education: developing the REFLECT rubric for assessing reflective writing

PURPOSE

Reflective writing (RW) curriculum initiatives to promote reflective capacity are proliferating within medical education. The authors developed a new evaluative tool that can be effectively applied to assess students' reflective levels and assist with the process of providing individualized written feedback to guide reflective capacity promotion.

METHOD

Following a comprehensive search and analysis of the literature, the authors developed an analytic rubric through repeated iterative cycles of development, including empiric testing and determination of interrater reliability, reevaluation and refinement, and redesign. Rubric iterations were applied in successive development phases to Warren Alpert Medical School of Brown University students' 2009 and 2010 RW narratives with determination of intraclass correlations (ICCs).

RESULTS

The final rubric, the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT), consisted of four reflective capacity levels ranging from habitual action to critical reflection, with focused criteria for each level. The rubric also evaluated RW for transformative reflection and learning and confirmatory learning. ICC ranged from 0.376 to 0.748 for datasets and rater combinations and was 0.632 for the final REFLECT iteration analysis.

CONCLUSIONS

The REFLECT is a rigorously developed, theory-informed analytic rubric, demonstrating adequate interrater reliability, face validity, feasibility, and acceptability. The REFLECT rubric is a reflective analysis innovation supporting development of a reflective clinician via formative assessment and enhanced crafting of faculty feedback to reflective narratives.