Assessing the level of student reflection from reflective journals

Reflective Engagement With a Digital Physical Activity Intervention Among People Living With and Beyond Breast Cancer: Mixed Methods Study

BACKGROUND

People living with and beyond breast cancer can face internal barriers to physical activity (eg, fatigue and pain). Digital interventions that promote psychological acceptance and motivation may help this population navigate these barriers. The degree to which individuals (1) adhere to intervention protocols and (2) reflect on and internalize intervention content may predict intervention efficacy.

OBJECTIVE

The objective of this study was to characterize the nature of reflective processes brought about by an 8-week acceptance- and mindfulness-based physical activity intervention for insufficiently active survivors of breast cancer (n=75). Furthermore, we explored the potential utility of a metric of reflective processes for predicting study outcomes.

METHODS

Of the intervention's 8 weekly modules, 7 (88%) included an item that asked participants to reflect on what they found to be most useful. Two coders conducted directed content analysis on participants' written responses. They assessed each comment's depth of reflection using an existing framework (ranging from 0 to 4, with 0=simple description and 4=fundamental change with consideration of social and ethical issues). The coders identified themes within the various levels of reflection. We fit multiple linear regression models to evaluate whether participants' (1) intervention adherence (ie, number of modules completed) and (2) the mean level of the depth of reflection predicted study outcomes.

RESULTS

Participants were aged on average 57.2 (SD 11.2) years, mostly non-Hispanic White (58/75, 77%), and mostly overweight or obese (54/75, 72%). Of the 407 responses to the item prompting personal reflection, 70 (17.2%) were rated as reflection level 0 (ie, description), 247 (60.7%) were level 1 (ie, reflective description), 74 (18.2%) were level 2 (ie, dialogic reflection), 14 (3.4%) were level 3 (ie, transformative reflection), and 2 (0.5%) were level 4 (ie, critical reflection). Lower levels of reflection were characterized by the acquisition of knowledge or expressing intentions. Higher levels were characterized by personal insight, commentary on behavior change processes, and a change of perspective. Intervention adherence was associated with increases in self-reported weekly bouts of muscle-strengthening exercise (B=0.26, SE 0.12, 95% CI 0.02-0.50) and decreases in sleep disturbance (B=-1.04, SE 0.50, 95% CI -0.06 to -2.02). The mean level of reflection was associated with increases in psychological acceptance (B=3.42, SE 1.70, 95% CI 0.09-6.75) and motivation for physical activity (ie, integrated regulation: B=0.55, SE 0.25, 95% CI 0.06-1.04).

CONCLUSIONS

We identified a useful method for understanding the reflective processes that can occur during digital behavior change interventions serving people living with and beyond breast cancer. Intervention adherence and the depth of reflection each predicted changes in study outcomes. Deeper reflection on intervention content was associated with beneficial changes in the determinants of sustained behavior change. More research is needed to investigate the relations among digital behavior change intervention use, psychological processes, and intervention efficacy.

The use of artificial intelligence to detect students' sentiments and emotions in gross anatomy reflections

Students' reflective writings in gross anatomy provide a rich source of complex emotions experienced by learners. However, qualitative approaches to evaluating student writings are resource heavy and timely. To overcome this, natural language processing, a nascent field of artificial intelligence that uses computational techniques for the analysis and synthesis of text, was used to compare health professional students' reflections on the importance of various regions of the body to their own lives and those of the anatomical donor dissected. A total of 1365 anonymous writings (677 about a donor, 688 about self) were collected from 132 students. Binary and trinary sentiment analysis was performed, as well as emotion detection using the National Research Council Emotion Lexicon which classified text into eight emotions: anger, fear, sadness, disgust, surprise, anticipation, trust, and joy. The most commonly written about body regions were the hands, heart, and brain. The reflections had an overwhelming positive sentiment with major contributing words "love" and "loved." Predominant words such as "pain" contributed to the negative sentiments and reflected various ailments experienced by students and revealed through dissections of the donors. The top three emotions were trust, joy, and anticipation. Each body region evoked a unique combination of emotions. Similarities between student self-reflections and reflections about their donor were evident suggesting a shared view of humanization and person centeredness. Given the pervasiveness of reflections in anatomy, adopting a natural language processing approach to analysis could provide a rich source of new information related to students' previously undiscovered experiences and competencies.

Teacher-led versus student-led seminar blended with portfolio for "assessment of learning": An interventional study

BACKGROUND

Increase in competitive demands has led to the promotion of seminar presentation by the students to increase active learning and for the assessment of learning. Portfolios are an important tool for assessment for learning. The objectives of the study were to compare the gain in knowledge among the conventional and the blended seminar groups (intervention group), analyze the working portfolios to assess for learning, and to gather the perception of students.

MATERIALS AND METHODS

This was an interventional study (2016) with convenience sampling that included 27 students each in the conventional and interventional groups. It was conducted in a medical college in central India. The conventional group was teacher led and the blended seminar group was student led with interaction with the students regarding the contents of the portfolio. Student's unpaired and paired t tests were used for statistical analysis. P < 0.05 was considered as the level of significance. Class average normalized gain (g) was used as a measure of effectiveness of the intervention. Quantitative questions were analyzed using percentages and qualitative data using categorization.

RESULTS

Significant difference was found between the conventional and intervention groups (P < 0.05) with gain "g" being 0.52 for the intervention group. On evaluation of the feedback, students commented regarding its interactive nature and progress during the learning process. The reflections were coded as text as the unit of coding and student as the unit of coding. It was also found that the students who were critical reflectors were the ones who scored > 50% in the posttest scores.

CONCLUSION

The present study showed that seminar when blended with portfolio yielded positive results in the process of learning, and hence was effective in assessment for learning.

Assessment tools for reflection in healthcare learners: A scoping review

Reflection has been integrated in many healthcare educations programs to achieve deeper learning and improve professional practice. A variety of evaluation tools are used to assess reflection, but few guides are available to inform educators in their choice of a relevant evaluation tool. The aim of this paper is to identify all existing evaluation tools published along with their strengths and weaknesses. A review strategy retrieved tools available in Medline, PsychInfo, CINALH and Eric databases. The procedures outlined by Munn and colleagues were used to synthetize the information. Additionally, the reflection dimensions assessed in each tool (when sufficient information was available) were analyzed deductively, using thematic analysis according to the Killion and Todnem framework. Subthemes were identified inductively. Forty-five papers were identified, reporting on 34 different tools. The tools were based on a variety of theoretical models. Some had evidence of adequate validity and fidelity. Eleven components of reflection were identified across tools. No tool encompassed all components, but most tools included between three and five components. Context surrounding evaluation should be carefully considered when choosing an evaluation tool for reflection. There is a need for further research to validate the psychometric properties of reflection evaluation tools.

Using theory-informed data science methods to trace the quality of dental student reflections over time

This study describes a theory-informed application of data science methods to analyze the quality of reflections made in a health professions education program over time. One thousand five hundred reflections written by a cohort of 369 dental students over 4 years of academic study were evaluated for an overall measure of reflection depth (No, Shallow, Deep) and the presence of six theoretically-indicated elements of reflection quality (Description, Analysis, Feeling, Perspective, Evaluation, Outcome). Machine learning models were then built to automatically detect these qualities based on linguistic features in the reflections. Results showed a dramatic increase from No to Shallow reflections from the start to end of year one (20%  →  66%), but only a limited gradual rise in Deep reflections across all four years (2%  →  26%). The presence of all six reflection elements increased over time, but inclusion of Feelings and Analysis remained relatively low even at the end of year four (found in 44% and 60% of reflections respectively). Models were able to reliably detect the presence of Description (κ_TEST = 0.70) and Evaluation (κ_TEST = 0.65) in reflections; models to detect the presence of Analysis (κ_TEST = 0.50), Feelings (κ_TEST = 0.54), and Perspectives (κ_TEST = 0.53) showed moderate performance; the model to detect Outcomes suffered from overfitting (κ_TRAIN = 0.90, κ_TEST = 0.53). A classifier for overall depth built on the reflection elements showed moderate performance across all time periods (κ_TEST > 0.60) but relied almost exclusively on the presence of Description. Implications for the conceptualization of reflection quality and providing personalized learning support to help students develop reflective skills are discussed.

Reflections of psychomotor skill development in preclinical simulation: A qualitative analysis

INTRODUCTION

Reflection on competency helps students become more proficient in performing skills, with the ultimate goal of better retention of knowledge and skill. Competency progression involves a complex interplay of factors, and not having the insight of such multiple perspectives of the same experience, we will not be able to understand students as learners fully and consequently may rely on assumption and may not be able to provide the necessary interventions for growth and progress towards competency. This study aims to investigate students' insight of their own progression during psychomotor skill development.

MATERIALS AND METHODS

Data were obtained from scanned reflective dialogue log document portfolios from the preclinical fixed dental prosthodontics section, which is part of a comprehensive dental care course during the second year. Data were coded using NVivo software version 12 plus (QSR International Pty Ltd, Vic, Australia), and subsequent thematic analyses identified emerging themes.

RESULTS

Three themes emerged and these were (a) challenges to overcome, (b) feed-back and self-appraisal, and (c) progress and growth. The qualitative data generated did not reveal considerable variation in the students' reflections, and the three themes seem to interrelate.

CONCLUSIONS

The main challenge was the concept of the single path of insertion and the perception that it was a source of frustration during this course. Self-appraisal identified time management issues and the transformation from preconceived or learned concepts. As the semester progressed, students reflected progress and growth.

Higher Education Students' Reflective Journal Writing and Lifelong Learning Skills: Insights From an Exploratory Sequential Study

Reflective journal (RJ) writing has been recognized as an effective pedagogical tool for nurturing students’ lifelong learning skills. With the paucity of empirical work on the dimensionality of reflective writing, this research sought to qualitatively analyze students’ RJ writing and design a generic reflection scheme for identifying dimensions of reflective thinking. Drawing on the theoretical scheme, another aim was to design and validate a questionnaire to measure students’ perceptions of their reflective writing experiences. The last aim was to quantitatively measure the link between perceived reflective writing and students’ tendency to use RJs in their future careers and personal lives. This exploratory sequential research included the following steps: First, experts’ review and analysis of 1312 RJ entries were attained. This step led to the design of a theoretical scheme of reflective writing and a 31-item questionnaire, used to gather data from 171 students (second-year pre-service teachers and third-year health managers). Partial Least Squares analysis corroborated the structure suggested by the theoretical scheme: two timelines–reflections regarding the current course assignments and those related to the student’s future development. Students’ tendency to use reflective skills in their future professional lives was highly connected to their long-term reflections, including learning experiences linked to academic, professional, personal, and multicultural development. The current study’s suggested validated generic scheme can be adapted and integrated into different curricula, thereby possibly increasing the potential of infusing RJ instructional strategies into higher education curricula, improving the quality of reflection in student journals, and promoting lifelong learning skills.

Reflection as a learning strategy for the training of nurses in clinical practice setting: a scoping review

OBJECTIVE

The objective of this review is to identify and map the tools and approaches used for reflection as a learning strategy for nurses and nursing students in clinical settings.

BACKGROUND

Several types of reflection interventions exist to develop nurses' reflective competencies in clinical practice. This scoping review mapped the available literature on the use of reflection in the education of nurses and nursing students in clinical settings. Three questions guided the review: i) What tools and approaches for reflection have been reported? ii) What levels of reflection have been identified (low/descriptive, middle/application, or high/evaluative)? iii) What outcomes of reflection have been reported?

INCLUSION CRITERIA

Studies investigated nursing students or nurses working in clinical settings involved in patient care, and described reflective interventions and definitions of reflection, descriptions of the contents reflected on, measurement instruments, and outcomes of interventions.

METHODS

MEDLINE (PubMed), Emcare (Ovid), MEDLINE (Ovid), ERIC (ProQuest), CINAHL (Ovid), and Web of Science databases were searched for full-text studies published in English, Danish, Swedish, or Norwegian. Studies from any country and published from 2007 to September 2020 were eligible for inclusion. The review followed the JBI methodology for scoping reviews and was based on an a priori protocol. Two independent reviewers assessed studies against the inclusion criteria and extracted data. Inclusion for review was determined according to the pre-determined eligibility criteria and consensus between researchers. Data were extracted using a standardized data extraction table and results were summarized narratively with accompanying tables.

RESULTS

Seventeen articles were included; nine quantitative studies and eight qualitative studies. Findings are presented separately due to varying study designs and outcomes. Most studies used physical tools and reflective dialogues in clinical settings. Some tools included specific questions for participants to reflect on, while others included open questions or no questions. Reflective approaches included lectures, workshops, or reflection with supervisors or peers in clinical settings. None of the 17 studies included the same combination of tools and approaches. Measurement of reflection was varied across articles, and there was a lack of standardization in how reflection outcomes were classified.

CONCLUSIONS

Interventions, tools, approaches, and the measurement of nurses' reflection in clinical settings are highly varied, and none of the included studies had the same combination of elements. The studies indicate that reflection skills are achieved potentially according to the tools and approaches used, and the foci for reflection. There is little agreement among the research community on how best to assess the measurement and quality of reflection. It is recommended that future studies are explicit on the classification of low-, middle-, and high-level reflection according to standardized criteria.

Do critical incidents lead to critical reflection among medical students?

Context

Medical students are exposed during their training to a wide range of experiences and behaviors that can affect their learning regarding professionalism and their behavior and attitudes towards patient-centered care. The aim of the study is to explore learning associated with critical incidents and levels of critical reflection among medical students.

Approach

Medical students' were invited to narrate a critical incident and reflect on the learning associated with it. All students' narratives were audio-recorded and analyzed thematically. Mezirow's theory of transformative learning was used to analyze the level of reflection reached in students' narratives.

Findings

For the present analysis critical incidents narrated by 70 clinical students (4th-6th year) were included. Fifty-two of them were females. Students' experiences are derived from three types of interactions: observed interactions between doctors and patients, personal interactions between students and patients, and interactions between doctors and students. Reflections deriving from the experiences included: behaving to patients as humans not as cases, emotional aspects of care, doctors as role models, skills needed when working under pressure, 'tasting' the real profession, emotional management, the importance of communication skills, teaching qualities of doctors, becoming a doctor, and reflections of future practice. Analyzing the actual level of reflection indicated that only 32 (45.7%) students were categorized as reflectors.

Conclusions

Student interactions with doctors and patients provide insights about; the daily experience of being a doctor, the most common challenges, what qualities are necessary for being a doctor and what do they need to develop their professional identity. However, it is noteworthy that while the majority of students shared a critical incident crucial to their professional development, there is little evidence of critical reflection.

A framework for scaffolding academic reflective writing in dentistry

INTRODUCTION

Being able to demonstrate reflection on clinical practice is a key competency required of undergraduate dental students and registrants alike. Academic reflective writing (ARW) is an increasingly widely used genre of writing which evidences reflective practice. ARW is complex, as students need to achieve balance from a challenging mix of personal, academic and evaluative writing. The aim of this study is to analyse four student ARW tasks from a UK Dental School in order to make recommendations regarding the systematic development of student ARW through the dental curriculum.

METHODS

In this study, a detailed linguistic analysis of high-scoring examples of two first year and two fourth year ARW tasks was undertaken, complemented by comments from interviews with markers of the tasks.

RESULTS

The linguistic analysis of high-scoring student ARW revealed five consistent reflective aspects common to all tasks, viz description, feelings, analysis, evaluation and conclusion and action planning. Entry-level and final-year ARW showed a different balance between reflective aspects. Markers' expectations in the lower years were found to equate to high-quality technical reflection, whereas markers in the higher years expected students to have shifted their perspectives to achieve critical reflection.

CONCLUSION

Whilst this study is limited in its generalisability due to its small-scale exploratory nature, it is hoped that the five reflective aspects, and the categories of reflection put forward here will be useful in stimulating discussion about how to scaffold the development of student ARW (and associated reflective thinking) longitudinally through the dental curriculum.

Developing Student Metacognition through Reflective Writing in an Upper Level Undergraduate Anatomy Course

Incorporating metacognitive strategies in the classroom helps students monitor and adjust their learning strategies throughout the semester, and helps students progress from novice to expert learners in a subject. Journaling (i.e., reflective writing) is one metacognitive task that allows students to contemplate and articulate their skill development as they learn a new subject. The research reported here examines the use of 'blogs' (i.e., online journals) in an upper level undergraduate human anatomy course. The blogs both facilitated development of students' metacognitive skills and provided researchers insight into student metacognitive process. Data were examined from 92 students from three successive semesters (spring 2010, 2012 and 2014). Each student reviewed 10 radiology online cases throughout the semester and then reflected on their understanding of anatomy and radiology in an online blog for each case. A total of 927 blogs were examined for this research. The researchers used a grounded theory approach to analyze the blog narratives and develop a codebook based on common themes. The 927 blogs yielded 11,082 statements that were coded with the codebook. As the semester progressed, the blog entries showed that students demonstrated greater self-confidence in their abilities to understand the subject matter, expressed greater enthusiasm for anatomy in general, and they improved their metacognitive skills. This research illustrates that reflective writing in an undergraduate anatomy course not only facilitates improvement in student metacognitive skills, but also provides the instructor with evidence how a student progresses from novice to more experienced learner in anatomy.

Value creation programmes: lessons from an early-stage implementation

This paper explores the authors’ efforts to design and deliver a Venture Creation Programme (VCP) at the University of Worcester between 2015 and 2017. The purpose of the paper is to present critical insights into the manner in which learning can be facilitated through VCPs using our own experiences as an example. The paper begins with a discussion of the authors' methodology before moving to a literature review addressing VCP’s and related elements of entrepreneurial education best practice. It then shifts to a critical discussion of the authors' experiences based on insights, student and staff evaluations, and course data. In the process of conducting this review the authors found that good pedagogical design, in a VCP context, needs to focus on ensuring that academics promote constructive alignment between the curriculum that is ‘taught’, the teaching methods used, the learning environment chosen and the assessment procedures adopted. Only by minimising inconsistencies which link these factors will students’ experiences of venture creation be fully engaging, promoting more effective, experiential and entrepreneurial development. This paper provides educators with an original and experiential insight into the design of a VCP programme and the pedagogical developments required for their successful implementation. There is considerable value herein for higher education institutions which want to design a new VCP, especially with reference to the curriculum design, experiential learning activities, types and methods of assessments and online learning environments for entrepreneurial students.

A compass for care: Evaluating first-year pharmacy students' philosophy of practice

INTRODUCTION

A philosophy of practice serves as the foundation for any patient care practice. Understanding the philosophies of practice held by students entering the profession is important as this presents potential opportunities for educators to help shape students' professional values/beliefs. The objective of this work was to evaluate how first-year pharmacy students (P1s) conceptualize their philosophy of practice as future pharmacists.

METHODS

P1s at the University of Minnesota College of Pharmacy were asked to write their philosophy of practice that best described them as a future practitioner. A sample of 80 student responses were inductively coded to develop an initial coding scheme. These codes were applied and refined to a random sample of 32 responses (20%) where inter-coder reliability (ICR) testing was carried out at three stages. The final codebook was applied to all responses.

RESULTS

The exercise was completed by 155 (92%) students, including the 80 student responses used for developing the initial coding scheme and the 32 responses used for ICR. Thirteen unique codes emerged from student responses describing their philosophy of practice that were grouped into four themes: community partner, commitment to the patient, professional responsibilities, and interprofessional team-based care.

CONCLUSIONS

There was variability in how pharmacy students articulated their philosophy of practice. This presents an opportunity to shape students' philosophy of practice as they progress through pharmacy school so that they graduate with a uniform set of professional values and beliefs.

Automated Analysis of Middle School Students’ Written Reflections During Game-Based Learning

Game-based learning environments enable students to engage in authentic, inquiry-based learning. Reflective thinking serves a critical role in inquiry-based learning by encouraging students to think critically about their knowledge and experiences in order to foster deeper learning processes. Free-response reflection prompts can be embedded in game-based learning environments to encourage students to engage in reflection and externalize their reflection processes, but automatically assessing student reflection presents significant challenges. In this paper, we present a framework for automatically assessing students’ written reflection responses during inquiry-based learning in Crystal Island, a game-based learning environment for middle school microbiology. Using data from a classroom study involving 153 middle school students, we compare the effectiveness of several computational representations of students’ natural language responses to reflection prompts—GloVe, ELMo, tf-idf, unigrams—across several machine learning-based regression techniques (i.e., random forest, support vector machine, multi-layer perceptron) to assess the depth of student reflection responses. Results demonstrate that assessment models based on ELMo deep contextualized word representations yield more accurate predictions of students’ written reflection depth than competing techniques. These findings point toward the potential of leveraging automated assessment of student reflection to inform real-time adaptive support for inquiry-based learning in game-based learning environments.

Experiences of recent graduates: reframing reflection as purposeful, social activity

BACKGROUND

During the Royal College of Veterinary Surgeons' (RCVS) Professional Development Phase, graduates are required to reflect on their progress. Reflection is often conceptualised as a solitary activity, which may contrast with day-to-day reflective activities in the workplace. This study drew on cultural-historical activity theory to understand how recently graduated veterinary surgeons engage in reflective activity.

METHODS

Data comprised RCVS documentation and semistructured interviews with 15 recent graduates from one veterinary school. Thematic analysis was used to describe a collective system of reflective activity and to identify contradictions in the system with the potential to limit outcomes of reflective activity.

RESULTS

Two overarching themes of contradictions were identified: 'social reflection' and 'formalising the informal'. Graduates need opportunities for talking and/or writing to progress worries into purposeful reflection, underpinned by a shared understanding of reflective activity with colleagues, and by working practices which prioritise and normalise reflective interaction.

CONCLUSION

These findings identify potential avenues to better support veterinary graduates as they negotiate the transition to working life, and suggest that reconsideration of the formal expectations of new veterinary graduates and their employers is timely.

Can a reflective rubric be applied consistently with raters globally? A study across three countries

INTRODUCTION

Reflection is a powerful tool for assisting students to develop the skills to make better informed decisions. As a pharmacy competency standard, reliable and fair assessment strategies are required to measure reflective skills and support students in developing their reflective capacity. The aim of this research was to explore whether we can extend the applicability of a previously tested rubric to a range of educational settings, to account for diversity of pharmacy educators and curricula internationally.

METHODS

Four raters from three countries applied a reflective rubric to assess a sample (n = 43) of reflective accounts, representing 41% of a cohort of 105 second-year undergraduate pharmacy students. The interrater reliability (IRR) was measured utilizing the intra-class correlation coefficient (ICC), using a two-way random effects model with absolute agreement, to determine the level of agreement between the raters' absolute scores. Generalizability Theory analysis was used to estimate generalizability of raters and stages.

RESULTS

Results indicated agreement of raters for (i) each of the seven stages of reflection and (ii) overall score for the reflective account, with moderate to substantial agreement (ICC = 0.55-0.69, p < 0.001); and high agreement for all raters for the overall score (ICC = 0.96, p < 0.001), respectively. The G-Study estimated a relative error coefficient of 0.78.

CONCLUSION

This additional analysis further confirms the reliability and applicability of the rubric to a range of rater academic backgrounds.

Pharmacy Students' Perceptions of Reflective Portfolios and their Effect on Students' Deep Information-Processing Skills

Objective. To evaluate pharmacy students' perceptions of the educational value of reflective portfolio and to gain an understanding of the factors that might influence these perceptions. Methods. Bachelor of Pharmacy (BPharm) students' perceptions of using reflective portfolios were evaluated by administering the same questionnaire at the beginning of years 2, 3 and 4 of the curriculum. Statistical analysis was carried out to determine the differences among the perception scores of different academic years. Semi-structured interviews were completed with fourth-year students to further explore their experiences with the reflective portfolio. Students' deep information processing (DIP) skills were compared with those of students from another pharmacy cohort whose curriculum did not include a reflective portfolio. Results. The students' perceptions of the reflective portfolio improved significantly as they progressed from year 2 to year 4 of the curriculum. The factors that contributed to a positive experience were a clear understanding of objectives and guidelines for the reflective portfolio, useful mentor feedback, a positive learning attitude and motivation, and having a user-friendly technology platform for submission of the portfolio. The students' DIP skills after completing the reflective portfolio were higher than those of students who did not have a reflective portfolio assignment in their curriculum. Conclusion. Pharmacy students' appreciation of the educational value of a reflective portfolio increased as they progressed to their final year, and their DIP skills improved. These findings support the use of a reflective portfolio as a learning tool for BPharm students' personal and professional development.

Applications of the reflective practice questionnaire in medical education

BACKGROUND

We sought to determine whether the Reflective Practice Questionnaire (RPQ) is a reliable measure of reflective capacity and related characteristics in medical students. We also planned to learn how the RPQ could be used in medical education.

METHODS

The RPQ is a 40 item self-report questionnaire that includes a multi-faceted approach to measuring reflective capacity. It also includes sub-scales on several other theoretically relevant constructs such as desire for improvement, confidence, stress, and job satisfaction. The reliabilities of reflective capacity and other sub-scales were determined by calculating their Cronbach alpha reliability values. In the present study, the RPQ was answered by 98 graduating fourth-year medical students from an American University, and these RPQ scores were compared with general public and mental health practitioner samples from a prior study using ANOVA and Bonferroni adjusted comparisons.

RESULTS

Medical students reported a higher reflective capacity than the general public sample, but students were statistically indistinguishable from the mental health practitioner sample. For medical students, reflective capacity was associated with features of confidence, stress, and desire for improvement. Job satisfaction was positively associated with confidence in communication with patients, and negatively associated with stress when interacting with patients. A cluster analysis revealed that around 19% of the medical students exhibited a relatively high level of anxiety interacting with patients, 23% were less engaged, 5% were dissatisfied, and 7% expressed a level of over-confidence in their knowledge and skills that was concerning.

CONCLUSIONS

The RPQ is a reliable measure of reflective capacity (Chronbach's alpha value = 0.84) and related characteristics (Cronbach's alpha values from 0.75 to 0.83) in medical students. The RPQ can be used as part of pre-post evaluations of medical education initiatives, to complement student self-reflection activities in the curriculum, and to identify students who might benefit from targeted intervention.

Reflection in the training of nurses in clinical practice settings: a scoping review protocol

This scoping review will seek to find answers for the following questions which will focus on the use of reflection in the education of nurses in clinical settings:The review will also extract and map data regarding: i) what outcomes have been found in relation to the use of different tools and approaches (e.g. dialogues, diaries, case studies); ii) how approaches and tools have been implemented as interventions; iii) details of the topic or focus of reflection (e.g. ethical issues, care of older adults etc.); iv) details about the participants involved in reflection activities (e.g. first or second year undergraduate nursing students etc.); and v) barriers/challenges to the use of reflection approaches/tools. Additional details may also be extracted and mapped during the process of the scoping review and this will be explained in the final scoping review report.

Inter-rater reliability of a reflective rubric to assess pharmacy students' reflective thinking

INTRODUCTION

Many health education programs aspire to foster reflective practices to enhance the reflective thinking of students during their study years and throughout their professional careers. Given the increasing attention paid to incorporating reflective practice activities into health education and evidence attributed to rater variability, it is important for educators to carefully consider the assessment criteria and reliability of assessment tools. A reflective rubric was developed to assess pharmacy students' levels of reflection from a reflective writing task. This study investigates the inter-rater reliability (IRR) of a rubric as a tool to assess the levels of reflective thinking apparent in students' written statements.

METHODS

The research involved four raters from different disciplines utilizing a rubric to assess a random sample of the same forty-three reflective statements from a cohort of two hundred and sixty-four students. The IRR was measured using the intra-class correlation coefficient (ICC), using a two-way random effects model (ANOVA) with absolute agreement, to determine reliability of the assessment tool among the raters.

RESULTS

Results showed measures between the raters for (i) overall reflective statement scores, and (ii) average scores for stages of reflection with an "almost perfect" agreement, ICC = 0.81 (95% CI 0.61-0.90), (F(42, 126) = 7.83, p < 0.01); and ICC = 0.89 (95% CI 0.83-0.93), (F(42, 840) = 12.49, p < 0.01) respectively.

DISCUSSION AND CONCLUSIONS

The proposed rubric utilized by four raters showed high agreement with each other's scores, and is a tool for academic assessment of pharmacy students' reflective thinking processes.

Value of Online Group Reflection After International Service-Learning Experiences: I Never Thought of That

Reflection is a widely accepted learning tool and a component of competent professional practice. An exploratory descriptive study was conducted to compare the breadth and level of reflection between students engaged in individual reflection papers and students engaged in an online group reflection discussion after an international cultural immersion service-learning program. Results indicated that students participating in the online group discussion had a higher level of reflective thinking and discussed more topics in their written reflections.

Can reflexivity be learned? An experience with tobacco control practitioners in Canada

Summary

To explore an example of a reflexive intervention with health professionals working in tobacco control (TC). This study reports the perceived intervention effects regarding: (i) participants' understanding of reflexivity and personal learning and (ii) conditions needed in order to integrate reflexivity into professional and organizational practices. This is a qualitative study using an interpretative evaluation framework to assess the perceived effects of a reflexive intervention in Montréal, Québec. Semi-structured qualitative interviews (n = 8) gathered data. Data analysis began deductively, guided by the broad categories found in research questions. Sub-categories to populate these broad categories captured the inhibitors and facilitators through an inductive thematic analysis. Our study reveals that, following the intervention, most participants had a generally good understanding of reflexivity and described concrete learning in association with the intervention. Main facilitators and inhibitors to conducting a reflexive workshop pertained to the organizational context as well as to the professional and individual characteristics of the participants. Some participants implemented sustainable changes as a result of the intervention, such as creating a tool, reviewing work plans and developing new mechanisms to integrate the voice of their clientele in the planning process. The need and interest for dialogue among health professionals about how TC intervention activities may inadvertently contribute to social inequalities in smoking is apparent. While there appears to be potential for reflexive practice, the integration of reflexivity into practice is reliant upon the organizational context (financial and time constraints, culture, support, and climate) and the reflexivity concept itself (intangibility, complexity and fuzziness).

The Effect of Reflective Activities on Reflective Thinking Ability in an Undergraduate Pharmacy Curriculum

Objective. To determine the effectiveness of integrating reflective practice activities into a second-year undergraduate pharmacy curriculum and their impact on reflective thinking ability. Design. A cross-over design with repeated measures was employed. Newly developed reflective modules based on real hospital and community pharmacy cases were integrated into the second-year pharmacy practice curriculum. A novel strategy, the Reflective Ability Clinical Assessment (RACA), was introduced to enhance self- and peer reflection. Assessment. Student responses (n=214) to the adapted Kember et al(1) Reflective Thinking Questionnaire (RTQ) were compared before and after reflective activities were undertaken. Significant improvement in three indicators of reflective thinking was shown after students engaged in reflective activities. Conclusion. Integration of reflective activities into a pharmacy curriculum increased the reflective thinking capacity of students. Enhancing reflective thinking ability may help students make better informed decisions and clinical judgments, thus improving future practice.

Reflective Writing for Medical Students on the Surgical Clerkship: Oxymoron or Antidote?

OBJECTIVES

Reflective writing has emerged as a solution to declining empathy during clinical training. However, the role for reflective writing has not been studied in a surgical setting. The aim of this proof-of-concept study was to assess receptivity to a reflective-writing intervention among third-year medical students on their surgical clerkship.

STUDY DESIGN

The reflective-writing intervention was a 1-hour, peer-facilitated writing workshop. This study employed a pre-post-intervention design. Subjects were surveyed on their experience 4 weeks before participation in the intervention and immediately afterwards. Surveys assessed student receptivity to reflective writing as well as self-perceived empathy, writing habits, and communication behaviors using a Likert-response scale. Quantitative responses were analyzed using paired t tests and linear regression. Qualitative responses were analyzed using an iterative consensus model.

SETTING

Yale-New Haven hospital, a tertiary care academic center.

PARTICIPANTS

All medical students of Yale School of Medicine, rotating on their surgical clerkship during a 9-month period (74 in total) were eligible. In all, 25 students completed this study.

RESULTS

The proportion of students desiring more opportunities for reflective writing increased from 32%-64%. The proportion of students receptive to a mandatory writing workshop increased from 16%-40%. These differences were both significant (p = 0.003 and p = 0.001). In all, 88% of students also reported new insight as a result of the workshop. In total, 39% of students reported a more positive impression of the surgical profession after participation.

CONCLUSION

Overall, the workshop was well-received by students and improved student attitudes toward reflective writing and the surgical profession. Larger studies are required to validate the effect of this workshop on objective empathy measures. This study demonstrates how reflective writing can be incorporated into a presurgical curriculum.

Qualitative Analysis of Written Reflections during a Teaching Certificate Program

OBJECTIVE

To evaluate the success of a teaching certificate program by qualitatively evaluating the content and extent of participants' reflections.

METHODS

Two investigators independently identified themes within midpoint and final reflection essays across six program years. Each essay was evaluated to determine the extent of reflection in prompted teaching-related topic areas (strengths, weaknesses, assessment, feedback).

RESULTS

Twenty-eight themes were identified within 132 essays. Common themes encompassed content delivery, student assessment, personal successes, and challenges encountered. Deep reflection was exhibited, with 48% of essays achieving the highest level of critical reflection. Extent of reflection trended higher from midpoint to final essays, with significant increases in the strengths and feedback areas.

CONCLUSION

The teaching certificate program fostered critical reflection and self-reported positive behavior change in teaching, thus providing a high-quality professional development opportunity. Such programs should strongly consider emphasizing critical reflection through required reflective exercises at multiple points within program curricula.

Students' and teachers' experiences of participating in the reflection process "THiNK"

OBJECTIVES

Reflective journal writing in clinical practice is used as a tool in bachelor programs in nursing. The reflection process THiNK was developed to enhance students' reflection processes. It was tailored to the needs of a nursing programme (in Norway) as former studies showed that many students had superficial level of reflection in their reflective journals, few students applied knowledge to their reflections and some met unprepared in the guided reflection groups. The teachers had inconsistent focus on the importance of reflection as a way of learning. This study aimed to describe students' and teachers' experiences of participation in reflection processes.

DESIGN AND METHOD

The development of THiNK is inspired by educational design research. This study used focus groups to gain insight into students' and teachers' experiences of the reflection processes. Data were analysed using qualitative content analysis.

PARTICIPANTS AND SETTING

The empirical sample included 5 focus groups with 28 students and 7 teachers in a bachelor nursing program at a University College in Norway.

RESULTS

The themes were as follows: stop and verbalise the experiences, stimulate and support the reflection processes and develop a conscious attitude. The students became aware of themselves and their own execution. The teachers and reflection groups were crucial in order to learn to see the complexity of a given situation and helped draw connection between the situation and the theoretical knowledge.

CONCLUSIONS

Enhancing students' professional development requires attention to facilitation skills as well as other contextual factors. Readiness in the culture can be ensured by tailoring frameworks of reflection that replies to students' and teachers' requests. Participating in reflection processes facilitates integration of modes of thinking when dealing with clinical situations.

A Model for Assessing Reflective Practices in Pharmacy Education

OBJECTIVE. To research the literature and examine assessment strategies used in health education that measure reflection levels and to identify assessment strategies for use in pharmacy education. METHODS. A simple systematic review using a 5-step approach was employed to locate peer-reviewed articles addressing assessment strategies in health education from the last 20 years. RESULTS. The literature search identified assessment strategies and rubrics used in health education for assessing levels of reflection. There is a significant gap in the literature regarding reflective rubric use in pharmacy education. CONCLUSION. Two assessment strategies to assess levels of reflection, including a reflective rubric tailored for pharmacy education, are proposed.

Reflective Writing: A Potential Tool to Improve Interprofessional Teamwork with Radiologists

RATIONALE AND OBJECTIVES

Studies show that problems with interprofessional collaboration can result in adverse patient outcomes. These problems are common in the field of radiology, where technology has decreased opportunities for direct communication and collaboration with referring physicians. To our knowledge, critical reflection has not been studied as an intervention to better understand one's own and/or others' roles in the context of an interprofessional team, or more specifically, to improve interprofessional collaboration between radiologists and other physicians.

MATERIALS AND METHODS

We trialed a reflective journaling assignment in our fourth year medical student general radiology elective. Student journal content was scored by percentage of comments reflecting on elective experiences versus recounting events. Content was categorized as "reflection" using an established measurement tool. Reflective content was evaluated to identify common themes.

RESULTS

A total of 31 journals (178 entries and 26,749 words) were analyzed. Reflective content accounted for 43% of overall content and was subdivided into three categories: insight into one's own role and responsibilities as an ordering physician (20%), insight into a radiologist's role and responsibilities (12%), and thoughts on improving interprofessional collaboration with radiologists (11%).

CONCLUSIONS

Reflective writing allows students to explore their own role and responsibilities in the context of an interprofessional team and may improve interprofessional teamwork with radiologists.

Dental students' reflective habits: is there a relation with their academic achievements?

INTRODUCTION

Reflection is regarded as an important and essential component of healthcare professionals' education and practice. This cross-sectional study aimed to explore the levels of reflection habits of dental students and clinical and PhD postgraduate trainees and to establish the relationship between students' reflection and their academic performance.

MATERIALS AND METHODS

A total of 324 subjects (208 females, 56% ≥23 years of age) from King's College London Dental Institute, consisting of 281 undergraduate and 43 postgraduate trainees, responded to the invitation to participate and completed an online self-reported Reflection Questionnaire; this assesses two levels of non-reflective actions (Habitual Action and Understanding) and two levels of reflective actions (Reflection and Critical Reflection). Reflection results were compared amongst different cohorts and correlated with students' academic performance.

RESULTS

Reflection Questionnaire mean scores were 10.7 for Habitual Action, 17.3 for Understanding, 17.0 for Reflection and 14.4 for Critical Reflection, with significant differences between cohorts of undergraduate students, clinical postgraduates and PhD trainees. Reflection and Critical Reflection mean scores were higher as students climbed courses. Further, those ≥24 years of age and those with previous university degrees demonstrated higher reflective habits, whilst there was no gender difference. Additionally, the assessment programme methods correlated differently to the Reflection Questionnaire scores.

DISCUSSION

The Reflection Questionnaire's internal reliability was acceptable. The most common approaches used by both students and trainees were Understanding and Reflection, whilst those with high Understanding scores also tended to have good Reflective scores. Further, multiple choice questions encouraged students to 'understand', whilst ill-defined problems, such as those from essays to clinical reasoning cases, increased this to the reflection scale.

CONCLUSION

To foster reflective thinking, tutoring should primarily be focused on younger students (≤23 years old) and those without a previous university degree. Further, a wide variety of assessment methods is suggested to stimulate different reflective constructs.

Essential steps in developing best practices to assess reflective skill: A comparison of two rubrics

PURPOSE

Medical education lacks best practices for evaluating reflective writing skill. Reflection assessment rubrics include the holistic, reflection theory-based Reflection-on-Action and the analytic REFLECT developed from both reflection and narrative-medicine literatures. To help educators move toward best practices, we evaluated these rubrics to determine (1) rater requirements; (2) score comparability; and (3) response to an intervention.

METHODS

One-hundred and forty-nine third-year medical students wrote reflections in response to identical prompts. Trained raters used each rubric to score 56 reflections, half written with structured guidelines and half without. We used Pearson's correlation coefficients to associate overall rubric levels and independent t-tests to compare structured and unstructured reflections.

RESULTS

Reflection-on-Action training required for two hours; two raters attained an interrater-reliability = 0.91. REFLECT training required six hours; three raters achieved an interrater-reliability = 0.84. Overall rubric correlation was 0.53. Students given structured guidelines scored significantly higher (p < 0.05) on both rubrics.

CONCLUSIONS

Reflection-on-Action and REFLECT offer unique educational benefits and training challenges. Reflection-on-Action may be preferred for measuring overall quality of reflection given its ease of use. Training on REFLECT takes longer but it yields detailed data on multiple dimensions of reflection that faculty can reference when providing feedback.

Web-based apps for reflection: a longitudinal study with hospital staff

BACKGROUND

Reflection is an important cognitive process in workplace learning; however, it occurs only rarely on its own and therefore needs additional support.

OBJECTIVE

In this study, we investigated the effect of software applications (apps) that aim to support reflection on hospital staff's actual reflection behavior. In doing so, we also analyzed the relationship between reflection and the job satisfaction of health care professionals.

METHODS

Reflective learning was introduced in the ward of a neurological hospital by providing apps that aimed to foster particular aspects of individual and collaborative reflection. Data were collected repeatedly: once before the introduction of the apps and again 2 years after the initial measure. We used a questionnaire with subjective ratings of reflection and job satisfaction. Response rates were 34.4% (167/485) for the first and 40.6% (210/517) for the second measure.

RESULTS

Collaborative reflection was increased (P=.047) after the provision of the apps (2010: mean 2.84, SD 0.72; 2012: mean 3.06, SD 0.63) in contrast to a control group of other wards of the same hospital (2010: mean 2.68, SD 0.67; 2012: mean 2.63, SD 0.68). In addition, we revealed a positive correlation between collaborative reflection and job satisfaction (r=.61, P<.001).

CONCLUSIONS

The findings provide evidence for an effect of the apps on hospital employees' reflection behavior. Apps that foster reflective learning can increase health care professionals' reflection about work experiences and support them in discussing experiences in teams or with their supervisors. The relationship between collaborative reflection and job satisfaction suggests that opportunities for joint reflection on work experiences in a hospital have further impact over and above fostering reflective learning per se. We discuss the limitations of our study and provide suggestions for both future research and the development of Web-based apps.

Medical students' reflective writing about a task-based learning experience on public health communication

BACKGROUND

Medical educators constantly face the challenge of preparing students for public health practice.

AIMS

This study aimed to analyze students' reflections to gain insight into their task-based experiences in the public health communication selective. We have also examined their self-reported learning outcomes and benefits with regard to application of public health communication.

METHOD

Each student wrote a semi-structured reflective journal about his or her experiences leading to the delivery of a public health talk by the group. Records from 41 students were content-analyzed for recurring themes and sub-themes.

RESULTS

Students reported a wide range of personal and professional issues. Their writings were characterized by a deep sense of self-awareness and social relatedness such as increased self-worth, communications skills, and collaborative learning. The learning encounter challenged assumptions, and enhanced awareness of the complexity of behaviour change Students also wrote about learning being more enjoyable and how the selective had forced them to adopt a more thoughtful stance towards knowledge acquisition and assimilation.

CONCLUSIONS

Task-based learning combined with a process for reflection holds promise as an educational strategy for teaching public health communication, and cultivating the habits of reflective practice.

Defining, illustrating and reflecting on logic analysis with an example from a professional development program

Program designers and evaluators should make a point of testing the validity of a program's intervention theory before investing either in implementation or in any type of evaluation. In this context, logic analysis can be a particularly useful option, since it can be used to test the plausibility of a program's intervention theory using scientific knowledge. Professional development in public health is one field among several that would truly benefit from logic analysis, as it appears to be generally lacking in theorization and evaluation. This article presents the application of this analysis method to an innovative public health professional development program, the Health Promotion Laboratory. More specifically, this paper aims to (1) define the logic analysis approach and differentiate it from similar evaluative methods; (2) illustrate the application of this method by a concrete example (logic analysis of a professional development program); and (3) reflect on the requirements of each phase of logic analysis, as well as on the advantages and disadvantages of such an evaluation method. Using logic analysis to evaluate the Health Promotion Laboratory showed that, generally speaking, the program's intervention theory appeared to have been well designed. By testing and critically discussing logic analysis, this article also contributes to further improving and clarifying the method.

Learning reflexively from a health promotion professional development program in Canada

In recent decades, reflexivity has received much attention in the professional education and training literature, especially in the public health and health promotion fields. Despite general agreement on the importance of reflexivity, there appears to be no consensus on how to assess reflexivity or to conceptualize the different forms developed among professionals and participants of training programs. This paper presents an analysis of the reflexivity outcomes of the Health Promotion Laboratory, an innovative professional development program aimed at supporting practice changes among health professionals by fostering competency development and reflexivity. More specifically, this paper explores the difference between two levels of reflexivity (formative and critical) and highlights some implications of each for practice. Data were collected through qualitative interviews with participants from two intervention sites. Results showed that involvement in the Health Promotion Laboratory prompted many participants to modify their vision of their practice and professional role, indicating an impact on reflexivity. In many cases, new understandings seem to have played a formative function in enabling participants to improve their practice and their role as health promoters. The reflective process also served a critical function culminating in a social and moral understanding of the impacts on society of the professionals' practices and roles. This type of outcome is greatly desired in health promotion, given the social justice and equity concerns of this field of practice. By redefining the theoretical concept of reflexivity on two levels and discussing their impacts on practice, this study supports the usefulness of both levels of reflexivity.

A pilot study to determine the validity and reliability of the Level of Reflection-on-Action Assessment

The purpose of this study was to determine the reliability of the Level of Reflection-on-Action Assessment (LORAA). The LORAA is an instrument developed to identify the level of reflection achieved by students on their reflective journal entries and includes prompts for the educator to guide students to higher levels of reflection. The LORAA was developed based on a literature review and was content validated by a panel of three experts in reflective learning. Interrater reliability was achieved when pilot tested using five sample student journal entries rated at 0.67 by three independent raters on four entries and at 0.80 with two raters on all five entries. On further testing using a larger sample of 18 student journal entries, interrater reliability was achieved at 0.94 with two raters. The results suggest that the LORAA can be used to determine the level of reflection achieved by students on reflective journal entries.

First reflections: third-year dentistry students' introduction to reflective practice

INTRODUCTION

To develop both professionally and personally, health professionals need to build essential skills in reflective practice. Educators generally agree that these skills should be developed as part of curricula for health professional students. The aims were to introduce reflective practice to third-year dentistry students and to evaluate the students' self-perceived reflective skills before and after their reflective activities.

MATERIALS AND METHODS

Using a written questionnaire, third-year dentistry students' perceptions of reflective practice were gathered before and after their first clinic placement. Students also completed two reflective writing pieces about their clinical placements.

RESULTS

The students' reflective writing pieces were allocated into one of the three categories: non-reflector, reflector and critical reflector. Content analysis of students' perceptions of reflective practice revealed nine themes. Eight main themes emerged from students' suggestions for improving their reflective abilities.

DISCUSSION AND CONCLUSION

Issues around the assessment of reflective writing are discussed, and students' suggestions for improving their reflective abilities are presented.

Reflective components in undergraduate mental health nursing curricula: some issues for consideration

After reflective practice took nursing by storm 20 years ago it is timely to re-examine its influence and implementation on undergraduate teaching. Elements of reflection in relation to learning, matters that affect students and teachers in the classroom and on clinical placements, are reiterated. Interpersonal ethical concerns and issues associated with students' emotional responses to the situations on which they reflect also are considered. We have identified a number of elements that must be considered from both the student's and educator's perspectives to ensure safe and appropriate reflection activities are used in mental health-specific or non-specific undergraduate nursing subjects.

The influence of reflection on portfolio learning in undergraduate dental education

INTRODUCTION

Disparity exists between the growing consensus about the positive effects of reflection on performance and the scarcity of empirical evidence demonstrating this effect. Portfolios are considered a useful instrument to assess and supervise competence-based education and to stimulate reflection. The present study describes the introduction of a portfolio in a social dentistry and oral health promotion course and investigates student reflection as a predictor for the acquisition of the other competences in the course.

METHODS

Fourth year undergraduate dental students (n = 110) in the course 'Society and Health' between 2008 and 2011 collected evidence in their portfolios, demonstrating the acquisition of five competences: the ability to (1) assess the oral health profile of a target group; (2) integrate theoretical models in health promotion; (3) search for and apply scientific evidence; (4) work trans-, multi- and/or trans-disciplinarily; (5) reflect on personal development. Linear regression analysis was used to investigate the predictive value of reflection on the other course related competences.

RESULTS

Reflection scores proved to significantly predict other course-related competences, when analysing all students between 2008 and 2011 and for each year separately, explaining between 10.7% and 25.5% of the variance in the other competences.

CONCLUSION

Undergraduate dental students' competences related to social dentistry and oral health promotion were significantly predicted by the reflection scores obtained in a portfolio-based context. In line with the growing consensus about the benefits of reflection for dental students and professionals, results suggest the value to further develop the integration of reflection in dental education and practice.

Reflexivity and habitus: opportunities and constraints on transformative learning

This paper will explore the relationship between Mezirow's concept of reflexivity and Bourdieu's theory of habitus in order to develop a more robust framework within which critical reflection can take place. Nurse educators have sought to close the theory practice gap through the use of critical reflection. However, we are not convinced that this has produced the depth and quality of reflection required. Furthermore, the contexts in which critical reflection takes place is often sidelined or erased so that the whole impetus in the literature is to educate nurses in reflection rather than empower nurses to understand the complex circumstances and barriers that obstruct critical reflection. This paper argues that the reason for this position is that nurse education does not always acknowledge the role that personal and cultural values systems have on reflective practices. The literature search was undertaken using CINHAL and MEDLINE. Keywords included: values systems, habitus, and critical reflection. Inclusion criteria were determined by the theoretical approach and included seminal texts, from as far back as 1956, to identify key themes. Although critical reflection is a potentially powerful way of enhancing care it has often failed to do so. It is suggested that this is because nurse educators have frequently used models of reflection without considering the impact that student's personal values systems has on their perceptions of care and subsequently care delivery. The purpose of this paper is to promote deeper levels of reflection and is part of a programme of research aimed at developing a more robust approach to reflection in educational practice.