Improving medical students’ written communication skills: design and evaluation of an educational curriculum

How Do Learners Receive Feedback on Note Writing? A Scoping Review

PURPOSE

The literature assessing the process of note-writing based on gathered information is scant. This scoping review investigates methods of providing feedback on learners' note-writing abilities.

METHOD

Scopus and Web of Science were searched for studies that investigated feedback on student notes or reviewed notes written on an information or data-gathering activity in health care and other fields in August 2022. Of 426 articles screened, 23 met the inclusion criteria. Data were extracted on the article title, publication year, study location, study aim, study design, number of participants, participant demographics, level of education, type of note written, field of study, form of feedback given, source of the feedback, and student or participant rating of feedback method from the included articles. Then possible themes were identified and a final consensus-based thematic analysis was performed.

RESULTS

Themes identified in the 23 included articles were as follows: (1) learners found faculty and peer feedback beneficial; (2) direct written comments and evaluation tools, such as rubrics or checklists, were the most common feedback methods; (3) reports on notes in real clinical settings were limited (simulated clinical scenarios in preclinical curriculum were the most studied); (4) feedback providers and recipients benefit from having prior training on providing and receiving feedback; (5) sequential or iterative feedback was beneficial for learners but can be time intensive for faculty and confounded by maturation effects; and (6) use of technology and validated assessment tools facilitate the feedback process through ease of communication and improved organization.

CONCLUSIONS

The various factors influencing impact and perception of feedback include the source, structure, setting, use of technology, and amount of feedback provided. As the utility of note-writing in health care expands, studies are needed to clarify the value of note feedback in learning and the role of innovative technologies in facilitating note feedback.

Automated Patient Note Grading: Examining Scoring Reliability and Feasibility

PURPOSE

Scoring postencounter patient notes (PNs) yields significant insights into student performance, but the resource intensity of scoring limits its use. Recent advances in natural language processing (NLP) and machine learning allow application of automated short answer grading (ASAG) for this task. This retrospective study evaluated psychometric characteristics and reliability of an ASAG system for PNs and factors contributing to implementation, including feasibility and case-specific phrase annotation required to tune the system for a new case.

METHOD

PNs from standardized patient (SP) cases within a graduation competency exam were used to train the ASAG system, applying a feed-forward neural networks algorithm for scoring. Using faculty phrase-level annotation, 10 PNs per case were required to tune the ASAG system. After tuning, ASAG item-level ratings for 20 notes were compared across ASAG-faculty (4 cases, 80 pairings) and ASAG-nonfaculty (2 cases, 40 pairings). Psychometric characteristics were examined using item analysis and Cronbach's alpha. Inter-rater reliability (IRR) was examined using kappa.

RESULTS

ASAG scores demonstrated sufficient variability in differentiating learner PN performance and high IRR between machine and human ratings. Across all items the ASAG-faculty scoring mean kappa was .83 (SE ± .02). The ASAG-nonfaculty pairings kappa was .83 (SE ± .02). The ASAG scoring demonstrated high item discrimination. Internal consistency reliability values at the case level ranged from a Cronbach's alpha of .65 to .77. Faculty time cost to train and supervise nonfaculty raters for 4 cases was approximately $1,856. Faculty cost to tune the ASAG system was approximately $928.

CONCLUSIONS

NLP-based automated scoring of PNs demonstrated a high degree of reliability and psychometric confidence for use as learner feedback. The small number of phrase-level annotations required to tune the system to a new case enhances feasibility. ASAG-enabled PN scoring has broad implications for improving feedback in case-based learning contexts in medical education.

Twelve tips for interfacing with the new generation of medical students: iGen

iGen, or Generation Z, is the newest generation of health professions students to enter the classroom. This generation represents the first cohort of students in which technology has been present in all aspects of their lives. Since birth, they have been influenced by the boom of social media and wide-spread internet availability, leading to decreased face-to-face interactions and a desire for immediate access to information. Health professions educators should recognize the unique attributes of iGen students in order to foster student success and create a more positive learning environment. The following twelve tips examine the research-based distinctive characteristics of iGen students and highlight important concepts to consider when modifying current pedagogy to better support their needs. Incorporating these tips as an educator can promote lifelong learning and skill development for iGen students and empower this generation to thrive.

Skill Session on Writing Patient Assessments for Pediatric Clerkship Students

Introduction

Formulating written patient assessments requires the student to synthesize subjective and objective information and use clinical reasoning to reach a diagnosis. Medical students lack this skill, and clinical experience is not enough to acquire it. This session provides a structured process for learning how to formulate a patient assessment.

Methods

Third-year medical students participated in a large-group interactive skill session at the beginning of their pediatrics clerkship. Instructors following a scripted model walked students through practice examples to ultimately formulate a complete written patient assessment. The session covered data synthesis, use of appropriate medical terminology, and differential diagnosis development. Students used an audience response system to practice these skills.

Results

Over 1 academic year, 250 medical students participated in six sessions, with an average of 40-50 attendees per session. Over 90% of students completed pre- and postsession written patient assessments. These assessments were evaluated using portions of the Pediatric History and Physical Exam Evaluation grading rubric. The session had a positive effect on patient assessment formulation skills, with a significant increase in scores after the session.

Discussion

The session improved students' skill in generating more complete written patient assessments. Almost all students found the session valuable regardless of prior clinical experience. Nearly 50% of students felt inadequately prepared to formulate a written patient assessment prior to the session, revealing a skills gap for learners and a curricular teaching gap. This skill session provided a structured method and active learning format for teaching this essential clinical skill.

Duly noted: Lessons from a two-site intervention to assess and improve the quality of clinical documentation in the electronic health record

BACKGROUND

Communication errors are identified as a root cause contributing to a majority of sentinel events. The clinical note is a cornerstone of physician communication, yet there are few published interventions on teaching note writing in the electronic health record (EHR). This is a prospective, two-site, quality improvement project to assess and improve the quality of clinical documentation in the EHR using a validated assessment tool.

METHODS

Internal Medicine (IM) residents at the University of Kentucky College of Medicine (UK) and Montefiore Medical Center/Albert Einstein College of Medicine (MMC) received one of two interventions during an inpatient ward month: either a lecture, or a lecture and individual feedback on progress notes. A third group of residents in each program served as control. Notes were evaluated with the Physician Documentation Quality Instrument 9 (PDQI-9).

RESULTS

Due to a significant difference in baseline PDQI-9 scores at MMC, the sites were not combined. Of 75 residents at the UK site, 22 were eligible, 20 (91%) enrolled, 76 notes in total were scored. Of 156 residents at MMC, 22 were eligible, 18 (82%) enrolled, 40 notes in total were scored. Note quality did not improve as measured by the PDQI-9.

CONCLUSION

This educational quality improvement project did not improve the quality of clinical documentation as measured by the PDQI-9. This project underscores the difficulty in improving note quality. Further efforts should explore more effective educational tools to improve the quality of clinical documentation in the EHR.

Medicine Clerkship Implementation in a Hospitalist Group: Curricular Innovation and Review

BACKGROUND

In 2008, the Department of Hospital Medicine at Ochsner Clinic Foundation in New Orleans, LA, began training its own students for the first time as a result of the partnership between our institution and the University of Queensland (UQ) in Brisbane, Australia, that established a global medical school. The Department of Hospital Medicine is responsible for the Medicine clerkship for third-year medical students. We have 5 resident teams at the main hospital in the system, but the majority of our hospitalists work alone. Because of staffing issues, we have had to change our mentality from having teaching hospitalists and nonteaching hospitalists to viewing all hospitalists as potential educators.

METHODS

The department has slowly increased the number of students in the Medicine clerkship each year with the goal of training 120 third-year students in the New Orleans area in 2016. The students in the Medicine clerkship will be divided into five 8-week rotations, allowing for 25 students to be trained at one time.

RESULTS

The UQ curriculum is similar to that of most 4-year American schools, but some differences in methods, such as a heavy emphasis on bedside instruction and oral summative assessments, are novel to us. These differences have provided our department with new goals for professional and instructor development. For the actual instruction, we pair students one on one with hospitalists and also assign them to resident teams. Student placement has been a challenge, but we are making improvements as we gain experience and explore opportunities for placement at our community hospitals.

CONCLUSION

Our arrangement may be adapted to other institutions in the future as the number of students increases and the availability of resident teachers becomes more difficult nationwide.