A collaborative model for developing and maintaining virtual patients for medical education

[Blended Learning With Virtual Pediatric Emergency Patients for Medical Students]

Blended Learning With Virtual Pediatric Emergency Patients for Medical Students Abstract. Treating critically ill children is a major challenge for learners. Medical Students often feel inadequately prepared for their later role as physicians. This article describes the implementation and evaluation of blended learning using virtual patients (VP) during the student rotation at the pediatric emergency department Inselspital Bern. Students rated the project as highly beneficial and recommended its integration into the entire clinical curriculum.

Development and initial validation of an online engagement metric using virtual patients

BACKGROUND

Considerable evidence in the learning sciences demonstrates the importance of engagement in online learning environments. The purpose of this work was to demonstrate feasibility and to develop and collect initial validity evidence for a computer-generated dynamic engagement score based on student interactions in an online learning environment, in this case virtual patients used for clinical education.

METHODS

The study involved third-year medical students using virtual patient cases as a standard component of their educational program at more than 125 accredited US and Canadian medical schools. The engagement metric algorithm included four equally weighted components of student interactions with the virtual patient. We developed a self-report measure of motivational, emotional, and cognitive engagement and conducted confirmatory factor analysis to assess the validity of the survey responses. We gathered additional validity evidence through educator reviews, factor analysis of the metric, and correlations between student use of the engagement metric and self-report measures of learner engagement.

RESULTS

Confirmatory factor analysis substantiated the hypothesized four-factor structure of the survey scales. Educator reviews demonstrated a high level of agreement with content and scoring cut-points (mean Pearson correlation 0.98; mean intra-class correlation 0.98). Confirmatory factor analysis yielded an acceptable fit to a one-factor model of the engagement score components. Correlations of the engagement score with self-report measures were statistically significant and in the predicted directions.

CONCLUSIONS

We present initial validity evidence for a dynamic online engagement metric based on student interactions in a virtual patient case. We discuss potential uses of such an engagement metric including better understanding of student interactions with online learning, improving engagement through instructional design and interpretation of learning analytics output.

Evaluating the Difference between Virtual and Paper-Based Clinical Cases in Family Medicine Undergraduate Education

INTRODUCTION

A "virtual patient" is defined as a computer program which simulates real patients' cases. The aim of this study was to determine whether the inclusion of virtual patients affects the level of factual knowledge of family medicine students at the undergraduate level.

METHODS

This was a case-controlled prospective study. The students were randomly divided into experimental (EG: N = 51) and control (CG: N = 48) groups. The students in the EG were asked to practice diagnosis using virtual patients instead of the paper-based clinical cases which were solved by the students in the CG. The main observed variable in the study was knowledge of family medicine, determined by 50 multiple choice questions (MCQs) about knowledge of family medicine.

RESULTS

There were no statistically significant differences in the groups' initial knowledge. At the final assessment of knowledge, there were no statistically significant differences between the groups, but there was a statistically significant difference between their initial and final knowledge.

CONCLUSIONS

The study showed that adding virtual patient cases to the curriculum, instead of paper clinical cases, did not affect the level of factual knowledge about family medicine. Virtual patients can be used, but a significant educational outcome is not expected.

An investigation of professionalism reflected by student comments on formative virtual patient encounters

BACKGROUND

This study explored the use of virtual patient generated data by investigating the association between students' unprofessional patient summary statements, which they entered during an on-line virtual patient case, and detection of their future unprofessional behavior.

METHOD

At the USUHS, students complete a number of virtual patient encounters, including a patient summary, to meet the clerkship requirements of Internal Medicine, Family Medicine, and Pediatrics. We reviewed the summary statements of 343 students who graduated in 2012 and 2013. Each statement was rated with regard to four features: Unprofessional, Professional, Equivocal (could be construed as unprofessional), and Unanswered (students did not enter a statement). We also combined Unprofessional and Equivocal into a new category to indicate a statement receiving either rating. We then examined the associations of students' scores on these categories (i.e. whether received a particular rating or not) and Expertise score and Professionalism score reflected by a post-graduate year one (PGY-1) program director (PD) evaluation form. The PD forms contained 58 Likert-scale items designed to measure the two constructs (Expertise and Professionalism).

RESULTS

The inter-rater reliability of statements coding was high (Cohen's Kappa = .97). The measure of receiving an Unprofessional or Equivocal rating was significantly correlated with lower Expertise score (r = -.19, P < .05) as well as lower Professionalism score (r = -.17, P < .05) during PGY-1.

CONCLUSION

Incident reports and review of routine student evaluations are what most schools rely on to identify the majority of professionalism lapses. Unprofessionalism reflected in student entries may provide additional markers foreshadowing subsequent unprofessional behavior.

Using Foreign Virtual Patients With Medical Students in Germany: Are Cultural Differences Evident and Do They Impede Learning?

Background

Learning with virtual patients (VPs) is considered useful in medical education for fostering clinical reasoning. As the authoring of VPs is highly demanding, an international exchange of cases might be desirable. However, cultural differences in foreign VPs might hamper learning success.

Objective

We investigated the need for support for using VPs from the United States at a German university, with respect to language and cultural differences. Our goal was to better understand potential implementation barriers of a intercultural VP exchange.

Methods

Two VPs were presented to 30 German medical students featuring a cultural background different from German standards with respect to diagnostic and therapeutic procedures, ethical aspects, role models, and language (as identified by a cultural adaptation framework). Participants were assigned to two groups: 14 students were advised to complete the cases without further instructions (basic group), and 16 students received written explanatory supplemental information specifically with regard to cultural differences (supplement group). Using a 6-point scale (6=strongly agree), we analyzed the results of an integrated assessment of learning success as well as an evaluation of cases by the students on usefulness for learning and potential issues regarding the language and cultural background.

Results

The German students found it motivating to work with cases written in English (6-point scale, 4.5 points). The clinical relevance of the VPs was clearly recognized (6 points), and the foreign language was considered a minor problem in this context (3 points). The results of the integrated learning assessment were similar in both groups (basic 53% [SD 4] vs supplement 52% [SD 4] correct answers, P=.32). However, students using the supplemental material more readily realized culturally different diagnostic and therapeutic strategies (basic 4 vs supplement 5 points, P=.39) and were less affirmative when asked about the transferability of cases to a German context (basic 5 vs supplement 3 points, P=.048).

Conclusions

German students found English VPs to be highly clinically relevant, and they rated language problems much lower than they rated motivation to work on cases in English. This should encourage the intercultural exchange of VPs. The provision of supplemental explanatory material facilitates the recognition of cultural differences and might help prevent unexpected learning effects.

The Role for Virtual Patients in the Future of Medical Education

The medical education community is working-across disciplines and across the continuum-to address the current challenges facing the medical education system and to implement strategies to improve educational outcomes. Educational technology offers the promise of addressing these important challenges in ways not previously possible. The authors propose a role for virtual patients (VPs), which they define as multimedia, screen-based interactive patient scenarios. They believe VPs offer capabilities and benefits particularly well suited to addressing the challenges facing medical education. Well-designed, interactive VP-based learning activities can promote the deep learning that is needed to handle the rapid growth in medical knowledge. Clinically oriented learning from VPs can capture intrinsic motivation and promote mastery learning. VPs can also enhance trainees' application of foundational knowledge to promote the development of clinical reasoning, the foundation of medical practice. Although not the entire solution, VPs can support competency-based education. The data created by the use of VPs can serve as the basis for multi-institutional research that will enable the medical education community both to better understand the effectiveness of educational interventions and to measure progress toward an improved system of medical education.

A qualitative analysis of virtual patient descriptions in healthcare education based on a systematic literature review

BACKGROUND

Virtual Patients (VPs) have been in the focus of research in healthcare education for many years. The aim of our study was to analyze how virtual patients are described in the healthcare education literature, and how the identified concepts relate to each other.

METHODS

We performed a literature review and extracted 185 descriptions of virtual patients from the articles. In a qualitative content analysis approach we inductively-deductively developed categories and deducted subcategories. We constructed a concept map to illustrate these concepts and their interrelations.

RESULTS

We developed the following five main categories: Patient, Teacher, Virtual Patient, Curriculum, and Learner. The concept map includes these categories and highlights aspects such as the under-valued role of patients in shaping their virtual representation and opposing concepts, such as standardization of learner activity versus learner-centeredness.

CONCLUSIONS

The presented concept map synthesizes VP descriptions and serves as a basis for both, VP use and discussions of research topics related to virtual patients.

The Development and Preliminary Validation of a Rubric to Assess Medical Students' Written Summary Statements in Virtual Patient Cases

PURPOSE

The ability to create a concise summary statement can be assessed as a marker for clinical reasoning. The authors describe the development and preliminary validation of a rubric to assess such summary statements.

METHOD

Between November 2011 and June 2014, four researchers independently coded 50 summary statements randomly selected from a large database of medical students' summary statements in virtual patient cases to each create an assessment rubric. Through an iterative process, they created a consensus assessment rubric and applied it to 60 additional summary statements. Cronbach alpha calculations determined the internal consistency of the rubric components, intraclass correlation coefficient (ICC) calculations determined the interrater agreement, and Spearman rank-order correlations determined the correlations between rubric components. Researchers' comments describing their individual rating approaches were analyzed using content analysis.

RESULTS

The final rubric included five components: factual accuracy, appropriate narrowing of the differential diagnosis, transformation of information, use of semantic qualifiers, and a global rating. Internal consistency was acceptable (Cronbach alpha 0.771). Interrater reliability for the entire rubric was acceptable (ICC 0.891; 95% confidence interval 0.859-0.917). Spearman calculations revealed a range of correlations across cases. Content analysis of the researchers' comments indicated differences in their application of the assessment rubric.

CONCLUSIONS

This rubric has potential as a tool for feedback and assessment. Opportunities for future study include establishing interrater reliability with other raters and on different cases, designing training for raters to use the tool, and assessing how feedback using this rubric affects students' clinical reasoning skills.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

BACKGROUND

Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available.

AIMS

We examined the validity of an instrument for assessing the perception of VP design by learners.

METHODS

Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis.

RESULTS

Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors.

CONCLUSION

The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

The evolving role of online virtual patients in internal medicine clerkship education nationally

PURPOSE

Despite the significant resources required to develop and maintain virtual patient (VP) programs, little is known about why this innovation has been adopted and how it is implemented. Understanding needs and implementation strategies is important for effective curriculum planning.

METHOD

In 2009 and 2011, surveys were offered to 110 U.S. internal medicine clerkship directors regarding their goals for adoption of Simulated Internal Medicine Patient Learning Experience VPs. In 2011, respondents were asked how they implemented VPs in their curricula. Results were analyzed using chi-square and Fisher exact test.

RESULTS

Responses were obtained from 33 clerkship directors in 2009 and 45 in 2011. Comparing 2009 with 2011, improving students' knowledge (29/33 [88%] versus 40/45 [91%]), differential diagnoses (27/33 [82%] versus 38/45 [86%]), and ability to identify key findings (26/33 [79%] versus 38/45 [86%]) remained somewhat or very important reasons for adopting VPs. Meeting Liaison Committee on Medical Education ED-2 (31/33 [94%] versus 33/45 [73%], P = .011) and ED-8 requirements (25/33 [76%] versus 25/45 [56%], P = .004) declined in importance. Eight of 38 (21%) replaced a learning activity with VPs, 9/38 (24%) integrated VPs into other learning activities, and 21/38 (55%) simply added VPs onto their curricula.

CONCLUSIONS

This large, multi-institutional study reports national trends in VP adoption and integration. Meeting cognitive learning objectives remained an important reason for adopting VPs, whereas meeting regulatory requirements decreased significantly in importance. Opportunities remain for m ore systematically integrating VPs into clerkship curricula. Clarifying the changing goals may help with this process.

Language, culture and international exchange of virtual patients

BACKGROUND

Language and cultural differences could be a limiting factor for the international exchange of Virtual Patients (VPs), especially for small countries and languages of limited circulation. Our research evaluated whether it would be feasible to develop a VP based educational program in our Romanian institution, with cases in English and developed in a non-Romanian setting.

METHOD

The participants in the research comprised 4th year Romanian medical students from the Faculty of Medicine in Cluj-Napoca, Romania, with previous training exclusively in Romanian, good English proficiency and no experience with VPs. The students worked on eight VPs in two identical versions, Romanian and English. The first group (2010) of 136 students worked with four VPs developed in Cluj and the second group (2011) of 144 students with four VPs originally developed at an US University. Every student was randomly assigned two different VPs, one in Romanian and another in English. Student activity throughout the case, the diagnosis, therapeutic plan and diagnosis justification were recorded. We also compared student performance on the two VPs versions, Romanian and English and the student performance on the two sets of cases, originally developed in Romania, respectively USA.

RESULTS

We found no significant differences between the students' performance on the Romanian vs. English version of VPs. Regarding the students' performance on the two sets of cases, in those originally developed in Romania, respectively in the USA, we found a number of statistically significant differences in the students' activity through the cases. There were no statistically significant differences in the students' ability to reach the correct diagnosis and therapeutic plan.

CONCLUSION

The development of our program with VPs in English would be feasible, cost-effective and in accordance with the globalization of medical education.

Adoption of computer-assisted learning in medical education: the educators' perspective

CONTEXT

Computer-assisted learning (CAL) in medical education has been shown to be effective in the achievement of learning outcomes, but requires the input of significant resources and development time. This study examines the key elements and processes that led to the widespread adoption of a CAL program in undergraduate medical education, the Computer-assisted Learning in Paediatrics Program (CLIPP). It then considers the relative importance of elements drawn from existing theories and models for technology adoption and other studies on CAL in medical education to inform the future development, implementation and testing of CAL programs in medical education.

METHODS

The study used a mixed-methods explanatory design. All paediatric clerkship directors (CDs) using CLIPP were recruited to participate in a self-administered, online questionnaire. Semi-structured interviews were then conducted with a random sample of CDs to further explore the quantitative results.

RESULTS

Factors that facilitated adoption included CLIPP's ability to fill gaps in exposure to core clinical problems, the use of a national curriculum, development by CDs, and the meeting of CDs' desires to improve teaching and student learning. An additional facilitating factor was that little time and effort were needed to implement CLIPP within a clerkship. The quantitative findings were mostly corroborated by the qualitative findings.

CONCLUSIONS

This study indicates issues that are important in the consideration and future exploration of the development and implementation of CAL programs in medical education. The promise of CAL as a method of enhancing the process and outcomes of medical education, and its cost, increase the need for future CAL funders and developers to pay equal attention to the needs of potential adopters and the development process as they do to the content and tools in the CAL program. Important questions that remain on the optimal design, use and integration of CAL should be addressed in order to adequately inform future development. Support is needed for studies that address these critical areas.