Case-based MCQ generator: A custom ChatGPT based on published prompts in the literature for automatic item generation

WHAT IS THE EDUCATIONAL CHALLENGE?

A fundamental challenge in medical education is creating high-quality, clinically relevant multiple-choice questions (MCQs). ChatGPT-based automatic item generation (AIG) methods need well-designed prompts. However, the use of these prompts is hindered by the time-consuming process of copying and pasting, a lack of know-how among medical teachers, and the generalist nature of standard ChatGPT, which often lacks the medical context.

WHAT ARE THE PROPOSED SOLUTIONS?

The Case-based MCQ Generator, a custom GPT, addresses these challenges. It has been trained by using GPT Builder, which is a platform designed by OpenAI for customizing ChatGPT to meet specific needs, in order to allow users to generate case-based MCQs. By using this free tool for those who have ChatGPT Plus subscription, health professions educators can easily select a prompt, input a learning objective or item-specific test point, and generate clinically relevant questions.

WHAT ARE THE POTENTIAL BENEFITS TO A WIDER GLOBAL AUDIENCE?

It enhances the efficiency of MCQ generation and ensures the generation of contextually relevant questions, surpassing the capabilities of standard ChatGPT. It streamlines the MCQ creation process by integrating prompts published in medical education literature, eliminating the need for manual prompt input.

WHAT ARE THE NEXT STEPS?

Future development aims at sustainability and addressing ethical and accessibility issues. It requires regular updates, integration of new prompts from emerging health professions education literature, and a supportive digital ecosystem around the tool. Accessibility, especially for educators in low-resource countries, is vital, demanding alternative access models to overcome financial barriers.

Clinical reasoning: What do nurses, physicians, and students reason about

Clinical reasoning is a core ability in the health professions, but the term is conceptualised in multiple ways within and across professions. For interprofessional teamwork it is indispensable to recognise the differences in understanding between professions. Therefore, our aim was to investigate how nurses, physicians, and medical and nursing students define clinical reasoning. We conducted 43 semi-structured interviews with an interprofessional group from six countries and qualitatively analysed their definitions of clinical reasoning based on a coding guide. Our results showed similarities across professions, such as the emphasis on clinical skills as part of clinical reasoning. But we also revealed differences, such as a more patient-centered view and a broader understanding of the clinical reasoning concept in nurses and nursing students. The explicit sharing and discussion of differences in the understanding of clinical reasoning across health professions can provide valuable insights into the perspectives of different team members on clinical practice and education. This understanding may lead to improved interprofessional collaboration, and our study's categories and themes can serve as a basis for such discussions.

The association between quality of connections and diagnostic accuracy in student-generated concept maps for clinical reasoning education with virtual patients

Objectives

Concept maps are a learning tool that fosters clinical reasoning skills in healthcare education. They can be developed by students in combination with virtual patients to create a visual representation of the clinical reasoning process while solving a case. However, in order to optimize feedback, there is a need to better understand the role of connections between concepts in student-generated maps. Therefore, in this study we investigated whether the quality of these connections is indicative of diagnostic accuracy.

Methods

We analyzed 40 concept maps created by fifth-year medical students in the context of four virtual patients with commonly encountered diagnoses. Half of the maps were created by students who made a correct diagnosis on the first attempt; the other half were created by students who made an error in their first diagnosis. The connections in the maps were rated by two reviewers using a relational scoring system. Analysis of covariance was employed to examine the difference in mean connection scores among groups while controlling for the number of connections.

Results

There were no differences between the groups in the number of concepts or connections in the maps; however, maps made by students who made a correct first diagnosis had higher scores for the quality of connections than those created by students who made an incorrect first diagnosis (12.13 vs 9.09; p=0.03). We also observed students' general reluctance to use connections in their concept maps.

Conclusion

Our results suggest that the quality, not the quantity, of connections in concept maps is indicative of their diagnostic accuracy.

Game Elements in the Design of Simulations in Military Trauma Management Training: Protocol for a Systematic Review

BACKGROUND

Military trauma teams are commonly operating in civilian hospitals during peacetime; in a war situation they must adjust their practices to the austere conditions. Simulations can replicate austere conditions to allow training in a safe environment that tolerates errors. Gamification, understood as the use of game elements to motivate and engage learners in nongame contexts, is gaining interest in medical education and military training. Applying game elements in the design of military trauma management simulations has the potential to provide learners with active learning opportunities and prepare them for providing medical services under austere conditions. Although gamification is known for its engaging and motivational benefits, there are controversies about its pedagogical value. The controversies can be attributed to the fact that various gamification strategies may consist of a different combination of game elements, leading to different outcomes.

OBJECTIVE

This systematic review aims to understand how game elements are used in the design of simulations in military trauma management training and their reported outcomes.

METHODS

We have designed a search strategy for the purpose of the review. Two researchers will independently assess the identified studies based on the defined inclusion and exclusion criteria. The selection process will be represented using a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram. The search will be repeated and updated as necessary prior to publication of the review. Two reviewers will independently extract and manage the data for each of the articles using a structured data extraction form. Any disagreement that arises between reviewers will be resolved through discussion, and a third review author will be consulted when needed. We are going to conduct a thematic synthesis of the extracted game element descriptions. The results are going to be presented in a diagrammatic or tabular form, alongside a narrative summary. The quality of the studies will be assessed.

RESULTS

We implemented and tested the developed search strategy in May 2023. We retrieved 1168 study abstracts, which were reduced to 630 abstracts after deduplication. We have piloted the screening on 20% (126/630) of the identified abstracts in groups of 2 reviewers.

CONCLUSIONS

Although gamification has the potential to motivate learners in various ways, there is a lack of understanding about specific game elements and how they can inform instructional design in different contexts. Our findings will increase the understanding of how game elements are used in the design of simulations in military trauma management training and, thus, contribute to more effective development of future simulations.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/45969.

Developing a European longitudinal and interprofessional curriculum for clinical reasoning

Clinical reasoning is a complex and crucial ability health professions students need to acquire during their education. Despite its importance, explicit clinical reasoning teaching is not yet implemented in most health professions educational programs. Therefore, we carried out an international and interprofessional project to plan and develop a clinical reasoning curriculum with a train-the-trainer course to support educators in teaching this curriculum to students. We developed a framework and curricular blueprint. Then we created 25 student and 7 train-the-trainer learning units and we piloted 11 of these learning units at our institutions. Learners and faculty reported high satisfaction and they also provided helpful suggestions for improvements. One of the main challenges we faced was the heterogeneous understanding of clinical reasoning within and across professions. However, we learned from each other while discussing these different views and perspectives on clinical reasoning and were able to come to a shared understanding as the basis for developing the curriculum. Our curriculum fills an important gap in the availability of explicit clinical reasoning educational materials both for students and faculty and is unique with having specialists from different countries, schools, and professions. Faculty time and time for teaching clinical reasoning in existing curricula remain important barriers for implementation of clinical reasoning teaching.

Teams managing civilian and military complex trauma: What are the competencies required in austere environments and the potential of simulation technology to address them?

Surgical training in civilian hospitals may not be sufficient for managing complex trauma in a setting where such care is not commonly practiced. Understanding the challenges that civilian teams face when moving to austere environments can inform the competencies that need to be trained. The aim of this study was to explore the competencies required in austere environments for teams managing complex trauma, and how they can be trained with simulation technologies. Ethnographic field observations were conducted, and field notes were synthesized. The field notes were structured with the elements of Activity Theory to generate the teams' competencies that need to be trained. A literature review was conducted to verify the results and identify examples of relevant simulation modalities. The analysis resulted in a structured list of competencies for civilian teams to manage complex trauma in an austere environment and recommendations which simulation technologies could be used in training of those competencies based on published studies. Our study contributes to understanding the challenges that civilian teams face when operating in an austere environment. A systematized list of competencies with suggested simulation technologies directs future research to improve quality of complex trauma training in civilian and military collaboration.

Development and implementation of an online platform for curriculum mapping in medical education

Objectives

Nowadays universities face ever-increasing demands on quality of education, which is crucial from perspective of future graduates. In face of the need of constant quality improvements of medical curricula, it is important to seek strategies for their efficient management. The general trend is to develop electronic support tools to streamline the curricular design, analysis and harmonization.

Methods

Based on the requirements we have identified by the needs analysis among curriculum designers, teachers and managers at five universities involved in the Building Curriculum Infrastructure in Medical Education (BCIME) project, and evidence published in literature on curriculum development, we have developed methodological guidelines on curriculum innovations and a software-based tools that help manage, map and analyse curricula in the medical and healthcare study fields.

Results

In this paper, we share our experiences with building and implementation of EDUportfolio, an online platform developed within our consortium and intended to facilitate harmonisation and optimisation of medical outcome-based curricula. Its functionalities and outputs were verified by pilot mapping of Anatomy curricula as taught at partner universities in five European countries.

Conclusions

The visualisation and the analysis of described curriculum data using natural language processing techniques revealed both the hidden relations between curriculum building blocks and a set of overlaps and gaps in curricula. In addition, we demonstrate both the usability of the platform in the context of the involved academic environments and the capability to map and compare curricula across different institutions and different countries.

Why is it so difficult to implement a longitudinal clinical reasoning curriculum? A multicenter interview study on the barriers perceived by European health professions educators

BACKGROUND

Effective clinical reasoning is a core competency of health professionals that is necessary to assure patients' safety. Unfortunately, adoption of longitudinal clinical reasoning curricula is still infrequent. This study explores the barriers that hinder the explicit teaching of clinical reasoning from a new international perspective.

METHODS

The context of this study was a European project whose aim is to develop a longitudinal clinical reasoning curriculum. We collected data in semi-structured interviews with responders from several European countries who represent various health professions and have different backgrounds, roles and experience. We performed a qualitative content analysis of the gathered data and constructed a coding frame using a combined deductive/inductive approach. The identified themes were validated by parallel coding and in group discussions among project members.

RESULTS

A total of 29 respondents from five European countries participated in the interviews; the majority of them represent medicine and nursing sciences. We grouped the identified barriers into eight general themes: Time, Culture, Motivation, Clinical Reasoning as a Concept, Teaching, Assessment, Infrastructure and Others. Subthemes included issues with discussing errors and providing feedback, awareness of clinical reasoning teaching methods, and tensions between the groups of professionals involved.

CONCLUSIONS

This study provides an in-depth analysis of the barriers that hinder the teaching of explicit clinical reasoning. The opinions are presented from the perspective of several European higher education institutions. The identified barriers are complex and should be treated holistically due to the many interconnections between the identified barriers. Progress in implementation is hampered by the presence of reciprocal causal chains that aggravate this situation. Further research could investigate the perceptual differences between health professions regarding the barriers to clinical reasoning. The collected insights on the complexity and diversity of these barriers will help when rolling out a long-term agenda for overcoming the factors that inhibit the implementation of clinical reasoning curricula.

Clinical Reasoning Needs to Be Explicitly Addressed in Health Professions Curricula: Recommendations from a European Consortium

Clinical reasoning entails the application of knowledge and skills to collect and integrate information, typically with the goal of arriving at a diagnosis and management plan based on the patient's unique circumstances and preferences. Evidence-informed, structured, and explicit teaching and assessment of clinical reasoning in educational programs of medical and other health professions remain unmet needs. We herein summarize recommendations for clinical reasoning learning objectives (LOs), as derived from a consensus approach among European and US researchers and health professions educators. A four-step consensus approach was followed: (1) identification of a convenience sample of the most relevant and applied national LO catalogues for health professions educational programs (N = 9) from European and US countries, (2) extraction of LOs related to clinical reasoning and translation into English, (3) mapping of LOs into predefined categories developed within the Erasmus+ Developing, implementing, and disseminating an adaptive clinical reasoning curriculum for healthcare students and educators (DID-ACT) consortium, and (4) synthesis of analysis findings into recommendations for how LOs related to clinical reasoning could be presented and incorporated in LO catalogues, upon consensus. Three distinct recommendations were formulated: (1) make clinical reasoning explicit, (2) emphasize interprofessional and collaboration aspects of clinical reasoning, and (3) include aspects of teaching and assessment of clinical reasoning. In addition, the consortium understood that implementation of bilingual catalogues with English as a common language might contribute to lower heterogeneity regarding amount, structure, and level of granularity of clinical reasoning LOs across countries. These recommendations will hopefully motivate and guide initiatives towards the implementation of LOs related to clinical reasoning in existing and future LO catalogues.

Adaptation of an international virtual patient collection to the COVID-19 pandemic

The COVID-19 pandemic posed new global challenges for teaching. We met these challenges as an international collaboration by adapting a collection of virtual patients for clinical reasoning training to this novel context.

The need for longitudinal clinical reasoning teaching and assessment: Results of an international survey

Background: Clinical reasoning is a key ability essential for practising health professionals. However, little is known about the current global adoption of clinical reasoning teaching and assessment.Purpose: We aimed to provide insights into how clinical reasoning is deliberately taught and assessed in curricula worldwide and to identify needs and perceived barriers for teaching clinical reasoning to students and educators.Methods: A questionnaire was devised by an international expert group and distributed in a large international medical education community. Data were collected in 2018 and analysed using descriptive statistics. We identified themes in free-text responses using content analysis.Results: Three hundred and thirteen responses from 76 countries were collected. Most respondents were from Europe (34%). While the presence of a longitudinal clinical reasoning curriculum was only reported by 28%, 85% stated that such a curriculum was needed. The lack of awareness of the need to explicitly teach clinical reasoning was the most commonly identified barrier. For assessment, the greatest need identified was for more workplace-based assessment.Conclusions: Global respondents indicate the need to implement explicit longitudinal clinical reasoning curricula. Our findings suggest that efforts should be put into improving faculty development, including evidence-based materials on how to teach and assess clinical reasoning.

Virtual Patient Simulations in Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration

Background

Virtual patients are interactive digital simulations of clinical scenarios for the purpose of health professions education. There is no current collated evidence on the effectiveness of this form of education.

Objective

The goal of this study was to evaluate the effectiveness of virtual patients compared with traditional education, blended with traditional education, compared with other types of digital education, and design variants of virtual patients in health professions education. The outcomes of interest were knowledge, skills, attitudes, and satisfaction.

Methods

We performed a systematic review on the effectiveness of virtual patient simulations in pre- and postregistration health professions education following Cochrane methodology. We searched 7 databases from the year 1990 up to September 2018. No language restrictions were applied. We included randomized controlled trials and cluster randomized trials. We independently selected studies, extracted data, and assessed risk of bias and then compared the information in pairs. We contacted study authors for additional information if necessary. All pooled analyses were based on random-effects models.

Results

A total of 51 trials involving 4696 participants met our inclusion criteria. Furthermore, 25 studies compared virtual patients with traditional education, 11 studies investigated virtual patients as blended learning, 5 studies compared virtual patients with different forms of digital education, and 10 studies compared different design variants. The pooled analysis of studies comparing the effect of virtual patients to traditional education showed similar results for knowledge (standardized mean difference [SMD]=0.11, 95% CI −0.17 to 0.39, I²=74%, n=927) and favored virtual patients for skills (SMD=0.90, 95% CI 0.49 to 1.32, I²=88%, n=897). Studies measuring attitudes and satisfaction predominantly used surveys with item-by-item comparison. Trials comparing virtual patients with different forms of digital education and design variants were not numerous enough to give clear recommendations. Several methodological limitations in the included studies and heterogeneity contributed to a generally low quality of evidence.

Conclusions

Low to modest and mixed evidence suggests that when compared with traditional education, virtual patients can more effectively improve skills, and at least as effectively improve knowledge. The skills that improved were clinical reasoning, procedural skills, and a mix of procedural and team skills. We found evidence of effectiveness in both high-income and low- and middle-income countries, demonstrating the global applicability of virtual patients. Further research should explore the utility of different design variants of virtual patients.

Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration

Background

Virtual reality (VR) is a technology that allows the user to explore and manipulate computer-generated real or artificial three-dimensional multimedia sensory environments in real time to gain practical knowledge that can be used in clinical practice.

Objective

The aim of this systematic review was to evaluate the effectiveness of VR for educating health professionals and improving their knowledge, cognitive skills, attitudes, and satisfaction.

Methods

We performed a systematic review of the effectiveness of VR in pre- and postregistration health professions education following the gold standard Cochrane methodology. We searched 7 databases from the year 1990 to August 2017. No language restrictions were applied. We included randomized controlled trials and cluster-randomized trials. We independently selected studies, extracted data, and assessed risk of bias, and then, we compared the information in pairs. We contacted authors of the studies for additional information if necessary. All pooled analyses were based on random-effects models. We used the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach to rate the quality of the body of evidence.

Results

A total of 31 studies (2407 participants) were included. Meta-analysis of 8 studies found that VR slightly improves postintervention knowledge scores when compared with traditional learning (standardized mean difference [SMD]=0.44; 95% CI 0.18-0.69; I²=49%; 603 participants; moderate certainty evidence) or other types of digital education such as online or offline digital education (SMD=0.43; 95% CI 0.07-0.79; I²=78%; 608 participants [8 studies]; low certainty evidence). Another meta-analysis of 4 studies found that VR improves health professionals’ cognitive skills when compared with traditional learning (SMD=1.12; 95% CI 0.81-1.43; I²=0%; 235 participants; large effect size; moderate certainty evidence). Two studies compared the effect of VR with other forms of digital education on skills, favoring the VR group (SMD=0.5; 95% CI 0.32-0.69; I²=0%; 467 participants; moderate effect size; low certainty evidence). The findings for attitudes and satisfaction were mixed and inconclusive. None of the studies reported any patient-related outcomes, behavior change, as well as unintended or adverse effects of VR. Overall, the certainty of evidence according to the GRADE criteria ranged from low to moderate. We downgraded our certainty of evidence primarily because of the risk of bias and/or inconsistency.

Conclusions

We found evidence suggesting that VR improves postintervention knowledge and skills outcomes of health professionals when compared with traditional education or other types of digital education such as online or offline digital education. The findings on other outcomes are limited. Future research should evaluate the effectiveness of immersive and interactive forms of VR and evaluate other outcomes such as attitude, satisfaction, cost-effectiveness, and clinical practice or behavior change.

Uncovering the relation between clinical reasoning and diagnostic accuracy - An analysis of learner's clinical reasoning processes in virtual patients

BACKGROUND

Clinical reasoning is an important topic in healthcare training, assessment, and research. Virtual patients (VPs) are a safe environment to teach, assess and perform research on clinical reasoning and diagnostic accuracy. Our aim was to explore the details of the clinical reasoning process and diagnostic accuracy of undergraduate medical students when working with VPs using a concept mapping tool.

METHODS

Over seven months we provided access to 67 German and 30 English VPs combined with a concept mapping tool to visualize and measure the clinical reasoning process of identifying problems, differential diagnoses, recommended tests and treatment options, and composing a summary statement about a VP. A final diagnosis had to be submitted by the learners in order to conclude the VP scenario. Learners were allowed multiple attempts or could request the correct diagnosis from the system.

RESULTS

We analyzed 1,393 completed concept maps from 317 learners. We found significant differences between maps with a correct final diagnosis on one or multiple attempts and maps in which learners gave up and requested the solution from the system. These maps had lower scores, fewer summary statements, and fewer problems, differential diagnoses, tests, and treatments.

CONCLUSIONS

The different use patterns and scores between learners who had the correct final diagnosis on one or multiple attempts and those who gave up, indicate that diagnostic accuracy in the form of a correct final diagnosis on the first attempt has to be reconsidered as a sole indicator for clinical reasoning competency. For the training, assessment, and research of clinical reasoning we suggest focusing more on the details of the process to reach a correct diagnosis, rather than whether it was made in the first attempt.

Creating and validating e-cases as educational tools in general practitioners’ continuing medical education context

Objective

This study aimed at creating electronic cases (e-cases) and analyzing their validity as a diagnostic assessment tool within the context of continuing medical education (CME) to measure general practitioners’ (GPs) knowledge of common infectious disease.

Methods

We designed assessment e-cases in an electronic CME platform. The e-cases were designed to measure GPs’ knowledge about diagnosis and treatment of common infectious disease in outpatient settings. The data collected were analyzed for five forms of evidence: content, response process, internal structure, relations with other variables and consequences.

Results

A total of 46 GPs participated in the study. Among them, 87% perceived the e-cases as resembling the patients whom they visit in their everyday practice. Although attendance in this activity made 85% of the participants more cautious about prescription of antibiotics, we could not detect any statistically significant association between the assessment scores and the physicians’ previous antibiotic prescription. The diagnostic assessment with e-cases was supported by most of the elements of validity evidence, including content, response process, internal structure and consequences.

Conclusions

Overall, evidence suggests that using e-cases might be a valid diagnostic assessment CME activity to measure GPs’ knowledge of common infectious disease, but more research is necessary.

Advancing clinical reasoning in virtual patients - development and application of a conceptual framework

Background: Clinical reasoning is a complex skill students have to acquire during their education. For educators it is difficult to explain their reasoning to students, because it is partly an automatic and unconscious process. Virtual Patients (VPs) are used to support the acquisition of clinical reasoning skills in healthcare education. However, until now it remains unclear which features or settings of VPs optimally foster clinical reasoning. Therefore, our aims were to identify key concepts of the clinical reasoning process in a qualitative approach and draw conclusions on how each concept can be enhanced to advance the learning of clinical reasoning with virtual patients. Methods: We chose a grounded theory approach to identify key categories and concepts of learning clinical reasoning and develop a framework. Throughout this process, the emerging codes were discussed with a panel of interdisciplinary experts. In a second step we applied the framework to virtual patients. Results: Based on the data we identified the core category as the "multifactorial nature of learning clinical reasoning". This category is reflected in the following five main categories: Psychological Theories, Patient-centeredness, Context, Learner-centeredness, and Teaching/Assessment. Each category encompasses between four and six related concepts. Conclusions: With our approach we were able to elaborate how key categories and concepts of clinical reasoning can be applied to virtual patients. This includes aspects such as allowing learners to access a large number of VPs with adaptable levels of complexity and feedback or emphasizing dual processing, errors, and uncertainty.

Modelling Feedback in Virtual Patients: An Iterative Approach

Virtual Patients (VPs) offer learners the opportunity to practice clinical reasoning skills and have recently been integrated in Massive Open Online Courses (MOOCs). Feedback is a central part of a branched VP, allowing the learner to reflect on the consequences of their decisions and actions. However, there is insufficient guidance on how to design feedback models within VPs and especially in the context of their application in MOOCs. In this paper, we share our experiences from building a feedback model for a bladder cancer VP in a Urology MOOC, following an iterative process in three steps. Our results demonstrate how we can systematize the process of improving the quality of VP components by the application of known literature frameworks and extend them with a feedback module. We illustrate the design and re-design process and exemplify with content from our VP. Our results can act as starting point for discussions on modelling feedback in VPs and invite future research on the topic.

A Clinical Reasoning Tool for Virtual Patients: Design-Based Research Study

BACKGROUND

Clinical reasoning is a fundamental process medical students have to learn during and after medical school. Virtual patients (VP) are a technology-enhanced learning method to teach clinical reasoning. However, VP systems do not exploit their full potential concerning the clinical reasoning process; for example, most systems focus on the outcome and less on the process of clinical reasoning.

OBJECTIVES

Keeping our concept grounded in a former qualitative study, we aimed to design and implement a tool to enhance VPs with activities and feedback, which specifically foster the acquisition of clinical reasoning skills.

METHODS

We designed the tool by translating elements of a conceptual clinical reasoning learning framework into software requirements. The resulting clinical reasoning tool enables learners to build their patient's illness script as a concept map when they are working on a VP scenario. The student's map is compared with the experts' reasoning at each stage of the VP, which is technically enabled by using Medical Subject Headings, which is a comprehensive controlled vocabulary published by the US National Library of Medicine. The tool is implemented using Web technologies, has an open architecture that enables its integration into various systems through an open application program interface, and is available under a Massachusetts Institute of Technology license.

RESULTS

We conducted usability tests following a think-aloud protocol and a pilot field study with maps created by 64 medical students. The results show that learners interact with the tool but create less nodes and connections in the concept map than an expert. Further research and usability tests are required to analyze the reasons.

CONCLUSIONS

The presented tool is a versatile, systematically developed software component that specifically supports the clinical reasoning skills acquisition. It can be plugged into VP systems or used as stand-alone software in other teaching scenarios. The modular design allows an extension with new feedback mechanisms and learning analytics algorithms.

Virtual Patients in a Behavioral Medicine Massive Open Online Course (MOOC): A Qualitative and Quantitative Analysis of Participants' Perceptions

OBJECTIVE

The purpose of this article is to explore learners' perceptions of using virtual patients in a behavioral medicine Massive Open Online Course (MOOCs) and thereby describe innovative ways of disseminating knowledge in health-related areas.

METHODS

A 5-week MOOC on behavioral medicine was hosted on the edX platform. The authors developed two branched virtual patients consisting of video recordings of a live standardized patient, with multiple clinical decision points and narration unfolding depending on learners' choices. Students interacted with the virtual patients to treat stress and sleep problems. Answers to the exit survey and participant comments from the discussion forum were analyzed qualitatively and quantitatively.

RESULTS

In total, 19,236 participants enrolled in the MOOC, out of which 740 received the final certificate. The virtual patients were completed by 2317 and 1640 participants respectively. Among survey respondents (n = 442), 83.1% agreed that the virtual patient exercise was helpful. The qualitative analysis resulted in themes covering what it was like to work with the virtual patient, with subthemes on learner-centered education, emotions/eustress, game comparisons, what the participants learned, what surprised them, how confident participants felt about applying interventions in practice, suggestions for improvement, and previous experiences of virtual patients.

CONCLUSIONS

Students were enthusiastic about interacting with the virtual patients as a means to apply new knowledge about behavioral medicine interventions. The most common suggestion was to incorporate more interactive cases with various levels of complexity. Further research should include patient outcomes and focus on interprofessional aspects of learning with virtual patients in a MOOC.

Implementation of targeted temperature management after cardiac arrest in Polish intensive care units. What has changed in the last five years?

BACKGROUND

Studies conducted up to 2010 indicate the underuse of targeted temperature management (TTM) in Poland.

AIM

This study evaluated the current degree of TTM implementation in Polish intensive care units (ICUs) and analysed the implementation process since 2010.

METHODS

A telephone survey, conducted from December 2014 to July 2015, was carried out to determine the number of ICUs using TTM in patients after cardiac arrest. We collected data on the details and prevalence of TTM, and the impact of organisational and financial issues and recently published papers on its use.

RESULTS

We obtained data from 271 of 396 ICUs (68.4%). In total, 79 (29.2%) ICUs indicated TTM use and 27 (34.2%) used dedicated TTM equipment. Overall, 62% of the ICUs used TTM regardless of the cardiac arrest rhythm. Target temperatures of 32-34°C and 34.1-36°C were reached by 44.3% and 43.0% of ICUs, respectively. The duration of TTM was 12-24 h in 58.2% of the ICUs. The most common barriers to TTM implementation were a lack of dedicated devices (36.3%) and organ-isational and logistical issues (31.2%). Any influence of recently published data on TTM practice modifications was reported by only 23.4% of the ICUs.

CONCLUSIONS

Targeted temperature management is underused in Polish ICUs. There is a need for additional educational and implementation efforts supporting the translation of knowledge into clinical practice at the regional and national levels.

A qualitative analysis of virtual patient descriptions in healthcare education based on a systematic literature review

BACKGROUND

Virtual Patients (VPs) have been in the focus of research in healthcare education for many years. The aim of our study was to analyze how virtual patients are described in the healthcare education literature, and how the identified concepts relate to each other.

METHODS

We performed a literature review and extracted 185 descriptions of virtual patients from the articles. In a qualitative content analysis approach we inductively-deductively developed categories and deducted subcategories. We constructed a concept map to illustrate these concepts and their interrelations.

RESULTS

We developed the following five main categories: Patient, Teacher, Virtual Patient, Curriculum, and Learner. The concept map includes these categories and highlights aspects such as the under-valued role of patients in shaping their virtual representation and opposing concepts, such as standardization of learner activity versus learner-centeredness.

CONCLUSIONS

The presented concept map synthesizes VP descriptions and serves as a basis for both, VP use and discussions of research topics related to virtual patients.

Barriers in the implementation of the Resuscitation Guidelines: European survey of defibrillation techniques

Background

The European Resuscitation Council (ERC) Guidelines recommend providing chest compressions during defibrillator charging and using adhesive pads for defibrillation to increase the effectiveness of resuscitation. However, the most common defibrillation technique in each European country is unknown, as are the potential barriers in implementation of the guidelines. The aim of this study was to assess the techniques of defibrillation procedures performed by professional European healthcare providers and to estimate how frequently adhesive pads are used.

Methods

We sent an online questionnaire to the ERC National Representatives that contained 12 questions regarding the techniques of defibrillation and monitoring heart rhythm during cardiac arrest. We also evaluated the frequency and indications of manual paddles use.

Results

We collected questionnaires from 27 out of 33 invited ERC member countries. The response rate was 82 %. Seventeen (17/27; 63 %) declared the use of adhesive pads. The leading cause for not using adhesive pads was economic reason (9/17; 53 %). Some respondents declared resistance to using adhesive pads by healthcare providers or tradition connected with manual paddles use. We found three leading techniques of defibrillation with manual paddles: Charging paddles keeping them on the defibrillator during chest compressions being delivered (9/21; 43 %), Charging paddles keeping them on the patient chest during chest compressions being delivered (6/21; 29 %), Charging paddles on the patient chest without chest compressions (5/21; 24 %). Respondents from 11 countries declared the use of gel or electrode pastes during defibrillation with manual paddles.

Discussion

This study collected preliminary data showing how defibrillation is performed in Europe. It revealed the recommeded techniques underuse and identyfied barriers in the Resuscitation Guidelines implementation. The survey should be open to a wider group of respondents. in each country in future.

Conclusions

There are limitations and barriers in the implementation of the defibrillation technique guidelines. There are still countries where the use of adhesive pads is low due to economic and traditional reasons. There is a need for further efforts focused on guidelines implementation.

Virtual Patients in a Behavioral Medicine Massive Open Online Course (MOOC): A Case-Based Analysis of Technical Capacity and User Navigation Pathways

BACKGROUND

Massive open online courses (MOOCs) have been criticized for focusing on presentation of short video clip lectures and asking theoretical multiple-choice questions. A potential way of vitalizing these educational activities in the health sciences is to introduce virtual patients. Experiences from such extensions in MOOCs have not previously been reported in the literature.

OBJECTIVE

This study analyzes technical challenges and solutions for offering virtual patients in health-related MOOCs and describes patterns of virtual patient use in one such course. Our aims are to reduce the technical uncertainty related to these extensions, point to aspects that could be optimized for a better learner experience, and raise prospective research questions by describing indicators of virtual patient use on a massive scale.

METHODS

The Behavioral Medicine MOOC was offered by Karolinska Institutet, a medical university, on the EdX platform in the autumn of 2014. Course content was enhanced by two virtual patient scenarios presented in the OpenLabyrinth system and hosted on the VPH-Share cloud infrastructure. We analyzed web server and session logs and a participant satisfaction survey. Navigation pathways were summarized using a visual analytics tool developed for the purpose of this study.

RESULTS

The number of course enrollments reached 19,236. At the official closing date, 2317 participants (12.1% of total enrollment) had declared completing the first virtual patient assignment and 1640 (8.5%) participants confirmed completion of the second virtual patient assignment. Peak activity involved 359 user sessions per day. The OpenLabyrinth system, deployed on four virtual servers, coped well with the workload. Participant survey respondents (n=479) regarded the activity as a helpful exercise in the course (83.1%). Technical challenges reported involved poor or restricted access to videos in certain areas of the world and occasional problems with lost sessions. The visual analyses of user pathways display the parts of virtual patient scenarios that elicited less interest and may have been perceived as nonchallenging options. Analyzing the user navigation pathways allowed us to detect indications of both surface and deep approaches to the content material among the MOOC participants.

CONCLUSIONS

This study reported on first inclusion of virtual patients in a MOOC. It adds to the body of knowledge by demonstrating how a biomedical cloud provider service can ensure technical capacity and flexible design of a virtual patient platform on a massive scale. The study also presents a new way of analyzing the use of branched virtual patients by visualization of user navigation pathways. Suggestions are offered on improvements to the design of virtual patients in MOOCs.

Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning

BACKGROUND

Virtual patients (VPs) are increasingly used to train clinical reasoning. So far, no validated evaluation instruments for VP design are available.

AIMS

We examined the validity of an instrument for assessing the perception of VP design by learners.

METHODS

Three sources of validity evidence were examined: (i) Content was examined based on theory of clinical reasoning and an international VP expert team. (ii) The response process was explored in think-aloud pilot studies with medical students and in content analyses of free text questions accompanying each item of the instrument. (iii) Internal structure was assessed by exploratory factor analysis (EFA) and inter-rater reliability by generalizability analysis.

RESULTS

Content analysis was reasonably supported by the theoretical foundation and the VP expert team. The think-aloud studies and analysis of free text comments supported the validity of the instrument. In the EFA, using 2547 student evaluations of a total of 78 VPs, a three-factor model showed a reasonable fit with the data. At least 200 student responses are needed to obtain a reliable evaluation of a VP on all three factors.

CONCLUSION

The instrument has the potential to provide valid information about VP design, provided that many responses per VP are available.

Virtual patients--what are we talking about? A framework to classify the meanings of the term in healthcare education

BACKGROUND

The term "virtual patients" (VPs) has been used for many years in academic publications, but its meaning varies, leading to confusion. Our aim was to investigate and categorize the use of the term "virtual patient" and then classify its use in healthcare education.

METHODS

A literature review was conducted to determine all articles using the term "virtual patient" in the title or abstract. These articles were categorized into: Education, Clinical Procedures, Clinical Research and E-Health. All educational articles were further classified based on a framework published by Talbot et al. which was further developed using a deductive content analysis approach.

RESULTS

536 articles published between 1991 and December 2013 were included in the study. From these, 330 were categorized as educational. Classifying these showed that 37% articles used VPs in the form of Interactive Patient Scenarios. VPs in form of High Fidelity Software Simulations (19%) and Virtual Standardized Patients (16%) were also frequent. Less frequent were other forms, such as VP Games. Analyzing the literature across time shows an overall trend towards the use of Interactive Patient Scenarios as the predominant form of VPs in healthcare education.

CONCLUSIONS

The main form of educational VPs in the literature are Interactive Patient Scenarios despite rapid technical advances that would support more complex applications. The adapted classification provides a valuable model for VP developers and researchers in healthcare education to more clearly communicate the type of VP they are addressing avoiding misunderstandings.

Evaluation of three educational use cases for using Virtual Patients in Massive Open Online Courses (MOOCs): a Delphi study

Massive Open Online Courses (MOOCs) extended with Virtual Patients (VPs) may foster specific medical skills. In particular, three educational use cases have been proposed to enable interactivity and foster clinical reasoning skills training: collective evaluation of decision making in the context of uncertainty, collective repurposing of cases with division of discussion into subgroups, and computational models in short cases for flexible selection and adaptive learning with VPs. The aim of this study was to evaluate the educational strengths and weaknesses of the proposed use cases.

We went through a two-round modified Delphi process. A panel of experts was formed and asked with open-ended questions to identify the strengths and weaknesses of each use case. The obtained responses were categorized thematically; four specific aspects of the use cases were isolated. In the second phase, the panel was asked to read the collected, categorized responses and prioritize the use cases focusing on each of the four identified aspects.

Six experts participated in the process. According to their opinion, decision making in uncertain context was the most feasible in implementation and in fostering clinical reasoning skills training; cultural repurposing was judged to leverage the MOOC potential the most; and computational models in short cases were considered the most interesting use case for the learners.

The use cases were validated and prioritized; the Delphi approach brought insights into the use cases’ potential benefits, threats, and challenges.

Beyond xMOOCs in healthcare education: study of the feasibility in integrating virtual patient systems and MOOC platforms

Background. Massive Open Online Courses (MOOCs) are an emerging trend in online learning. However, their technology is not yet completely adjusted to the needs of healthcare education. Integration of Virtual Patients within MOOCs to increase interactivity and foster clinical reasoning skills training, has been discussed in the past, but not verified by a practical implementation.

Objective. To investigate the technical feasibility of integrating MOOCs with Virtual Patients for the purpose of enabling further research into the potential pedagogical benefits of this approach.

Methods. We selected OpenEdx and Open Labyrinth as representative constituents of a MOOC platform and Virtual Patient system integration. Based upon our prior experience we selected the most fundamental technical requirement to address. Grounded in the available literature we identified an e-learning standard to guide the integration. We attempted to demonstrate the feasibility of the integration by designing a “proof-of-concept” prototype. The resulting pilot implementation was subject of verification by two test cases.

Results. A Single Sign-On mechanism connecting Open Labyrinth with OpenEdx and based on the IMS LTI standard was successfully implemented and verified.

Conclusion. We investigated the technical perspective of integrating Virtual Patients with MOOCs. By addressing this crucial technical requirement we set a base for future research on the educational benefits of using virtual patients in MOOCs. This provides new opportunities for integrating specialized software in healthcare education at massive scale.

A framework for different levels of integration of computational models into web-based virtual patients

BACKGROUND

Virtual patients are increasingly common tools used in health care education to foster learning of clinical reasoning skills. One potential way to expand their functionality is to augment virtual patients' interactivity by enriching them with computational models of physiological and pathological processes.

OBJECTIVE

The primary goal of this paper was to propose a conceptual framework for the integration of computational models within virtual patients, with particular focus on (1) characteristics to be addressed while preparing the integration, (2) the extent of the integration, (3) strategies to achieve integration, and (4) methods for evaluating the feasibility of integration. An additional goal was to pilot the first investigation of changing framework variables on altering perceptions of integration.

METHODS

The framework was constructed using an iterative process informed by Soft System Methodology. The Virtual Physiological Human (VPH) initiative has been used as a source of new computational models. The technical challenges associated with development of virtual patients enhanced by computational models are discussed from the perspectives of a number of different stakeholders. Concrete design and evaluation steps are discussed in the context of an exemplar virtual patient employing the results of the VPH ARCH project, as well as improvements for future iterations.

RESULTS

The proposed framework consists of four main elements. The first element is a list of feasibility features characterizing the integration process from three perspectives: the computational modelling researcher, the health care educationalist, and the virtual patient system developer. The second element included three integration levels: basic, where a single set of simulation outcomes is generated for specific nodes in the activity graph; intermediate, involving pre-generation of simulation datasets over a range of input parameters; advanced, including dynamic solution of the model. The third element is the description of four integration strategies, and the last element consisted of evaluation profiles specifying the relevant feasibility features and acceptance thresholds for specific purposes. The group of experts who evaluated the virtual patient exemplar found higher integration more interesting, but at the same time they were more concerned with the validity of the result. The observed differences were not statistically significant.

CONCLUSIONS

This paper outlines a framework for the integration of computational models into virtual patients. The opportunities and challenges of model exploitation are discussed from a number of user perspectives, considering different levels of model integration. The long-term aim for future research is to isolate the most crucial factors in the framework and to determine their influence on the integration outcome.

Virtual patient repositories--a comparative analysis

Virtual Patients (VPs) are an important component of medical education. One way to reduce the costs for creating VPs is sharing through repositories. We conducted a literature review to identify existing repositories and analyzed the 17 included repositories in regards to the search functions and metadata they provide. Most repositories provided some metadata such as title or description, whereas other data, such as educational objectives, were less frequent. Future research could, in cooperation with the repository provider, investigate user expectations and usage patterns.

Virtual patients in massive open online courses--design implications and integration strategies

Massive Open Online Courses (MOOCs) are raising extensive attention across disciplines, while it becomes evident that rethinking of learning designs that work well in these environments is needed. In the field of medical education, where the technology of MOOCs is not widely adopted yet, we wish to investigate the potential offered by virtual patients for the purpose of clinical reasoning skills training. In this paper we describe three use case scenarios employing virtual patients' features in MOOCs: (1) collective evaluation of decision making in the context of uncertainty; (2) collective repurposing of cases and division of discussion into subgroups focusing on local variances in healthcare; (3) division of content in short cases for flexible selection and adaptive learning with virtual patients. We also present technical requirements for implementing the use case scenarios and future work plans.

Effects of introducing a voluntary virtual patient module to a basic life support with an automated external defibrillator course: a randomised trial

BACKGROUND

The concept of virtual patients (VPs) encompasses a great variety of predominantly case-based e-learning modules with different complexity and fidelity levels. Methods for effective placement of VPs in the process of medical education are sought. The aim of this study was to determine whether the introduction of a voluntary virtual patients module into a basic life support with an automated external defibrillator (BLS-AED) course improved the knowledge and skills of students taking the course.

METHODS

Half of the students were randomly assigned to an experimental group and given voluntary access to a virtual patient module consisting of six cases presenting BLS-AED knowledge and skills. Pre- and post-course knowledge tests and skills assessments were performed, as well as a survey of students' satisfaction with the VP usage. In addition, time spent using the virtual patient system, percentage of screen cards viewed and scores in the formative questions in the VP system throughout the course were traced and recorded.

RESULTS

The study was conducted over a six week period and involved 226 first year medical students. The voluntary module was used by 61 (54%) of the 114 entitled study participants. The group that used VPs demonstrated better results in knowledge acquisition and in some key BLS-AED action skills than the group without access, or those students from the experimental group deliberately not using virtual patients. Most of the students rated the combination of VPs and corresponding teaching events positively.

CONCLUSIONS

The overall positive reaction of students and encouraging results in knowledge and skills acquisition suggest that the usage of virtual patients in a BLS-AED course on a voluntary basis is feasible and should be further investigated.

New approaches to linking clinical guidelines to virtual patients

An often reported challenge of evidence-based medicine concerns increasing use of clinical guidelines in practice. One of the proposed improvements is to promote guidelines by presenting them in conjunction with virtual patients. Three approaches to linking clinical guidelines to virtual patients are presented in this paper: (1) guidelines as a source for generating virtual patients; (2) guidelines hyper-flowchart as a virtual patient progress indicator; (3) guidelines flowchart reconstruction as a learning activity in virtual patient systems. The scenarios have been preliminarily evaluated using two demonstrator applications: Bit Pathways and CASUS. Challenges and direction for further development are proposed.

Criteria to assess the quality of virtual patients

Quality assessment of virtual patients (VP) is important but still insufficiently standardized. We developed categories and criteria to assess the quality of virtual patients. These categories reflect the life cycle of a VP from the planning to the development, implementation and evaluation. Such elaborated criteria will help authors to create VPs and curriculum planners to assess the quality of implemented VPs and choose high quality VPs from repositories.

Push and pull models to manage patient consent and licensing of multimedia resources in digital repositories for case-based reasoning

Patient consents for distribution of multimedia constitute a significant element of medical case-based repositories in medicine. A technical challenge is posed by the right of patients to withdraw permission to disseminate their images or videos. A technical mechanism for spreading information about changes in multimedia usage licenses is sought. The authors gained their experience by developing and managing a large (>340 cases) repository of virtual patients within the European project eViP. The solution for dissemination of license status should reuse and extend existing metadata standards in medical education. Two methods: PUSH and PULL are described differing in the moment of update and the division of responsibilities between parties in the learning object exchange process. The authors recommend usage of the PUSH scenario because it is better adapted to legal requirements in many countries. It needs to be stressed that the solution is based on mutual trust of the exchange partners and therefore is most appropriate for use in educational alliances and consortia. It is hoped that the proposed models for exchanging consents and licensing information will become a crucial part of the technical frameworks for building case-based repositories.

Development and validation of strategies to test for interoperability of virtual patients

Interoperability of e-learning resources requires the adherence to specific standards. In the domain of virtual patients (VP) a central role is played by the MedBiquitous' MVP specification and its application profile proposed by the eViP (Electronic Virtual Patients) project. An important factor in promoting a standard is the use of metrics for assessing the conformity of the resources to the constraints imposed by the specifications. The overall aim of this study was to explore strategies to test for conformance and investigate the capabilities and limitations of automated conformance testing. A four-level scale of conformance of virtual patient packages to the eViP profile is presented, as well as two implementations of conformance testing applications. The developed tools have been tested upon level two on a sample of four VP cases acquired from the eViP repository of virtual patients.

Enabling interoperability, accessibility and reusability of virtual patients across Europe - design and implementation

Virtual Patients (VPs) have successfully been integrated into medical and healthcare curricula for a number of years. Lack of time and resources is a frequently reported problem encountered when developing VPs for teaching and learning. Consequently there is a need for cross-institutional repositories of VPs. The aims of the study were two-fold: to enable interoperability between virtual patient systems and to investigate if (and how) an application profile is implemented in four different types of VP systems. This European collaborative implementation of a blend of several specifications (Medbiquitous VP XML, Medbiquitous Healthcare LOM, and SCORM) is innovative and the study has shown a variation in how the application profile could be implemented.