Challenges and Perspectives of Computerassisted Instruction in Medical Education

Exploring students' [pre-pandemic] use and the impact of commercial-off-the-shelf learning platforms on students' national licensing exam performance: A focused review - BEME Guide No. 72

BACKGROUND

Commercial-off-the-shelf learning platforms developed for medical education (herein referred to as MedED-COTS) have emerged as a resource used by a majority of medical students to prepare for licensing examinations. As MedED-COTS proliferate and include more functions and features, there is a need for an up-to-date review to inform medical educators on (a) students' use of MedED-COTS outside the formal medical school curriculum, (b) the integration of MedED-COTS into the formal curriculum, and (c) the potential effects of MedED-COTS usage on students' national licensing exam scores in the USA.

METHODS

Due to the limited number of studies published on either the use or integration of MedED-COTS, a focused review of literature was conducted to guide future research and practice. Data extraction and quality appraisal were conducted independently by three reviewers; with disagreements resolved by a fourth reviewer. A narrative synthesis was completed to answer research questions, contextualize results, and identify trends and issues in the findings reported by the studies included in the review.

RESULTS

Results revealed consistent positive correlations between students' use of question banks and their licensing exam performance. The limited number of integration studies, combined with a number of methodological issues, makes it impossible to isolate specific effects or associations of integrated commercial resources on standardized test or course outcomes. However, consistent positive correlations, along with students' pervasive use and strong theoretical foundations explaining the results, provide evidence for integrating MedED-COTS into medical school curricula and highlight the need for further research.

CONCLUSIONS

Based on findings, we conclude that students use exam preparation materials broadly and they have a positive impact on exam results; the literature on integration of MedED-COTS into formal curriculum and the use by students of resources outside of exam preparation is scant.

Improving resident's skills in the management of circulatory shock with a knowledge-based e-learning tool

BACKGROUND

Correct clinical management of circulatory shock in emergency departments (ER) and intensive care units (ICU) is critical. In this context, the transmission of professional skills by means of the practical supervision of real cases at the point of care entails important issues that can be widely overcome with the use of computer knowledge-based e-learning tools.

OBJECTIVE

Shock-Instructor is a web-based e-learning tool implementing the already tested training program model (TPM) that uses a knowledge base about the evidence found in the clinical practice guidelines about seven types of shock. This tool is expected to reduce the learning times and to improve the skills of hospital residents with regard to both the correct application of the guidelines and patient recovery, suppressing the risks of direct interventions.

METHODS

Shock-Instructor has been used to train residents in the Emergency Department of the Hospital Clínic de Barcelona (Spain) in order to reduce the learning cycle without affecting quality. A case-base with the description of 51 cases with shock and a knowledge-base with 137 clinical rules about the treatment of shock were incorporated to the Shock-Instructor system. A group of 33 residents was involved in a randomized controlled trial to check whether the use of Shock-Instructor can significantly improve the skills of clinicians after one week of problem-based training.

RESULTS

No significant differences were found in the skill levels of the intervention (IG) and control (CG) groups prior to learning. However, we observed an improvement of the IG clinicians capacity to stabilize patients with shock in better clinical conditions (5% improvement, p = 0.004), and to reduce the risk of death in 19.52% (p = 0.004), after training. First-year residents in IG enhanced 14.3% their sensitivity in the correct application of guidelines (p = 0.01), and 14.9% the mean survival rate of their patients (p = 0.01), after being trained with Shock-Instructor for a week. Residents with specialties different from ER and ICU enhanced 16.1% their application of guidelines (p = 0.04), and 14.5% the survival rate of the cases attended (p = 0.01).

CONCLUSIONS

Web-oriented knowledge-based e-learning computer tools such as Shock-Instructor can significantly reduce the learning time of ER and ICU residents, while enhancing their assimilation of evidence-based medicine and both the survival rate and health condition of patients with shock.

Evidence of virtual patients as a facilitative learning tool on an anesthesia course

Virtual patients are computerised representations of realistic clinical cases. They were developed to teach clinical reasoning skills through delivery of multiple standardized patient cases. The anesthesia course at The Chinese University of Hong Kong developed two novel types of virtual patients, formative assessment cases studies and storyline, to teach its final year medical students on a 2 week rotational course. Acute pain management cases were used to test if these two types of virtual patient could enhance student learning. A 2 × 2 cross over study was performed in academic year 2010-2011 on 130 students divided into four groups of 32-34. Performance was evaluated by acute pain management items set within three examinations; an end of module 60-item multiple choice paper, a short answer modified essay paper and the end of year final surgery modified essay paper. The pain management case studies were found to enhanced student performance in all three examinations, whilst the storyline virtual patient had no demonstrable effect. Student-teaching evaluation questionnaires showed that the case studies were favored more than the storyline virtual patient. Login times showed that students on average logged onto the case studies for 6 h, whereas only half the students logged on and used the storyline virtual patient. Formative assessment case studies were well liked by the students and reinforced learning of clinical algorithms through repetition and feedback, whereas the educational role of the more narrative and less interactive storyline virtual patient was less clear .

Learner preferences regarding integrating, sequencing and aligning virtual patients with other activities in the undergraduate medical curriculum: A focus group study

CONTEXT

E-learning resources, such as virtual patients (VPs), can be more effective when they are integrated in the curriculum. To gain insights that can inform guidelines for the curricular integration of VPs, we explored students' perceptions of scenarios with integrated and non-integrated VPs aimed at promoting clinical reasoning skills.

METHODS

During their paediatric clerkship, 116 fifth-year medical students were given at least ten VPs embedded in eight integrated scenarios and as non-integrated add-ons. The scenarios differed in the sequencing and alignment of VPs and related educational activities, tutor involvement, number of VPs, relevance to assessment and involvement of real patients. We sought students' perceptions on the VP scenarios in focus group interviews with eight groups of 4-7 randomly selected students (n = 39). The interviews were recorded, transcribed and analysed qualitatively.

RESULTS

The analysis resulted in six themes reflecting students' perceptions of important features for effective curricular integration of VPs: (i) continuous and stable online access, (ii) increasing complexity, adapted to students' knowledge, (iii) VP-related workload offset by elimination of other activities, (iv) optimal sequencing (e.g.: lecture--1 to 2 VP(s)--tutor-led small group discussion--real patient) and (V) optimal alignment of VPs and educational activities, (vi) inclusion of VP topics in assessment.

CONCLUSIONS

The themes appear to offer starting points for the development of a framework to guide the curricular integration of VPs. Their impact needs to be confirmed by studies using quantitative controlled designs.

Integrating virtual patients into courses: follow-up seminars and perceived benefit

CONTEXT

  The use of virtual patients (VPs) suggests promising effects on student learning. However, currently empirical data on how best to use VPs in practice are scarce. More knowledge is needed regarding aspects of integrating VPs into a course, of which student acceptance is one key issue. Several authors call for looking beyond technology to see VPs in relation to the course context. The follow-up seminar is proposed as an important aspect of integration that warrants investigation.

METHODS

  A cross-sectional explanatory study was performed in a clinical clerkship introduction course at four teaching hospitals affiliated to the same medical faculty. The VP-related activities were planned collaboratively by teachers from all four settings. However, each setting employed a different strategy to follow up the activity in the course. Sixteen questionnaire items were grouped into three scales pertaining to: perceived benefit of VPs; wish for more guidance on using VPs, and wish for assessment and feedback on VPs. Scale scores were compared across the four settings, which were ranked according to the level of intensity of students' processing of cases during VP follow-up activities.

RESULTS

  The perceived benefit of VPs and their usage were higher in the two intense-use settings compared with the moderate- and low-intensity settings. The wish for more guidance was high in the low- and one of the high-intensity settings. Students in all settings displayed little interest in more assessment and feedback regarding VPs.

CONCLUSIONS

  High case processing intensity was related to positive perceptions of the benefit of VPs. However, the low interest in more assessment and feedback on the use of VPs indicates the need to clearly communicate the added value of the follow-up seminar. The findings suggest that a more intense follow-up pays off in terms of the benefit perceived by students. This study illustrates the need to consider VPs from the perspective of a holistic course design and not as isolated add-ons.

A collaborative model for developing and maintaining virtual patients for medical education

There is great interest in using computer-assisted instruction in medical education, but getting computer-assisted instruction materials used broadly is difficult to achieve. We describe a successful model for the development and maintenance of a specific type of computer-assisted instruction - virtual patients - in medical education. The collaborative model's seven key components are described and compared to other models of diffusion of innovation and curriculum development. The collaborative development model that began in one medical discipline is now extended to two additional disciplines, through partnerships with their respective clerkship director organizations. We believe that the ability to achieve broad use of virtual patients, and to transition the programs from successfully relying on grant funding to financially self-sustaining, resulted directly from the collaborative development and maintenance process. This process can be used in other learning environments and for the development of other types of computer-assisted instruction programs.

As time goes by: Stakeholder opinions on the implementation and use of a virtual patient simulation system

BACKGROUND

Stakeholder opinions on the implementation of a particular virtual patient application might prove important for decision-making and implementation efforts in general. This study aimed to capitalize on experiences originated from two post-implementation settings of Web-based Simulation of Patients (Web-SP).

METHOD

We conducted a cross-sectional survey of stakeholder opinions (39 students, teachers, course directors, and university leaders) on the implementation and use of Web-SP in Colombia and Sweden, using a mixed method approach.

RESULTS

The respondents did not show equal preference in rating the ranking of the order of importance included in the variables (Friedman's Chi square: 26.5 to 115.1, df = 6, p < 0.001, Kendall's coefficient of concordance ranging from 0.11 to 0.50). The answers provided as free comments supported the statistical findings on the importance of end-user customization, need for authenticity in software design, and use of virtual patient simulations in a curricular context, for clinical reasoning development.

CONCLUSIONS

Virtual Patient design should allow extensive editing, support case authenticity and enhance clinical reasoning abilities, in an effort for ensuring accountability and sustainable development of the field.

Web-based collaborative training of clinical reasoning: a randomized trial

BACKGROUND

Clinical reasoning skills are essential for medical practice. Problem-based collaborative learning via the internet might prove useful in imparting these skills.

AIM

This randomized study assessed whether web-based learning (WBL) is superior to face-to-face problem-based learning (PBL) in the setting of a 6-week cardio-respiratory course.

METHODS

During winter term 2007/08, all 148 fourth-year medical students enrolled in the 6-week course consented to be randomized in small groups to diagnose a patient complaining of dyspnoea either using a virtual collaborative online module or a traditional PBL session. Clinical reasoning skills were assessed by means of a key feature examination at the end of the course.

RESULTS

No significant difference between the mean scores of both study groups was detected (p = 0.843). In virtual learning groups, costs for diagnostic tests were significantly correlated to the number of contributions to online group discussions (r = 0.881; p = 0.002). Evaluation data favored traditional PBL sessions over virtual collaborative learning.

CONCLUSION

While virtual collaborative learning was as effective as traditional PBL regarding the acquisition of clinical reasoning skills, it was less well accepted than traditional PBL. Future research needs to determine the ideal format and time-point for computer-assisted learning in medical education.

Integration strategies for using virtual patients in clinical clerkships

PURPOSE

To explore students' perceptions of virtual patient use in the clinical clerkship and develop a framework to evaluate effects of different integration strategies on students' satisfaction and perceptions of learning effectiveness with this innovation.

METHOD

A prospective, multiinstitutional study was conducted at six schools' pediatric clerkships to assess the impact of integrating Web-based virtual patient cases on students' perceptions of their learning during 2004-2005 and 2005-2006. Integration strategies were designed to meet the needs of each school, and integration was scored for components of virtual patient use and elimination of other teaching methodologies. A student survey was developed, validated, and administered at the end of the clerkship to 611 students. Data were analyzed using confirmatory factor analysis and structural equation modeling.

RESULTS

A total of 545 students (89%) completed the survey. Overall student satisfaction with the virtual patients was high; students reported that they were more effective than traditional methods. The structural model demonstrated that elimination of other teaching methodologies was directly associated with perceived effectiveness of the integration strategies. A higher use score had a significant negative effect on perceived integration, but a positive effect on perceived knowledge and skills gain. Students' positive perceptions of integration directly affected their satisfaction and perception of the effectiveness of their learning.

CONCLUSIONS

Integration strategies balancing the use of virtual patients with elimination of some other requirements were significantly associated with students' satisfaction and their perceptions of improved knowledge and skills.

Virtual patients: a critical literature review and proposed next steps

CONTEXT

The opposing forces of increased training expectations and reduced training resources have greatly impacted health professions education. Virtual patients (VPs), which take the form of interactive computer-based clinical scenarios, may help to reconcile this paradox.

METHODS

We summarise research on VPs, highlight the spectrum of potential variation and identify an agenda for future research. We also critically consider the role of VPs in the educational armamentarium.

RESULTS

We propose that VPs' most unique and cost-effective function is to facilitate and assess the development of clinical reasoning. Clinical reasoning in experts involves a non-analytical process that matures through deliberate practice with multiple and varied clinical cases. Virtual patients are ideally suited to this task. Virtual patients can also be used in learner assessment, but scoring rubrics should emphasise non-analytical clinical reasoning rather than completeness of information or algorithmic approaches. Potential variations in VP design are practically limitless, yet few studies have rigorously explored design issues. More research is needed to inform instructional design and curricular integration.

CONCLUSIONS

Virtual patients should be designed and used to promote clinical reasoning skills. More research is needed to inform how to effectively use VPs.