Librarian-Led Assessment of Medical Students' Evidence-Based Medicine Competency: Facilitators and Barriers

Introduction

We must ensure, through rigorous assessment that physicians have the evidence-based medicine (EBM) skills to identify and apply the best available information to their clinical work. However, there is limited guidance on how to assess EBM competency. With a better understanding of their current role in EBM education, Health Sciences Librarians (HSLs), as experts, should be able to contribute to the assessment of medical student EBM competence. The purpose of this study is to explore the HSLs perspective on EBM assessment practices, both current state and potential future activities.

Methods

We conducted focus groups with librarians from across the United States to explore their perceptions of assessing EBM competence in medical students. Participants had been trained to be raters of EBM competence as part of a novel Objective Structured Clinical Examination (OSCE). This OSCE was just the starting point and the discussion covered topics of current EBM assessment and possibility for expanded responsibilities at their own institutions. We used a reflexive thematic analysis approach to construct themes from our conversations.

Results

We constructed eight themes in four broad categories that influence the success of librarians being able to engage in effective assessment of EBM: administrative, curricular, medical student, and librarian.

Conclusion

Our results inform medical school leadership by pointing out the modifiable factors that enable librarians to be more engaged in conducting effective assessment. They highlight the need for novel tools, like EBM OSCEs, that can address multiple barriers and create opportunities for deeper integration of librarians into assessment processes.

Exploring the use of metacognitive monitoring cues following a diagram completion intervention

Studying texts constitutes a significant part of student learning in health professions education. Key to learning from text is the ability to effectively monitor one's own cognitive performance and take appropriate regulatory steps for improvement. Inferential cues generated during a learning experience typically guide this monitoring process. It has been shown that interventions to assist learners in using comprehension cues improve their monitoring accuracy. One such intervention is having learners to complete a diagram. Little is known, however, about how learners use cues to shape their monitoring judgments. In addition, previous research has not examined the difference in cue use between categories of learners, such as good and poor monitors. This study explored the types and patterns of cues used by participants after being subjected to a diagram completion task prior to their prediction of performance (PoP). Participants' thought processes were studied by means of a think-aloud method during diagram completion and the subsequent PoP. Results suggest that relying on comprehension-specific cues may lead to a better PoP. Poor monitors relied on multiple cue types and failed to use available cues appropriately. They gave more incorrect responses and made commission errors in the diagram, which likely led to their overconfidence. Good monitors, on the other hand, utilized cues that are predictive of learning from the diagram completion task and seemed to have relied on comprehension cues for their PoP. However, they tended to be cautious in their judgement, which probably made them underestimate themselves. These observations contribute to the current understanding of the use and effectiveness of diagram completion as a cue-prompt intervention and provide direction for future research in enhancing monitoring accuracy.

Twelve tips for applying the think-aloud method to capture cognitive processes

The think-aloud method is an established technique for studying human thought (cognitive) processes. Problem-solving and decision-making are essential skills for medical professionals, and the cognitive processes underlying these skills are complex. Studying these thought processes would enable educators, clinicians, and researchers to modify or refine their approaches and interventions. The think-aloud method has been utilized for capturing cognitive processes in a variety of fields, including computer usability, sports and cognitive psychology. Medical education also recognizes thought processes as valuable data for research and education. This article aims to guide researchers and educators through the preparation and implementation of a think-aloud method to record participants' thought processes during an activity.

Dealing with Desirable Difficulties: Supporting Students to Accept, Reduce, or Silence Effort

In this writing, I summarize the insights from my keynote lecture at the annual 2023 IAMSE Meeting in Cancún, Mexico, titled "Effort is the new smart. Supporting students in the self-regulated use of desirable difficulties." I explain how self-regulated learning is challenging for many students in higher education and even more so under learning conditions that create desirable difficulties: conditions that foster long-term learning and transfer of knowledge and skills, but that are generally more effortful to engage in. I describe how the intricate relation between perceived effort and perceived learning determines students' use of desirable difficulties. Finally, I outline promising interventional approaches academic teachers can employ to support students to seek out and engage in desirable difficulties.

Basic psychological needs satisfaction as a mediator between clinical learning climate, self-regulated learning and perceived learning in the nursing education context

PURPOSE

Self-regulated learning (SRL) can enhance students' learning process. Students need support to effectively regulate their learning. However, the effect of learning climate on SRL behavior, its ultimate effect on learning and the underlying mechanisms have not yet been established. We explored these relationships using self-determination theory.

MATERIALS AND METHODS

Nursing students (N = 244) filled in questionnaires about SRL behavior, perceived learning, perceived pedagogical atmosphere and Basic Psychological Needs (BPN) satisfaction after their clinical placement. Structural equation modelling was used to test a model in which perceived pedagogical atmosphere affects SRL behavior and subsequent perceived learning through BPN satisfaction.

RESULTS

The tested model had an adequate fit (RMSEA = 0.080, SRMR = 0.051; CFI = 0.972; TLI = 0.950). A positively perceived pedagogical atmosphere contributed to SRL behavior, which was fully explained by BPN satisfaction. SRL partially mediated the contribution of pedagogical atmosphere/BPN to perceived learning.

CONCLUSIONS

A learning climate that satisfies students' BPN contributes to their SRL behavior. SRL behavior plays a positive but modest role in the relationship between climate and perceived learning. Without a culture that is supportive of learning, implementation of tools to apply SRL behavior may not be effective. Study limitations include reliance on self-report scales and the inclusion of a single discipline.

Understanding effort regulation: Comparing 'Pomodoro' breaks and self-regulated breaks

BACKGROUND

During self-study, students need to monitor and regulate mental effort to replete working memory resources and optimize learning results. Taking breaks during self-study could be an effective effort regulation strategy. However, little is known about how breaktaking relates to self-regulated learning.

AIMS

We investigated the effects of taking systematic or self-regulated breaks on mental effort, task experiences and task completion in real-life study sessions for 1 day.

SAMPLE

Eighty-seven bachelor's and master's students from a Dutch University.

METHODS

Students participated in an online intervention during their self-study. In the self-regulated-break condition (n = 35), students self-decided when to take a break; in the systematic break conditions, students took either a 6-min break after every 24-min study block (systematic-long or 'Pomodoro technique', n = 25) or a 3-min break after every 12-min study block (systematic-short, n = 27).

RESULTS

Students had longer study sessions and breaks when self-regulating. This was associated with higher levels of fatigue and distractedness, and lower levels of concentration and motivation compared to those in the systematic conditions. We found no difference between groups in invested mental effort or task completion.

CONCLUSIONS

Taking pre-determined, systematic breaks during a study session had mood benefits and appeared to have efficiency benefits (i.e., similar task completion in shorter time) over taking self-regulated breaks. Measuring how mental effort dynamically fluctuates over time and how effort spent on the learning task differs from effort spent on regulating break-taking requires further research.

Worth the Effort: the Start and Stick to Desirable Difficulties (S2D2) Framework

Desirable difficulties are learning conditions that are often experienced as effortful, but have a positive effect on learning results and transfer of knowledge and skills (Bjork & Bjork, 2011; Bjork, 1994). Learners often do not appreciate the beneficial effects of desirable difficulties, and the negative experiences of high effort and perceived low learning make them resistant to engage in desirable difficulties (Biwer et al., 2020a). This ultimately limits learning outcomes and academic achievement. With the increasing emphasis on self-regulation in education, characterized by higher learner agency and abundant choices in what, when, and how to study, the field of educational psychology is in need of theoretical and empirically testable assumptions that improve self-regulation in desirably difficult learning conditions with the aim to foster self-regulation abilities, learning outcomes, and academic achievement. Here, we present a framework that describes how to support self-regulation of effort when engaging in desirable difficulties: the “Start and Stick to Desirable Difficulties (S2D2)” framework. The framework builds on the Effort Monitoring and Regulation model (de Bruin et al., 2020). The aim of this framework is (1) to describe evidence for the central role of perceived effort and perceived learning in (dis)engagement in desirable difficulties, and (2) to review evidence on, and provide an agenda for research to improve learners’ self-regulated use of desirable difficulties to help them start and persist when learning feels tough, but is actually effective.

Supported Independence: The Role of Supervision to Help Trainees Manage Uncertainty

PURPOSE

Safe and effective supervised practice requires a negotiated partnership between trainees and their supervisors. Substantial work has explored how supervisors make judgments about trainees' readiness to safely engage in critical professional activities, yet less is known about how trainees leverage the support of supervisors when they perceive themselves to be at the limits of their abilities. The purpose of this study is to explore how trainees use supervisory support to navigate experiences of clinical uncertainty.

METHOD

Using a constructivist grounded theory approach, the authors explored how novice emergency medicine trainees conceptualized the role of their supervisors during experiences of clinical uncertainty. They employed a critical incident technique to elicit stories from participants immediately following clinical shifts between July and September 2020, and asked participants to describe their experiences of uncertainty within the context of supervised practice. Using constant comparison, 2 investigators coded line-by-line and organized these stories into focused codes. The relationships between these codes were discussed by the research team, and this enabled them to theorize about the relationships between the emergent themes.

RESULTS

Participants reported a strong desire for supported independence, where predictable and accessible supervisory structures enabled them to work semiautonomously through challenging clinical situations. They described a process of borrowing their supervisors' comfort during moments of uncertainty and mechanisms to strategically broadcast their evolving understanding of a situation to implicitly invoke (the right level of) support from their supervisors. They also highlighted challenges they faced when they felt insufficiently supported.

CONCLUSIONS

By borrowing comfort from-or deliberately projecting their thinking to-supervisors, trainees aimed to strike the appropriate balance between independence for the purposes of learning and support to ensure safety. Understanding these strategic efforts could help educators to better support trainees in their growth toward self-regulation.

Holistic processing only? The role of the right fusiform face area in radiological expertise

Radiologists can visually detect abnormalities on radiographs within 2s, a process that resembles holistic visual processing of faces. Interestingly, there is empirical evidence using functional magnetic resonance imaging (fMRI) for the involvement of the right fusiform face area (FFA) in visual-expertise tasks such as radiological image interpretation. The speed by which stimuli (e.g., faces, abnormalities) are recognized is an important characteristic of holistic processing. However, evidence for the involvement of the right FFA in holistic processing in radiology comes mostly from short or artificial tasks in which the quick, ‘holistic’ mode of diagnostic processing is not contrasted with the slower ‘search-to-find’ mode. In our fMRI study, we hypothesized that the right FFA responds selectively to the ‘holistic’ mode of diagnostic processing and less so to the ‘search-to-find’ mode. Eleven laypeople and 17 radiologists in training diagnosed 66 radiographs in 2s each (holistic mode) and subsequently checked their diagnosis in an extended (10-s) period (search-to-find mode). During data analysis, we first identified individual regions of interest (ROIs) for the right FFA using a localizer task. Then we employed ROI-based ANOVAs and obtained tentative support for the hypothesis that the right FFA shows more activation for radiologists in training versus laypeople, in particular in the holistic mode (i.e., during 2s trials), and less so in the search-to-find mode (i.e., during 10-s trials). No significant correlation was found between diagnostic performance (diagnostic accuracy) and brain-activation level within the right FFA for both, short-presentation and long-presentation diagnostic trials. Our results provide tentative evidence from a diagnostic-reasoning task that the FFA supports the holistic processing of visual stimuli in participants’ expertise domain.

Skeptical self-regulation: Resident experiences of uncertainty about uncertainty

OBJECTIVES

Managing uncertainty is central to expert practice, yet how novice trainees navigate these moments is likely different than what has been described by experienced clinicians. Exploring trainees' experiences with uncertainty could therefore help explicate the unique cues that they attend to, how they appraise their comfort in these moments and how they enact responses within the affordances of their training environment.

METHODS

Informed by constructivist grounded theory, we explored how novice emergency medicine trainees experienced and managed clinical uncertainty in practice. We used a critical incident technique to prompt participants to reflect on experiences with uncertainty immediately following a clinical shift, exploring the cues they attended to and the approaches they used to navigate these moments. Two investigators coded line-by-line using constant comparison, organising the data into focused codes. The research team discussed the relationships between these codes and developed a set of themes that supported our efforts to theorise about the phenomenon.

RESULTS

We enrolled 13 trainees in their first two years of postgraduate training across two institutions. They expressed uncertainty about the root causes of the patient problems they were facing and the potential management steps to take, but also expressed a pervasive sense of uncertainty about their own abilities and their appraisals of the situation. This, in turn, led to challenges with selecting, interpreting and using the cues in their environment effectively. Participants invoked several approaches to combat this sense of uncertainty about themselves, rehearsing steps before a clinical encounter, checking their interpretations with others and implicitly calibrating their appraisals to those of more experienced team members.

CONCLUSIONS

Trainees' struggles with the legitimacy of their interpretations impact their experiences with uncertainty. Recognising these ongoing struggles may enable supervisors and other team members to provide more effective scaffolding, validation and calibration of clinical judgments and patient management.

Does Individual Performance Feedback Increase the Use of Retrieval Practice?

The testing effect—the power of retrieval practice to enhance long-term knowledge retention more than restudying does—is a well-known phenomenon in learning. However, retrieval practice is hardly appreciated by students and underutilized when studying. One of the reasons is that learners usually do not experience immediate benefits of such practice which often present only after a delay. We therefore conducted 2 experiments to examine whether students choose retrieval practice more often as their learning strategy after having experienced its benefits. In Experiment 1, students received individual feedback about the extent to which their 7-day delayed test scores after retrieval practice differed from their test scores after restudy. Those students who had actually experienced the benefits of retrieval practice appreciated the strategy more and used it more often after receiving feedback. In Experiment 2, we compared the short-term and long-term effects on retrieval practice use of individual performance feedback and general instruction about the testing effect. Although both interventions enhanced its use in the short term, only the individual feedback led to enhanced use in the long term by those who had actually experienced its benefits, demonstrating the superiority of the individual feedback in terms of its ability to promote retrieval practice use.

Warning bells: How clinicians leverage their discomfort to manage moments of uncertainty

OBJECTIVES

It remains unclear how medical educators can more effectively bridge the gap between trainees' intolerance of uncertainty and the tolerance that experienced physicians demonstrate in practice. Exploring how experienced clinicians experience, appraise and respond to discomfort arising from uncertainty could provide new insights regarding the kinds of behaviours we are trying to help trainees achieve.

METHODS

We used a constructivist grounded theory approach to explore how emergency medicine faculty experienced, managed and responded to discomfort in settings of uncertainty. Using a critical incident technique, we asked participants to describe case-based experiences of uncertainty immediately following a clinical shift. We used probing questions to explore cognitive, emotional and somatic manifestations of discomfort, how participants had appraised and responded to these cues, and how they had used available resources to act in these moments of uncertainty. Two investigators coded the data line by line using constant comparative analysis and organised transcripts into focused codes. The entire research team discussed relationships between codes and categories, and developed a conceptual framework that reflected the possible relationships between themes.

RESULTS

Participants identified varying levels of discomfort in their case descriptions. They described multiple cues alerting them to problems that were evolving in unexpected ways or problems with aspects of management that were beyond their abilities. Discomfort served as a trigger for participants to monitor a situation with greater attention and to proceed more intentionally. It also served as a prompt for participants to think deliberately about the types of human and material resources they might call upon strategically to manage these uncertain situations.

CONCLUSIONS

Discomfort served as a dynamic means to manage and respond to uncertainty. To be 'tolerant' of uncertainty thus requires clinicians to embrace discomfort as a powerful tool with which to grapple with the complex problems pervasive in clinical practice.

"I Was Worried About the Patient, but I Wasn't Feeling Worried": How Physicians Judge Their Comfort in Settings of Uncertainty

PURPOSE

Clinical educators often raise concerns that learners are not comfortable with uncertainty in clinical work, yet existing literature provides little insight into practicing clinicians' experiences of comfort when navigating the complex, ill-defined problems pervasive in practice. Exploring clinicians' comfort as they identify and manage uncertainty in practice could help us better support learners through their discomfort.

METHOD

Between December 2018 and April 2019, the authors employed a constructivist grounded theory approach to explore experiences of uncertainty in emergency medicine faculty. The authors used a critical incident technique to elicit narratives about decision making immediately following participants' clinical shifts, exploring how they experienced uncertainty and made real-time judgments regarding their comfort to manage a given problem. Two investigators analyzed the transcripts, coding data line-by-line using constant comparative analysis to organize narratives into focused codes. These codes informed the development of conceptual categories that formed a framework for understanding comfort with uncertainty.

RESULTS

Participants identified multiple forms of uncertainty, organized around their understanding of the problems they were facing and the potential actions they could take. When discussing their comfort in these situations, they described a fluid, actively negotiated state. This state was informed by their efforts to project forward and imagine how a problem might evolve, with boundary conditions signaling the borders of their expertise. It was also informed by ongoing monitoring activities pertaining to patients, their own metacognitions, and their environment.

CONCLUSIONS

The authors' findings offer nuances to current notions of comfort with uncertainty. Uncertainty involved clinical, environmental, and social aspects, and comfort dynamically evolved through iterative cycles of forward planning and monitoring.

Synthesizing Cognitive Load and Self-regulation Theory: a Theoretical Framework and Research Agenda

An exponential increase in the availability of information over the last two decades has asked for novel theoretical frameworks to examine how students optimally learn under these new learning conditions, given the limitations of human processing ability. In this special issue and in the current editorial introduction, we argue that such a novel theoretical framework should integrate (aspects of) cognitive load theory and self-regulated learning theory. We describe the effort monitoring and regulation (EMR) framework, which outlines how monitoring and regulation of effort are neglected but essential aspects of self-regulated learning. Moreover, the EMR framework emphasizes the importance of optimizing cognitive load during self-regulated learning by reducing the unnecessary load on the primary task or distributing load optimally between the primary learning task and metacognitive aspects of the learning task. Three directions for future research that derive from the EMR framework and that are discussed in this editorial introduction are: (1) How do students monitor effort? (2) How do students regulate effort? and (3) How do we optimize cognitive load during self-regulated learning tasks (during and after the primary task)? Finally, the contributions to the current special issue are introduced.

Undergraduate Radiology Education During the COVID-19 Pandemic: A Review of Teaching and Learning Strategies [Formula: see text]

The Coronavirus disease 2019 (COVID-19) pandemic has altered how medical education is delivered, worldwide. Didactic sessions have transitioned to electronic/online platforms and clinical teaching opportunities are limited. These changes will affect how radiology is taught to medical students at both the pre-clerkship (ie, year 1 and 2) and clinical (ie, year 3 and 4) levels. In the pre-clerkship learning environment, medical students are typically exposed to radiology through didactic lectures, integrated anatomy laboratories, case-based learning, and ultrasound clinical skills sessions. In the clinical learning environment, medical students primarily shadow radiologists and radiology residents and attend radiology resident teaching sessions. These formats of radiology education, which have been the tenets of the specialty, pose significant challenges during the pandemic. This article reviews how undergraduate radiology education is affected by COVID-19 and explores solutions for teaching and learning based on e-learning and blended learning theory.

Corrigendum: How and Why Do Students Use Learning Strategies? A Mixed Methods Study on Learning Strategies and Desirable Difficulties With Effective Strategy Users

[This corrects the article DOI: 10.3389/fpsyg.2018.02501.].

Thinking fast or slow? Functional magnetic resonance imaging reveals stronger connectivity when experienced neurologists diagnose ambiguous cases

For ∼40 years, thinking about reasoning has been dominated by dual-process theories. This model, consisting of two distinct types of human reasoning, one fast and effortless and the other slow and deliberate, has also been applied to medical diagnosis. Medical experts are trained to diagnose patients based on their symptoms. When symptoms are prototypical for a certain diagnosis, practitioners may rely on fast, recognition-based reasoning. However, if they are confronted with ambiguous clinical information slower, analytical reasoning is required. To examine the neural underpinnings of these two hypothesized forms of reasoning, 16 highly experienced clinical neurologists were asked to diagnose two types of medical cases, straightforward and ambiguous cases, while functional magnetic resonance imaging was being recorded. Compared with reading control sentences, diagnosing cases resulted in increased activation in brain areas typically found to be active during reasoning such as the caudate nucleus and frontal and parietal cortical regions. In addition, we found vast increased activity in the cerebellum. Regarding the activation differences between the two types of reasoning, no pronounced differences were observed in terms of regional activation. Notable differences were observed, though, in functional connectivity: cases containing ambiguous information showed stronger connectivity between specific regions in the frontal, parietal and temporal cortex in addition to the cerebellum. Based on these results, we propose that the higher demands in terms of controlled cognitive processing during analytical medical reasoning may be subserved by stronger communication between key regions for detecting and resolving uncertainty.

Communication skills supervisors' monitoring of history-taking performance: an observational study on how doctors and non-doctors use cues to prepare feedback

BACKGROUND

Medical students need feedback to improve their patient-interviewing skills because self-monitoring is often inaccurate. Effective feedback should reveal any discrepancies between desired and observed performance (cognitive feedback) and indicate metacognitive cues which are diagnostic of performance (metacognitive feedback). We adapted a cue-utilization model to studying supervisors' cue-usage when preparing feedback and compared doctors' and non-doctors' cue usage.

METHOD

Twenty-one supervisors watched a video of a patient interview, choose scenes for feedback, and explained their selection. We applied content analysis to categorize and count cue-use frequency per communication pattern (structuring/facilitating) and scene performance rating (positive/negative) for both doctors and non-doctors.

RESULTS

Both groups used cognitive cues more often than metacognitive cues to explain their scene selection. Both groups also used metacognitive cues such as subjective feelings and mentalizing cues, but mainly the doctors mentioned 'missing information' as a cue. Compared to non-doctors, the doctors described more scenes showing negative performance and fewer scenes showing positive narrative-facilitating performance.

CONCLUSIONS

Both groups are well able to communicate their observations and provide cognitive feedback on undergraduates' interviewing skills. To improve their feedback, supervisors should be trained to also recognize metacognitive cues, such as subjective feelings and mentalizing cues, and learn how to convert both into metacognitive feedback.

Debunking myths in medical education: The science of refutation

de Bruin argues that efforts to eradicate erroneous assumptions students and teachers hold about learning requires refutation, a process that combines both correct scientific knowledge and rejection of misconceptions.

Developing the Evidence Base for M-Learning in Undergraduate Radiology Education: Identifying Learner Preferences for Mobile Apps

PURPOSE

There is a lack of evidence for developing radiology mobile apps for medical students. This study identifies the characteristics which students perceive as most valuable to teaching radiology with mobile apps (m-learning).

METHODS

An online anonymous survey was administered to second- to fourth-year medical students at a single institution. The survey, which was based on established theoretical framework, collected students' preferred content organization, content presentation, and delivery strategies. The Copeland method was used to rank student preferences and a 2-tailed t test was used to determine if student responses were related to their clinical experience, with statistical significance at P < .05.

RESULTS

The response rate was 25.6% (163/635). For content organization, image interpretation (66.9%), imaging anatomy (61.3%), and common pathological conditions (50.3%) were selected as the most important. For content presentation, quizzes (49.1%) and case presentations (46.0%) were selected as the most useful. Students with clinical experience rated algorithms as more important (P < .01) and quizzes as less important (P = .03). For delivery strategies, ease of use (92.6%), navigation (90.8%), and gestural design (74.8%) were deemed the most applicable.

CONCLUSION

This study documents medical students' preferences for m-learning in radiology. Although learner preferences are not the only feature to consider in the development of educational technology, these provide the initial framework for radiologists wishing to develop and incorporate mobile apps into their teaching.

How and Why Do Students Use Learning Strategies? A Mixed Methods Study on Learning Strategies and Desirable Difficulties With Effective Strategy Users

In order to ensure long-term retention of information students must move from relying on surface-level approaches that are seemingly effective in the short-term to “building in” so called “desirable difficulties,” with the aim of achieving understanding and long-term retention of the subject matter. But how can this level of self-regulation be achieved by students when learning? Traditionally, research on learning strategy use is performed using self-report questionnaires. As this method is accompanied by several drawbacks, we chose a qualitative, in-depth approach to inquire about students' strategies and to investigate how students successfully self-regulate their learning. In order to paint a picture of effective learning strategy use, focus groups were organized in which previously identified, effectively self-regulating students (N = 26) were asked to explain how they approach their learning. Using a constructivist grounded theory methodology, a model was constructed describing how effective strategy users manage their learning. In this model, students are driven by a personal learning goal, adopting a predominantly qualitative, or quantitative approach to learning. While learning, students are continually engaged in active processing and self-monitoring. This process is guided by a constant balancing between adhering to established study habits, while maintaining a sufficient degree of flexibility to adapt to changes in the learning environment, assessment demands, and time limitations. Indeed, students reported using several strategies, some of which are traditionally regarded as “ineffective” (highlighting, rereading etc.). However, they used them in a way that fit their learning situation. Implications are discussed for the incorporation of desirable difficulties in higher education.

Teachers' monitoring of students' text comprehension: can students' keywords and summaries improve teachers' judgment accuracy?

We investigated intra-individual monitoring and regulation in learning from text in sixth-grade students and their teachers. In Experiment 1, students provided judgments of learning (JOLs) for six texts in one of three cue-prompt conditions (after writing delayed keywords or summaries or without a cue prompt) and then selected texts for restudy. Teachers also judged their students' learning for each text, while seeing - if present - the keywords or summaries each student had written for each text, and also selected texts for restudy. Overall, monitoring accuracy was low (.10 for students, -.02 for teachers) and did not differ between cue-prompt conditions. Regulation, indexed by the correlation between JOLs and restudy selections, was significant (-.38 for students, -.60 for teachers), but was also not affected by cue-prompt condition. In Experiment 2, teachers judged students' comprehension of six texts without knowing the students' names, so that only the keywords and summaries, not prior impressions, could inform judgments. Again, monitoring accuracy was generally low (.06), but higher for keywords (.23) than for summaries (-.10). These results suggest that monitoring intra-individual differences in students' learning is challenging for teachers. Analyses of the diagnosticity and utilization of keywords suggest that these may contain insufficient cues for improving teacher judgments at this level of specificity.

Is there a superior simulator for human anatomy education? How virtual dissection can overcome the anatomic and pedagogic limitations of cadaveric dissection

Educators must select the best tools to teach anatomy to future physicians and traditionally, cadavers have always been considered the "gold standard" simulator for living anatomy. However, new advances in technology and radiology have created new teaching tools, such as virtual dissection, which provide students with new learning opportunities. Virtual dissection is a novel way of studying human anatomy through patient computed tomography (CT) scans. Through touchscreen technology, students can work together in groups to "virtually dissect" the CT scans to better understand complex anatomic relationships. This article presents the anatomic and pedagogic limitations of cadaveric dissection and explains what virtual dissection is and how this new technology may be used to overcome these limitations.

Monitoring and regulation of learning in medical education: the need for predictive cues

CONTEXT

Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research.

COGNITIVE PSYCHOLOGY

Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used.

APPLICATION TO MEDICAL EDUCATION

Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning.

CONCLUSIONS

Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is applicable to learning in several areas, including the development of clinical reasoning.

The potential of neuroscience for health sciences education: towards convergence of evidence and resisting seductive allure

Since emergence of the field 'Educational Neuroscience' (EN) in the late nineties of the previous century, a debate has emerged about the potential this field holds to influence teaching and learning in the classroom. By now, most agree that the original claims promising direct translations to teaching and learning were too strong. I argue here that research questions in (health professions) education require multi-methodological approaches, including neuroscience, while carefully weighing what (combination of) approaches are most suitable given a research question. Only through a multi-methodological approach will convergence of evidence emerge, which is so desperately needed for improving teaching and learning in the classroom. However, both researchers and teachers should become aware of the so-called 'seductive allure' of EN; that is, the demonstrable physical location and apparent objectivity of the measurements can be interpreted as yielding more powerful evidence and warranting stronger conclusions than, e.g., behavioral experiments, where in fact oftentimes the reverse is the case. I conclude that our tendency as researchers to commit ourselves to one methodological approach and to addressing educational research questions from a single methodological perspective is limiting progress in educational science and in translation to education.

Systematic viewing in radiology: seeing more, missing less?

To prevent radiologists from overlooking lesions, radiology textbooks recommend "systematic viewing," a technique whereby anatomical areas are inspected in a fixed order. This would ensure complete inspection (full coverage) of the image and, in turn, improve diagnostic performance. To test this assumption, two experiments were performed. Both experiments investigated the relationship between systematic viewing, coverage, and diagnostic performance. Additionally, the first investigated whether systematic viewing increases with expertise; the second investigated whether novices benefit from full-coverage or systematic viewing training. In Experiment 1, 11 students, ten residents, and nine radiologists inspected five chest radiographs. Experiment 2 had 75 students undergo a training in either systematic, full-coverage (without being systematic) or non-systematic viewing. Eye movements and diagnostic performance were measured throughout both experiments. In Experiment 1, no significant correlations were found between systematic viewing and coverage, r = -.10, p = .62, and coverage and performance, r = -.06, p = .74. Experts were significantly more systematic than students F2,25 = 4.35, p = .02. In Experiment 2, significant correlations were found between systematic viewing and coverage, r = -.35, p < .01, but not between coverage and performance, r = .13, p = .31. Participants in the full-coverage training performed worse compared with both other groups, which did not differ between them, F2,71 = 3.95, p = .02. In conclusion, the data question the assumption that systematic viewing leads to increased coverage, and, consequently, to improved performance. Experts inspected cases more systematically, but students did not benefit from systematic viewing training.

Why verifying diagnostic decisions with a checklist can help: insights from eye tracking

Making a diagnosis involves ratifying or verifying a proposed answer. Formalizing this verification process with checklists, which highlight key variables involved in the diagnostic decision, is often advocated. However, the mechanisms by which a checklist might allow clinicians to improve their verification process have not been well studied. We hypothesize that using a checklist to verify diagnostic decisions enhances analytic scrutiny of key variables, thereby improving clinicians' ability to find and fix mistakes. We asked 16 participants to verify their interpretation of 12 electrocardiograms, randomly assigning half to be verified with a checklist and half with an analytic prompt. While participants were verifying their interpretation, we tracked their eye movements. We analyzed these eye movements using a series of eye tracking variables theoretically linked to analytic scrutiny of key variables. We found that more errors were corrected using a checklist compared to an analytic prompt (.27 ± .53 errors per ECG vs. .04 ± .43, F 1,15 = 8.1, p = .01, η (2) = .20). Checklist use was associated with enhanced analytic scrutiny in all eye tracking measures assessed (F 6,10 = 6.0, p = .02). In this experiment, using a key variable checklist to verify diagnostic decisions improved error detection. This benefit was associated with enhanced analytic scrutiny of those key variables as measured by eye tracking.

Effects of learning content in context on knowledge acquisition and recall: a pretest-posttest control group design

BACKGROUND

It is generally assumed that learning in context increases performance. This study investigates the relationship between the characteristics of a paper-patient context (relevance and familiarity), the mechanisms through which the cognitive dimension of context could improve learning (activation of prior knowledge, elaboration and increasing retrieval cues), and test performance.

METHODS

A total of 145 medical students completed a pretest of 40 questions, of which half were with a patient vignette. One week later, they studied musculoskeletal anatomy in the dissection room without a paper-patient context (control group) or with (ir)relevant-(un)familiar context (experimental groups), and completed a cognitive load scale. Following a short delay, the students completed a posttest.

RESULTS

Surprisingly, our results show that students who studied in context did not perform better than students who studied without context. This finding may be explained by an interaction of the participants' expertise level, the nature of anatomical knowledge and students' approaches to learning. A relevant-familiar context only reduced the negative effect of learning the content in context. Our results suggest discouraging the introduction of an uncommon disease to illustrate a basic science concept. Higher self-perceived learning scores predict higher performance. Interestingly, students performed significantly better on the questions with context in both tests, possibly due to a 'framing effect'.

CONCLUSIONS

Since studies focusing on the physical and affective dimensions of context have also failed to find a positive influence of learning in a clinically relevant context, further research seems necessary to refine our theories around the role of context in learning.

Can students evaluate their understanding of cause-and-effect relations? The effects of diagram completion on monitoring accuracy

For effective self-regulated study of expository texts, it is crucial that learners can accurately monitor their understanding of cause-and-effect relations. This study aimed to improve adolescents' monitoring accuracy using a diagram completion task. Participants read six texts, predicted performance, selected texts for restudy, and were tested for comprehension. Three groups were compared, in which learners either completed causal diagrams immediately after reading, completed them after a delay, or received no-diagram control instructions. Accuracy of predictions of performance was highest for learning of causal relations following delayed diagram completion. Completing delayed diagrams focused learners specifically on their learning of causal relations, so this task did not improve monitoring of learning of factual information. When selecting texts for restudy, the participants followed their predictions of performance to the same degree, regardless of monitoring accuracy. Fine-grained analyses also showed that, when completing delayed diagrams, learners based judgments on diagnostic cues that indicated actual understanding of connections between events in the text. Most important, delayed diagram completion can improve adolescents' ability to monitor their learning of cause-and-effect relations.

Twelve tips on engaging learners in checking health care decisions

BACKGROUND

Checking diagnostic and management decisions can help reduce medical error, however, little literature explores how this is best taught.

AIMS

To provide practical advice to direct teaching practices.

METHODS

The authors conducted a literature review using Medline and PsychInfo using search terms: check or checklist and medical error or diagnostic error, supplemented by a manual search through cited literature.

CONCLUSION

Twelve tips for teaching how to check diagnostic and management decisions are presented.

Eye movements reveal differences in children's referential processing during narrative comprehension

Children differ in their ability to build referentially coherent discourse representations. Using a visual world paradigm, we investigated how these differences might emerge during the online processing of spoken discourse. We recorded eye movements of 69 children (6-11 years of age) as they listened to a 7-min story and concurrently viewed a display containing line drawings of the protagonists. Throughout the story, the protagonists were referenced by either a name (e.g., rabbit) or an anaphoric pronoun (e.g., he). Results showed that the probability of on-target fixations increased after children heard a proper name, but not after they heard an anaphoric pronoun. However, differences in the probability of on-target fixation at word onset indicate that the referents of anaphoric pronouns were anticipated by good comprehenders, but less so by poor comprehenders. These findings suggest that comprehension outcomes are related to the online processing of discourse-level cues that regulate the accessibility of entities.

Checklists improve experts' diagnostic decisions

CONTEXT

Checklists are commonly proposed tools to reduce error. However, when applied by experts, checklists have the potential to increase cognitive load and result in 'expertise reversal'. One potential solution is to use checklists in the verification stage, rather than in the initial interpretation stage of diagnostic decisions. This may avoid expertise reversal by preserving the experts' initial approach. Whether checklist use during the verification stage of diagnostic decision making improves experts' diagnostic decisions is unknown.

METHODS

Fifteen experts interpreted 18 electrocardiograms (ECGs) in four different conditions: undirected interpretation; verification without a checklist; verification with a checklist, and interpretation combined with verification with a checklist. Outcomes included the number of errors, cognitive load, interpretation time and interpretation length. Outcomes were compared in two analyses: (i) a comparison of verification conditions with and without a checklist, and (ii) a comparison of all four conditions. Standardised scores for each outcome were used to calculate the efficiency of a checklist and to weigh its relative benefit against its relative cost in terms of cognitive load imposed, interpretation time and interpretation length.

RESULTS

In both analyses, checklist use was found to reduce error (more errors were corrected in verification conditions with checklists [0.29 ± 0.77 versus 0.03 ± 0.61 errors per ECG], and fewer net errors occurred in all conditions with checklists [0.39 ± 1.14 versus 1.04 ± 1.49 errors per ECG]; p < 0.01 for both). Checklists were not associated with increased cognitive load (verifications with and without checklists: 3.7 ± 1.9 and 3.3 ± 2.0, respectively; conditions with and without checklists: 4.0 ± 1.8 versus 3.9 ± 2.0, respectively [p = not significant for both]). Checklists resulted in greater interpretation times and lengths (p < 0.01 for all). However, checklists were efficient in terms of the cognitive load invested, interpretation time and interpretation length (p < 0.01 for all).

CONCLUSIONS

Among ECG interpretation experts, checklist use during the verification stage of diagnostic decisions did not increase cognitive load or cause expertise reversal, but did reduce diagnostic error.

Constructive, collaborative, contextual, and self-directed learning in surface anatomy education

Anatomy education often consists of a combination of lectures and laboratory sessions, the latter frequently including surface anatomy. Studying surface anatomy enables students to elaborate on their knowledge of the cadaver's static anatomy by enabling the visualization of structures, especially those of the musculoskeletal system, move and function in a living human being. A recent development in teaching methods for surface anatomy is body painting, which several studies suggest increases both student motivation and knowledge acquisition. This article focuses on a teaching approach and is a translational contribution to existing literature. In line with best evidence medical education, the aim of this article is twofold: to briefly inform teachers about constructivist learning theory and elaborate on the principles of constructive, collaborative, contextual, and self-directed learning; and to provide teachers with an example of how to implement these learning principles to change the approach to teaching surface anatomy. Student evaluations of this new approach demonstrate that the application of these learning principles leads to higher student satisfaction. However, research suggests that even better results could be achieved by further adjustments in the application of contextual and self-directed learning principles. Successful implementation and guidance of peer physical examination is crucial for the described approach, but research shows that other options, like using life models, seem to work equally well. Future research on surface anatomy should focus on increasing the students' ability to apply anatomical knowledge and defining the setting in which certain teaching methods and approaches have a positive effect.

Feasibility of self-reflection as a tool to balance clinical reasoning strategies

Clinicians are believed to use two predominant reasoning strategies: system 1 based pattern recognition, and system 2 based analytical reasoning. Balancing these cognitive reasoning strategies is widely believed to reduce diagnostic error. However, clinicians approach different problems with different reasoning strategies. This study explores whether clinicians have insight into their problem specific reasoning strategy, and whether this insight can be used to balance their reasoning and reduce diagnostic error. In Experiment 1, six medical residents interpreted eight ECGs and self-reported their predominant reasoning strategy using a four point scale (4S). Self-assessed reasoning strategy correlated with objective assessment by two clinical experts using a post hoc talk-aloud protocol (ρ = 0.69, p < 0.0001). Reporting an analytic strategy was also associated with 40% longer interpretation times (p = 0.01). In Experiment 2, twenty-four residents were asked to reinterpret eight ECGs with instructions customized to their 4S. Half of the ECGs were reinterpreted with instructions to use the opposite reasoning strategy to that reported, and half with instructions to use the same reasoning strategy. ECG reinterpretation scores did not differ with potentiating compared to balancing reasoning instructions (F(1,188) = 0.22, p = 0.64). However, analytic instructions were associated with improved scores (F(1,188) = 15, p < 0.0001). These data suggest that clinicians are able to recognize their reasoning strategies. However, attempting to balance reasoning strategies through customizable instructions did not result in a reduction in diagnostic errors. This suggests important limitations to the widespread belief in balancing reasoning strategies to reduce diagnostic error.

Deliberate practice predicts performance over time in adolescent chess players and drop-outs: a linear mixed models analysis

In this study, the longitudinal relation between deliberate practice and performance in chess was examined using a linear mixed models analysis. The practice activities and performance ratings of young elite chess players, who were either in, or had dropped out of the Dutch national chess training, were analysed since they had started playing chess seriously. The results revealed that deliberate practice (i.e. serious chess study alone and serious chess play) strongly contributed to chess performance. The influence of deliberate practice was not only observable in current performance, but also over chess players' careers. Moreover, although the drop-outs' chess ratings developed more slowly over time, both the persistent and drop-out chess players benefited to the same extent from investments in deliberate practice. Finally, the effect of gender on chess performance proved to be much smaller than the effect of deliberate practice. This study provides longitudinal support for the monotonic benefits assumption of deliberate practice, by showing that over chess players' careers, deliberate practice has a significant effect on performance, and to the same extent for chess players of different ultimate performance levels. The results of this study are not in line with critique raised against the deliberate practice theory that the factors deliberate practice and talent could be confounded.

The influence of achievement motivation and chess-specific motivation on deliberate practice

Although the importance of high motivation to engage in deliberate practice has been acknowledged, no research has directly tested this hypothesis. Therefore, the present study examined this relation in adolescent elite chess players by means of a questionnaire. In addition, to provide an explanation for dropout among promising chess players, differences in motivation between persistent and dropout chess players were analyzed. Competitiveness and the will to excel proved to be predictors of investments in deliberate practice. Moreover, achievement motivation and chess-specific motivation differed to a certain extent between persisters and dropouts. Our results suggest that motivation to engage in deliberate practice not only contains elements of the will to improve performance, but also of the will to attain exceptional levels of performance.

Introduction to the special issue on innovations in problem-based learning

This special issue on Innovations in Problem-Based Learning is based on a symposium that was organized during the 11th biennial meeting of the European Association for Research on Learning and Instruction (EARLI) in Nicosia (Cyprus). Most papers in this special issue were presented at this symposium. The aim of this special issue in Advances in Health Sciences Education is to provide an overview of the state of the art in PBL research, and to discuss implications of these recent developments for educational practice. This introductory paper presents a short overview of all contributions to the special issue.

The role of basic science knowledge and clinical knowledge in diagnostic reasoning: a structural equation modeling approach

PURPOSE

To examine four theories on the role of basic science knowledge and clinical knowledge in diagnostic reasoning.

METHOD

In 2000-01, the authors tested the basic science and clinical knowledge and diagnostic performances of 59 family physicians and 184 second- to sixth-year medical students at Maastricht University, The Netherlands. Structural equation modeling was used to analyze the data. Four theoretical models were tested. In the first model only basic science knowledge is involved in diagnostic reasoning; in the second model only clinical knowledge is related to diagnostic reasoning; in the third model, clinical knowledge is related to diagnostic reasoning, but basic science knowledge is integrated in clinical knowledge; and in the fourth model, both basic science knowledge and clinical knowledge independently influence diagnostic reasoning.

RESULTS

Forty-four (75%) of the family physicians and 184 (100%) of the students responded. The results indicated that the third model, which is based on the knowledge encapsulation theory, provided the best fit to the data, whereas the models that had directly related basic science knowledge with diagnostic performance did not fit the data adequately.

CONCLUSION

The results generally supported the third model by Schmidt and Boshuizen of knowledge encapsulation theory suggesting that basic science knowledge is activated in expert diagnostic reasoning through its relation with clinical knowledge.