How does error correction occur during lexical learning?

We examined two theories of the mechanisms that enable error correction via corrective feedback. One theory focuses on enhancing the encoding of corrective feedback (corrective feedback-encoding facilitation account). The other is the recursive reminding theory, which considers memory integration between an initial event with error generation and a subsequent event involving correct answer feedback. The Japanese idiom pronunciation task was used in two experiments, in which it was manipulated whether the generated errors were visually presented, as well as corrective feedback. In an immediate retest after a five-minute retention interval, participants recalled their errors in the initial test and their correct answers. In addition, error trials fell into three ordinal confidence categories (low, medium, and high). First, a typical hypercorrection was replicated in which higher-confidence errors are more likely to be corrected. However, this was not observed when errors from the initial test were not recalled in the final test, which does not align with the corrective feedback-encoding facilitation account. The second issue was whether additional experience with the generated errors would enhance the error correction. Given the recursive reminding theory, the additional experience of errors should reinforce the mutual dependence between an error and the correct answer provided by feedback, improving cued recall performance later. This prediction is supported. The present findings suggest that the recursive reminding theory can explain the benefits of generating errors when learning through corrective feedback and can also be expanded to understand the hypercorrection effect.

Evidence-Based Learning Strategies in Medicine Using AI

Unlabelled

Large language models (LLMs), like ChatGPT, are transforming the landscape of medical education. They offer a vast range of applications, such as tutoring (personalized learning), patient simulation, generation of examination questions, and streamlined access to information. The rapid advancement of medical knowledge and the need for personalized learning underscore the relevance and timeliness of exploring innovative strategies for integrating artificial intelligence (AI) into medical education. In this paper, we propose coupling evidence-based learning strategies, such as active recall and memory cues, with AI to optimize learning. These strategies include the generation of tests, mnemonics, and visual cues.

Guessing as a learning intervention: A meta-analytic review of the prequestion effect

Giving students test questions before they have learned the correct answers (i.e., prequestions) enhances learning. However, existing research has provided conflicting evidence on whether the benefits of prequestions are specific to the initially tested material or if they generalize to new, nontested material. In this review, we summarize the literature on the prequestion effect, describe the attention-based account underlying this effect, report a meta-analysis of the magnitude of the specific and general effects, and explore theoretically and empirically relevant moderator variables that influence the size and direction of the prequestion effect. This preregistered meta-analysis demonstrated a moderate specific effect (g = 0.54, k = 97) but a virtually nonexistent general effect (g = 0.04, k = 91). Overall, the attention-based account received support from some theoretically relevant moderator analyses. Future researchers are encouraged to conduct theoretically motivated studies to help clarify the mechanisms that underlie the attention-enhancing effects of prequestions and to explore the benefits of prequestions in educational domains to establish the extent to which these effects translate into the classroom.

Learning from errors versus explicit instruction in preparation for a test that counts

BACKGROUND

Although the generation of errors has been thought, traditionally, to impair learning, recent studies indicate that, under particular feedback conditions, the commission of errors may have a beneficial effect.

AIMS

This study investigates the teaching strategies that facilitate learning from errors.

MATERIALS AND METHODS

This 2-year study, involving two cohorts of ~88 students each, contrasted a learning-from-errors (LFE) with an explicit instruction (EI) teaching strategy in a multi-session implementation directed at improving student performance on the high-stakes New York State Algebra 1 Regents examination. In the LFE condition, instead of receiving instruction on 4 sessions, students took mini-tests. Their errors were isolated to become the focus of 4 teacher-guided feedback sessions. In the EI condition, teachers explicitly taught the mathematical material for all 8 sessions.

RESULTS

Teacher time-on in the LFE condition produced a higher rate of learning than did teacher time-on in the EI condition. The learning benefit in the LFE condition was, however, inconsistent across teachers. Second-by-second analyses of classroom activities, directed at isolating learning-relevant differences in teaching style revealed that a highly interactive mode of engaging the students in understanding their errors was more conducive to learning than was teaching directed at getting to the correct solution, either by lecturing about corrections or by interaction focused on corrections.

CONCLUSION

These results indicate that engaging the students interactively to focus on errors, and the reasons for them, facilitates productive failure and learning from errors.

Factors related to the performance of laypersons diagnosing pigmented skin cancer: an explorative study

It is important but challenging for prospective health professionals to learn the visual distinction between potentially harmful and harmless skin lesions, such as malignant melanomas and benign nevi. Knowledge about factors related to diagnostic performance is sparse but a prerequisite for designing and evaluating evidence-based educational interventions. Hence, this study explored how the characteristics of 240 skin lesions, the number of classified lesions and the response times of 137 laypeople were related to performance in diagnosing pigmented skin cancer. Our results showed large differences between the lesions, as some were classified correctly by more than 90% and others by less than 10% of the participants. A t-test showed that for melanomas, the correct diagnosis was provided significantly more often than for nevi. Furthermore, we found a significant Pearson correlation between the number of solved tasks and performance in the first 50 diagnostic tasks. Finally, t-tests for investigating the response times revealed that compared to true decisions, participants spent longer on false-negative but not on false-positive decisions. These results provide novel knowledge about performance-related factors that can be useful when designing diagnostic tests and learning interventions for melanoma detection.

The effects of procedural and conceptual knowledge on visual learning

Even though past research suggests that visual learning may benefit from conceptual knowledge, current interventions for medical image evaluation often focus on procedural knowledge, mainly by teaching classification algorithms. We compared the efficacy of pure procedural knowledge (three-point checklist for evaluating skin lesions) versus combined procedural plus conceptual knowledge (histological explanations for each of the three points). All students then trained their classification skills with a visual learning resource that included images of two types of pigmented skin lesions: benign nevi and malignant melanomas. Both treatments produced significant and long-lasting effects on diagnostic accuracy in transfer tasks. However, only students in the combined procedural plus conceptual knowledge condition significantly improved their diagnostic performance in classifying lesions they had seen before in the pre- and post-tests. Findings suggest that the provision of additional conceptual knowledge supported error correction mechanisms.

Are you sure? Examining the potential benefits of truth-checking as a learning activity

Learners may be uncertain about whether encountered information is true. Uncertainty may encourage people to critically assess information's accuracy, serving as a kind of desirable difficulty that benefits learning. Uncertainty may also have negative effects, however, leading people to mistrust true information or to later misremember false information as true. In three experiments, participants read history statements. In one condition, all statements were true, and the participants knew it. In the other two conditions, some statements were true, and others were false. Participants were either told the statements' accuracy or they guessed the statements' accuracy prior to feedback, a manipulation we refer to as truth-checking. All participants were then tested on recalling the true information and on recognising true versus false statements. We observed a significant benefit of truth-checking in one of the three experiments, suggesting that truth-checking may have some potential to enhance learning, perhaps by inducing people to encode to-be-learned information more deeply than they would otherwise. Even so, the benefit may come at a cost-truth-checking took significantly longer than study alone, and it led to a greater likelihood of thinking false information was true, suggesting costs of truth-checking may tend to outweigh benefits.

Using insights from cognitive science for the teaching of clinical skills: AMEE Guide No. 155

Students have to develop a wide variety of clinical skills, from cannulation to advanced life support, prior to entering clinical practice. An important challenge for health professions' educators is the implementation of strategies for effectively supporting students in their acquisition of different types of clinical skills and also to minimize skill decay over time. Cognitive science provides a unified approach that can inform how to maximize clinical skill acquisition and also minimize skill decay. The Guide discusses the nature of expertise and mastery development, the key insights from cognitive science for clinical skill development and skill retention, how these insights can be practically applied and integrated with current approaches used in clinical skills teaching.

Improving mastery and retention of knowledge and complex skills among sterile processing professionals: A pilot study on borescope training and competency testing

BACKGROUND

Healthcare is shifting toward minimally invasive procedures requiring increasingly complex instruments and sophisticated processing technologies. Effective training methods are needed to ensure sterile processing professionals acquire and retain essential skills.

OBJECTIVE

This study aimed to develop and evaluate a new training model that supports mastery and retention of complex key skills.

METHODS

The model was pilot-tested with training focused on visual inspection of endoscopes. Pre- and post-training tests were administered to enhance learning during a face-to-face workshop that interspersed lectures and hands-on practice, followed by structured homework, and an online booster session. Surveys assessed satisfaction and confidence levels.

RESULTS

Mean test scores for nine certified sterile processing employees increased significantly following the workshop (41% vs. 84%, p<0.001). After the workshop, all trainees identified actionable visible defects on patient-ready endoscopes in their facilities. Test scores remained high after two months (90%), and trainees reported higher technical confidence and satisfaction levels after training.

CONCLUSIONS

This study demonstrated effectiveness and clinical relevance of a new, evidence-based model for training sterile processing professionals that incorporated pre-testing, lectures, hands-on practice, a training booster, and post-testing to enhance learning. This model may be applicable to other complex skills necessary for infection prevention and patient safety.

Pre-testing effects are target-specific and are not driven by a generalised state of curiosity

Guessing an answer to an unfamiliar question prior to seeing the answer leads to better memory than studying alone (the pre-testing effect), which some theories attribute to increased curiosity. A similar effect occurs in general knowledge learning: people are more likely to recall information that they were initially curious to learn. Gruber and Ranganath [(2019). How curiosity enhances hippocampus-dependent memory: The prediction, appraisal, curiosity, and exploration (PACE) framework. Trends in Cognitive Sciences, 23(12), 1014-1025] argued that unanswered questions can cause a state of curiosity during which encoding is enhanced for the missing answer, but also for incidental information presented at the time. If pre-testing similarly induces curiosity, then it too should produce better memory for incidental information. We tested this idea in three experiments that varied the order, nature and timing of the incidental material presented within a pre-testing context. All three experiments demonstrated a reliable pre-testing effect for the targets, but no benefit for the incidental material presented before the target. This pattern suggests that the pre-testing effect is highly specific and is not consistent with a generalised state of curiosity.

Metacognitive awareness of the pretesting effect improves with self-regulation support

The pretesting or prequestion effect refers to the counterintuitive finding that taking tests on information that one has yet to learn, during which many erroneous responses typically occur, can benefit learning relative to nontesting methods (e.g., reading) if the correct answers are studied afterwards. Using a knowledge updating approach that entailed two or three cycles of pretesting versus reading followed by a criterial test, we investigated (a) the extent to which learners develop metacognitive awareness of the pretesting effect through experience (as evidenced by predictions of criterial test performance) and (b) three forms of external support-namely, performance feedback (displaying criterial test performance for pretested versus read items), prediction reminders (displaying learners' predictions alongside performance feedback), and recall prompts (asking learners to remember criterial test performance during the first cycle prior to making predictions for the second cycle)-that might improve, or provide insights into, such awareness. Across five experiments, we found that learners generally lack awareness of the memorial benefits of pretesting, are predisposed to believing that reading is more effective even after repeatedly experiencing both techniques, and need support before they recognize that pretesting is more beneficial. Overall, these results underscore the challenge of, and highlight several means of dislodging, learners' inaccurate beliefs about the efficacy of pretesting.

Attitudes of an international student cohort to the Quizlet study system employed in an advanced clinical health care review course

Computer-based learning applications and mobile technology have transformed many aspects of the educational experience over the last decade, producing software aimed at improving learning efficiency and streamlining the presentation of course materials. One such class of software, purpose-created to take advantage of spaced learning and spaced testing principles, are electronic flashcard applications. We provide a perspective on the novel use of the Quizlet flashcard application in a tertiary educational setting. To reduce cognitive load for international graduate dental students taking a pharmacology review course, we implemented Quizlet, which integrates both spaced learning and self-testing, to improve the student learning experience. This study assessed students' perceptions of the Quizlet flashcard system in a student cohort comprised of two consecutive years' classes (n = 51 students in total). Results indicated broad acceptance of Quizlet based on ease of use of the software and ease of study of the material. Our data provide insight into the use of this common software in a professional healthcare tertiary education setting and further demonstrate the successful application of electronic flashcards for a mixed international student cohort. Further research should include an assessment of the impact of flashcard on long-term knowledge retention in this setting.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s10639-022-11371-3.

Can we learn from errors? Retrieval facilitates the correction of false memories for pragmatic inferences

Errorful learning suggests that, when perfect learning has not yet been attained, errors can enhance future learning if followed by corrective feedback. Research on memory updating has shown that after retrieval, memory becomes more malleable and prone to change. Thus, retrieval of a wrong answer might provide a good context for the incorporation of feedback. Here, we tested this hypothesis using sentences including pragmatic sentence implications, commonly used for the study of false memories. Across two experiments with young adults, we hypothesized that corrective feedback would be more efficient at reducing false memories if provided immediately after retrieval, when memory is more malleable than after being exposed to the material. Participants’ memory was assessed as a function of the type of learning task (Experiment 1: retrieval vs. restudy; and Experiment 2: active vs. passive recognition); and whether participants received corrective feedback or not. In both experiments, we observed that retrieval not only improved correct recall (replicating the testing effect) but also promoted the correction of false memories. Notably, corrective feedback was more effective when given after errors that were committed during retrieval rather than after restudy (Experiment 1) or after passive recognition (Experiment 2). Our results suggest that the benefits of retrieval go beyond the testing effect since it also facilitates false memories correction. Retrieval seems to enhance memory malleability, thus improving the incorporation of feedback, compared to the mere presentation of the information. Our results support the use of learning strategies that engage in active and explicit retrieval because, even if the retrieved information is wrong—when immediate feedback is provided—memory updating is promoted and errors are more likely to be corrected.

Vocabulary Learning During Reading: Benefits of Contextual Inferences Versus Retrieval Opportunities

Retrieval practice of isolated words (e.g., with flashcards) enhances foreign vocabulary learning. However, vocabulary is often encountered in context. We investigated whether retrieval opportunities also enhance contextualized word learning. In two within‐subjects experiments, participants encoded 24 foreign words and then read a story to further strengthen word knowledge. The story contained eight target words in a retrieval context, which required participants to recall word meaning from memory to understand the text (e.g., “She borrowed aknyga”), and eight target words in an inference context from which meaning could be inferred (e.g., “She read aknyga” [book]). After 1 to 2 days, a posttest measured word retention. Reading the words in either the retrieval or inference context increased retention, compared to control words not included in the story. Moreover, in Experiment 1, retention was significantly higher in the inference than in the retrieval condition. In Experiment 2, in which encoding before reading was more extensive and feedback was available, no differences in retention were found between the inference and retrieval + feedback condition (both increased retention, compared to control words). Overall, the findings suggest that the benefits of retrieval may be less pronounced during incidental, contextualized learning than during intentional exercises and that retrieval success must be considered when adding retrieval opportunities to contextualized learning. Under low retrieval success, the better comprehension afforded by an informative context may outweigh the benefits of retrieval opportunities (Exp.1). Yet even when retrieval success was enhanced and feedback was added (Exp. 2), retrieval opportunities were only as beneficial as exposure to rich contextual information.

Interpolated testing and content pretesting as interventions to reduce task-unrelated thoughts during a video lecture

Considerable research has examined the prevalence and apparent consequences of task-unrelated thoughts (TUTs) in both laboratory and authentic educational settings. Few studies, however, have explored methods to reduce TUTs during learning; those few studies tested small samples or used unvalidated TUT assessments. The present experimental study attempted to conceptually replicate or extend previous findings of interpolated testing and pretesting effects on TUT and learning. In a study of 195 U.S. undergraduates, we investigated whether interpolated testing (compared to interpolated restudy) and pretesting on lecture-relevant materials (compared to pretesting on conceptually related but lecture-irrelevant materials) would reduce TUTs during a video lecture on introductory statistics. Subjects completed either a content-matched or content-mismatched pretest on statistics concepts and then watched a narrated lecture slideshow. During the lecture, half of the sample completed interpolated tests on the lecture material and half completed interpolated restudy of that material. All subjects responded to unpredictably presented thought probes during the video to assess their immediately preceding thoughts, including TUTs. Following the lecture, students reported on their situational interest elicited by the lecture and then completed a posttest. Interpolated testing significantly reduced TUT rates during the lecture compared to restudying, conceptually replicating previous findings—but with a small effect size and no supporting Bayes-factor evidence. We found statistical evidence for neither an interpolated testing effect on learning, nor an effect of matched-content pretesting on TUT rates or learning. Interpolated testing might have limited utility to support students’ attention, but varying effect sizes across studies warrants further work.

Scalable Science Education via Online Cooperative Questioning

A critical goal for science education is to design and implement learning activities that develop a deep conceptual understanding, are engaging for students, and are scalable for large classes or those with few resources. Approaches based on peer learning and online technologies show promise for scalability but often lack a grounding in cognitive learning principles relating to conceptual understanding. Here, we present a novel design for combining these elements in a principled way. The design centers on having students author multiple-choice questions for their peers using the online platform PeerWise, where beneficial forms of cognitive engagement are encouraged via a series of supporting activities. We evaluated an implementation of this design within a cohort of 632 students in an undergraduate biochemistry course. Our results show a robust relationship between the quality of question authoring and relevant learning outcomes, even after controlling for the confounding influence of prior grades. We conclude by discussing practical and theoretical implications.

A Retrospective Analysis of Pre-/Post-Test Scores of Students Participating in Online Asynchronous Shelter Surgery Coursework

The University of Pennsylvania School of Veterinary Medicine first offered the elective Student Shelter Opportunities I (SSOI) course in 2016 to provide pre-clinical students with an opportunity to engage with shelter medicine and high-quality, high-volume surgery (HQHVS) concepts. The course utilized online asynchronous coursework to deliver content that was completed on a self-guided timeline by students. With most of the veterinary medical curriculum delivered in a traditional classroom format, it is important to assess learning in this unique course format. There is also limited information on educational experiences in online shelter medicine coursework. This retrospective study aimed to evaluate student learning in the asynchronous online portion of the SSOI elective course using paired pre- and post-test scores from a multiple-choice type assessment. The study investigated how students' pre-test and post-test scores compared and whether time to completion of material influenced student assessment performance. Paired assessments from 400 students were analyzed, and a statistically significant increase was found in post-test scores compared to pre-test following completion of the online coursework (p < .001). There was no significant difference in the mean change in score from pre-test to post-test for students who completed the online course material in 30 days or less compared to those who completed it in greater than 30 days. This study's findings support online asynchronous learning as an effective option to teach veterinary students and can be considered in the development of veterinary coursework, including for curricular adjustments to increase online learning during the SARS-CoV-2 pandemic.

Unraveling the benefits of experiencing errors during learning: Definition, modulating factors, and explanatory theories

Making errors is part of human nature, and it is thus important to know how to get the best out of them. Experimental evidence has shown that generating errors can enhance learning when these are followed by corrective feedback. However, little is known about the specific conditions and mechanisms that underlie this benefit of experiencing errors. This review aimed to shed some light on this type of learning. First, we highlight certain conditions that may influence errorful learning. These include the timing of corrective feedback, error types, learner awareness about errorful learning, motivation to learn the study material, differences in special populations (e.g., amnesia), incidental versus intentional encoding, the importance of selecting an appropriate final test procedure, whether the study material needs to be semantically related, and if it is necessary to recover the previous errors at the time of retrieval. We then consider four explanatory theories of errorful learning: (1) The Mediator Effectiveness hypothesis, (2) the Search Set theory, (3) the Recursive Reminding theory, and (4) the Error Prediction theory. According to these theories, two factors are decisive for observing the benefits of errorful learning: the level of a pre-existing semantic relationship between the study materials, and whether the error must be explicitly recovered on the final test. To conclude, we discuss some limitations of using a pretesting procedure to study errorful learning and we reflect on further research. This review brings us closer to understanding why experiencing errors confers a memory advantage.

Prior episodic learning and the efficacy of retrieval practice

In three experiments we investigated how the level of study-based, episodic knowledge influences the efficacy of subsequent retrieval practice (testing) as a learning event. Possibilities are that the efficacy of a test, relative to a restudy control, decreases, increases, or is independent of the degree of prior study-based learning. The degree of study-based learning was manipulated by varying the number of item repetitions in the initial study phase between one and eight. Predictions of the dual-memory model of test-enhanced learning for the case of one study-phase repetition were used as a reference. Results support the hypothesis that the advantage of testing over restudy is independent of the degree of prior episodic learning, and they suggest that educators can apply cued-recall testing with the expectation that its efficacy is similar across varying levels of prior content learning. Implications for testing effect theory are discussed.

Effectiveness of pediatric learning modules in enhancing competencies among nurses in a specialized unit

The global shortage of highly competent nurses has forced healthcare organizations to adapt to various strategies, including rotating nurses to different units, impacting their competency levels. To equip nurses with the needed competencies, the researchers developed the pediatric learning modules and conducted this study to analyze their effectiveness. Researchers employed purposive and random sampling to identify study participants, and utilized a mixed-method design, including Solomon Four Group Design and a narrative approach. A researcher-made evaluation questionnaire, competency checklists, and guide questions for a written narrative were used to collect data. Descriptive statistics and factorial analysis of covariance were utilized for data analysis. The outcomes revealed that demographic variables have no effect on knowledge and competence (p > 0.05). It was also discovered that exposure to learning modules enhanced the competence levels of nurses who were inexperienced in the pediatric setting (F = 41.183, df = 1, p = 0.000). Furthermore, results revealed that enhancing knowledge and competencies is not wholly dependent on well-structured educational sessions and training modules, but is also influenced by other factors, including time allocation for module delivery, protected time to practice, and the opportunity to apply new skills in the clinical area.

A little can go a long way: giving learners some context can enhance the benefits of pretesting

Tests given to learners before they study new information can enhance the learning of that information. When responding to these pretests, learners typically generate answers that are incorrect but that are nevertheless helpful for improving the learning of the correct answers. The present research examined how providing learners with context prior to pretesting can enhance the benefits of pretesting. Across two experiments, participants were given a pretest for half of the to-be-learned information and then asked to read a passage about a fictional topic, an alien civilisation known as Yoffas (Experiment 1 and Experiment 2), or a unique fruit called the Anona (Experiment 2). Participants who read a short paragraph contextualising the to-be-learned information exhibited a significantly larger pretesting effect than participants who did not, with this interaction being observed regardless of whether memory was tested after a 5-min delay or 1-week delay, and regardless of whether contextualisation was manipulated between-subjects or within-subjects. These results suggest that what learners know prior to a pretest can have an impact on the extent to which learners benefit from that pretest.

Memory updating after retrieval: when new information is false or correct

We conducted three experiments testing the malleability of memory in incorporating new information following retrieval. All experiments used associative lists typical of the DRM paradigm [Deese, J. (1959). On the prediction of occurrence of particular verbal intrusions in immediate recall. Journal of Experimental Psychology, 58(1), 17-22; Roediger, H. L., & McDermott, K. B. (1995). Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(4), 803-814]. This paradigm enabled the evaluation of the integration of false information and correct information with the original information. In Experiment 1, participants studied DRM lists, and in a later phase either retrieved or restudied the lists and were presented with never-presented critical lures. The results of Experiment 1 showed that compared to restudy, retrieval enhanced the integration of subsequent false information, as measured by later recall in a follow-up test. In Experiments 2 and 3, after initial study, participants retrieved or studied incorrect information and received corrective feedback. The results showed that retrieval led to more error correction than restudy, when feedback was presented immediately. In general, this research suggests retrieval facilitates incorporation of new, related information, regardless of whether it is false or correct.

The benefits of impossible tests: Assessing the role of error-correction in the pretesting effect

Relative to studying alone, guessing the meanings of unknown words can improve later recognition of their meanings, even if those guesses were incorrect – the pretesting effect (PTE). The error-correction hypothesis suggests that incorrect guesses produce error signals that promote memory for the meanings when they are revealed. The current research sought to test the error-correction explanation of the PTE. In three experiments, participants studied unfamiliar Finnish-English word pairs by either studying each complete pair or by guessing the English translation before its presentation. In the latter case, the participants also guessed which of two categories the word belonged to. Hence, guesses from the correct category were semantically closer to the true translation than guesses from the incorrect category. In Experiment 1, guessing increased subsequent recognition of the English translations, especially for translations that were presented on trials in which the participants’ guesses were from the correct category. Experiment 2 replicated these target recognition effects while also demonstrating that they do not extend to associative recognition performance. Experiment 3 again replicated the target recognition pattern, while also examining participants’ metacognitive recognition judgments. Participants correctly judged that their memory would be better after small than after large errors, but incorrectly believed that making any errors would be detrimental, relative to study-only. Overall, the data are inconsistent with the error-correction hypothesis; small, within-category errors produced better recognition than large, cross-category errors. Alternative theories, based on elaborative encoding and motivated learning, are considered.

Perceptions Regarding Pediatric and Adolescent Gynecology Training among Obstetrics and Gynecology Residents in Hawai'i

A pediatric and adolescent gynecology curriculum was implemented for obstetrics/gynecology residents at a single institution in Hawai'i. The primary objective of this study was to evaluate the effectiveness of this pediatric and adolescent gynecology (PAG) formal curriculum in improving knowledge. Twelve residents at the University of Hawai'i completed the curriculum during their second year of training. Of these, 6 residents completed an examination assessing knowledge both before and immediately after the curriculum. Five of these residents completed the examination 1-2 years after completing the curriculum. Average pre-rotation examination scores were 57%, which increased significantly immediately after the curriculum to 87% (P=.003) and remained significantly higher after 1-2 years with average scores of 71% (P=.025). A secondary objective was to assess resident comfort and confidence levels in the subject. All residents who completed the curriculum completed surveys assessing comfort in their pediatric and adolescent gynecology training and both perceived confidence and competence in this area. Despite completing the curriculum, 7 of 12 (58%) did not feel comfortable evaluating pediatric gynecology patients, and 5 of 12 (42%) did not feel prepared nor competent to care for pediatric and adolescent patients. A formal pediatric and adolescent gynecology curriculum increased examination performance, which suggests knowledge retention. However, despite curriculum completion, residents still desired more training in the subject area. Residency programs should consider innovative ways to incorporate inpatient and outpatient educational experiences, periodic testing, and feedback longitudinally throughout residency training to improve long-term retention, confidence, and competence in providing gynecologic care in the pediatric and adolescent gynecology population.

Neurocognitive modeling of latent memory processes reveals reorganization of hippocampal-cortical circuits underlying learning and efficient strategies

Efficient memory-based problem-solving strategies are a cardinal feature of expertise across a wide range of cognitive domains in childhood. However, little is known about the neurocognitive mechanisms that underlie the acquisition of efficient memory-based problem-solving strategies. Here we develop, to the best of our knowledge, a novel neurocognitive process model of latent memory processes to investigate how cognitive training designed to improve children's problem-solving skills alters brain network organization and leads to increased use and efficiency of memory retrieval-based strategies. We found that training increased both the use and efficiency of memory retrieval. Functional brain network analysis revealed training-induced changes in modular network organization, characterized by increase in network modules and reorganization of hippocampal-cortical circuits. Critically, training-related changes in modular network organization predicted performance gains, with emergent hippocampal, rather than parietal cortex, circuitry driving gains in efficiency of memory retrieval. Our findings elucidate a neurocognitive process model of brain network mechanisms that drive learning and gains in children's efficient problem-solving strategies.

Predicting as a learning strategy

This article attempts to delineate the procedural and mechanistic characteristics of predicting as a learning strategy. While asking students to generate a prediction before presenting the correct answer has long been a popular learning strategy, the exact mechanisms by which it improves learning are only beginning to be unraveled. Moreover, predicting shares many features with other retrieval-based learning strategies (e.g., practice testing, pretesting, guessing), which begs the question of whether there is more to it than getting students to engage in active retrieval. I argue that active retrieval as such does not suffice to explain beneficial effects of predicting. Rather, the effectiveness of predicting is also linked to changes in the way the ensuing feedback is processed. Initial evidence suggests that predicting boosts surprise about unexpected answers, which leads to enhanced attention to the correct answer and strengthens its encoding. I propose that it is this affective aspect of predicting that sets it apart from other retrieval-based learning strategies, particularly from guessing. Predicting should thus be considered as a learning strategy in its own right. Studying its unique effects on student learning promises to bring together research on formal models of learning from prediction error, epistemic emotions, and instructional design.

Micro design in inter-organizational hybrid governance: a study on product adaptation, reverse knowledge transfer and integration mechanism

Purpose

This study aims to explore the extent to which inter-organizational hybrid governance manages the micro design for optimum reverse knowledge transfer in the open innovation context. The authors use two essential facets of micro design in hybrid governance: product adaptation and integration mechanism.

Design/methodology/approach

Data for this study were collected from franchisees through structured questionnaires in Indonesia.

Findings

Results indicated that product adaptation has a positive relationship with reverse knowledge transfer. This study also found that the formalization strengthens the relationship between product adaptation and reverse knowledge transfer. However, the socialization does not have a moderation effect.

Research limitations/implications

This research estimates the knowledge transfer from the agent’s side only. Therefore, further research is expected to estimate the reverse knowledge transfer in dyads (from agent and principal) to get a detailed understanding of reverse knowledge transfer.

Practical implications

This study offers guidelines to managers, especially in inter-organizational hybrid governance. The authors suggest reverse knowledge transfer as a form to manage the dispersed knowledge from their agents. Governing institutions should change their view that agents have diverse knowledgebase from experience adapting to local conditions and can improve their open innovation through reverse knowledge transfer. From the results, it is found that giving agents the flexibility to adapt products can boost reverse knowledge transfer to support open innovation.

Originality/value

This study provides an understanding of the utilization of external knowledge sourcing in the context of open innovation from agent to principal in hybrid governance through reverse knowledge transfer, which has thus far been empirically under-researched.

Answer First or Google First? Using the Internet in ways that Enhance, not Impair, One’s Subsequent Retention of Needed Information

Technological advances have given us tools—Google, in particular—that can both augment and free up our cognitive resources. Research has demonstrated, however, that some cognitive costs may arise from our reliance on such external memories. We examined whether pretesting—asking participants to solve a problem before consulting Google for needed information—can enhance participants’ subsequent recall for the searched-for content as well as for relevant information previously studied. Two groups of participants, one with no programming knowledge and one with some programming knowledge, learned several fundamental programming concepts in the context of a problem-solving task. On a later multiple-choice test with transfer questions, participants who attempted the task before consulting Google for help out-performed participants who were allowed to search Google right away. The benefit of attempting to solve the problem before googling appeared larger with some degree of programming experience, consistent with the notion that some prior knowledge can help learners integrate new information in ways that benefit its learning as well as that of previously studied related information.

Epistemic Curiosity and the Region of Proximal Learning

We propose a framework for understanding epistemic curiosity as a metacognitive feeling state that is related to the individual's Region of Proximal Learning (RPL), an adaptive mental space where we feel we are on the verge of knowing or understanding. First, we review several historical views, contrasting the RPL perspective with alternative views of curiosity. Second, we detail the processes, conditions, and outcomes within the RPL framework which are proposed to be related to curiosity. Finally, we review several lines of evidence relevant to the relation between RPL and curiosity. These include (1) differences in the conditions under which experts and novices mind wander, (2) experiments investigating people's choices of whether to study materials for which they have high versus low feelings of knowing, (3) results related to people's engagement with corrections to errors made with high confidence, and (4) curiosity, attention, and learning data related to the tip-of-the-tongue state.

Learning from errors: students' and instructors' practices, attitudes, and beliefs

In some educational contexts, such as during assessments, it is essential to avoid errors. In other contexts, however, generating an error can foster valuable learning opportunities. For instance, generating errors can improve memory for correct answers. In two surveys conducted at three large public universities in North America, we investigated undergraduate students' and instructors' awareness of the pedagogical benefits of generating errors, as well as related practices, attitudes, and beliefs. Surveyed topics included the incorporation of errors into learning activities, opinions about the consequences of studying errors, and approaches to feedback. Many students had an aversion towards making errors during learning and did not use opportunities to engage in errorful generation, yet studied or analysed errors when they occurred. Many instructors had a welcoming attitude towards errors that occur during learning, yet varied in providing students with resources that facilitate errorful generation. Overall, these findings reveal the prevalence of an ambivalent approach to errors: Students and instructors avoid generating errors but prioritise learning from them when they occur. These results have important implications for the implementation of pretesting, productive failure, and other error-focused learning techniques in educational contexts.

Cognitive Prerequisites for Generative Learning: Why Some Learning Strategies Are More Effective Than Others

This study examined age-related differences in the effectiveness of two generative learning strategies (GLSs). Twenty-five children aged 9-11 and 25 university students aged 17-29 performed a facts learning task in which they had to generate either a prediction or an example before seeing the correct result. We found a significant Age × Learning Strategy interaction, with children remembering more facts after generating predictions rather than examples, whereas both strategies were similarly effective in adults. Pupillary data indicated that predictions stimulated surprise, whereas the effectiveness of example-based learning correlated with children's analogical reasoning abilities. These findings suggest that there are different cognitive prerequisites for different GLSs, which results in varying degrees of strategy effectiveness by age.

Searching for an Alternative Method to Evaluate Surgical Knowledge: Does a Drawing and Labeling Quiz Predict In-Service Exam Performance and Enhance Learning?

BACKGROUND

The best methods to assess surgical knowledge are still debated. The authors used a non-multiple-choice test as a pre- and post-conference assessment to measure residents' knowledge gains with comparison to a standard summative assessment tool.

METHODS

At one didactic conference, plastic surgery residents at a single institution were given a pre-test of drawing and labeling structures in the extensor mechanism of the finger and within the carpal tunnel. The quiz was followed by a lecture on the same material and a subsequent post-test. Scores were correlated with in-service exam performance.

RESULTS

Pre-test scores (n = 13) were positively correlated with postgraduate year (PGY) until PGY-3. Performance on labeling structures was higher than performance on the respective drawing prompt. Residents' ability to label structures increased more strongly with PGY than their ability to draw structures. The post-test (n = 8) demonstrated that teaching improves performance on labeling questions (pre-test score = 62%; post-test score = 87%). Improvement was observed across all PGYs. Pretest results were positively correlated with in-service exam performance.

CONCLUSIONS

Our study suggests that a knowledge test focused on drawing and labeling structures given to surgical residents is a valid, nontraditional method for assessing resident knowledge. Such a quiz would offer programs an alternative method for regularly evaluating residents aside from in-service questions, in order to identify residents who may need targeted training for the in-service exam and to inform teaching plans.Additionally, residents could use quiz feedback to guide study efforts and prime conference-related learning.

Learning from errors is attributable to episodic recollection rather than semantic mediation

Many recent studies have shown that memory for correct answers is enhanced when an error is committed and then corrected, as compared to when the correct answer is provided without intervening error commission. The fact that the kind of errors that produced such a benefit, in past research, were those that were semantically related to the correct answer suggested that the effect may occur because the error provides a semantic stepping stone to the correct answer: the Semantic Mediation hypothesis. This hypothesis seems at odds with the finding that amnesicsgenerate answers, again including those studied by Tulving and his colleagues-who purportedly have spared semantic/implicit memory-experience enormous difficulties when they commit errors. Accordingly, the present experiments investigated whether the error-generation benefit seen in typicals was attributable Semantic Mediation or to Episodic Recollection. In Experiment 1, we used polysemous materials to create Congruent (e.g., wrist-palm) and Incongruent (e.g., tree-palm) cues for target words (e.g., HAND). In the Congruent condition, participants generated errors that were semantically related to the target (e.g., finger), and which could have provided a semantic mediator. In the Incongruent condition they generated errors that were unrelated to the target (e.g., coconut), and which, therefore, should not have provided a semantic mediator. The Congruent and Incongruent conditions both produced an error-generation benefit-contradicting the Semantic Mediation hypothesis. Experiment 2 showed that the error-generation benefit only occurred when the original error was also recollected on the final memory test. Indeed, in the Incongruent condition, when the error was not, itself, recalled, error generation resulted in a deficit in memory for the correct response. These results point to episodic/explicit, rather than semantic/implicit memory, as the locus of the 'learning from errors' benefits.

Characteristics of Effective Continuing Medical Education for Physician Assistants and Nurse Practitioners in Hospital Medicine

PURPOSE

The purpose of this study was to describe participant characteristics and effective teaching methods at a national continuing medical education (CME) conference on hospital medicine for physician assistants (PAs) and nurse practitioners (NPs).

METHODS

In this cross-sectional study, participants provided demographic information and teaching effectiveness scores for each presentation. Associations between teaching effectiveness score and presentation characteristics were determined.

RESULTS

In total, 163 of 253 participants (64.4%) completed evaluations of 28 presentations. Many of the participants were younger than 50 years (69.0%), had practiced for fewer than 5 years (41.5%), and worked in nonacademic settings (76.7%). Teaching effectiveness scores were significantly associated with the use of clinical cases (perfect scores for 68.8% of presentations with clinical cases vs. 59.8% without; P = .04).

CONCLUSION

Many PAs and NPs at an HM CME conference were early-career clinicians working in nonacademic settings. Presenters at CME conferences in hospital medicine should consider using clinical cases to improve their teaching effectiveness among PA and NP learners.

Does pre-testing promote better retention than post-testing?

Compared with other learning strategies, retrieval practice seems to promote superior long-term retention. This has been found mostly in conditions where learners take tests after being exposed to learning content. However, a pre-testing effect has also been demonstrated, with promising results. This raises the question, for a given amount of time dedicated to retrieval practice, whether learners should be tested before or after an initial exposure to learning content. Our experiment directly compares the benefits of post-testing and pre-testing relative to an extended reading condition, on a retention test 7 days later. We replicated both post-testing (d = 0.74) and pre-testing effects (d = 0.35), with significantly better retention in the former condition. Post-testing also promoted knowledge transfer to previously untested questions, whereas pre-testing did not. Our results thus suggest that it may be more fruitful to test students after than before exposure to learning content.

Selective effects of errorful generation on recognition memory: the role of motivation and surprise

The current research examined the effects of errorful generation on memory, focusing particularly on the roles of motivation and surprise. In two experiments, participants were first presented with photographs of faces and were asked to associate four facts with each photograph. On Generate trials, the participants guessed two of the facts (Guess targets) before those correct facts, and another two correct facts (Study targets), were revealed. On the remaining Read trials, all four facts were presented without a guessing stage. In Experiment 1, participants also ranked their motivation to know the answers before they were revealed, or their surprise on learning the true answers. Guess targets were subsequently better recognised than the concurrently presented, non-guessed Study targets. Guess targets were also better recognised than Read targets, and recognition of Study and Read targets did not differ. Errorful generation also increased self-reported motivation, but not surprise. Experiment 2 showed that the results of Experiment 1 can outlive a 20-minute delay, and that they generalise to a more challenging recognition test. Together, the results suggest that errorful generation improves memory specifically for the guessed fact, and this may be linked to an increase in motivation to learn that fact.

Conditions of highly specific learning through cued recall

After studying a stimulus (e.g., a word triplet such as gift, rose, wine), taking a cued recall test on that stimulus (e.g., gift, rose, ?) improves later recall of the retrieved term (e.g., wine) relative to a restudy control. That testing effect, however, is specific to the tested term: later recall of a previously untested term from the same stimulus (e.g., gift or rose) is not enhanced. In the present research, two possibilities for that highly specific learning were investigated: (a) learning through cued recall is always highly specific to the tested term, or, alternatively, (b) learning specificity is unique to the case of retrieving a term from an episodic memory of a study event. We addressed those possibilities by using the pretesting paradigm, in which there is no study opportunity prior to cued recall testing, and hence retrieval occurs through semantic memory. The results of two experiments supported the latter hypothesis. Thus, it is not the recall attempt per se that produces highly specific learning, but rather the attempt to recall the response by accessing an episodic memory of a particular study event. Theoretical and practical implications for learning through cued recall are discussed.

Using prequestions to enhance learning from reading passages: the roles of question type and structure building ability

Answering questions before learning something ("prequestions") enhances learning. However, these benefits usually occur for information that was asked in the prequestions (i.e. prequestioned material), and not for non-prequestioned material. We reasoned that this narrow benefit may be due to the fact that studies typically use fairly simple prequestions that have a clear answer within one part of the learning material - isolative prequestions. We explored the effects of integrative prequestions that required participants to make connections across different parts of a reading passage. Experiment 1 showed the usual benefit of isolative prequestions on prequestioned but not on non-prequestioned material, but no benefit of integrative prequestions. However, in Experiment 2 when participants were given instructions to seek the answers while reading, integrative prequestions benefited learning of both prequestioned and non-prequestioned material. Individual differences in structure building positively predicted performance, but did not interact with the effects of prequestions.

Effects of retrieval practice and presentation modality on verbal learning: testing the limits of the testing effect

The testing effect refers to improved memory after retrieval practice and has been researched primarily with visual stimuli. In two experiments, we investigated whether the testing effect can be replicated when the to-be-learned information is presented auditorily, or visually + auditorily. Participants learned Swahili-English word pairs in one of three presentation modalities - visual, auditory, or visual + auditory. This was manipulated between-participants in Experiment 1 and within-participants in Experiment2. All participants studied the word pairs during three study trials. Half of participants practiced recalling the English translations in response to the Swahili cue word twice before the final test whereas the other half simply studied the word pairs twice more. Results indicated an improvement in final test performance in the repeated test condition, but only in the visual presentation modality (Experiments 1 and 2) and in the visual + auditory presentation modality (Experiment 2). This suggests that the benefits of practiced retrieval may be limited to information presented in a visual modality.

How Does Pretesting for PubMed Knowledge Spark Student Learning?

Predoctoral students enter dental school with varying skill levels for searching biomedical databases and a tendency to overestimate their abilities. Accordingly, PubMed instruction is embedded within a required dental course and includes a graded component. This article describes a pretest/intervention/posttest developed for the PubMed session. The expectation for this new assessment was that motivation to learn PubMed would increase during the intervention if pretesting objectively showed students the difference between their self-perceived versus actual PubMed abilities. The goals were to help students better self-assess their genuine searching abilities, spark learning during the instruction session, and elicit measurable improvement in skills.

Response Format, Not Semantic Activation, Influences the Failed Retrieval Effect

In educational settings, tests are mainly used to measure the extent to which learners' knowledge and skill have been acquired. However, the act of taking a test also promotes learning itself. In particular, making errors on tests (i.e., searching for erroneous information) promotes learning. This is called the "failed retrieval effect" (FRE) and has been the subject of considerable study. Previous research shows that enhanced learning does not occur if feedback correcting an error is delayed. This is attributed to the relative absence of activated information. In this study, we manipulated both the amount of information to be retrieved prior to learning and the delay time until feedback is given to investigate their effects on learning. As a result, even when multiple incorrect answers were given to increase the degree of semantic activation, learning was not promoted beyond that found with traditional procedures that rely on only one incorrect response. The timing of feedback (immediate, short-delay, long-delay) also did not impact FRE. However, the manipulation of response format for erroneous information resulted in degraded performance when responses were typed and feedback was delayed. Based on this result, we suggested that the failed retrieval effect was not affected by semantic activation at the time of retrieval but was affected by response format. Moreover, the processing necessary for typing may affect FRE under the delayed feedback condition.

Testing Test-Enhanced Continuing Medical Education: A Randomized Controlled Trial

PURPOSE

The authors investigated the impact of the use of an efficient multiple-choice question (MCQ) test-enhanced learning (TEL) intervention for continuing professional development (CPD) on knowledge retention as well as self-reported learning behaviors.

METHOD

The authors conducted a randomized controlled trial comparing knowledge retention among learners who registered for an annual CPD conference at the University of Toronto in April 2016. Participants were randomized to receive an online preworkshop stand-alone MCQ test (no feedback) and a postworkshop MCQ test (with feedback) after a 14-day delay. Controls received no pre-/posttesting. The primary outcome measure was performance on a clinical vignette-based retention and application test delivered to all participants four weeks post conference. Secondary outcomes included self-reported changes in learning behavior, satisfaction, and efficiency of TEL.

RESULTS

Three hundred eight physicians from across Canada registered for the four-day conference; 186 physicians consented to participate in the study and were randomized to receive TEL or to the control group in 1 of 15 workshops, with 126 providing complete data. A random-effects meta-analysis demonstrated a pooled effect size indicating moderate effect of TEL (Hedges g of 0.46; 95% CI: 0.26-0.67). The majority of respondents (65%) reported improved CPD learning resulting from pretesting.

CONCLUSIONS

Testing for learning can be leveraged to efficiently and effectively improve outcomes for CPD. Testing remains an underused education intervention in CPD, and the use of formative assessment to enhance professional development should be a key target for research.

Memory and availability-biased metacognitive illusions for flags of varying familiarity

Research on everyday attention suggests that frequent interaction with objects often does not benefit memory or metamemory for them. Across three experiments, participants gave confidence judgments and completed eight-alternative forced-choice tests of the US, Canadian, and Mexican flags. In Experiment 1, environmental availability was correlated with confidence for the US flag, despite similar recognition performance at a saturated time point in the US (July 4th) and a neutral time point (August 6th). In Experiment 2, participants that were asked to verbally describe the flags before judging and remembering them were less accurate and more overconfident than were controls. Experiment 3 utilized a draw-study paradigm wherein participants who first drew the flag had reliably more accurate recognition and confidence scores than those who only studied it. These findings illuminate a persistent metacognitive bias, demonstrate a powerful learning intervention, and extend theories of errorful learning by highlighting the role of attention.

Word repetition and retrieval practice effects in aphasia: Evidence for use-dependent learning in lexical access

This study tested the hypothesis that a use-dependent learning mechanism operates at each of two stages of lexical access: retrieval of a word from semantics ("Stage 1"), followed by retrieval of the word's constituent phonemes ("Stage 2"). Two participants with aphasia were selected due to their contrasting types of naming impairment (Stage 1 versus Stage 2 difficulty). For each participant, items were assigned to naming training that involved retrieval practice (retrieval of the name from semantics) or repetition practice (hear the name and orally repeat it). Naming tests were administered one day and one week after training. The results supported the predicted training effects: (a) Because successful naming via retrieval practice requires both Stage 1 and Stage 2, this technique uses and strengthens item-specific connections in both stages. (b) Because word repetition circumvents semantically driven retrieval, this technique primarily uses and strengthens item-specific connections in Stage 2.