The effects of procedural and conceptual knowledge on visual learning

Even though past research suggests that visual learning may benefit from conceptual knowledge, current interventions for medical image evaluation often focus on procedural knowledge, mainly by teaching classification algorithms. We compared the efficacy of pure procedural knowledge (three-point checklist for evaluating skin lesions) versus combined procedural plus conceptual knowledge (histological explanations for each of the three points). All students then trained their classification skills with a visual learning resource that included images of two types of pigmented skin lesions: benign nevi and malignant melanomas. Both treatments produced significant and long-lasting effects on diagnostic accuracy in transfer tasks. However, only students in the combined procedural plus conceptual knowledge condition significantly improved their diagnostic performance in classifying lesions they had seen before in the pre- and post-tests. Findings suggest that the provision of additional conceptual knowledge supported error correction mechanisms.

Reconsidering conceptual knowledge: Heterogeneity of its components

Cognitive arithmetic classically distinguishes procedural and conceptual knowledge as two determinants of the acquisition of flexible expertise. Whereas procedural knowledge relates to algorithmic routines, conceptual knowledge is defined as the knowledge of core principles, referred to as fundamental structures of arithmetic. To date, there is no consensus regarding their number, list, or even their definition, partly because they are difficult to measure. Recent findings suggest that among the most complex of these principles, some might not be "fundamental structures" but rather may articulate several components of conceptual knowledge, each specific to the arithmetic operation involved. Here, we argue that most of the arithmetic principles similarly may rather articulate several core concepts specific to the operation involved. Data were collected during a national mathematics contest based on an arithmetic game involving a large sample of 9- to 11-year-old students (N = 11,243; 53.1% boys) over several weeks. The purpose of the game was to solve complex arithmetic problems using five numbers and the four operations. A principal component analysis (PCA) was performed. The results show that both conceptual and procedural knowledge were used by children. Moreover, the PCA sorted conceptual and procedural knowledge together, with dimensions being defined by the operation rather than by the concept. This implies that "fundamental structures" rather regroup different concepts that are learned separately. This opens the way to reconsider the very nature of conceptual knowledge and has direct pedagogical implications.

Visual illusions influence proceduralized sports performance

Does the Ebbinghaus visual illusion really influence sports performances? Does the influence depend on the type of knowledge (procedural vs. declarative) that guides movement? To address these questions, we evaluated the knowledge hypothesis, a novel hypothesis according to which the more sports performance relies on procedural knowledge, the more it will be influenced by visual illusions. In the context of golf putting, we first used the high-error/low-error motor-learning technique (Experiment 1) or varied the number of practice trials (Experiment 2) to induce novice participants to rely more on procedural knowledge than on declarative knowledge (or vice versa). We then manipulated the perceived size of two golf holes by projecting a ring of small or large circles around them, which caused the holes to appear larger or smaller, respectively. This Ebbinghaus visual illusion had an influence on putting in both experiments. We also observed a pattern of findings consistent with the knowledge hypothesis: the procedural groups were moderately influenced by the illusion when putting, but the declarative groups were influenced only weakly, at best. Among the participants most sensitive to the illusion, the analyses confirmed a significantly stronger influence for the procedural group. Overall, these findings demonstrate that the effect of visual illusions on sports performance is a reliable phenomenon for proceduralized actions. The knowledge hypothesis represents an attractive way of reconciling earlier divergent findings.

Machine understanding surgical actions from intervention procedure textbooks

The automatic extraction of procedural surgical knowledge from surgery manuals, academic papers or other high-quality textual resources, is of the utmost importance to develop knowledge-based clinical decision support systems, to automatically execute some procedure's step or to summarize the procedural information, spread throughout the texts, in a structured form usable as a study resource by medical students. In this work, we propose a first benchmark on extracting detailed surgical actions from available intervention procedure textbooks and papers. We frame the problem as a Semantic Role Labeling task. Exploiting a manually annotated dataset, we apply different Transformer-based information extraction methods. Starting from RoBERTa and BioMedRoBERTa pre-trained language models, we first investigate a zero-shot scenario and compare the obtained results with a full fine-tuning setting. We then introduce a new ad-hoc surgical language model, named SurgicBERTa, pre-trained on a large collection of surgical materials, and we compare it with the previous ones. In the assessment, we explore different dataset splits (one in-domain and two out-of-domain) and we investigate also the effectiveness of the approach in a few-shot learning scenario. Performance is evaluated on three correlated sub-tasks: predicate disambiguation, semantic argument disambiguation and predicate-argument disambiguation. Results show that the fine-tuning of a pre-trained domain-specific language model achieves the highest performance on all splits and on all sub-tasks. All models are publicly released.

Videos of demonstration versus text and image-based material for pre-skill conceptualisation in flipped newborn resuscitation training for medical students: a pilot study

BACKGROUND

The flipped skills lab is a student-centred approach which incorporates pre-class preparation (pre-skill conceptualization) followed by repeated, hands-on practice for practical skills training. Objective measures of skills acquisition in the flipped literature are few and conflicting. The importance of pre-skill conceptualization in flipped skills training suggests that pedagogically informed pre-skill conceptualization can enhance outcomes.

METHODS

A mixed quasi-experimental study was conducted on 41 final year medical students who followed a flipped newborn resuscitation skills lab. Pre-class preparatory material covered conceptual and procedural knowledge. Students in the traditional group (n = 19) and those in the interventionalmental group (n = 22) received identical reading material covering conceptual knowledge. Procedural knowledge was shared with the interventional group as demonstration videos, while the traditional group received a PowerPoint presentation with text and images covering the same material. Knowledge acquisition was assessed by 20 single best answer questions before and after hands-on practice in the skills lab and skill performance was tested post-intervention with a simulated scenario. Students' perceptions were collected by survey. Quantitative data was analysed using Wilcoxon Signed Ranks test and Mann-Whitney U test as appropriate. Qualitative data was analysed by thematic analysis.

RESULTS

Overall student rating of the intervention was positive with ratings of 4.54 and 4.46 out of 5 by the traditional group and the experimental group respectively. Post-intervention skill performance in the experimental group was significantly better (p < .05) in the interventional group (M = 87.86%, SD = 5.89) than in the traditional group (M = 83.44, SD = 5.30) with a medium effect size (r = .40). While both groups showed significant knowledge gains, only students in the experimental group showed a statistically significant gain in procedural knowledge (p < .05) following the flipped skills lab. Finally, while both groups self-reported feeling more knowledgeable and confident following the intervention, the level of confidence was superior in the experimental group.

CONCLUSIONS

Flipping the skills lab with pre-skill conceptualisation combining text-based conceptual knowledge and video-based procedural knowledge followed by simulation-based hands-on practice improves procedural knowledge and skills acquisition in newborn resuscitation training for medical students. This study shows that in addition to temporal benefits, pedagogically informed pre-skill conceptualization can confer procedure-specific cognitive and emotional benefits supporting skills acquisition.

Conceptual Knowledge, Procedural Knowledge, and Metacognition in Routine and Nonroutine Problem Solving

When, how, and why students use conceptual knowledge during math problem solving is not well understood. We propose that when solving routine problems, students are more likely to recruit conceptual knowledge if their procedural knowledge is weak than if it is strong, and that in this context, metacognitive processes, specifically feelings of doubt, mediate interactions between procedural and conceptual knowledge. To test these hypotheses, in two studies (Ns = 64 and 138), university students solved fraction and decimal arithmetic problems while thinking aloud; verbal protocols and written work were coded for overt uses of conceptual knowledge and displays of doubt. Consistent with the hypotheses, use of conceptual knowledge during calculation was not significantly positively associated with accuracy, but was positively associated with displays of doubt, which were negatively associated with accuracy. In Study 1, participants also explained solutions to rational arithmetic problems; using conceptual knowledge in this context was positively correlated with calculation accuracy, but only among participants who did not use conceptual knowledge during calculation, suggesting that the correlation did not reflect "online" effects of using conceptual knowledge. In Study 2, participants also completed a nonroutine problem-solving task; displays of doubt on this task were positively associated with accuracy, suggesting that metacognitive processes play different roles when solving routine and nonroutine problems. We discuss implications of the results regarding interactions between procedural knowledge, conceptual knowledge, and metacognitive processes in math problem solving.

Automatic detection of procedural knowledge in robotic-assisted surgical texts

Purpose

The automatic extraction of knowledge about intervention execution from surgical manuals would be of the utmost importance to develop expert surgical systems and assistants. In this work we assess the feasibility of automatically identifying the sentences of a surgical intervention text containing procedural information, a subtask of the broader goal of extracting intervention workflows from surgical manuals.

Methods

We frame the problem as a binary classification task. We first introduce a new public dataset of 1958 sentences from robotic surgery texts, manually annotated as procedural or non-procedural. We then apply different classification methods, from classical machine learning algorithms, to more recent neural-network approaches and classification methods exploiting transformers (e.g., BERT, ClinicalBERT). We also analyze the benefits of applying balancing techniques to the dataset.

Results

The architectures based on neural-networks fed with FastText’s embeddings and the one based on ClinicalBERT outperform all the tested methods, empirically confirming the feasibility of the task. Adopting balancing techniques does not lead to substantial improvements in classification.

Conclusion

This is the first work experimenting with machine / deep learning algorithms for automatically identifying procedural sentences in surgical texts. It also introduces the first public dataset that can be used for benchmarking different classification methods for the task.

Use of a Secure Web-Based Data Management Platform to Track Resident Operative Performance and Program Educational Quality Over Time

OBJECTIVE

In surgery residency programs, Accreditation Council for Graduate Medical Education mandated performance assessment can include assessment in the operating room to demonstrate that necessary quality and autonomy goals are achieved by the conclusion of training. For the past 3 years, our institution has used The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) instrument to assess and track operative skills. Evaluation is accomplished in near real-time using a secure web-based platform for data management and analytics (Firefly). Simultaneous to access of the platform's case logging function, the O-SCORE instrument is delivered to faculty members for rapid completion, facilitating quality, and timeliness of feedback. We sought to demonstrate the platform's utility in detecting operative performance changes over time in response to focused educational interventions based on stored case log and O-SCORE data.

DESIGN

Stored resident performance assessments for the most frequently performed laparoscopic procedures (cholecystectomy, appendectomy, inguinal hernia repair, ventral hernia repair) were examined for 3 successive academic years (2016-2019). During this time, 4 of 36 residents had received program-assigned supplemental simulation training to improve laparoscopic skills. O-SCORE data for these residents were extracted from peer data, which were used for comparisons. Assigned training consisted of a range of videoscopic and virtual reality skills drills with performance objectives. O-SCORE responses were converted to integers and autonomy scores for items pertaining to technical skill were compared before and after educational interventions (Student's t-tests). These scores were also compared to aggregate scores in the nonintervention group. Bayesian-modeled learning curves were used to characterize patterns of improvement over time.

SETTING

University of Massachusetts Medical School-Baystate Surgery Residency and Baystate Medical Center PARTICIPANTS: General surgery residents (n = 36) RESULTS: During the period of review, 3325 resident cases were identified meeting the case type criteria. As expected, overall autonomy increased with the number of cases performed. The 4 residents who had been assigned supplemental training (6-18 months) had preintervention score averages that were lower than that of the nonintervention group (2.25 ± 0.43 vs 3.57 ± 1.02; p < 0.0001). During the respective intervention periods, all 4 residents improved autonomy scores (increase to 3.40 ± 0.61; p < 0.0001). Similar improvements were observed for tissue handling, instrument handling, bimanual dexterity, visuospatial skill, and operative efficiency component skills. Postintervention scores were not significantly different compared to scores for the non-intervention group. Bayesian-modeled learning curves showed a similar pattern of postintervention performance improvement.

CONCLUSIONS

The data management platform proved to be an effective tool to track responses to supplemental training that was deemed necessary to close defined skills gaps in laparoscopic surgery. This could be seen both in individual and in aggregated data. We were gratified that at the conclusion of the supplemental training, O-SCORE results for the intervention group were not different than those seen in the non-intervention group.

How do marine and coastal citizen science experiences foster environmental engagement?

Citizen science programs enable community involvement in scientific research. In addition to fostering greater science literacy, some citizen science programs aim to foster engagement in environmental issues. However, few data are available to indicate whether and how citizen science programs can achieve greater environmental engagement. We survey individuals choosing to attend one of seventeen reef citizen science events and examine the extent to which attendees reported three indicators of greater environmental engagement: (i) willingness to share information, (ii) increased support for marine conservation and citizen science, and (iii) intentions to adopt a new behavior. Most participants reported being willing to share information about reef conservation (91%) and described increased support for marine science and conservation (87%). Half of participants (51%) reported intentions to adopt a new conservation behavior. We found that key elements of the citizen science experience associated with these outcomes were learning about actions to protect reefs and coasts (procedural learning), experiencing surprise, and experiencing negative emotions about environmental problems. Excitement was also associated with positive outcomes, but only in participants who were less likely to see themselves as environmental, or were less frequent visitors to reefs and coasts. Importantly, the association between factual learning and environmental engagement outcomes was limited or negative. These findings suggest that the way citizen science experiences make people feel, may be more important for fostering future environmental engagement than factual-based learning. When designing citizen science programs for community members, these findings provide a reminder to not focus on provision of factual information alone, but to highlight environmental impacts while providing meaningful experiences and building environmental skills.