Effects of the Physical Environment on Cognitive Load and Learning: Towards a New Model of Cognitive Load

Influence of learner knowledge and case complexity on handover accuracy and cognitive load: results from a simulation study

CONTEXT

The handover represents a high-risk event in which errors are common and lead to patient harm. A better understanding of the cognitive mechanisms of handover errors is essential to improving handover education and practice.

OBJECTIVES

This paper reports on an experiment conducted to study the effects of learner knowledge, case complexity (i.e. cases with or without a clear diagnosis) and their interaction on handover accuracy and cognitive load.

METHODS

Participants were 52 Dutch medical students in Years 2 and 6. The experiment employed a repeated-measures design with two explanatory variables: case complexity (simple or complex) as the within-subject variable, and learner knowledge (as indicated by illness script maturity) as the between-subject covariate. The dependent variables were handover accuracy and cognitive load. Each participant performed a total of four simulated handovers involving two simple cases and two complex cases.

RESULTS

Higher illness script maturity predicted increased handover accuracy (p < 0.001) and lower cognitive load (p = 0.007). Case complexity did not independently affect either outcome. For handover accuracy, there was no interaction between case complexity and illness script maturity. For cognitive load, there was an interaction effect between illness script maturity and case complexity, indicating that more mature illness scripts reduced cognitive load less in complex cases than in simple cases.

CONCLUSIONS

Students with more mature illness scripts performed more accurate handovers and experienced lower cognitive load. For cognitive load, these effects were more pronounced in simple than complex cases. If replicated, these findings suggest that handover curricula and protocols should provide support that varies according to the knowledge of the trainee.

Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings

As interest in applying cognitive load theory (CLT) to the study and design of pedagogic and technological approaches in healthcare simulation grows, suitable measures of cognitive load (CL) are needed. Here, we report a two-phased study investigating the sensitivity of subjective ratings of mental effort (SRME) and secondary-task performance (signal detection rate, SDR and recognition reaction time, RRT) as measures of CL. In phase 1 of the study, novice learners and expert surgeons attempted a visual-monitoring task under two conditions: single-task (monitoring a virtual patient's heart-rate) and dual-task (tying surgical knots on a bench-top simulator while monitoring the virtual patient's heart-rate). Novices demonstrated higher mental effort and inferior secondary-task performance on the dual-task compared to experts (RRT 1.76 vs. 0.73, p = 0.012; SDR 0.27 vs. 0.97, p < 0.001; SRME 7.75 vs. 2.80, p < 0.001). Similarly, secondary task performance deteriorated from baseline to dual-task among novices (RRT 0.63 vs. 1.76 s, p < 0.006 and SDR 1.00 vs. 0.27, p < 0.001), but not experts (RRT 0.63 vs. 0.73 s, p = 0.124 and SDR 1.00 vs. 0.97, p = 0.178). In phase 2, novices practiced surgical knot-tying on the bench top simulator during consecutive dual-task trials. A significant increase in SDR (F(9,63) = 6.63, p < 0.001, f = 0.97) and decrease in SRME (F(9,63) = 9.39, p < 0.001, f = 1.04) was observed during simulation training, while RRT did not change significantly (F(9,63) = 1.18, p < 0.32, f = 0.41). The results suggest subjective ratings and dual-task performance can be used to track changes in CL among novices, particularly in early phases of simulation-based skills training. The implications for measuring CL in simulation instructional design research are discussed.

Validity of Cognitive Load Measures in Simulation-Based Training: A Systematic Review

BACKGROUND

Cognitive load theory (CLT) provides a rich framework to inform instructional design. Despite the applicability of CLT to simulation-based medical training, findings from multimedia learning have not been consistently replicated in this context. This lack of transferability may be related to issues in measuring cognitive load (CL) during simulation. The authors conducted a review of CLT studies across simulation training contexts to assess the validity evidence for different CL measures.

METHOD

PRISMA standards were followed. For 48 studies selected from a search of MEDLINE, EMBASE, PsycInfo, CINAHL, and ERIC databases, information was extracted about study aims, methods, validity evidence of measures, and findings. Studies were categorized on the basis of findings and prevalence of validity evidence collected, and statistical comparisons between measurement types and research domains were pursued.

RESULTS

CL during simulation training has been measured in diverse populations including medical trainees, pilots, and university students. Most studies (71%; 34) used self-report measures; others included secondary task performance, physiological indices, and observer ratings. Correlations between CL and learning varied from positive to negative. Overall validity evidence for CL measures was low (mean score 1.55/5). Studies reporting greater validity evidence were more likely to report that high CL impaired learning.

CONCLUSIONS

The authors found evidence that inconsistent correlations between CL and learning may be related to issues of validity in CL measures. Further research would benefit from rigorous documentation of validity and from triangulating measures of CL. This can better inform CLT instructional design for simulation-based medical training.

Measuring cognitive load: performance, mental effort and simulation task complexity

CONTEXT

Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation.

OBJECTIVES

We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training.

METHODS

We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task.

RESULTS

Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant.

CONCLUSIONS

Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.

Limitations of subjective cognitive load measures in simulation-based procedural training

CONTEXT

The effective implementation of cognitive load theory (CLT) to optimise the instructional design of simulation-based training requires sensitive and reliable measures of cognitive load. This mixed-methods study assessed relationships between commonly used measures of total cognitive load and the extent to which these measures reflected participants' experiences of cognitive load in simulation-based procedural skills training.

METHODS

Two groups of medical residents (n = 38) completed three questionnaires after participating in simulation-based procedural skills training sessions: the Paas Cognitive Load Scale; the NASA Task Load Index (TLX), and a cognitive load component (CLC) questionnaire we developed to assess total cognitive load as the sum of intrinsic load (how complex the task is), extraneous load (how the task is presented) and germane load (how the learner processes the task for learning). We calculated Pearson's correlation coefficients to assess agreement among these instruments. Group interviews explored residents' perceptions about how the simulation sessions contributed to their total cognitive load. Interviews were audio-recorded, transcribed and subjected to qualitative content analysis.

RESULTS

Total cognitive load scores differed significantly according to the instrument used to assess them. In particular, there was poor agreement between the Paas Scale and the TLX. Quantitative and qualitative findings supported intrinsic cognitive load as synonymous with mental effort (Paas Scale), mental demand (TLX) and task difficulty and complexity (CLC questionnaire). Additional qualitative themes relating to extraneous and germane cognitive loads were not reflected in any of the questionnaires.

CONCLUSIONS

The Paas Scale, TLX and CLC questionnaire appear to be interchangeable as measures of intrinsic cognitive load, but not of total cognitive load. A more complete understanding of the sources of extraneous and germane cognitive loads in simulation-based training contexts is necessary to determine how best to measure and assess their effects on learning and performance outcomes.

Operationalising elaboration theory for simulation instruction design: a Delphi study

OBJECTIVE

The aim of this study was to assess the feasibility of incorporating the Delphi process within the simplifying conditions method (SCM) described in elaboration theory (ET) to identify conditions impacting the complexity of procedural skills for novice learners.

METHODS

We generated an initial list of conditions impacting the complexity of lumbar puncture (LP) from key informant interviews (n = 5) and a literature review. Eighteen clinician-educators from six different medical specialties were subsequently recruited as expert panellists. Over three Delphi rounds, these panellists rated: (i) their agreement with the inclusion of the simple version of the conditions in a representative ('epitome') training scenario, and (ii) how much the inverse (complex) version increases LP complexity for a novice. Cronbach's α-values were used to assess inter-rater agreement.

RESULTS

All panellists completed Rounds 1 and 2 of the survey and 17 completed Round 3. In Round 1, Cronbach's α-values were 0.89 and 0.94 for conditions that simplify and increase LP complexity, respectively; both values increased to 0.98 in Rounds 2 and 3. With the exception of 'high CSF (cerebral spinal fluid) pressure', panellists agreed with the inclusion of all conditions in the simplest (epitome) training scenario. Panellists rated patient movement, spinal anatomy, patient cooperativeness, body habitus, and the presence or absence of an experienced assistant as having the greatest impact on the complexity of LP.

CONCLUSIONS

This study demonstrated the feasibility of using expert consensus to establish conditions impacting the complexity of procedural skills, and the benefits of incorporating the Delphi method into the SCM. These data can be used to develop and sequence simulation scenarios in a progressively challenging manner. If the theorised learning gains associated with ET are realised, the methods described in this study may be applied to the design of simulation training for other procedural and non-procedural skills, thereby advancing the agenda of theoretically based instruction design in health care simulation.

Attention and working memory in elderly: the influence of a distracting environment

The present work investigated the effect of a distracting environment in the performance of attentional and working memory (WM) tasks in elderly participants. To this end, forty elderly performed two attentional tasks (simple reaction time and go/no-go tasks), and three WM tasks (arithmetic, memory for digits and sequences of letters and numbers). Each participant performed the tasks in a distracting and a non-distracting environment, with an interval of 14-21 days between sessions. The results revealed better performance in the attentional tasks when these were done in the non-distracting environment, as compared to when they were done in the distracting environment. Specifically, participants provided more accurate responses, fewer false alarms and omissions when responding in the non-distracting environment than when responding in the distracting environment. Participants were also faster at providing correct responses in the go/no-go task when it was performed in the non-distracting environment. As for the memory tasks, the effect of type of environment was significant only in the memory for digits in a forward direction task. Our data suggest the need to consider the potential damaging consequences of distracting environments when the elderly have to perform tasks that demand their attention. Specific examples of such situations are presented in the discussion (e.g., distracting effect of environment on medical and on psychological evaluations).