Moderators of learning and performance trajectories in microworld simulations: Too soon to give up on intellect!?

Do Judgments of Learning Impair Recall When Uninformative Cues Are Salient?

Judgments of learning (JOL) are one of the most commonly used measures of metamemory. There is mixed evidence that eliciting JOLs while participants are studying word pairs influences their subsequent recall, a phenomenon known as reactivity. The majority of studies have found that positive reactivity occurs when word pairs are related. This finding suggests that when the final test is sensitive to the cues used to make a JOL (e.g., pair relatedness), a benefit to recall is observed. Here, in three experiments, JOL reactivity is examined in the presence of a salient, yet non-diagnostic cue-font size. The results show that when study time is limited and font size is made salient, eliciting JOLs impairs future recall. It is argued that JOLs prompt participants to evaluate salient cues in the learning environment to evaluate whether they will affect future recall. This increased processing of salient cues can impair recall if it comes at the expense of processing less salient but more informative cues. These findings suggest that the relevance to the test of the cues processed when JOLs are performed determines the direction of reactivity effects, with both positive and negative reactivity being possible depending on how diagnostic the salient metacognitive cues are for recall.

Within-Individual Variation in Cognitive Performance Is Not Noise: Why and How Cognitive Assessments Should Examine Within-Person Performance

Despite evidence that it exists, short-term within-individual variability in cognitive performance has largely been ignored as a meaningful component of human cognitive ability. In this article, we build a case for why this within-individual variability should not be viewed as mere measurement error and why it should be construed as a meaningful component of an individual's cognitive abilities. We argue that in a demanding and rapidly changing modern world, between-individual analysis of single-occasion cognitive test scores does not account for the full range of within-individual cognitive performance variation that is implicated in successful typical cognitive performance. We propose that short-term repeated-measures paradigms (e.g., the experience sampling method (ESM)) be used to develop a process account of why individuals with similar cognitive ability scores differ in their actual performance in typical environments. Finally, we outline considerations for researchers when adapting this paradigm for cognitive assessment and present some initial findings from two studies in our lab that piloted the use of ESM to assess within-individual cognitive performance variation.

Intelligence IS Cognitive Flexibility: Why Multilevel Models of Within-Individual Processes Are Needed to Realise This

Despite substantial evidence for the link between an individual’s intelligence and successful life outcomes, questions about what defines intelligence have remained the focus of heated dispute. The most common approach to understanding intelligence has been to investigate what performance on tests of intellect is and is not associated with. This psychometric approach, based on correlations and factor analysis is deficient. In this review, we aim to substantiate why classic psychometrics which focus on between-person accounts will necessarily provide a limited account of intelligence until theoretical considerations of within-person accounts are incorporated. First, we consider the impact of entrenched psychometric presumptions that support the status quo and impede alternative views. Second, we review the importance of process-theories, which are critical for any serious attempt to build a within-person account of intelligence. Third, features of dynamic tasks are reviewed, and we outline how static tasks can be modified to target within-person processes. Finally, we explain how multilevel models are conceptually and psychometrically well-suited to building and testing within-individual notions of intelligence, which at its core, we argue is cognitive flexibility. We conclude by describing an application of these ideas in the context of microworlds as a case study.

Reconsidering the Use of the Mindset Assessment Profile in Educational Contexts

The Mindset Assessment Profile is a popular questionnaire purportedly designed to measure mindset-an individual's belief in whether intelligence is malleable or stable. Despite its widespread use, the questionnaire appears to assess an individual's need for cognition and goal orientation more than mindset. We assessed the reliability, construct validity, and factor structure of the Mindset Assessment Profile in a sample of 992 undergraduates. The reliability of the Mindset Assessment Profile was questionable (α = .63) and significantly lower than the reliability of the Implicit Theories of Intelligence Questionnaire (α = .94), an established measure of mindset. The Mindset Assessment Profile also lacked convergent and discriminant validity. Overall scores on the Mindset Assessment Profile correlated significantly more strongly with need for cognition than with mindset. Item-level analyses supported this finding: most items correlated weakly or not at all with mindset, and correlated significantly more strongly with need for cognition and learning goal orientation. Exploratory factor analysis indicated that three factors were underlying scores on the Mindset Assessment Profile: need for cognition, mindset, and performance goal orientation. Based on its questionable reliability and poor construct validity, we do not recommend that researchers and educators use the Mindset Assessment Profile to measure mindset.