Towards a theory of algorithm-determined cognitive test construction

Intelligence, Personality and Tolerance of Ambiguity

In this study, 3836 adults completed a personality test (the HPTI) and a multidimensional intelligence test (GIA). Two prominent theories that link personality traits to intelligence (compensation and investment) were tested. There were more sex differences in the personality traits than in the IQ scores. Correlational and regression analyses results provided little evidence for either theory but pointed to the role of tolerance of ambiguity as a consistently significant, positive correlate of IQ at both the facet and domain levels. The role of this neglected trait is discussed. Limitations of various aspects of this study and its implications are considered.

How Task Features Impact Evidence From Assessments Embedded in Simulations and Games

One of the key ideas of evidence-centered assessment design (ECD) is that task features can be deliberately manipulated to change the psychometric properties of items. ECD identifies a number of roles that task-feature variables can play, including determining the focus of evidence, guiding form creation, determining item difficulty and discrimination, characterizing proficiency, and producing task variants. Assessment developers can use these task features to manipulate the psychometric properties of both conventional assessment formats and complex tasks embedded in simulations and games. Simulations and games tasks present additional challenges: even defining what corresponds to an item can be difficult. Often task features are determined by game or simulation logic rather than psychometric design. Despite these difficulties, the roles for task features identified in ECD are useful for analyzing the psychometric properties of embedded assessments in simulations and games. This article compares a conventional test of mathematics-problem-solving ability using word problems to an assessment of conceptual physics, creativity, and conscientiousness embedded in the game Newton's Playground, describing how the task roles play out in each setting and how they can be used to manipulate the evidence provided by an assessment.

Validity of measures of cognitive processes and general ability for learning and performance on highly complex computerized tutors: is the g factor of intelligence even more general?

Theoretical arguments and analyses from 2 studies provide compelling evidence that computerized measures of information-processing skills and abilities are highly useful supplements to more traditional, paper-based measures of general mental ability for predicting individuals' capacity to learn from and perform on highly challenging, multifaceted tutors. These tutors were designed to emulate learning and performance in complex, real-world settings. Hierarchical confirmatory factor analysis provided evidence that a general, higher order factor model with general ability at the apex could quite adequately and singularly account for the individual-differences data, both traditional and cognitive-process measures. Results are interpreted in light of the utility and generality of human cognitive abilities.

Can reading ability be measured with tests of memory and processing speed?

The author investigated the possibility of estimating reading ability using tests of memory and processing speed that have significantly reduced reading requirements. Four hundred three air force trainees were given tests of their working memory and long-term memory processes, as well as a standardized reading test. A multiple correlation of .79 indicated that global reading ability is substantially correlated with memory processes. Additional analyses indicated that word knowledge and comprehension, the two major components of reading ability, were both explained by the same set of memory process factors, namely, long-term memory processes, semantic memory retrieval speed, and working memory capacity. These results, which are consistent with the view that memory ability underlies reading ability, support the endeavor of measuring reading ability with memory and processing speed tasks.