Modern Assessments of Intelligence Must Be Fair and Equitable

Historically, assessments of human intelligence have been virtually synonymous with practices that contributed to forms of inequality and injustice. As such, modern considerations for assessing human intelligence must focus on equity and fairness. First, we highlight the array of diversity, equity, and inclusion concerns in assessment practices and discuss strategies for addressing them. Next, we define a modern, non-g, emergent view of intelligence using the process overlap theory and argue for its use in improving equitable practices. We then review the empirical evidence, focusing on sub-measures of g to highlight the utility of non-g, emergent models in promoting equity and fairness. We conclude with suggestions for researchers and practitioners.

Remote Assessment: Origins, Benefits, and Concerns

Although guidelines surrounding COVID-19 have relaxed and school-aged students are no longer required to wear masks and social distance in schools, we have become, as a nation and as a society, more comfortable working from home, learning online, and using technology as a platform to communicate ubiquitously across ecological environments. In the school psychology community, we have also become more familiar with assessing students virtually, but at what cost? While there is research suggesting score equivalency between virtual and in-person assessment, score equivalency alone is not sufficient to validate a measure or an adaptation thereof. Furthermore, the majority of psychological measures on the market are normed for in-person administration. In this paper, we will not only review the pitfalls of reliability and validity but will also unpack the ethics of remote assessment as an equitable practice.

Confirmatory Factor Analysis of the WJ IV Cognitive: What Does the Standard Battery Measure at School Age?

This study aimed to evaluate the tenability of the proposed scoring/interpretive structure for the Woodcock-Johnson IV Test of Cognitive Abilities (WJ IV COG) Standard Battery configuration of subtests using confirmatory factor analysis (CFA) at school age. Results indicated that a three-factor hierarchical model, consistent with the CHC theory (Crystallized Ability, Fluid Reasoning, Short-Term Memory/Working Memory), provided the best fit to the WJ IV COG normative data. Whereas the preferred CHC interpretive structure was largely replicated, indices of interpretive relevance indicated that, among the Stratum II/III attributes that were located, only the omnibus general intelligence dimension should be interpreted with confidence. Nevertheless, several subtests contained adequate specificity to be interpreted in isolation apart from broad abilities. Implications for clinical interpretation are discussed.

A Psychometric Network Analysis of CHC Intelligence Measures: Implications for Research, Theory, and Interpretation of Broad CHC Scores "Beyond g"

For over a century, the structure of intelligence has been dominated by factor analytic methods that presume tests are indicators of latent entities (e.g., general intelligence or g). Recently, psychometric network methods and theories (e.g., process overlap theory; dynamic mutualism) have provided alternatives to g-centric factor models. However, few studies have investigated contemporary cognitive measures using network methods. We apply a Gaussian graphical network model to the age 9-19 standardization sample of the Woodcock-Johnson Tests of Cognitive Ability-Fourth Edition. Results support the primary broad abilities from the Cattell-Horn-Carroll (CHC) theory and suggest that the working memory-attentional control complex may be central to understanding a CHC network model of intelligence. Supplementary multidimensional scaling analyses indicate the existence of possible higher-order dimensions (PPIK; triadic theory; System I-II cognitive processing) as well as separate learning and retrieval aspects of long-term memory. Overall, the network approach offers a viable alternative to factor models with a g-centric bias (i.e., bifactor models) that have led to erroneous conclusions regarding the utility of broad CHC scores in test interpretation beyond the full-scale IQ, g.

Math anxiety as an independent psychological construct among social-emotional attitudes: An exploratory factor analysis

Math anxiety is a widespread problem for children and adolescents worldwide. However, the psychological trait of math anxiety is poorly understood. The present study aimed to examine the psychological construct of math anxiety among social-emotional attitudes. A total of 28,726 students, including 17,378 fourth graders and 11,348 eighth graders, were selected from the Qingdao Basic Education Quality Assessment database using multi-stage cluster random sampling. There were 10 questionnaires assessing social-emotional attitudes in the database. Pearson's correlation analyses were performed to examine the intercorrelations between social-emotional attitudes and mathematical performance. Exploratory factor analysis (EFA) was conducted to examine the psychological structure of these 10 social-emotional attitudes. After controlling for other social-emotional attitudes, math anxiety had a weak but significant correlation to mathematical performance in most subtests across grades. Among three-factor, four-factor, and five-factor EFA models, math anxiety was an independent factor that was separate from other social-emotional attitudes across six parallel subtests in grades four and eight. Math anxiety is a stable and independent psychological construct that is separate from other social-emotional attitudes. It suggests that math anxiety should be considered as a distinct anxiety disorder specific to mathematical learning in subsequent versions of the Diagnostic and Statistical Manual of Mental Disorders.

Dimensional Evaluation of Cognitive Measures: Methodological Confounds and Theoretical Concerns

The current study provides a methodological review of studies supporting a general factor of intelligence as the primary model for contemporary measures of cognitive abilities. A further evaluation is provided by an empirical evaluation that compares statistical estimates using different approaches in a large sample of children (ages 9–13 years, N = 780) administered a comprehensive battery of cognitive measures. Results from this study demonstrate the ramifications of using the bifactor and Schmid–Leiman (BF/SL) technique and suggest that using BF/SL methods limit interpretation of cognitive abilities to only a general factor. The inadvertent use of BF/SL methods is demonstrated to impact both model dimensionality and variance estimates for specific measures. As demonstrated in this study, conclusions from both exploratory and confirmatory studies using BF/SL methods are significantly questioned, especially for studies with a questionable theoretical basis. Guidelines for the interpretation of cognitive test scores in applied practice are discussed.

The Effects of Using Partial or Uncorrected Correlation Matrices When Comparing Network and Latent Variable Models

Network models of the WAIS-IV based on regularized partial correlation matrices have been reported to outperform latent variable models based on uncorrected correlation matrices. The present study sought to compare network and latent variable models using both partial and uncorrected correlation matrices with both types of models. The results show that a network model provided better fit to matrices of partial correlations but latent variable models provided better fit to matrices of full correlations. This result is due to the fact that the use of partial correlations removes most of the covariance common to WAIS-IV tests. Modeling should be based on uncorrected correlations since these represent the majority of shared variance between WAIS-IV test scores.

Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory

Researchers continue to debate the constructs measured by commercial ability tests. Factor analytic investigations of these measures have been used to develop and refine widely adopted psychometric theories of intelligence particularly the Cattell-Horn-Carroll (CHC) model. Even so, this linkage may be problematic as many of these investigations examine a particular instrument in isolation and CHC model specification across tests and research teams has not been consistent. To address these concerns, the present study used Monte Carlo resampling to investigate the latent structure of four of the most widely used intelligence tests for children and adolescents. The results located the approximate existence of the publisher posited CHC theoretical group factors in the Differential Abilities Scales-Second edition and the Kaufman Assessment Battery for Children-Second edition but not in the Wechsler Intelligence Scale for Children-Fifth edition or the Woodcock-Johnson IV Tests of Cognitive Abilities. Instead, the results supported alternative conceptualizations from independent factor analytic research. Additionally, whereas a bifactor model produced superior fit indices in two instruments (Wechsler Intelligence Scale for Children-Fifth edition and Woodcock-Johnson IV Tests of Cognitive Abilities), a higher order structure was found to be superior in the Kaufman Assessment Battery for Children-Second edition and the Differential Abilities Scales-Second edition. Regardless of the model employed, the general factor captured a significant portion of each instrument's variance. Implications for IQ test assessment, interpretation, and theory are discussed.

Exploratory Factor Analyses of the Intelligence and Development Scales-2: Implications for Theory and Practice

The factor structure of the intelligence and scholastic skills domains of the Intelligence and Development Scales-2 was examined using exploratory factor analyses with the standardization and validation sample (N = 2,030, aged 5 to 20 years). Results partly supported the seven proposed intelligence group factors. However, the theoretical factors Visual Processing and Abstract Reasoning as well as Verbal Reasoning and Long-Term Memory collapsed, resulting in a five-factor structure for intelligence. Adding the three scholastic skills subtests resulted in an additional factor Reading/Writing and in Logical-Mathematical Reasoning showing a loading on abstract Visual Reasoning and the highest general factor loading. A data-driven separation of intelligence and scholastic skills is not evident. Omega reliability estimates based on Schmid-Leiman transformations revealed a strong general factor that accounted for most of the true score variance both overall and at the group factor level. The possible usefulness of factor scores is discussed.

Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

Independent exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) research with the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) standardization sample has failed to provide support for the five group factors proposed by the publisher, but there have been no independent examinations of the WISC-V structure among clinical samples. The present study examined the latent structure of the 10 WISC-V primary subtests with a large (N = 2,512), bifurcated clinical sample (EFA, n = 1,256; CFA, n = 1,256). EFA did not support five factors as there were no salient subtest factor pattern coefficients on the fifth extracted factor. EFA indicated a four-factor model resembling the WISC-IV with a dominant general factor. A bifactor model with four group factors was supported by CFA as suggested by EFA. Variance estimates from both EFA and CFA found that the general intelligence factor dominated subtest variance and omega-hierarchical coefficients supported interpretation of the general intelligence factor. In both EFA and CFA, group factors explained small portions of common variance and produced low omega-hierarchical subscale coefficients, indicating that the group factors were of poor interpretive value.

The Woodcock-Johnson IV Tests of Achievement Provides Too Many Scores for Clinical Interpretation

The Woodcock-Johnson Tests of Achievement, Fourth Edition (WJ IV ACH) is purported to align with Cattell–Horn–Carroll (CHC) theory and offers upward of 20 scores within its interpretive and scoring system. The Technical Manual does not furnish validity evidence for the scores reported by the scoring system, suggesting that evidentiary support may be incomplete. Exploratory bifactor analysis (EBFA; maximum likelihood extraction with a bigeomin [orthogonal] rotation) was applied to the two school-aged correlation matrices at ages 9 to 19. Results indicated nonalignment with CHC theory and do not support the interpretation of most of the scores suggested by the scoring system. Instead, the results of this study suggest that the loading patterns diverge significantly from the interpretive system produced by the WJ IV ACH. Only the academic fluency and academic knowledge clusters emerged following the use of EBFA. Implications for clinical interpretation of the WJ IV ACH are offered.

Investigating the Theoretical Structure of the Differential Ability Scales—Second Edition Through Hierarchical Exploratory Factor Analysis

When the Differential Ability Scales–Second Edition (DAS-II) was developed, the instrument’s content, structure, and theoretical orientation were amended. Despite these changes, the Technical Handbook did not report results from exploratory factor analytic investigations, and confirmatory factor analyses were implemented using selected subtests across the normative age groups from the total battery. To address these omissions, the present study investigated the theoretical structure of the DAS-II using principal axis factoring followed by the Schmid–Leiman procedure with participants from the 5- to 8-year-old age range to determine the degree to which the DAS-II theoretical structure proposed in the Technical Handbook could be replicated. Unlike other age ranges investigated where at most 14 subtests were administered, the entire DAS-II battery was normed on participants aged 5 to 8 years, making it well suited to test the full instrument’s alignment with theory. Results suggested a six-factor solution that was essentially consistent with the Cattell–Horn–Carroll (CHC)-based theoretical structure suggested by the test publisher and simple structure was attained. The only exception involved two subtests (Picture Similarities and Early Number Concepts) that did not saliently load on a group factor. Implications for clinical practice are discussed.

What Does the WRAML2 Core Battery Measure? Utilizing Exploratory and Confirmatory Techniques to Disclose Higher Order Structure

The present study examined the factor structure of the Wide Range Assessment of Memory and Learning-Second Edition (WRAML2) core battery with participants from the normative sample aged 9 to 90 years ( n = 880) using higher order exploratory and confirmatory factor analytic techniques that were not reported in the in the WRAML2 Administration and Technical Manual. Exploratory factor analysis results suggested only one factor, whereas confirmatory factor analysis results favored the three factors posited by the test authors. Although model fit statistics were equivalent for the oblique, indirect hierarchical, and direct hierarchical measurement models, it was determined that the bifactor model best disclosed the influence of latent dimensions on WRAML2 manifest variables. In the three-factor bifactor model, the general factor accounted for 31% of the total variance and 69% of the common variance, whereas the three first-order factors combined accounted for 41% of the total variance and 31% of the common variance. Latent factor reliability coefficients (as estimated by ω_h) indicated that only the general factor was measured with enough precision to warrant confident clinical interpretation. Implications for clinical interpretation of WRAML2 scores and the procedures utilized in the development of related measures are discussed.