Hierarchical exploratory factor analyses of the Woodcock-Johnson IV Full Test Battery: Implications for CHC application in school psychology

Modern Assessments of Intelligence Must Be Fair and Equitable

Historically, assessments of human intelligence have been virtually synonymous with practices that contributed to forms of inequality and injustice. As such, modern considerations for assessing human intelligence must focus on equity and fairness. First, we highlight the array of diversity, equity, and inclusion concerns in assessment practices and discuss strategies for addressing them. Next, we define a modern, non-g, emergent view of intelligence using the process overlap theory and argue for its use in improving equitable practices. We then review the empirical evidence, focusing on sub-measures of g to highlight the utility of non-g, emergent models in promoting equity and fairness. We conclude with suggestions for researchers and practitioners.

Remote Assessment: Origins, Benefits, and Concerns

Although guidelines surrounding COVID-19 have relaxed and school-aged students are no longer required to wear masks and social distance in schools, we have become, as a nation and as a society, more comfortable working from home, learning online, and using technology as a platform to communicate ubiquitously across ecological environments. In the school psychology community, we have also become more familiar with assessing students virtually, but at what cost? While there is research suggesting score equivalency between virtual and in-person assessment, score equivalency alone is not sufficient to validate a measure or an adaptation thereof. Furthermore, the majority of psychological measures on the market are normed for in-person administration. In this paper, we will not only review the pitfalls of reliability and validity but will also unpack the ethics of remote assessment as an equitable practice.

A Psychometric Network Analysis of CHC Intelligence Measures: Implications for Research, Theory, and Interpretation of Broad CHC Scores "Beyond g"

For over a century, the structure of intelligence has been dominated by factor analytic methods that presume tests are indicators of latent entities (e.g., general intelligence or g). Recently, psychometric network methods and theories (e.g., process overlap theory; dynamic mutualism) have provided alternatives to g-centric factor models. However, few studies have investigated contemporary cognitive measures using network methods. We apply a Gaussian graphical network model to the age 9-19 standardization sample of the Woodcock-Johnson Tests of Cognitive Ability-Fourth Edition. Results support the primary broad abilities from the Cattell-Horn-Carroll (CHC) theory and suggest that the working memory-attentional control complex may be central to understanding a CHC network model of intelligence. Supplementary multidimensional scaling analyses indicate the existence of possible higher-order dimensions (PPIK; triadic theory; System I-II cognitive processing) as well as separate learning and retrieval aspects of long-term memory. Overall, the network approach offers a viable alternative to factor models with a g-centric bias (i.e., bifactor models) that have led to erroneous conclusions regarding the utility of broad CHC scores in test interpretation beyond the full-scale IQ, g.

Age and Sex Invariance of the Woodcock-Johnson IV Tests of Cognitive Abilities: Evidence from Psychometric Network Modeling

The Woodcock-Johnson IV Tests of Cognitive Abilities (WJ IV COG) is a comprehensive assessment battery designed to assess broad and narrow cognitive abilities, as defined by the Cattell-Horn-Carroll (CHC) theory of intelligence. Previous studies examined the invariance of the WJ assessments across sex and age groups using factor analytic methods. Psychometric network modeling is an alternative methodology that can address both direct and indirect relationships among the observed variables. In this study, we employed psychometric network modeling to examine the invariance of the WJ IV COG across sex and age groups. Using a normative sample (n = 4212 participants) representative of the United States population, we tested the extent to which the factorial structure of the WJ IV COG aligned with CHC theory for the school-aged sample. Next, we used psychometric network modeling as a data-driven method to investigate whether the network structure of the WJ IV COG remains similar across different sex and age (age 6 to 19, inclusively) groups. Our results showed that the WJ IV COG maintained the same network structure across all age and sex groups, although the network structure at younger ages indicated weaker relationships among some subtests. Overall, the results provide construct validity evidence for the WJ IV COG, based on both theoretical and data-driven methods.

Exploratory Factor Analyses of the French WISC-V (WISC-VFR) for Five Age Groups: Analyses Based on the Standardization Sample

This study investigated the factor structure of the French Wechsler Intelligence Scale for Children–Fifth Edition with five standardization sample age groups (6-7, 8-9, 10-11, 12-13, 14-16 years) using hierarchical exploratory factor analysis followed by Schmid–Leiman procedure. The primary research questions included (a) how many French Wechsler Intelligence Scale for Children–Fifth Edition factors should be extracted and retained in each age subgroup, (b) how are subtests associated with the latent factors, (c) was there evidence for the publisher’s claim of five first-order factors and separate Visual Spatial and Fluid Reasoning factors, (d) what proportion of variance was due to general intelligence versus the first-order group ability factors following a Schmid–Leiman procedure, and (e) do results support the age differentiation hypothesis? Results suggested that four factors might be sufficient for all five age groups and results did not support the distinction between Visual Spatial and Fluid Reasoning factors. While the general factor accounted for the largest portions of variance, the four first-order factors accounted for small unique portions of variance. Results did not support the age differentiation hypothesis because the number of factors remained the same across age groups, and there was no change in the percentage of variance accounted for by the general factor across age groups.

Measurement Invariance of the Wechsler Intelligence Scale for Children, Fifth Edition 10-Subtest Primary Battery: Can Index Scores be Compared across Age, Sex, and Diagnostic Groups?

Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10 subtest primary battery was evaluated across sex, age (6–8, 9–11, 12–14, and 15–16 year-olds), and three diagnostic (attention-deficit/hyperactivity disorder, anxiety, and encephalopathy) groups within a large clinical sample ( N = 5359) referred to a children’s specialty hospital. Competing models were tested using confirmatory factor analysis (CFA), and a five-factor oblique model corresponding to the publisher’s hypothesized first-order measurement model (e.g., verbal comprehension, fluid reasoning, visual-spatial, working memory, and processing speed) was found to have the best model fit. Multigroup CFA was subsequently used to evaluate progressively more restrictive constraints on the measurement model. Results indicated that full metric invariance was attained across the three groups studied. Full scalar invariance was attained for sex and diagnostic groups. Partial scalar invariance was attained for age-group. The results of this study provide support for the first-order scoring structure of the five WISC-V factors in the 10 subtest primary battery with this large clinical sample.

Exploratory Factor Analyses of the Intelligence and Development Scales-2: Implications for Theory and Practice

The factor structure of the intelligence and scholastic skills domains of the Intelligence and Development Scales-2 was examined using exploratory factor analyses with the standardization and validation sample (N = 2,030, aged 5 to 20 years). Results partly supported the seven proposed intelligence group factors. However, the theoretical factors Visual Processing and Abstract Reasoning as well as Verbal Reasoning and Long-Term Memory collapsed, resulting in a five-factor structure for intelligence. Adding the three scholastic skills subtests resulted in an additional factor Reading/Writing and in Logical-Mathematical Reasoning showing a loading on abstract Visual Reasoning and the highest general factor loading. A data-driven separation of intelligence and scholastic skills is not evident. Omega reliability estimates based on Schmid-Leiman transformations revealed a strong general factor that accounted for most of the true score variance both overall and at the group factor level. The possible usefulness of factor scores is discussed.

Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

Independent exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) research with the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) standardization sample has failed to provide support for the five group factors proposed by the publisher, but there have been no independent examinations of the WISC-V structure among clinical samples. The present study examined the latent structure of the 10 WISC-V primary subtests with a large (N = 2,512), bifurcated clinical sample (EFA, n = 1,256; CFA, n = 1,256). EFA did not support five factors as there were no salient subtest factor pattern coefficients on the fifth extracted factor. EFA indicated a four-factor model resembling the WISC-IV with a dominant general factor. A bifactor model with four group factors was supported by CFA as suggested by EFA. Variance estimates from both EFA and CFA found that the general intelligence factor dominated subtest variance and omega-hierarchical coefficients supported interpretation of the general intelligence factor. In both EFA and CFA, group factors explained small portions of common variance and produced low omega-hierarchical subscale coefficients, indicating that the group factors were of poor interpretive value.

The Woodcock-Johnson IV Tests of Achievement Provides Too Many Scores for Clinical Interpretation

The Woodcock-Johnson Tests of Achievement, Fourth Edition (WJ IV ACH) is purported to align with Cattell–Horn–Carroll (CHC) theory and offers upward of 20 scores within its interpretive and scoring system. The Technical Manual does not furnish validity evidence for the scores reported by the scoring system, suggesting that evidentiary support may be incomplete. Exploratory bifactor analysis (EBFA; maximum likelihood extraction with a bigeomin [orthogonal] rotation) was applied to the two school-aged correlation matrices at ages 9 to 19. Results indicated nonalignment with CHC theory and do not support the interpretation of most of the scores suggested by the scoring system. Instead, the results of this study suggest that the loading patterns diverge significantly from the interpretive system produced by the WJ IV ACH. Only the academic fluency and academic knowledge clusters emerged following the use of EBFA. Implications for clinical interpretation of the WJ IV ACH are offered.

Investigating the Theoretical Structure of the Differential Ability Scales—Second Edition Through Hierarchical Exploratory Factor Analysis

When the Differential Ability Scales–Second Edition (DAS-II) was developed, the instrument’s content, structure, and theoretical orientation were amended. Despite these changes, the Technical Handbook did not report results from exploratory factor analytic investigations, and confirmatory factor analyses were implemented using selected subtests across the normative age groups from the total battery. To address these omissions, the present study investigated the theoretical structure of the DAS-II using principal axis factoring followed by the Schmid–Leiman procedure with participants from the 5- to 8-year-old age range to determine the degree to which the DAS-II theoretical structure proposed in the Technical Handbook could be replicated. Unlike other age ranges investigated where at most 14 subtests were administered, the entire DAS-II battery was normed on participants aged 5 to 8 years, making it well suited to test the full instrument’s alignment with theory. Results suggested a six-factor solution that was essentially consistent with the Cattell–Horn–Carroll (CHC)-based theoretical structure suggested by the test publisher and simple structure was attained. The only exception involved two subtests (Picture Similarities and Early Number Concepts) that did not saliently load on a group factor. Implications for clinical practice are discussed.