1
|
Holden LR, Tanenbaum GJ. Modern Assessments of Intelligence Must Be Fair and Equitable. J Intell 2023; 11:126. [PMID: 37367528 DOI: 10.3390/jintelligence11060126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2023] [Revised: 05/28/2023] [Accepted: 06/16/2023] [Indexed: 06/28/2023] Open
Abstract
Historically, assessments of human intelligence have been virtually synonymous with practices that contributed to forms of inequality and injustice. As such, modern considerations for assessing human intelligence must focus on equity and fairness. First, we highlight the array of diversity, equity, and inclusion concerns in assessment practices and discuss strategies for addressing them. Next, we define a modern, non-g, emergent view of intelligence using the process overlap theory and argue for its use in improving equitable practices. We then review the empirical evidence, focusing on sub-measures of g to highlight the utility of non-g, emergent models in promoting equity and fairness. We conclude with suggestions for researchers and practitioners.
Collapse
Affiliation(s)
- LaTasha R Holden
- Department of Psychology, University of Illinois Urbana-Champaign, Champaign, IL 61820, USA
- Beckman Institute, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA
| | - Gabriel J Tanenbaum
- Department of Psychology, University of Illinois Urbana-Champaign, Champaign, IL 61820, USA
| |
Collapse
|
2
|
McGill RJ. Confirmatory Factor Analysis of the WJ IV Cognitive: What Does the Standard Battery Measure at School Age? JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT 2023. [DOI: 10.1177/07342829231159440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
This study aimed to evaluate the tenability of the proposed scoring/interpretive structure for the Woodcock-Johnson IV Test of Cognitive Abilities (WJ IV COG) Standard Battery configuration of subtests using confirmatory factor analysis (CFA) at school age. Results indicated that a three-factor hierarchical model, consistent with the CHC theory (Crystallized Ability, Fluid Reasoning, Short-Term Memory/Working Memory), provided the best fit to the WJ IV COG normative data. Whereas the preferred CHC interpretive structure was largely replicated, indices of interpretive relevance indicated that, among the Stratum II/III attributes that were located, only the omnibus general intelligence dimension should be interpreted with confidence. Nevertheless, several subtests contained adequate specificity to be interpreted in isolation apart from broad abilities. Implications for clinical interpretation are discussed.
Collapse
|
3
|
A Psychometric Network Analysis of CHC Intelligence Measures: Implications for Research, Theory, and Interpretation of Broad CHC Scores "Beyond g". J Intell 2023; 11:jintelligence11010019. [PMID: 36662149 PMCID: PMC9865475 DOI: 10.3390/jintelligence11010019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 01/07/2023] [Accepted: 01/10/2023] [Indexed: 01/18/2023] Open
Abstract
For over a century, the structure of intelligence has been dominated by factor analytic methods that presume tests are indicators of latent entities (e.g., general intelligence or g). Recently, psychometric network methods and theories (e.g., process overlap theory; dynamic mutualism) have provided alternatives to g-centric factor models. However, few studies have investigated contemporary cognitive measures using network methods. We apply a Gaussian graphical network model to the age 9-19 standardization sample of the Woodcock-Johnson Tests of Cognitive Ability-Fourth Edition. Results support the primary broad abilities from the Cattell-Horn-Carroll (CHC) theory and suggest that the working memory-attentional control complex may be central to understanding a CHC network model of intelligence. Supplementary multidimensional scaling analyses indicate the existence of possible higher-order dimensions (PPIK; triadic theory; System I-II cognitive processing) as well as separate learning and retrieval aspects of long-term memory. Overall, the network approach offers a viable alternative to factor models with a g-centric bias (i.e., bifactor models) that have led to erroneous conclusions regarding the utility of broad CHC scores in test interpretation beyond the full-scale IQ, g.
Collapse
|
4
|
Examination of differential effects of cognitive abilities on reading and mathematics achievement across race and ethnicity: Evidence with the WJ IV. J Sch Psychol 2022; 93:1-27. [DOI: 10.1016/j.jsp.2022.05.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 03/28/2022] [Accepted: 05/30/2022] [Indexed: 11/24/2022]
|
5
|
Watkins MW, Canivez GL. Are There Cognitive Profiles Unique to Students With Learning Disabilities? A Latent Profile Analysis of Wechsler Intelligence Scale for Children–Fourth Edition Scores. SCHOOL PSYCHOLOGY REVIEW 2021. [DOI: 10.1080/2372966x.2021.1919923] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
6
|
Dombrowski SC, Watkins MW, McGill RJ, Canivez GL, Holingue C, Pritchard AE, Jacobson LA. Measurement Invariance of the Wechsler Intelligence Scale for Children, Fifth Edition 10-Subtest Primary Battery: Can Index Scores be Compared across Age, Sex, and Diagnostic Groups? JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT 2020. [DOI: 10.1177/0734282920954583] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10 subtest primary battery was evaluated across sex, age (6–8, 9–11, 12–14, and 15–16 year-olds), and three diagnostic (attention-deficit/hyperactivity disorder, anxiety, and encephalopathy) groups within a large clinical sample ( N = 5359) referred to a children’s specialty hospital. Competing models were tested using confirmatory factor analysis (CFA), and a five-factor oblique model corresponding to the publisher’s hypothesized first-order measurement model (e.g., verbal comprehension, fluid reasoning, visual-spatial, working memory, and processing speed) was found to have the best model fit. Multigroup CFA was subsequently used to evaluate progressively more restrictive constraints on the measurement model. Results indicated that full metric invariance was attained across the three groups studied. Full scalar invariance was attained for sex and diagnostic groups. Partial scalar invariance was attained for age-group. The results of this study provide support for the first-order scoring structure of the five WISC-V factors in the 10 subtest primary battery with this large clinical sample.
Collapse
Affiliation(s)
| | | | | | | | - Calliope Holingue
- Kennedy Krieger Institute, Johns Hopkins University School of Medicine, MD, USA
| | - Alison E. Pritchard
- Kennedy Krieger Institute, Johns Hopkins University School of Medicine, MD, USA
| | - Lisa A. Jacobson
- Kennedy Krieger Institute, Johns Hopkins University School of Medicine, MD, USA
| |
Collapse
|
7
|
Decker SL, Bridges RM, Luedke JC, Eason MJ. Dimensional Evaluation of Cognitive Measures: Methodological Confounds and Theoretical Concerns. JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT 2020. [DOI: 10.1177/0734282920940879] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The current study provides a methodological review of studies supporting a general factor of intelligence as the primary model for contemporary measures of cognitive abilities. A further evaluation is provided by an empirical evaluation that compares statistical estimates using different approaches in a large sample of children (ages 9–13 years, N = 780) administered a comprehensive battery of cognitive measures. Results from this study demonstrate the ramifications of using the bifactor and Schmid–Leiman (BF/SL) technique and suggest that using BF/SL methods limit interpretation of cognitive abilities to only a general factor. The inadvertent use of BF/SL methods is demonstrated to impact both model dimensionality and variance estimates for specific measures. As demonstrated in this study, conclusions from both exploratory and confirmatory studies using BF/SL methods are significantly questioned, especially for studies with a questionable theoretical basis. Guidelines for the interpretation of cognitive test scores in applied practice are discussed.
Collapse
|
8
|
Hajovsky DB, Villeneuve EF, Schneider WJ, Caemmerer JM. An Alternative Approach to Cognitive and Achievement Relations Research: An Introduction to Quantile Regression. JOURNAL OF PEDIATRIC NEUROPSYCHOLOGY 2020. [DOI: 10.1007/s40817-020-00086-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
9
|
Kranzler JH, Gilbert K, Robert CR, Floyd RG, Benson NF. Further Examination of a Critical Assumption Underlying the Dual-Discrepancy/Consistency Approach to Specific Learning Disability Identification. SCHOOL PSYCHOLOGY REVIEW 2019. [DOI: 10.17105/spr-2018-0008.v48-3] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
10
|
McGill RJ. An instrument in search of a theory: Structural validity of the Kaufman assessment battery for children‐second edition normative update at school‐age. PSYCHOLOGY IN THE SCHOOLS 2019. [DOI: 10.1002/pits.22304] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Ryan J. McGill
- Department of School Psychology and Counselor EducationCollege of William and MaryWilliamsburg Virginia
| |
Collapse
|
11
|
Hajovsky DB, Niileksela CR, Villeneuve EF, Reynolds MR. Gender Matters in Neuropsychological Assessment of Child and Adolescent Writing Skill. JOURNAL OF PEDIATRIC NEUROPSYCHOLOGY 2019. [DOI: 10.1007/s40817-019-00071-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
12
|
Dombrowski SC, McGill RJ, Morgan GB. Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory. Assessment 2019; 28:977-993. [PMID: 31431055 DOI: 10.1177/1073191119869828] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Researchers continue to debate the constructs measured by commercial ability tests. Factor analytic investigations of these measures have been used to develop and refine widely adopted psychometric theories of intelligence particularly the Cattell-Horn-Carroll (CHC) model. Even so, this linkage may be problematic as many of these investigations examine a particular instrument in isolation and CHC model specification across tests and research teams has not been consistent. To address these concerns, the present study used Monte Carlo resampling to investigate the latent structure of four of the most widely used intelligence tests for children and adolescents. The results located the approximate existence of the publisher posited CHC theoretical group factors in the Differential Abilities Scales-Second edition and the Kaufman Assessment Battery for Children-Second edition but not in the Wechsler Intelligence Scale for Children-Fifth edition or the Woodcock-Johnson IV Tests of Cognitive Abilities. Instead, the results supported alternative conceptualizations from independent factor analytic research. Additionally, whereas a bifactor model produced superior fit indices in two instruments (Wechsler Intelligence Scale for Children-Fifth edition and Woodcock-Johnson IV Tests of Cognitive Abilities), a higher order structure was found to be superior in the Kaufman Assessment Battery for Children-Second edition and the Differential Abilities Scales-Second edition. Regardless of the model employed, the general factor captured a significant portion of each instrument's variance. Implications for IQ test assessment, interpretation, and theory are discussed.
Collapse
|
13
|
Grieder S, Grob A. Exploratory Factor Analyses of the Intelligence and Development Scales-2: Implications for Theory and Practice. Assessment 2019; 27:1853-1869. [PMID: 31023061 DOI: 10.1177/1073191119845051] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The factor structure of the intelligence and scholastic skills domains of the Intelligence and Development Scales-2 was examined using exploratory factor analyses with the standardization and validation sample (N = 2,030, aged 5 to 20 years). Results partly supported the seven proposed intelligence group factors. However, the theoretical factors Visual Processing and Abstract Reasoning as well as Verbal Reasoning and Long-Term Memory collapsed, resulting in a five-factor structure for intelligence. Adding the three scholastic skills subtests resulted in an additional factor Reading/Writing and in Logical-Mathematical Reasoning showing a loading on abstract Visual Reasoning and the highest general factor loading. A data-driven separation of intelligence and scholastic skills is not evident. Omega reliability estimates based on Schmid-Leiman transformations revealed a strong general factor that accounted for most of the true score variance both overall and at the group factor level. The possible usefulness of factor scores is discussed.
Collapse
|
14
|
McGill RJ, Dombrowski SC, Canivez GL. Cognitive profile analysis in school psychology: History, issues, and continued concerns. J Sch Psychol 2018; 71:108-121. [DOI: 10.1016/j.jsp.2018.10.007] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2018] [Revised: 10/13/2018] [Accepted: 10/29/2018] [Indexed: 11/24/2022]
|
15
|
Meta-analysis of the relationship between academic achievement and broad abilities of the Cattell-horn-Carroll theory. J Sch Psychol 2018; 71:42-56. [DOI: 10.1016/j.jsp.2018.10.001] [Citation(s) in RCA: 42] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2018] [Revised: 07/26/2018] [Accepted: 10/08/2018] [Indexed: 12/22/2022]
|
16
|
Dombrowski SC, Beaujean AA, McGill RJ, Benson NF. The Woodcock-Johnson IV Tests of Achievement Provides Too Many Scores for Clinical Interpretation. JOURNAL OF PSYCHOEDUCATIONAL ASSESSMENT 2018. [DOI: 10.1177/0734282918800745] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The Woodcock-Johnson Tests of Achievement, Fourth Edition (WJ IV ACH) is purported to align with Cattell–Horn–Carroll (CHC) theory and offers upward of 20 scores within its interpretive and scoring system. The Technical Manual does not furnish validity evidence for the scores reported by the scoring system, suggesting that evidentiary support may be incomplete. Exploratory bifactor analysis (EBFA; maximum likelihood extraction with a bigeomin [orthogonal] rotation) was applied to the two school-aged correlation matrices at ages 9 to 19. Results indicated nonalignment with CHC theory and do not support the interpretation of most of the scores suggested by the scoring system. Instead, the results of this study suggest that the loading patterns diverge significantly from the interpretive system produced by the WJ IV ACH. Only the academic fluency and academic knowledge clusters emerged following the use of EBFA. Implications for clinical interpretation of the WJ IV ACH are offered.
Collapse
|
17
|
Canivez GL, Watkins MW, McGill RJ. Construct validity of the Wechsler Intelligence Scale For Children - Fifth UK Edition: Exploratory and confirmatory factor analyses of the 16 primary and secondary subtests. BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY 2018; 89:195-224. [DOI: 10.1111/bjep.12230] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2017] [Revised: 05/11/2018] [Indexed: 11/30/2022]
|
18
|
Canivez GL, Dombrowski SC, Watkins MW. Factor structure of the WISC-V in four standardization age groups: Exploratory and hierarchical factor analyses with the 16 primary and secondary subtests. PSYCHOLOGY IN THE SCHOOLS 2018. [DOI: 10.1002/pits.22138] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|