Construct Validity of the WISC-V in Clinical Cases: Exploratory and Confirmatory Factor Analyses of the 10 Primary Subtests

Evaluating the Construct Validity of the Charité Alarm Fatigue Questionnaire using Confirmatory Factor Analysis

Background

The Charité Alarm Fatigue Questionnaire (CAFQa) is a 9-item questionnaire that aims to standardize how alarm fatigue in nurses and physicians is measured. We previously hypothesized that it has 2 correlated scales, one on the psychosomatic effects of alarm fatigue and the other on staff's coping strategies in working with alarms.

Objective

We aimed to validate the hypothesized structure of the CAFQa and thus underpin the instrument's construct validity.

Methods

We conducted 2 independent studies with nurses and physicians from intensive care units in Germany (study 1: n=265; study 2: n=1212). Responses to the questionnaire were analyzed using confirmatory factor analysis with the unweighted least-squares algorithm based on polychoric covariances. Convergent validity was assessed by participants' estimation of their own alarm fatigue and exposure to false alarms as a percentage.

Results

In both studies, the χ2 test reached statistical significance (study 1: χ226=44.9; P=.01; study 2: χ226=92.4; P<.001). Other fit indices suggested a good model fit (in both studies: root mean square error of approximation <0.05, standardized root mean squared residual <0.08, relative noncentrality index >0.95, Tucker-Lewis index >0.95, and comparative fit index >0.995). Participants' mean scores correlated moderately with self-reported alarm fatigue (study 1: r=0.45; study 2: r=0.53) and weakly with self-perceived exposure to false alarms (study 1: r=0.3; study 2: r=0.33).

Conclusions

The questionnaire measures the construct of alarm fatigue as proposed in our previous study. Researchers and clinicians can rely on the CAFQa to measure the alarm fatigue of nurses and physicians.

Intelligence Assessment of Children & Youth Benefiting from Psychological-Educational Support System in Poland

This article presents a unique dataset comprising 419,135 intelligence assessment results. The study utilised the Polish adaptation of the Stanford-Binet Intelligence Scale 5 during individual diagnostic sessions conducted under natural conditions. The research included children aged 3;0-18;11 of both genders who had been referred to support institutions (psychological-educational counselling centres, post-hospital clinics, hospital departments) by preschools or schools, or voluntarily requested by parents with their consent. The data collection spanned the entire country of Poland from 2018 to 2023. In addition to comprehensive intelligence assessment results, the dataset contains valuable demographic information, enabling in-depth analyses. The dataset's uniqueness lies in its impressive sample size, encompassing over four hundred thousand observations as well as the utilisation of time-consuming and thorough intelligence assessment procedures in settings that mimic the real world. Moreover, the context of the study is noteworthy, as the participants are individuals benefiting from the publicly - available Polish psychological support system.

Autism Symptom Presentation and Hierarchical Models of Intelligence

BACKGROUND

There is a substantial history studying the relationship between general intelligence and the core symptoms of autism. However, a gap in knowledge is how dimensional autism symptomatology associates with different components of clinically-relevant hierarchical models of intelligence.

METHOD

We examined correlations between autism diagnostic symptom magnitude (Autism Diagnostic Observational Schedule; ADOS) and a hierarchical statistical model of intelligence. One autistic cohort was tested on the fourth edition of Wechsler Intelligence Scale for Children (WISC-IV; N = 131), and another on the fifth edition (WISC-V; N = 83). We anticipated a convergent pattern of results between cohorts.

RESULTS

On WISC-IV, ADOS scores were correlated significantly with g and three out of four intermediate factor scores, which was a broader pattern of correlations than anticipated from the literature. In the WISC-V cohort, only one intermediate factor correlated significantly with the ADOS; correlations with g and the other intermediate factors were less statistically certain. ADOS-factor correlations were larger in the WISC-IV than WISC-V cohort; this difference was significant at the 90% level.

CONCLUSIONS

WISC-IV shows dimensional relationships with ADOS at multiple points in the hierarchical model of intelligence. Moreover, the current results provide evidence that relationship between core autism symptomatology and the construct of general intelligence may depend on how intelligence is measured. Known cohort effects in the relationship between categorical autism diagnosis and general intelligence have previously been attributed to changes in autism diagnostic practices. To our knowledge, this is the first evidence that differing versions of IQ tests may be implicated.

Measurement Invariance of the WISC-V across a Clinical Sample of Children and Adolescents with ADHD and a Matched Control Group

Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10-primary subtest battery was analyzed across a group of children and adolescents with ADHD (n = 91) and a control group (n = 91) matched by sex, age, migration background, and parental education or type of school. First, confirmatory factor analyses (CFAs) were performed to establish the model fit for the WISC-V second-order five-factor model in each group. A sufficiently good fit of the model was found for the data in both groups. Subsequently, multigroup confirmatory factor analyses (MGCFAs) were conducted to test for measurement invariance across the ADHD and control group. Results of these analyses indicated configural and metric invariance but did not support full scalar invariance. However, after relaxing equality constraints on the Vocabulary (VC), Digit Span (DS), Coding (CD), Symbol Search (SS), and Picture Span (PS) subtest intercepts as well as on the intercepts of the first-order factors Working Memory (WM) and Processing Speed (PS), partial scalar invariance could be obtained. Furthermore, model-based reliability coefficients indicated that the WISC-V provides a more precise measurement of general intelligence (e.g., represented by the Full-Scale IQ, FSIQ) than it does for cognitive subdomains (e.g., represented by the WISC-V indexes). Group comparisons revealed that the ADHD group scored significantly lower than the control group on four primary subtests, thus achieving significantly lower scores on the corresponding primary indexes and the FSIQ. Given that measurement invariance across the ADHD and the control group could not be fully confirmed for the German WISC-V, clinical interpretations based on the WISC-V primary indexes are limited and should only be made with great caution regarding the cognitive profiles of children and adolescents with ADHD.

Measuring intelligence in Autism and ADHD: Measurement invariance of the-Binet 5th edition and impact of subtest scatter on abbreviated IQ accuracy

Scatter and heterogeneity in cognitive profiles is thought to be common in autism spectrum disorder (ASD), which may indicate differences in the construct of IQ. However, less research has investigated IQ scatter in attention-deficit/hyperactivity disorder (ADHD). Scatter is also thought to negatively impact the predictive validity of IQ summary scores, although there is research refuting this notion. Abbreviated IQ tests, such as the Stanford-Binet fifth edition (SB-5) abbreviated battery IQ (ABIQ), may be especially susceptible to the influence of scatter. We tested the measurement invariance of the SB-5 as well as the predictive validity of the ABIQ in predicting FSIQ in 1679 youth (21% female) ages 2-16 years with a clinical diagnosis of ASD or ADHD. Results indicated the SB-5 is measuring IQ the same way in ASD and ADHD. There were no differences between diagnostic groups in scatter between ABIQ (i.e., routing) subtests. Additionally, scatter was not related to dimensional autistic traits. Higher degree of scatter was associated with poorer predictive validity of the ABIQ and a higher likelihood of overestimating FSIQ, regardless of diagnosis. Overall, we found more similarities than differences between the ASD and ADHD groups. Our results show that the SB-5 ABIQ is generally a strong predictor of FSIQ in youth with neurodevelopmental disorders. However, the use of the SB-5 ABIQ in research and clinical applications, without consideration of scatter on routing subtests, is potentially problematic.

Cross-National Generalizability of WISC-V and CHC Broad Ability Constructs across France, Spain, and the US

The Cattell-Horn-Carroll (CHC) model is based on psychometric cognitive ability research and is the most empirically supported model of cognitive ability constructs. This study is one in a series of cross-national comparisons investigating the equivalence and generalizability of psychological constructs which align with the CHC model. Previous research exploring the cross-cultural generalizability of cognitive ability measures concluded that the factor analytic models of cognitive abilities generalize across cultures and are compatible with well-established CHC constructs. The equivalence of the psychological constructs, as measured by the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V), has been established across English-speaking samples. However, few studies have explored the equivalence of psychological constructs across non-English speaking, nationally representative samples. This study explored the equivalence of the WISC-V five-factor model across standardization samples from France, Spain, and the US. The five-factor scoring model demonstrated excellent fit across the three samples independently. Factorial invariance was investigated and the results demonstrated strict factorial invariance across France, Spain, and the US. The results provide further support for the generalizability of CHC constructs across Western cultural populations that speak different languages and support the continued use and development of the CHC model as a common nomenclature and blueprint for cognitive ability researchers and test developers. Suggestions for future research on the CHC model of intelligence are discussed.

Factorial Validity of the German KABC-II at Ages 7 to 12 in a Clinical Sample: Four Factors Fit Better than Five

Multidimensional intelligence test batteries such as the KABC-II are widely used in clinical practice. Although validity evidence should be provided for all intended uses of a test, data on the factorial validity of the KABC-II mostly relies on the standardization samples and raises some concerns about the adequacy of the factor structure. Confirmatory factor analyses of the KABC-II core subtests were conducted in a sample of 627 children who had been assessed in German Centers for Social Pediatrics. The standard structure of the KABC-II was superior to unidimensional models but, as in previous research, evidenced cross-loadings and a high correlation between Planning/Gf and Simultaneous/Gv. Pattern Reasoning was more closely related to Simultaneous/Gv than to Planning/Gf. A four-factorial structure combining subtests from Planning/Gf and Simultaneous/Gv to form a common factor emerged as a better representation of the data. Story Completion showed a secondary loading on Knowledge/Gc. On average, most subtest variance was accounted for by the general factor. Models with bonus points for fast responses generally fitted worse than those without. Clinicians should be aware that Planning/Gf and Simultaneous/Gv measure both visual and fluid abilities. Scales of the KABC-II should not be interpreted as dimensions independent of the general factor.

Bootstrap Exploratory Graph Analysis of the WISC-V with a Clinical Sample

One important aspect of construct validity is structural validity. Structural validity refers to the degree to which scores of a psychological test are a reflection of the dimensionality of the construct being measured. A factor analysis, which assumes that unobserved latent variables are responsible for the covariation among observed test scores, has traditionally been employed to provide structural validity evidence. Factor analytic studies have variously suggested either four or five dimensions for the WISC-V and it is unlikely that any new factor analytic study will resolve this dimensional dilemma. Unlike a factor analysis, an exploratory graph analysis (EGA) does not assume a common latent cause of covariances between test scores. Rather, an EGA identifies dimensions by locating strongly connected sets of scores that form coherent sub-networks within the overall network. Accordingly, the present study employed a bootstrap EGA technique to investigate the structure of the 10 WISC-V primary subtests using a large clinical sample (N = 7149) with a mean age of 10.7 years and a standard deviation of 2.8 years. The resulting structure was composed of four sub-networks that paralleled the first-order factor structure reported in many studies where the fluid reasoning and visual-spatial dimensions merged into a single dimension. These results suggest that discrepant construct and scoring structures exist for the WISC-V that potentially raise serious concerns about the test interpretations of psychologists who employ the test structure preferred by the publisher.

Autism and Hierarchical Models of Intelligence

BACKGROUND

The Wechsler Intelligence Scale for Children (WISC) employs a hierarchical model of general intelligence in which index scores separate out different clinically-relevant aspects of intelligence; the test is designed such that index scores are statistically independent from one another within the normative sample. Whether or not the existing index scores meet the desired psychometric property of being statistically independent within autistic samples is unknown.

METHOD

We conducted a factor analysis on WISC fifth edition (WISC-V) (N = 83) and WISC fourth edition (WISC-IV) (N = 131) subtest data in children with autism. We compared the data-driven exploratory factor analysis with the manual-derived index scores, including in a typically developing (TD) WISC-IV cohort (N = 209).

RESULTS

The WISC-IV TD cohort showed the expected 1:1 relationship between empirically derived factors and manual-derived index scores. We observed less unique correlations between our data-driven factors and manualized IQ index scores in both ASD samples (WISC-IV and WISC-V). In particular, in both WISC-IV and -V, working memory (WM) influenced index scores in autistic individuals that do not load on WM in the normative sample.

CONCLUSIONS

WISC index scores do not show the desired statistical independence within autistic samples, as judged against an empirically-derived exploratory factor analysis. In particular, within the currently used WISC-V version, WM influences multiple index scores.

Assessing IQ in adolescents with mild to moderate cerebral palsy using the WISC-V

Objective: To examine the influence of subtests that require fine motor responses on measures of intellectual ability, and compare three approaches to minimizing motor demands while assessing cognitive abilities in adolescents with cerebral palsy (CP) to the traditional method of the Wechsler Intelligence Scale for Children - Fifth edition (WISC-V). Method: Seventy adolescents with CP (M = 14 years 6 months, SD = 10 months) who were able to provide either a verbal or point response were assessed using the WISC-V administered via Q-interactive. The pencil-to-paper version of Coding was also administered. Performance on Block Design and pencil-to-paper Coding was compared to Visual Puzzles and Coding on Q-interactive, respectively. Full Scale IQ (FSIQ) scores derived according to the Traditional method of the WISC-V were compared to alternative estimates of FSIQ derived according to the Q-interactive, Nonmotor, and Motor-free methods, which minimized motor demands. Results: An additional 7-12% of participants were able to respond to Visual puzzles and Coding on Q-interactive compared to Block Design and pencil-to-paper Coding, respectively, and performance was marginally but significantly better. For 54 adolescents (Gross Motor Function Classification System (GMFCS) Level I-III) who were able to obtain FSIQ scores, the Traditional method underestimated FSIQ by 3-6 points compared to the alternative methods and the difference was most pronounced for those with more severe CP as measured by the GMFCS. Conclusion: Adolescents with CP are at an inherent disadvantage when cognitive ability is assessed using the Traditional method of the WISC-V. Findings suggest clinicians should employ the Nonmotor or Motor-free methods when assessing IQ in adolescents with CP.

Evidence of the Internal Structure and Measurement Invariance of the BASC-3 Behavioral and Emotional Screening System Teacher Form

The Behavior Assessment System for Children, Third Edition (BASC-3) Behavioral and Emotional Screening System Teacher Form (BESS) is a universal screening measure designed to identify social, emotional, and behavioral risks in students. A series of confirmatory factor analyses (CFAs) were conducted to examine the extent to which the developer-proposed factor model fits the BASC-3 BESS data within an elementary K-5 sample of students ( N = 1472). Results suggested a higher-order factor structure, which was specified in accordance with developer recommendations, provided acceptable fit to the BASC-3 BESS data. Measurement invariance of the factor structure across student age levels was then examined via a series of multi-group CFA models. Introduction Multi-group CFA findings supported the configural, metric, and scalar invariance of the factor structure across age levels. Practical implications for educators and school psychologists interested in universal screening and directions for future research are discussed.

Long-term stability of Wechsler Intelligence Scale for Children-fifth edition scores in a clinical sample

This study investigated the stability of Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) scores for 225 children and adolescents from an outpatient neuropsychological clinic across, on average, a 2.6 year test-retest interval. WISC-V mean scores were relatively constant but subtest stability score coefficients were all below 0.80 (M = 0.66) and only the Verbal Comprehension Index (VCI), Visual Spatial Index (VSI), and omnibus Full Scale IQ (FSIQ) stability coefficients exceeded 0.80. Neither intraindividual subtest difference scores nor intraindividual composite difference scores were stable across time (M = 0.26 and 0.36, respectively). Rare and unusual subtest and composite score differences as well as subtest and index scatter at initial testing were unlikely to be repeated at retest (kappa = 0.03 to 0.49). It was concluded that VCI, VSI, and FSIQ scores might be sufficiently stable to support normative comparisons but that none of the intraindividual (i.e. idiographic, ipsative, or person-relative) measures were stable enough for confident clinical decision making.

Factor Structure of the KABC-II at Ages 5 and 6: Is It Valid in a Clinical Sample?

The factor structure of the German edition of the KABC-II for ages 5 and 6 was examined in a clinical sample. Participants were 450 children ages 5 and 6 who had been assessed due to various behavioral, emotional, or developmental disorders in five Centers for Social Pediatrics (SPCs). Confirmatory factor analyses of the standard test structure including core subtests of the Cattell-Horn-Carroll model and of the Luria model were conducted using maximum likelihood estimation. Several modified structures derived from CHC ability classifications were evaluated. Second-order factor structures corresponding to the standard test structure of the KABC-II demonstrated an adequate global fit for both theoretical models and were superior to unidimensional models. The fit of bifactor models was comparable to second-order models. In all subtests, the general factor accounted for more variance than group factors (broad abilities). However, in more than half of the subtests, unique variance explained the largest portion of the variance. The scale Learning/Glr showed a lack of convergent validity. At age 6, a model omitting subtest Rover significantly improved the fit. In the combined sample of 5- and 6-year-old children, both second-order and bifactor models with nine subtests demonstrated excellent fit.

A Validated WISC-V Short-Form to Estimate Intellectual Functioning in Very Preterm Children at Early School Age

Very preterm children (gestational age < 32 weeks) frequently show neurodevelopmental difficulties (Inattention/dysexecutiveness) throughout their life-stages. A scarcity of resources, along with this population’s cognitive vulnerability, makes the neuropsychological evaluation of these children both complicated and time-consuming. This study aimed to develop a specific and valid Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V) short-form to estimate intellectual functioning in this population. Eighty-four very preterm children (39 female; mean age = 6.50; SD: 0.06) were assessed with the WISC-V. Short-forms were developed following two independent strategies: a) multiple linear regressions for each index; b) correlational analyses between scores on all administered subtests and Full-Scale IQ. Validity of short-forms was analyzed. A short-form (Vocabulary, Matrix Reasoning, Picture Span, and Symbol Search) that satisfied 2/3 validation criteria was proposed. This validated short-form could facilitate the identification of cognitive difficulties in very preterm children, so that they could benefit from early care and support services, avoiding long assessment procedures.

Factor structure of the new Scandinavian WISC-V version: Support for a five-factor model

The fifth version of Wechsler Intelligence Scale for Children is designed to measure five distinct aspects of intelligence, incorporating a new fluid reasoning index to the four indexes of the previous fourth version. Several factor analyses, however, have failed to support the fifth factor. The Scandinavian version is the only national version not showing clear superiority for the five-factor solution in the Manual. In the present study, we analyze WISC-V protocols from a clinical sample of 237 children tested with the new Scandinavian version. We perform six confirmatory factor analyses (CFA) testing three hierarchical-, two bifactor-, and one correlated candidate factor models. The study shows that the three-factor model does not fit the data, and that all four- and five-factor models showed good fit. The four-factor bifactor model was somewhat better than the five-bifactor and hierarchical models, but the correlated five-factor model was the superior model. Finding support for five-factors in a clinical sample representative of those most probable to be tested with the test, strengthen the claim that also the Scandinavian version measure a distinct fluid factor as intended by the test owners, and thus that clinicians may use the index scores as their main level of analysis. Review of previous CFAs show that the choice of statistical methods for CFA, bifactor or hierarchical/correlated, influence whether second order factor models are better than g-factor models.

The dimensionality of the Behavior Rating Inventory of Executive Function, Second Edition in a clinical sample

The parent-report Behavior Rating Inventory of Executive Function, Second Edition (BRIEF-2) is a widely used pediatric neuropsychological measure. Unfortunately, despite meaningful changes from its predecessor, few studies have examined its internal factor structure (now with Behavioral Regulation [BRI], Emotion Regulation [ERI], and Cognitive Regulation [CRI]), and no available literature has investigated higher order models for the BRIEF-2. This study sought to address this shortcoming in the literature by investigating and reporting on the dimensionality of the parent-report BRIEF-2 in a clinical sample. Two hundred and two (202) pediatric neuropsychology examinees (M age = 9.90; 68% males) with complete data for the parent-report BRIEF-2 were included. Descriptive results revealed generally elevated scores across BRIEF-2 scales (Global Executive Composite M T = 70.16). Exploratory factor analyses suggested two factors (CRI and BRI/ERI) should be extracted, and that higher order models should be considered. Confirmatory factor analyses suggested that a direct hierarchical/bifactor two-factor structure (which was more parsimonious than the theoretical three-factor model) provided the best fit, with a bulk of the variance explained by the general GEC factor. The BRIEF-2 may be best interpreted at the overall level, with relatively less weight given to the index variables, particularly within clinical samples with high levels of reported executive functioning difficulties. Implications of these findings, limitations of the present study, and appropriate directions for future inquiry were discussed.

Exploratory Factor Analyses of the French WISC-V (WISC-VFR) for Five Age Groups: Analyses Based on the Standardization Sample

This study investigated the factor structure of the French Wechsler Intelligence Scale for Children–Fifth Edition with five standardization sample age groups (6-7, 8-9, 10-11, 12-13, 14-16 years) using hierarchical exploratory factor analysis followed by Schmid–Leiman procedure. The primary research questions included (a) how many French Wechsler Intelligence Scale for Children–Fifth Edition factors should be extracted and retained in each age subgroup, (b) how are subtests associated with the latent factors, (c) was there evidence for the publisher’s claim of five first-order factors and separate Visual Spatial and Fluid Reasoning factors, (d) what proportion of variance was due to general intelligence versus the first-order group ability factors following a Schmid–Leiman procedure, and (e) do results support the age differentiation hypothesis? Results suggested that four factors might be sufficient for all five age groups and results did not support the distinction between Visual Spatial and Fluid Reasoning factors. While the general factor accounted for the largest portions of variance, the four first-order factors accounted for small unique portions of variance. Results did not support the age differentiation hypothesis because the number of factors remained the same across age groups, and there was no change in the percentage of variance accounted for by the general factor across age groups.

Don’t Use a Bifactor Model Unless You Believe the True Structure Is Bifactor

The current article provides a response to concerns raised by Dombrowski, McGill, Canivez, Watkins, & Beaujean (2020) regarding the methodological confounds identified by Decker, Bridges, Luedke, and Eason (2020) for using a bifactor (BF) model and Schmid–Leiman (SL) procedure in previous studies supporting a general factor of intelligence (i.e., “g”). While Dombrowski et al. (2020) raised important theoretical and practical issues, the theoretical justification for using a BF model and SL procedure to identify cognitive dimensions remain unaddressed, as well as significant concerns for using these statistical methods as the basis for informing the use of cognitive tests in clinical applications.

Satter Eating Competence Inventory Subscale Restructure After Confirmatory Factor Analysis

OBJECTIVE

To examine the structural validity of the empirically and theoretically derived Satter Eating Competence Inventory (ecSI 2.0).

METHODS

Confirmatory factor analyses with 3 databases and a merger with 3 additional data sets (n = 2,010) from demographically heterogeneous samples.

RESULTS

Findings supported retaining all 16 items and migration of the item I trust myself to eat enough for me from the Internal Regulation to Eating Attitudes subscales, providing 4 distinct but correlated subscales and an adequately fitting model, χ²(98) = 1,364.27, P < .001, comparative fit index = .91, root mean square error of approximation = .07, 95% confidence interval, 0.07-0.08, standardized root mean square residual = .05.

CONCLUSION AND IMPLICATIONS

Psychometric integrity of the 16-item ecSI 2.0 was affirmed. Continued examination is recommended with cultural and linguistically diverse samples and network analyses to identify dynamic interactions among items.

Monte Carlo Modeling of Contemporary Intelligence Test (IQ) Factor Structure: Implications for IQ Assessment, Interpretation, and Theory

Researchers continue to debate the constructs measured by commercial ability tests. Factor analytic investigations of these measures have been used to develop and refine widely adopted psychometric theories of intelligence particularly the Cattell-Horn-Carroll (CHC) model. Even so, this linkage may be problematic as many of these investigations examine a particular instrument in isolation and CHC model specification across tests and research teams has not been consistent. To address these concerns, the present study used Monte Carlo resampling to investigate the latent structure of four of the most widely used intelligence tests for children and adolescents. The results located the approximate existence of the publisher posited CHC theoretical group factors in the Differential Abilities Scales-Second edition and the Kaufman Assessment Battery for Children-Second edition but not in the Wechsler Intelligence Scale for Children-Fifth edition or the Woodcock-Johnson IV Tests of Cognitive Abilities. Instead, the results supported alternative conceptualizations from independent factor analytic research. Additionally, whereas a bifactor model produced superior fit indices in two instruments (Wechsler Intelligence Scale for Children-Fifth edition and Woodcock-Johnson IV Tests of Cognitive Abilities), a higher order structure was found to be superior in the Kaufman Assessment Battery for Children-Second edition and the Differential Abilities Scales-Second edition. Regardless of the model employed, the general factor captured a significant portion of each instrument's variance. Implications for IQ test assessment, interpretation, and theory are discussed.