Test–retest reliability of Riding’s cognitive styles analysis test

Analytic and holistic cognitive style as a set of independent manifests: Evidence from a validation study of six measurement instruments

Cognitive styles are commonly studied constructs in cognitive psychology. The theory of field dependence-independence was one of the most important cognitive styles. Yet in the past, its measurement had significant shortcomings in validity and reliability. The theory of analytic and holistic cognitive styles attempted to extend this theory and overcome its shortcomings. Unfortunately, the psychometric properties of its measurement methods were not properly verified. Furthermore, new statistical approaches, such as analysis of reaction times, have been overlooked by current research. The aim of this pre-registered study was to verify the psychometric properties (i.e., factor structure, split-half reliability, test-retest reliability, discriminant validity with intelligence and personality, and divergent, concurrent and predictive validity) of several methods routinely applied in the field. We developed/adapted six methods based on self-report questionnaires, rod-and-frame principles, embedded figures, and hierarchical figures. The analysis was conducted on 392 Czech participants, with two data collection waves. The results indicate that the use of methods based on the rod-and-frame principle may be unreliable, demonstrating no absence of association with intelligence. The use of embedded and hierarchical figures is recommended. The self-report questionnaire used in this study showed an unsatisfactory factor structure and also cannot be recommended without futher validation on independent samples. The findings also did not correspond with the original two-dimensional theory.

Cognitive Styles, Gender, and Student Academic Performance in Engineering Education

Cognitive styles affect the learning process positively if tasks are matched to the cognitive style of learners. This effect becomes more pronounced in complex education, such as in engineering. We attempted to critically assess the effect of cognitive styles and gender on students’ academic performance in eight engineering majors to understand whether a cognitive style preference is associated with certain majors. We used the Cognitive Style Indicator (CoSI) with a sample of n = 584 engineering students. Multiple standard statistical tests, regression tree analysis, and cluster analysis showed that none of the three cognitive styles was exclusively associated with better performance. However, students who had a stronger preference for a cognitive style were more likely to perform better. Gender, the major, and students’ clarity about their cognitive style were shown to be the best predictors of academic performance. Female students performed better and were clearer about their preferred cognitive style, whereas male students were more capable of adapting to different learning tasks. Furthermore, certain engineering majors were shown to be associated with certain cognitive styles. We concluded the study with theoretical and practical implications for engineering education and suggestions for further research.

Unkept promises of cognitive styles: A new look at old measurements

Cognitive style is thought to be a stable marker of one's way to approach mental operations. While of wide interest over the last decades, its operationalization remains a challenge. The literature indicates that cognitive styles assessed via i) questionnaires are predicted by personality and ii) performance tests (e.g., Group Embedded Figures Test; GEFT) are related to general intelligence. In the first study, we tested the psychometric relationship between the Cognitive Style Index questionnaire (CSI) and personality inventories (NEO Five Factor Inventory; NEO-FFI, HEXACO Personality Inventory Revised; HEXACO-PI-R). In the second study, we assessed the CSI, NEO-FFI, GEFT and a general intelligence test (Raven's Standard Progressive Matrices Test; RSMT). We found that CSI scores were largely predicted by personality and that CSI was uncorrelated with GEFT performance. Instead, better performance on the GEFT was associated with better performance on the RSMT. We conclude that i) cognitive style questionnaires overlap with personality inventories, ii) cognitive style performance tests do not measure cognitive styles and should not be used as such and iii) the cognitive style concept needs to be assessed with alternative measurement types. We discuss possible future directions.

The impact of users’ verbal/imagery cognitive styles on their Web search behavior

Purpose

– The purpose of this paper is to investigate the effects of verbal-imagery cognitive styles of information searching behavior of users in using the Web.

Design/methodology/approach

– In all, 44 participants were recruited for this study. The participants’ cognitive styles were measured by using Riding's Cognitive Style Analysis test. Three search tasks were designed based on Kim's search task definitions. Moreover, an individual lab session was arranged and then participants’ memos were analyzed using content analysis.

Findings

– In all, 48 strategies in four categories of behaviors in searching the Web were identified. There were associations between users’ cognitive styles and their information searching behavior. The participants’ selection of the search initiation behaviors varied, so that imagers suffered from more varied initial behavior than verbalizers. The verbalizers tended to search in a narrow area, then broadening the area and following structured navigation and reading behavior to process information, while imagers tended to search in a general area, then narrowing down the search and adopting mixed navigational styles and mixed behaviors to process information. This study revealed that there was a difference in search performance of verbalizers and imagers descriptively, as verbalizers spent more time compared to imagers and imagers visited more nodes than verbalizers for the tasks completion. In addition, the task was an important variable influencing the search performance. Based on the key findings (search initiation behaviors, formulating search queries, navigational behaviors, information processing behaviors), a conceptual pattern of Web searching and cognitive styles is presented.

Research limitations/implications

– The study provides a new understanding of Web users’ information search behavior based on cognitive styles which contributes to the theoretical basis of Web search research. It also raises various questions within the context of user studies

Originality/value

– The paper adopted a mixed approach in the area of information searching on the Web. A valuable contribution lies in the methods developed.

Revisiting cognitive and learning styles in computer-assisted instruction: not so useful after all

In a previous systematic review, the author proposed that adaptation to learners' cognitive and learning styles (CLSs) could improve the efficiency of computer-assisted instruction (CAI). In the present article, he questions that proposition, arguing that CLSs do not make a substantive difference in CAI. To support this argument, the author performed an updated systematic literature search, pooled new findings with those from the previous review, and reinterpreted this evidence with a focus on aptitude-treatment interactions. (An aptitude-treatment interaction occurs when a student with attribute 1 learns better with instructional approach A than with approach B, whereas a student with attribute 2 learns better with instructional approach B).Of 65 analyses reported in 48 studies, only 9 analyses (14%) showed significant interactions between CLS and instructional approach. It seems that aptitude-treatment interactions with CLSs are at best infrequent and small in magnitude. There are several possible explanations for this lack of effect. First, the influence of strong instructional methods likely dominates the impact of CLSs. Second, current methods for assessing CLSs lack validity evidence and are inadequate to accurately characterize the individual learner. Third, theories are vague, and empiric evidence is virtually nonexistent to guide the planning of style-targeted instructional designs. Adaptation to learners' CLSs thus seems unlikely to enhance CAI. The author recommends that educators focus on employing strong instructional methods. Educators might also consider assessing and adapting to learners' prior knowledge or allowing learners to select among alternate instructional approaches.

The cognitive behavior survey: testing for factorial validity and time invariance across two years of medical school

BACKGROUND

The Cognitive Behavior Survey (CBS) assesses learner behavior in healthcare-related fields.

PURPOSE

The study aims were to evaluate the factorial validity of the CBS, which purports to measure three dimensions of learner behavior--conceptualization, reflection, and memorization--and propose and test an alternative model including its time invariance.

METHODS

The CBS was administered to 3 cohorts of medical students upon matriculation and at the end of their 1st and 2nd year.

RESULTS

Confirmatory factor analysis (CFA) did not support the original CBS model. Exploratory factor analysis (EFA) with an independent sample provided a new model. Retesting the EFA model using CFA with the original sample yielded a model with improved fit and time invariance.

CONCLUSIONS

This study provides evidence for the original CBS 3-factor structure but requires alternative scoring for a time-invariant model.

Scores from riding's cognitive styles analysis have poor test-retest reliability

BACKGROUND

The Cognitive Styles Analysis (CSA) purports to assess two cognitive style dimensions, wholist-analytic (WA) and verbalizer-imager (VI). CSA score reliability has not been studied in medical education.

PURPOSE

The objective of this study was to evaluate test-retest reliability and learner-perceived accuracy of CSA scores.

METHOD

CSA scores were measured twice and perceived accuracy of classifications once among 89 family medicine residents, internal medicine residents, and medical students.

RESULTS

Mean +/- standard deviation interval between tests was 564 +/- 136 days. Test-retest correlation for WA scores was 0.30, and for VI scores was 0.12. Upon retesting 44 learners (49%) were classified under a different WA style, and 56 learners (63%) were classified under a different VI style. There were 58 of 73 learners (79%) who agreed or strongly agreed with their WA classification, whereas 51 of 76 (67%) agreed with their VI classification.

CONCLUSIONS

CSA scores have poor test-retest reliability. Educators may wish to avoid using the CSA and should exercise caution when interpreting CSA scores.

Instructional methods and cognitive and learning styles in web-based learning: report of two randomised trials

CONTEXT

Adapting web-based (WB) instruction to learners' individual differences may enhance learning. Objectives This study aimed to investigate aptitude-treatment interactions between learning and cognitive styles and WB instructional methods.

METHODS

We carried out a factorial, randomised, controlled, crossover, post-test-only trial involving 89 internal medicine residents, family practice residents and medical students at 2 US medical schools. Parallel versions of a WB course in complementary medicine used either active or reflective questions and different end-of-module review activities ('create and study a summary table' or 'study an instructor-created table'). Participants were matched or mismatched to question type based on active or reflective learning style. Participants used each review activity for 1 course module (crossover design). Outcome measurements included the Index of Learning Styles, the Cognitive Styles Analysis test, knowledge post-test, course rating and preference.

RESULTS

Post-test scores were similar for matched (mean +/- standard error of the mean 77.4 +/- 1.7) and mismatched (76.9 +/- 1.7) learners (95% confidence interval [CI] for difference - 4.3 to 5.2l, P = 0.84), as were course ratings (P = 0.16). Post-test scores did not differ between active-type questions (77.1 +/- 2.1) and reflective-type questions (77.2 +/- 1.4; P = 0.97). Post-test scores correlated with course ratings (r = 0.45). There was no difference in post-test subscores for modules completed using the 'construct table' format (78.1 +/- 1.4) or the 'table provided' format (76.1 +/- 1.4; CI - 1.1 to 5.0, P = 0.21), and wholist and analytic styles had no interaction (P = 0.75) or main effect (P = 0.18). There was no association between activity preference and wholist or analytic scores (P = 0.37).

CONCLUSIONS

Cognitive and learning styles had no apparent influence on learning outcomes. There were no differences in outcome between these instructional methods.