Sophisticated Statistics Cannot Compensate for Method Effects If Quantifiable Structure Is Compromised

Application of the Rasch measurement model in rehabilitation research and practice: early developments, current practice, and future challenges

The application of the Rasch measurement model in rehabilitation is now well established. Both its dichotomous and polytomous forms provide for transforming ordinal scales into interval-level measures, consistent with the requirements of fundamental measurement. The growth of applying the model in rehabilitation spans 30 years, during which both the protocol has steadily developed and several software packages have emerged that provide for analysis, together with the "R" language that has an increasing set of codes for applying the model. This article reviews that development and highlights current practice requirements, including those for providing the relevant information for the methods, and what is expected of the analysis. In addition, this provides a worked example and looks at the remaining issues and current developments of its application.

Within-Individual Variation in Cognitive Performance Is Not Noise: Why and How Cognitive Assessments Should Examine Within-Person Performance

Despite evidence that it exists, short-term within-individual variability in cognitive performance has largely been ignored as a meaningful component of human cognitive ability. In this article, we build a case for why this within-individual variability should not be viewed as mere measurement error and why it should be construed as a meaningful component of an individual's cognitive abilities. We argue that in a demanding and rapidly changing modern world, between-individual analysis of single-occasion cognitive test scores does not account for the full range of within-individual cognitive performance variation that is implicated in successful typical cognitive performance. We propose that short-term repeated-measures paradigms (e.g., the experience sampling method (ESM)) be used to develop a process account of why individuals with similar cognitive ability scores differ in their actual performance in typical environments. Finally, we outline considerations for researchers when adapting this paradigm for cognitive assessment and present some initial findings from two studies in our lab that piloted the use of ESM to assess within-individual cognitive performance variation.

Long-term psychosocial consequences of false-positive screening mammography: a cohort study with follow-up of 12-14 years in Denmark

OBJECTIVE

To compare the long-term psychosocial consequences of mammography screening among women with breast cancer, normal results and false-positive results.

DESIGN

A matched cohort study with follow-up of 12-14 years.

SETTING

Denmark from 2004 to 2019.

PARTICIPANTS

1170 women who participated in the Danish mammography screening programme in 2004-2005.

INTERVENTION

Mammography screening for women aged 50-69 years.

OUTCOME MEASURES

We assessed the psychosocial consequences with the Consequences Of Screening-Breast Cancer, a condition-specific questionnaire that is psychometrically validated and encompasses 14 psychosocial dimensions.

RESULTS

Across all 14 psychosocial outcomes, women with false-positive results averagely reported higher psychosocial consequences compared with women with normal findings. Mean differences were statistically insignificant except for the existential values scale: 0.61 (95% CI (0.15 to 1.06), p=0.009). Additionally, women with false-positive results and women diagnosed with breast cancer were affected in a dose-response manner, where women diagnosed with breast cancer were more affected than women with false-positive results.

CONCLUSION

Our study suggests that a false-positive mammogram is associated with increased psychosocial consequences 12-14 years after the screening. This study adds to the harms of mammography screening. The findings should be used to inform decision-making among the invited women and political and governmental decisions about mammography screening programmes.

Methodological Quality of PROMs in Psychosocial Consequences of Colorectal Cancer Screening: A Systematic Review

Objective

This systematic review aimed to assess the adequacy of measurement properties in Patient-Reported Outcome Measures (PROMs) used to quantify psychosocial consequences of colorectal cancer screening among adults at average risk.

Methods

We searched four databases for eligible studies: MEDLINE, CINAHL, PsycINFO, and Embase. Our approach was inclusive and encompassed all empirical studies that quantified aspects of psychosocial consequences of colorectal cancer screening. We assessed the adequacy of PROM development and measurement properties for content validity using The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias checklist.

Results

We included 33 studies that all together used 30 different outcome measures. Two PROMs (6.7%) were developed in a colorectal cancer screening context. COSMIN rating for PROM development was inadequate for 29 out of 30 PROMs (97%). PROMs lacked proper cognitive interviews and pilot studies and therefore had no proven content validity. According to the COSMIN checklist, 27 out of 30 PROMs (90%) had inadequate measurement properties for content validity.

Discussion

The majority of included PROMs had inadequate development and measurement properties. These findings shed light on the trustworthiness of the included studies' findings and call for reevaluation of existing evidence on the psychosocial consequences of colorectal cancer screening. To provide trustworthy evidence about the psychosocial consequences of colorectal cancer screening, editors could require that studies provide evidence of the methodological quality of the PROM. Alternatively, authors should transparently disclose their studies' methodological limitations in measuring psychosocial consequences of screening validly.

Criterion-Referenced Assessment of Intelligence as Adaptation to the Environment: Is It Possible, Plausible, or Practical?

Criterion-referenced testing is usually applied to the assessment of achievement. In this article, we suggest how it can also be applied to the assessment of adaptive intelligence, that is, intelligence as adaptation to the environment. In the era of the Anthropocene, we argue that adaptive intelligence is what is most important not only for individual success, but also for success in terms of preservation of the world as we know it. We define criterion-referenced testing and compare it to norm-referenced testing. We then discuss two kinds of scoring of criterion-referenced testing, namely, with respect to external criteria and with respect to internal (theory-based) criteria. We then discuss past research on intelligence that could be viewed as criterion-referenced. Finally, we suggest how criterion-referencing could be applied to the assessment of adaptive intelligence.

Intelligence IS Cognitive Flexibility: Why Multilevel Models of Within-Individual Processes Are Needed to Realise This

Despite substantial evidence for the link between an individual’s intelligence and successful life outcomes, questions about what defines intelligence have remained the focus of heated dispute. The most common approach to understanding intelligence has been to investigate what performance on tests of intellect is and is not associated with. This psychometric approach, based on correlations and factor analysis is deficient. In this review, we aim to substantiate why classic psychometrics which focus on between-person accounts will necessarily provide a limited account of intelligence until theoretical considerations of within-person accounts are incorporated. First, we consider the impact of entrenched psychometric presumptions that support the status quo and impede alternative views. Second, we review the importance of process-theories, which are critical for any serious attempt to build a within-person account of intelligence. Third, features of dynamic tasks are reviewed, and we outline how static tasks can be modified to target within-person processes. Finally, we explain how multilevel models are conceptually and psychometrically well-suited to building and testing within-individual notions of intelligence, which at its core, we argue is cognitive flexibility. We conclude by describing an application of these ideas in the context of microworlds as a case study.