Structural Equation Modeling of Paired-Comparison and Ranking Data

The longitudinal relationships of problematic social media use, self-transcendence values and school adaptation: a two-wave study

This study aims to explore the temporal dynamics of the relationships between problematic social media use(PSMU), self-transcendence values, and school adaptation among college students.Methods This study employed longitudinal cross-lagged analysis and structural equation modeling to investigate whether there is a mutual influence among PSMU, self-transcendence values, and school adaptation. Additionally, the study explored whether these variables serve as intermediaries in the associations between the other two variables.Results Two thousand six hundred sixty-eight students were longitudinally followed up for two-wave.The results indicated PSMU predicted self-transcendence values and school adaptation 6 months later; self-transcendence values predicted PSMU and school adaptation 6 months later; and school adaptation predicted self-transcendence values and PSMU 6 months later. Moreover, each of them plays a mediating role in the association between the other two variables.Conclusions These results support and expand the Self-worth Theory and Self-determination Theory that there is a bidirectional relationship between PSMU, self-transcendence values, and school adaptation. This result has implications for helping college students adapt to college life and reduce PSMU.

Development of a Forced-Choice Personality Inventory via Thurstonian Item Response Theory (TIRT)

This study had two purposes: (1) to develop a forced-choice personality inventory to assess student personality characteristics based on the five-factor (FFM) personality model and (2) to examine its factor structure via the Thurstonian Item Response Theory (TIRT) approach based on Thurstone's law of comparative judgment. A total of 200 items were generated to represent the five dimensions, and through Principal Axis Factoring and the composite reliability index, a final pool of 75 items was selected. These items were then organized into 25 blocks, each containing three statements (triplets) designed to balance social desirability across the blocks. The study involved two samples: the first sample of 1484 students was used to refine the item pool, and the second sample of 823 university students was used to examine the factorial structure of the forced-choice inventory. After re-coding the responses into a binary format, the data were analyzed within a standard structural equation modeling (SEM) framework. Then, the TIRT model was applied to evaluate the factorial structure of the forced-choice inventory, with the results indicating an adequate fit. Further suggestions for future research with additional studies are provided to justify the scale's reliability (e.g., test-retest) and validity (e.g., concurrent, convergent, and divergent).

Linear Factor Analytic Thurstonian Forced-Choice Models: Current Status and Issues

Thurstonian forced-choice modeling is considered to be a powerful new tool to estimate item and person parameters while simultaneously testing the model fit. This assessment approach is associated with the aim of reducing faking and other response tendencies that plague traditional self-report trait assessments. As a result of major recent methodological developments, the estimation of normative trait scores has become possible in addition to the computation of only ipsative scores. This opened up the important possibility of comparisons between individuals with forced-choice assessment procedures. With item response theory (IRT) methods, a multidimensional forced-choice (MFC) format has also been proposed to estimate individual scores. Customarily, items to assess different traits are presented in blocks, often triplets, in applications of the MFC, which is an efficient form of item presentation but also a simplification of the original models. The present study provides a comprehensive review of the present status of Thurstonian forced-choice models and their variants. Critical features of the current models, especially the block models, are identified and discussed. It is concluded that MFC modeling with item blocks is highly problematic and yields biased results. In particular, the often-recommended presentation of blocks with items that are keyed in different directions of a trait proves to be counterproductive considering the goal to reduce response tendencies. The consequences and implications of the highlighted issues are further discussed.

Remarkable properties for diagnostics and inference of ranking data modelling

The Plackett-Luce model (PL) for ranked data assumes the forward order of the ranking process. This hypothesis postulates that the ranking process of the items is carried out by sequentially assigning the positions from the top (most liked) to the bottom (least liked) alternative. This assumption has been recently relaxed with the Extended Plackett-Luce model (EPL) through the introduction of the discrete reference order parameter, describing the rank attribution path. By starting from two formal properties of the EPL, the former related to the inverse ordering of the item probabilities at the first and last stage of the ranking process and the latter well-known as independence of irrelevant alternatives (or Luce's choice axiom), we derive novel diagnostic tools for testing the appropriateness of the EPL assumption as the actual sampling distribution of the observed rankings. These diagnostic tools can help uncovering possible idiosyncratic paths in the sequential choice process. Besides contributing to fill the gap of goodness-of-fit methods for the family of multistage models, we also show how one of the two statistics can be conveniently exploited to construct a heuristic method, that surrogates the maximum likelihood approach for inferring the underlying reference order parameter. The relative performance of the proposals, compared with more conventional approaches, is illustrated by means of extensive simulation studies.

A Meta-Analysis of the Faking Resistance of Forced-Choice Personality Inventories

This study presents a comprehensive meta-analysis on the faking resistance of forced-choice (FC) inventories. The results showed that (1) FC inventories show resistance to faking behavior; (2) the magnitude of faking is higher in experimental contexts than in real-life selection processes, suggesting that the effects of faking may be, in part, a laboratory phenomenon; and (3) quasi-ipsative FC inventories are more resistant to faking than the other FC formats. Smaller effect sizes were found for conscientiousness when the quasi-ipsative format was used (δ = 0.49 vs. δ = 1.27 for ipsative formats). Also, the effect sizes were smaller for the applicant samples than for the experimental samples. Finally, the contributions and practical implications of these findings are discussed.

Reviewing the Structure of Kolb’s Learning Style Inventory From Factor Analysis and Thurstonian Item Response Theory (IRT) Model Approaches

Kolb’s Learning Style Inventory (LSI) continues to generate a great debate among researchers, given the contradictory evidence resulting from its psychometric properties. One primary criticism focuses on the artificiality of the results derived from its internal structure because of the ipsative nature of the forced-choice format. This study seeks to contribute to the resolution of this debate. A short version of Kolb’s LSI with a forced-choice format and an additional inventory scored on a Likert scale was completed by a sample of students of the University Católica del Norte in Antofagasta, Chile. The data obtained from the two forms of the reduced version of the LSI were compared using principal component analysis, confirmatory factor analysis, and the Thurstonian Item Response Theory model. The results support the hypothesis of the existence of four learning mode dimensions. However, they do not support the existence of the learning styles as proposed by Kolb, indicating that said reports are the product of the artificial structure generated by the ipsative forced-choice format .

Detecting Heterogeneity of Intervention Effects in Comparative Judgments

Comparative measures such as paired comparisons and rankings are frequently used to evaluate health states and quality of life. The present article introduces log-linear Bradley-Terry (LLBT) models to evaluate intervention effectiveness when outcomes are measured as paired comparisons or rankings and presents a combination of the LLBT model and model-based recursive partitioning (MOB) to detect treatment effect heterogeneity. The MOB LLBT approach enables researchers to identify subgroups that differ in the preference order and in the effect an intervention has on choice behavior. Applicability of MOB LLBT models is demonstrated using an artificial data example with known data-generating mechanism and a real-world data example focusing on drug-harm perception among music festival visitors. In the artificial data example, the MOB LLBT model is able to adequately recover the "true" (population) model. In the real-world data example, the standard LLBT model confirms the existence of a situational willingness among festival visitors to trivialize drug harm when peer consumption behavior is made cognitively accessible. In addition, MOB LLBT results suggest that this trivialization effect is highly context-dependent and most pronounced for participants with low-to-moderate alcohol intoxication who also proactively contacted a substance counselor at the festival venue. Both data examples suggest that MOB LLBT models allow for more nuanced statements about the effectiveness of interventions. We provide R code examples to implement MOB LLBT models for paired comparisons, rankings, and rating (Likert-type) data.

An exploratory factor model for ordinal paired comparison indicators

Suppose the same contestants play in tournaments of chess, shogi, and Go. Per-tournament rankings can be estimated. We may also try to recover a latent board game skill that accounts for some proportion of the variance in per-board game rankings. To accomplish this, a factor model is introduced. Identification issues with the ordinal paired item model are discussed. Simulation studies are presented to provide some guidance about sample size requirements. Both single item and multivariate correlation and factor model are validated using simulation-based calibration. We recommend leave-one-out cross-validation to assess model fit. To ease application of the methods described, an open-source companion R extension, pcFactorStan, is published on the Comprehensive R Archive Network. Application of pcFactorStan is demonstrated by analysis of a real-world dataset.

On the Statistical and Practical Limitations of Thurstonian IRT Models

Forced-choice questionnaires have been proposed to avoid common response biases typically associated with rating scale questionnaires. To overcome ipsativity issues of trait scores obtained from classical scoring approaches of forced-choice items, advanced methods from item response theory (IRT) such as the Thurstonian IRT model have been proposed. For convenient model specification, we introduce the thurstonianIRT R package, which uses Mplus, lavaan, and Stan for model estimation. Based on practical considerations, we establish that items within one block need to be equally keyed to achieve similar social desirability, which is essential for creating forced-choice questionnaires that have the potential to resist faking intentions. According to extensive simulations, measuring up to five traits using blocks of only equally keyed items does not yield sufficiently accurate trait scores and inter-trait correlation estimates, neither for frequentist nor for Bayesian estimation methods. As a result, persons' trait scores remain partially ipsative and, thus, do not allow for valid comparisons between persons. However, we demonstrate that trait scores based on only equally keyed blocks can be improved substantially by measuring a sizable number of traits. More specifically, in our simulations of 30 traits, scores based on only equally keyed blocks were non-ipsative and highly accurate. We conclude that in high-stakes situations where persons are motivated to give fake answers, Thurstonian IRT models should only be applied to tests measuring a sizable number of traits.

Desaturation-Induced Brightness in Face Color Perception

The distinctiveness of perception of face from nonface objects has been noted previously. However, face brightness is often confounded with whiteness in the beauty industry; few studies have examined these perceptual differences. To investigate the interactions among face color attributes, we measured the effect of saturation on brightness and whiteness in both uniform color patches and face images to elucidate the relationship between these two perceptions. We found that, at constant luminance, a uniform color patch looked brighter with an increase in saturation (i.e., the Helmholtz-Kohlrausch effect occurred), while in contrast, brightness of a facial skin image looked less bright with increased saturation (i.e., contrary to the Helmholtz-Kohlrausch effect), which suggested this interaction of color attributes was influenced by top-down information. We conclude that this inverse effect of saturation on brightness for face images is not due to face recognition, color range of the skin tone, the luminance distribution, or recognition of human skin but due to the composite interactions of these facial skin factors in higher order recognition mechanisms.

My strong, Black daughter: Racial/ethnic differences in the attributes mothers value for their daughters

OBJECTIVES

The Strong Black Woman (SBW) ideal, which emphasizes self-reliance and strength, is considered an influential cultural image for many Black women. Research on how the SBW ideal may be reflected in maternal socialization is largely theoretical or qualitative.

METHODS

Guided by cultural models of parenting, we tested for racial/ethnic differences in the importance and meaning of SBW-related attributes among 194 low-income, Black (22%), White (20%), and Latina (57%) families. Mothers (M = 41.6 years) and daughters (M = 15.4 years) completed semistructured interviews, q-sort tasks, and self-report measures. Group differences were examined with analysis of covariance, logistic regression, and multigroup path models.

RESULTS

Black adolescents were not described by mothers or adolescents as possessing more SBW-related attributes (e.g., strong-willed, independent, assertive) compared to adolescents of other racial/ethnic groups; however, tests of moderation indicate group differences in how mothers perceived these attributes. Black adolescents with high SBW-related attributes were viewed by their mothers as showing leadership, whereas White and Latina adolescents with these attributes were viewed by mothers as having externalizing problems. Black mothers also rated these attributes as more important for young adult women to possess compared to other mothers. Finally, Black mothers described self-reliance as the critical developmental task for their daughter more than White and Latina mothers.

CONCLUSIONS

Findings suggest attributes consistent with the SBW ideal are valued by Black mothers more than Latina and White mothers from similar communities and provide empirical support about the potential importance of the SBW ideal in how Black mothers raise their daughters. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Why rate when you could compare? Using the "EloChoice" package to assess pairwise comparisons of perceived physical strength

We introduce "EloChoice", a package for R which uses Elo rating to assess pairwise comparisons between stimuli in order to measure perceived stimulus characteristics. To demonstrate the package and compare results from forced choice pairwise comparisons to those from more standard single stimulus rating tasks using Likert (or Likert-type) items, we investigated perceptions of physical strength from images of male bodies. The stimulus set comprised images of 82 men standing on a raised platform with minimal clothing. Strength-related anthropometrics and grip strength measurements were available for each man in the set. UK laboratory participants (Study 1) and US online participants (Study 2) viewed all images in both a Likert rating task, to collect mean Likert scores, and a pairwise comparison task, to calculate Elo, mean Elo (mElo), and Bradley-Terry scores. Within both studies, Likert, Elo and Bradley-Terry scores were closely correlated to mElo scores (all rs > 0.95), and all measures were correlated with stimulus grip strength (all rs > 0.38) and body size (all rs > 0.59). However, mElo scores were less variable than Elo scores and were hundreds of times quicker to compute than Bradley-Terry scores. Responses in pairwise comparison trials were 2/3 quicker than in Likert tasks, indicating that participants found pairwise comparisons to be easier. In addition, mElo scores generated from a data set with half the participants randomly excluded produced very comparable results to those produced with Likert scores from the full participant set, indicating that researchers require fewer participants when using pairwise comparisons.

Forced-Choice Assessment of Work-Related Maladaptive Personality Traits: Preliminary Evidence From an Application of Thurstonian Item Response Modeling

This article describes an investigation of whether Thurstonian item response modeling is a viable method for assessment of maladaptive traits. Forced-choice responses from 420 working adults to a broad-range personality inventory assessing six maladaptive traits were considered. The Thurstonian item response model’s fit to the forced-choice data was adequate, while the fit of a counterpart item response model to responses to the same items but arranged in a single-stimulus design was poor. Monotrait heteromethod correlations indicated corresponding traits in the two formats overlapped substantially, although they did not measure equivalent constructs. A better goodness of fit and higher factor loadings for the Thurstonian item response model, coupled with a clearer conceptual alignment to the theoretical trait definitions, suggested that the single-stimulus item responses were influenced by biases that the independent clusters measurement model did not account for. Researchers may wish to consider forced-choice designs and appropriate item response modeling techniques such as Thurstonian item response modeling for personality questionnaire applications in industrial psychology, especially when assessing maladaptive traits. We recommend further investigation of this approach in actual selection situations and with different assessment instruments.

Thurstonian Scaling of Compositional Questionnaire Data

To prevent response bias, personality questionnaires may use comparative response formats. These include forced choice, where respondents choose among a number of items, and quantitative comparisons, where respondents indicate the extent to which items are preferred to each other. The present article extends Thurstonian modeling of binary choice data to "proportion-of-total" (compositional) formats. Following the seminal work of Aitchison, compositional item data are transformed into log ratios, conceptualized as differences of latent item utilities. The mean and covariance structure of the log ratios is modeled using confirmatory factor analysis (CFA), where the item utilities are first-order factors, and personal attributes measured by a questionnaire are second-order factors. A simulation study with two sample sizes, N = 300 and N = 1,000, shows that the method provides very good recovery of true parameters and near-nominal rejection rates. The approach is illustrated with empirical data from N = 317 students, comparing model parameters obtained with compositional and Likert-scale versions of a Big Five measure. The results show that the proposed model successfully captures the latent structures and person scores on the measured traits.

Item Response Models for Forced-Choice Questionnaires: A Common Framework

In forced-choice questionnaires, respondents have to make choices between two or more items presented at the same time. Several IRT models have been developed to link respondent choices to underlying psychological attributes, including the recent MUPP (Stark et al. in Appl Psychol Meas 29:184-203, 2005) and Thurstonian IRT (Brown and Maydeu-Olivares in Educ Psychol Meas 71:460-502, 2011) models. In the present article, a common framework is proposed that describes forced-choice models along three axes: (1) the forced-choice format used; (2) the measurement model for the relationships between items and psychological attributes they measure; and (3) the decision model for choice behavior. Using the framework, fundamental properties of forced-choice measurement of individual differences are considered. It is shown that the scale origin for the attributes is generally identified in questionnaires using either unidimensional or multidimensional comparisons. Both dominance and ideal point models can be used to provide accurate forced-choice measurement; and the rules governing accurate person score estimation with these models are remarkably similar.

A Hierarchical Model for Accuracy and Choice on Standardized Tests

This paper assesses the psychometric value of allowing test-takers choice in standardized testing. New theoretical results examine the conditions where allowing choice improves score precision. A hierarchical framework is presented for jointly modeling the accuracy of cognitive responses and item choices. The statistical methodology is disseminated in the 'cIRT' R package. An 'answer two, choose one' (A2C1) test administration design is introduced to avoid challenges associated with nonignorable missing data. Experimental results suggest that the A2C1 design and payout structure encouraged subjects to choose items consistent with their cognitive trait levels. Substantively, the experimental data suggest that item choices yielded comparable information and discrimination ability as cognitive items. Given there are no clear guidelines for writing more or less discriminating items, one practical implication is that choice can serve as a mechanism to improve score precision.

Personality Testing and Industrial–Organizational Psychology: A Productive Exchange and Some Future Directions

The goal of our focal article was to provide a current perspective on personality testing and its use in organizational research and to elicit constructive discussion and suggestions for future research and practice. The present article caps off the discussion by integrating the main ideas presented in the commentaries within our original framework of questions and topics, with the immodest hope of advancing our understanding of personality and its measurement in the context of industrial–organizational psychology. In short, we recommend continuing to take advantage of the organizing framework of the Big Five while also pursuing more “bottom-up” approaches that examine facet-level relationships with multidimensional performance outcomes, in addition to developing process models that include more proximal motivational and situational variables. Work along these lines is valuable to both organizational science and practice.

Multidimensional item response theory models yielded good fit and reliable scores for the Short Form-12 questionnaire

OBJECTIVES

To propose a multidimensional item response theory (MIRT) scoring system for the Short Form 12 (SF-12) with good psychometric properties in terms of fit and reliability.

STUDY DESIGN AND SETTINGS

Two models, indicating physical (PCS) and mental component summary (MCS) dimensions, were fitted to SF-12 data from the European Study of the Epidemiology of Mental Disorders, a representative sample from European adult general population (n=21,425; response rate=61.2%). Goodness of fit, information, reliability, and agreement of individual scores were compared with the classical SF-12 and RAND-12 algorithms.

RESULTS

The bidimensional response process (BRP) model, where all items are indicators of both dimensions, yielded the best fit (root mean square error of approximation=0.057, comparative fit index=0.95, and Tucker-Lewis index=0.94), and highly agreed with PCS and MCS scores from the SF-12 (intraclass correlation coefficients of 0.92 and 0.88, respectively) and RAND-12 (0.88 and 0.95). Regarding reliability, the BRP yielded 0.75 and 0.77 (PCS and MCS, respectively), greater than SF-12 (0.65 and 0.66) and RAND-12 (0.65 and 0.67). As indicated by scale linking, MIRT scores can be interpreted similarly to the classical scores.

CONCLUSION

The MIRT models showed a clear construct structure for the PCS and MCS dimensions, defined by functional and role limitation content. Results support the use of SF-12 MIRT-based scores as a valid and reliable option to assess health status.

Probabilistic choice models in health-state valuation research: background, theories, assumptions and applications

Interest is rising in measuring subjective health outcomes, such as treatment outcomes that are not directly quantifiable (functional disability, symptoms, complaints, side effects and health-related quality of life). Health economists in particular have applied probabilistic choice models in the area of health evaluation. They increasingly use discrete choice models based on random utility theory to derive values for healthcare goods or services. Recent attempts have been made to use discrete choice models as an alternative method to derive values for health states. In this article, various probabilistic choice models are described according to their underlying theory. A historical overview traces their development and applications in diverse fields. The discussion highlights some theoretical and technical aspects of the choice models and their similarity and dissimilarity. The objective of the article is to elucidate the position of each model and their applications for health-state valuation.

Bayesian Thurstonian models for ranking data using JAGS

A Thurstonian model for ranking data assumes that observed rankings are consistent with those of a set of underlying continuous variables. This model is appealing since it renders ranking data amenable to familiar models for continuous response variables-namely, linear regression models. To date, however, the use of Thurstonian models for ranking data has been very rare in practice. One reason for this may be that inferences based on these models require specialized technical methods. These methods have been developed to address computational challenges involved in these models but are not easy to implement without considerable technical expertise and are not widely available in software packages. To address this limitation, we show that Bayesian Thurstonian models for ranking data can be very easily implemented with the JAGS software package. We provide JAGS model files for Thurstonian ranking models for general use, discuss their implementation, and illustrate their use in analyses.

Automated assembly of optimally spaced and balanced paired comparisons: controlling order effects

To control order effects in questionnaires containing paired comparisons, Ross (1934) described an optimal ordering of the pairings. The pairs can also be balanced so that every stimulus appears equal numbers of times as the first and the second member of a pair. First, we describe and illustrate the optimally spaced, balanced ordering of pairings. Then we show how the optimally spaced, balanced order can be used to implement a matrix-sampling design or a fully incomplete design when the number of stimuli n is so large that respondents cannot reasonably be expected to judge all n(n - 1)/2 pairs. The algorithm for balancing and optimally spacing the list of pairs is described.

Estimating Independent Locally Shifted Random Utility Models for Ranking Data

We consider the estimation of probabilistic ranking models in the context of conjoint experiments. By using approximate rather than exact ranking probabilities, we avoided the computation of high-dimensional integrals. We extended the approximation technique proposed by Henery (1981) in the context of the Thurstone-Mosteller-Daniels model to any independent locally shifted random utility model. In particular, this allowed us to estimate any independent random utility model with common shape (e.g., normal, logistic) and scale. Moreover, our approach also allows for the analysis of any partial ranking. Partial rankings are essential in practical conjoint analysis to collect data efficiently to relieve respondents' task burden. We applied the approach to the reanalysis of the career preference data set described in Maydeu-Olivares and Böckenholt (2005) , and to a holiday preferences data set.

Item Response Modeling of Paired Comparison and Ranking Data

The comparative format used in ranking and paired comparisons tasks can significantly reduce the impact of uniform response biases typically associated with rating scales. Thurstone's (1927, 1931) model provides a powerful framework for modeling comparative data such as paired comparisons and rankings. Although Thurstonian models are generally presented as scaling models, that is, stimuli-centered models, they can also be used as person-centered models. In this article, we discuss how Thurstone's model for comparative data can be formulated as item response theory models so that respondents' scores on underlying dimensions can be estimated. Item parameters and latent trait scores can be readily estimated using a widely used statistical modeling program. Simulation studies show that item characteristic curves can be accurately estimated with as few as 200 observations and that latent trait scores can be recovered to a high precision. Empirical examples are given to illustrate how the model may be applied in practice and to recommend guidelines for designing ranking and paired comparisons tasks in the future.

Assessing Goodness of Fit: Simple Rules of Thumb Simply Do Not Work

Confirmatory factor analytic (CFA) models are frequently used in many areas of organizational research. Due to their popularity, CFA models and issues about their fit have received a vast amount of attention during the past several decades. The purpose of this study was to examine several measures of fit and the appropriateness of previously developed ‘‘rules of thumb’’ for their interpretation. First, an empirical example is used to illustrate the effects of nonnormality on maximum likelihood (ML) estimation and to demonstrate the importance of diagonally weighted least squares (DWLS) estimation for organizational research. Then, the results of a simulation study are presented to show that appropriate cutoff values for DWLS estimation vary considerably across conditions. Finally, regression equations are described to aid researchers in selecting cutoff values for assessing the fit of DWLS solutions, given a desired level of Type I error. The results summarized here have important implications for the interpretation and use of CFA models.

[Subjective values of quality-of-life dimensions in the elderly]

OBJECTIVE

The present study analyzes the subjective importance of five quality-of-life dimensions among non-institutionalized elderly people (>65 years old).

MATERIAL AND METHODS

A Thurstonian design for paired comparisons was used to estimate subjective values for five quality-of-life dimensions (health, autonomy, housing, family and social support networks, and social activities) in a sample of 404 persons (168 men and 234 women) aged 65-94 years old.

RESULTS

The most important dimension was health, which was given the highest subjective value, followed by personal autonomy. The third most important dimensions were housing adapted to age-related needs and support networks, with practically equivalent values. The least important dimension was social activities.

CONCLUSIONS

Significant differences between genders were detected in educational level and income, but the rank order of the dimensions and the subjective values assigned to them were equivalent for men and women.

Modeling ranking, time trade-off, and visual analog scale values for EQ-5D health states: a review and comparison of methods

BACKGROUND

There is rising interest in eliciting health state valuations using rankings. Due to their relative simplicity, ordinal measurement methods may offer an attractive practical alternative to cardinal methods, such as time trade-off (TTO) and visual analog scale (VAS). In this article, we explore alternative models for estimating cardinal health state values from rank responses in a unique multicountry database. We highlight an estimation challenge pertaining to health states just below perfect health (the "nonoptimal gap") and propose an analytic solution to ameliorate this problem.

METHODS

Using a standardized protocol developed by the EuroQol Group, rank, VAS, and TTO responses were collected for 43 health states in 8 countries: Slovenia, Argentina, Denmark, Japan, Netherlands, Spain, United Kingdom, and United States, yielding a sample of 179,431 state responses from 11,483 subjects. States were described using the EQ-5D system, which allows for 3 different possible levels on 5 different dimensions of health. We estimated conditional logit and probit regression models for rank responses. The regressions included 17 health state attribute variables reflecting specific levels on each dimension and counts of different levels across dimensions. This flexible specification accommodates previously published valuation models, such as models applied in the United Kingdom and United States. In addition to fitting standard conditional logit and probit models, which assume equal variance across health states (homoscedasticity), we examined a heteroscedastic probit model that assumes no variance for the 2 points anchoring the scale ("optimal health" and "dead") and relaxes the equal-variance assumption for all other states. Rank-based predictions for the 243 unique states defined by the EQ-5D system were compared with predictions from conventional linear models fitted to TTO and VAS responses.

RESULTS

By construction, the TTO and VAS models assume no variance around the anchoring states of optimal health and dead. Mimicking this assumption in the probit rank models helps dissolve the nonoptimal gap. For all other states, variances in TTO and VAS were negatively associated with mean values, which contradict the assumption of homoscedasticity. Estimated health state values from the heteroscedastic probit model for the ranking data were highly correlated with predictions from both TTO and VAS models for the 243 EQ-5D states. Between VAS and rank-based estimates, Lin's rho, a measure of agreement, was over 0.98 with a mean absolute difference of 0.028. Corresponding measures of agreement between rank and TTO estimates were 0.96 and 0.12, which is similar to the agreement between VAS and TTO.

CONCLUSIONS

Rank-based valuation techniques, which offer advantages of flexibility, generalizability, and ease of administration, may be attractive substitutes for TTO and VAS in the measurement of societal values for health outcomes.

Thurstone scaling as a measurement method to quantify subjective health outcomes

BACKGROUND

Many objective health outcome measures are used to monitor patients or evaluate health interventions, but there are also subjective measures. For the latter, it is difficult to derive metric data, which are needed to quantify health outcomes such as functional disability, severity of side effects, and health status.

OBJECTIVE

Thurstone's Law of Comparative Judgment is presented as an alternative means to derive metric values for subjective health outcomes. The appeal of Thurstone's scaling model is that it can transform subjective individual rank order data or comparative preference data to a single group composite interval scale. To demonstrate its contribution, an empirical study was conducted, focusing on the valuation of health states.

RESEARCH DESIGN

Rank order data were collected for 18 health states and were then used as input for Thurstone scaling. Visual analogue scale (VAS) values were also collected for the same states.

SUBJECTS

An agency for market research recruited 212 Dutch respondents aged 18-75 years.

RESULTS

The derived Thurstone values showed a strong relationship with the VAS values. The positions of the 2 worst states were almost identical on the VAS and the Thurstone scale. Intermediate states were scaled somewhat differently by the 2 methods.

CONCLUSIONS

For many subjective health outcomes, Thurstone scaling and its derivatives may be an attractive methodology to arrive at quantitative measures.

Identification and Small Sample Estimation of Thurstone's Unrestricted Model for Paired Comparisons Data

The interpretation of a Thurstonian model for paired comparisons where the utilities' covariance matrix is unrestricted proved to be difficult due to the comparative nature of the data. We show that under a suitable constraint the utilities' correlation matrix can be estimated, yielding a readily interpretable solution. This set of identification constraints can recover any true utilities' covariance matrix, but it is not unique. Indeed, we show how to transform the estimated correlation matrix into alternative correlation matrices that are equally consistent with the data but may be more consistent with substantive theory. Also, we show how researchers can investigate the sample size needed to estimate a particular model by exploiting the simulation capabilities of a popular structural equation modeling statistical package.

Thurstonian-Based Analyses: Past, Present, and Future Utilities

Current psychometric models of choice behavior are strongly influenced by Thurstone's (1927, 1931) experimental and statistical work on measuring and scaling preferences. Aided by advances in computational techniques, choice models can now accommodate a wide range of different data types and sources of preference variability among respondents induced by such diverse factors as person-specific choice sets or different functional forms for the underlying utility representations. At the same time, these models are increasingly challenged by behavioral work demonstrating the prevalence of choice behavior that is not consistent with the underlying assumptions of these models. I discuss new modeling avenues that can account for such seemingly inconsistent choice behavior and conclude by emphasizing the interdisciplinary frontiers in the study of choice behavior and the resulting challenges for psychometricians.