A Matter of Context: A Meta-Analytic Investigation of the Relative Validity of Contextualized and Noncontextualized Personality Measures

Forced-Choice Versus Likert Responses on an Occupational Big Five Questionnaire

Abstract. Conventional self-report measures are prone to response biases, which distort measurement in any applied assessment. The forced-choice (FC) format was proposed as a potential remedy for these biases. The purpose of these studies was to develop and evaluate a FC questionnaire for the occupational context based on the five factor model of personality. A single-stimulus Likert questionnaire was contextualized for occupational settings and psychometrically optimized in Study 1 ( N = 401). Considering optimal design strategies, we subsequently used this questionnaire to construct and validate a FC questionnaire in Study 2 ( N = 517). Methodological add-ons to established approaches were applied to achieve decent confirmatory model fit. The new questionnaire shows good psychometric qualities and strong validity. We make suggestions for further applications and studies.

Best Practices in Data Collection and Preparation: Recommendations for Reviewers, Editors, and Authors

We offer best-practice recommendations for journal reviewers, editors, and authors regarding data collection and preparation. Our recommendations are applicable to research adopting different epistemological and ontological perspectives—including both quantitative and qualitative approaches—as well as research addressing micro (i.e., individuals, teams) and macro (i.e., organizations, industries) levels of analysis. Our recommendations regarding data collection address (a) type of research design, (b) control variables, (c) sampling procedures, and (d) missing data management. Our recommendations regarding data preparation address (e) outlier management, (f) use of corrections for statistical and methodological artifacts, and (g) data transformations. Our recommendations address best practices as well as transparency issues. The formal implementation of our recommendations in the manuscript review process will likely motivate authors to increase transparency because failure to disclose necessary information may lead to a manuscript rejection decision. Also, reviewers can use our recommendations for developmental purposes to highlight which particular issues should be improved in a revised version of a manuscript and in future research. Taken together, the implementation of our recommendations in the form of checklists can help address current challenges regarding results and inferential reproducibility as well as enhance the credibility, trustworthiness, and usefulness of the scholarly knowledge that is produced.

Advances in the Treatment of Context in Organizational Research

Although scholars in the field of organizational behavior have raised concerns about a lack of contextual appreciation, there has been a recent embrace of contextual thinking in the organizational sciences. In this review, I discuss several recent theories and measures of context. The added value of a contextual approach is illustrated by how context can shape personality, how it affects the emergence of work designs, and how it benefits the study of organizational demography. Future research topics include context cue sensitivity, the way context is shaped, the mediators of context effects, and the breadth and limits of contextual impact. A recurrent theme is that although context enables a demarcation of what is distinctive about situations, it also permits integration across research areas and levels of analysis, identifying what they have in common as settings for organizational behavior.

Contextualizing Interest Scales With Emojis: Implications for Measurement and Validity

The affective nature of emojis makes them suited for anchoring scales to measure the affective preferences of vocational interest. In Study 1, we conducted a content analysis to identify the five images that best represent a bipolar continuum from strongly like to strongly dislike. In Study 2, we compared the psychometric properties of traditional lexical anchors to those of the emoji anchors identified in Study 1 and tested the predictive validity of emoji-anchored scales. Results show emoji-anchored scales possess similar psychometric properties as lexical-anchored scales. Importantly, the interest-fit using contextualized emoji scales predicts job satisfaction better compared to lexical scales ( remoji (226) = .41 versus rlexical (226) = .13). Overall, this article (1) provides researchers with a validated public-domain emoji anchors, (2) demonstrates that emoji anchors possess the same the robust psychometric properties as lexical anchors, and (3) illustrates how emoji-anchored scales can be potentially better for measuring vocational interests.

Assessing Personality–Situation Interplay in Personnel Selection: Toward More Integration into Personality Research

Over the years, the personnel selection field has developed methods to assess trait expression in particular situations, but these approaches have evolved mostly outside the field of personality psychology. In this article, I review available personnel selection evidence regarding two such approaches: (i) situational judgement tests that present short scenarios and ask job candidates how they would handle the situations and (ii) assessment centre exercises requiring candidates to display behaviour in specified interactive situations. I describe these approaches and discuss their relations with personality research. I posit that adapting these approaches to personality research creates methodological diversity to address key research themes related to within–person variability, trait–behaviour links, personality disorders, and personality expression and perception. Copyright © 2017 European Association of Personality Psychology

Future directions in personality, occupational and medical selection: myths, misunderstandings, measurement, and suggestions

This paper has two objectives: (1) presenting recent advances in personality theory whereby personality traits are conceptualized within a framework that focuses on the dynamic interactions of behaviour, biology, context, and states, and (2) discussing the implications of these developments for measurement and medical selection. We start by presenting evidence that traits are no longer regarded as stable deterministic predictors of behaviour. Instead, traits are found to change across generations, the life span, and in response to environmental contingencies. Thus, there is an urgent need to explore how traits change as function of medical education. Second, drawing on recent theory and research (behavioural reaction norms and the density distribution model) we highlight evidence to show how the expression of trait relevant behaviour is dependent on context, and is distributed with an average (typical behaviour or personality) and a variance (plasticity or adaptability), with traditional personality measure associated with typical responding. Third, we demystify that some traits are better than others showing that so-called "good" traits have a dark-side. Fourth, we show how these developments impact on how personality might be assessed, thereby presenting recent evidence on the use of contextualized personality measures, situational judgment tests, other reports, and implicit measures. Throughout the paper, we outline the key implications of these developments for medical selection practices.

A review of entitlement

Interest in employee entitlement perceptions is increasing in academia and in organizations. Entitlement has a long history of being conceptualized as a personality trait in psychology closely aligned with narcissism. Research on workplace entitlement has generally revealed links with negative workplace behaviors, indicating costly outcomes for individuals, teams, and organizations. Our aim in this article is to review the literature on workplace entitlement perceptions, identifying how the construct has changed definition over time, and indicating related constructs that impact on research within industrial and organizational psychology. This review progresses research in this field by examining the nomological network around entitlement and resolving current inconsistencies in the construct definition of entitlement in the workplace, and establishing a set of firm future research directions for entitlement research.

Self-esteem and counterproductive work behaviors: a systematic review

Purpose

– Consistency theory and ego-defense theory have been used to examine the relationship between counterproductive work behavior (CWB) and self-esteem; however, these two theoretical approaches pose different directions for the expected relation. In line with this, previous research concerning the relationship between self-esteem and CWB has found inconsistent empirical results. The purpose of this paper is to summarize the relation between self-esteem and counterproductive behavior at work and draw conclusions about the merit of the competing theories. This study also examines the type of self-esteem as a potential moderator to this relationship.

Design/methodology/approach

– The authors performed a psychometric meta-analysis of the relation between self-esteem and CWB using 21 correlations with a total n of 5,135.

Findings

– The estimated population correlation was −0.26. The moderator analyses showed that global self-esteem had a stronger relation with CWB than organization-based self-esteem.

Practical implications

– The relation between self-esteem and counterproductive behavior at work is important to organizations for two reasons. First, CWBs are very costly at all levels of the organization. Second, organizations and managers have some control over the level of their employee’s self-esteem.

Originality/value

– Previous research has used both consistency theory and ego-defense theory to make predictions concerning the self-esteem and CWB relationship. This paper provides support for examining this relation using consistency theory due to the negative correlation the authors found between CWB and self-esteem.

Which Personality Attributes Are Most Important in the Workplace?

Employees face a variety of work demands that place a premium on personal attributes, such as the degree to which they can be depended on to work independently, deal with stress, and interact positively with coworkers and customers. We examine evidence for the importance of these personality attributes using research strategies intended to answer three fundamental questions, including (a) how well does employees' standing on these attributes predict job performance?, (b) what types of attributes do employers seek to evaluate in interviews when considering applicants?, and (c) what types of attributes are rated as important for performance in a broad sampling of occupations across the U.S. economy? We summarize and integrate results from these three strategies using the Big Five personality dimensions as our organizing framework. Our findings indicate that personal attributes related to Conscientiousness and Agreeableness are important for success across many jobs, spanning across low to high levels of job complexity, training, and experience necessary to qualify for employment. The strategies lead to differing conclusions about the relative importance of Emotional Stability and Extraversion. We note implications for job seekers, for interventions aimed at changing standing on these attributes, and for employers.

Cross-cultural adjustment to the United States: the role of contextualized extraversion change

Personality traits can predict how well-sojourners and expatriates adjust to new cultures, but the adjustment process remains largely unexamined. Based on recent findings that reveal personality traits predict as well as respond to life events and experiences, this research focuses on within-person change in contextualized extraversion and its predictive validity for cross-cultural adjustment in international students who newly arrived in US colleges. We proposed that the initial level as well as the rate of change in school extraversion (i.e., contextualized extraversion that reflects behavioral tendency in school settings) will predict cross-cultural adjustment, withdrawal cognitions, and school satisfaction. Latent growth modeling of three-wave longitudinal surveys of 215 new international students (54% female, M_age = 24 years) revealed that the initial level of school extraversion significantly predicted cross-cultural adjustment, (lower) withdrawal cognitions, and satisfaction, while the rate of change (increase) in school extraversion predicted cross-cultural adjustment and (lower) withdrawal cognitions. We further modeled global extraversion and cross-cultural motivation as antecedents and explored within-person change in school extraversion as a proximal factor that affects adjustment outcomes. The findings highlight the malleability of contextualized personality, and more importantly, the importance of understanding within-person change in contextualized personality in a cross-cultural adjustment context. The study points to more research that explicate the process of personality change in other contexts.

The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance

Introduction

Sensitivity analyses refer to investigations of the degree to which the results of a meta-analysis remain stable when conditions of the data or the analysis change. To the extent that results remain stable, one can refer to them as robust. Sensitivity analyses are rarely conducted in the organizational science literature. Despite conscientiousness being a valued predictor in employment selection, sensitivity analyses have not been conducted with respect to meta-analytic estimates of the correlation (i.e., validity) between conscientiousness and job performance.

Methods

To address this deficiency, we reanalyzed the largest collection of conscientiousness validity data in the personnel selection literature and conducted a variety of sensitivity analyses.

Results

Publication bias analyses demonstrated that the validity of conscientiousness is moderately overestimated (by around 30%; a correlation difference of about .06). The misestimation of the validity appears to be due primarily to suppression of small effects sizes in the journal literature. These inflated validity estimates result in an overestimate of the dollar utility of personnel selection by millions of dollars and should be of considerable concern for organizations.

Conclusion

The fields of management and applied psychology seldom conduct sensitivity analyses. Through the use of sensitivity analyses, this paper documents that the existing literature overestimates the validity of conscientiousness in the prediction of job performance. Our data show that effect sizes from journal articles are largely responsible for this overestimation.

Assessing Workplace Emotional Intelligence: Development and Validation of an Ability-based Measure

Existing measures of Emotional Intelligence (EI), defined as the ability to perceive, understand, and manage emotions for productive purposes, have displayed limitations in predicting workplace outcomes, likely in part because they do not target this context. Such considerations led to the development of an ability EI measure with work-related scenarios in which respondents infer the likely emotions (perception) and combinations of emotion (understanding) that would occur to protagonists while rating the effectiveness of ways of responding (management). Study 1 (n = 290 undergraduates) used item-total correlations to select scenarios from a larger pool and Study 2 (n = 578) reduced the measure-termed the NEAT-to 30 scenarios on the basis of structural equation modeling. Study 3 (n = 96) then showed that the NEAT had expected correlations with personality and cognitive ability and Study 4 (n = 85) demonstrated convergent validity with other ability EI measures. Last, study 5 (n = 91) established that the NEAT had predictive validity with respect to job satisfaction, job stress, and job performance. The findings affirm the importance of EI in the workplace in the context of a valid new instrument for assessing relevant skills.

Specificity matters: criterion-related validity of contextualized and facet measures of conscientiousness in predicting college student performance

To enhance the predictive validity of self-report personality measures, 2 distinct ways of increasing specificity of personality measures have been proposed in the literature-contextual specificity (i.e., providing a contextual referent) and content specificity (i.e., focusing on more specific constructs such as the Big Five facets). This study extends this line of research by examining whether there is an optimal way to configure, align, or integrate contextual and content specificity using measures of conscientiousness to predict college student success. A sample of 478 undergraduate students completed 4 measures of conscientiousness that varied in the level of content and contextual specificity. These forms of specificity were crossed to yield 4 distinct measures of conscientiousness. We then evaluated and compared the relative importance and the incremental importance of these different measures in the prediction of academic success. Superior predictive validity was found for both contextualized and facet measures of conscientiousness compared to a measure of global conscientiousness in predicting grade-point average and a broader behavioral criterion of student performance. When contextual and content specificity approaches were compared and combined, we observed the strongest predictive validity when the level of specificity is appropriately matched between predictor and criterion.

On Being the Same in Different Places: Evaluating Frame-of-Reference Effects Across Two Social Contexts

Frame-of-reference (FOR) effects in personality assessment are demonstrated when self-rated items oriented to specific contexts (e.g., workplace) show better predictive validity than noncontextualized items. Empirical support of FOR effects typically relies on job performance ratings or academic grades for criteria. The current study evaluates FOR effects using ratings of personality provided by informants from the home or school context. Items from the NEO Five-Factor Inventory (NEO-FFI; Costa & McCrae, 1992) were contextualized to the home and school contexts to create NEO-Home and NEO-School versions. One hundred fifty-eight college students completed the NEO-Home and NEO-School questionnaires, and 161 college students completed the standard, noncontextualized NEO-FFI. All participants recruited one peer from college and at least one parent to complete standard rater versions of the NEO-FFI. Contextualized self-ratings did not show FOR effects. NEO-Home self-ratings did not correlate higher with parent ratings than with peer ratings, and NEO-School self-ratings did not correlate higher with peer ratings than with parent ratings. Standard NEO-FFI self-ratings generally showed higher self-informant agreement with both types of informants than contextualized self-ratings. The pattern of correlations suggests that validity is enhanced more by specific trait-informant combinations than by the contextualization of items to social contexts.