1
|
Lei L, Wang C, Pinto J. Do Chameleons Lead Better? A Meta-Analysis of the Self-Monitoring and Leadership Relationship. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN 2023:1461672231210778. [PMID: 38006305 DOI: 10.1177/01461672231210778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2023]
Abstract
The relationship between self-monitoring and leadership has been debated. We attempt to resolve this debate through a meta-analysis (N = 9,029 across 55 samples). Since this is the first meta-analysis that focuses on this relationship, we were able to study both focal constructs at a granular level. As hypothesized, self-monitoring is positively associated with leadership emergence and leadership effectiveness. Whereas self-monitoring is positively related to managerial leadership, its relationship with transactional leadership is non-significant. Contrary to our prediction that self-monitoring is negatively related to authentic leadership and to transformational leadership, we found positive relationships. Importantly, the relationship between self-monitoring and leadership variables is typically non-significant when the latter is measured by subordinate ratings. This casts doubt on the general finding that self-monitoring is positively related to leadership. Also, the relationships significantly differ when self-monitoring was measured by different scales. Implications for theory and practice are discussed.
Collapse
Affiliation(s)
| | - Chen Wang
- Western Michigan University, Kalamazoo, USA
| | | |
Collapse
|
2
|
Keener SK, Kepes S, Torka AK. The trustworthiness of the cumulative knowledge in industrial/organizational psychology: The current state of affairs and a path forward. Acta Psychol (Amst) 2023; 239:104005. [PMID: 37625919 DOI: 10.1016/j.actpsy.2023.104005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 07/13/2023] [Accepted: 08/04/2023] [Indexed: 08/27/2023] Open
Abstract
The goal of industrial/organizational (IO) psychology, is to build and organize trustworthy knowledge about people-related phenomena in the workplace. Unfortunately, as with other scientific disciplines, our discipline may be experiencing a "crisis of confidence" stemming from the lack of reproducibility and replicability of many of our field's research findings, which would suggest that much of our research may be untrustworthy. If a scientific discipline's research is deemed untrustworthy, it can have dire consequences, including the withdraw of funding for future research. In this focal article, we review the current state of reproducibility and replicability in IO psychology and related fields. As part of this review, we discuss factors that make it less likely that research findings will be trustworthy, including the prevalence of scientific misconduct, questionable research practices (QRPs), and errors. We then identify some root causes of these issues and provide several potential remedies. In particular, we highlight the need for improved research methods and statistics training as well as a re-alignment of the incentive structure in academia. To accomplish this, we advocate for changes in the reward structure, improvements to the peer review process, and the implementation of open science practices. Overall, addressing the current "crisis of confidence" in IO psychology requires individual researchers, academic institutions, and publishers to embrace system-wide change.
Collapse
Affiliation(s)
- Sheila K Keener
- Department of Management, Old Dominion University, Norfolk, VA, United States of America.
| | - Sven Kepes
- Department of Management and Entrepreneurship, Virginia Commonwealth University, Richmond, VA, United States of America.
| | - Ann-Kathrin Torka
- Department of Social, Work, and Organizational Psychology, TU Dortmund University, Dortmund, Germany.
| |
Collapse
|
3
|
Wilmot MP, Ones DS. Agreeableness and Its Consequences: A Quantitative Review of Meta-Analytic Findings. PERSONALITY AND SOCIAL PSYCHOLOGY REVIEW 2022; 26:242-280. [DOI: 10.1177/10888683211073007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
Agreeableness impacts people and real-world outcomes. In the most comprehensive quantitative review to date, we summarize results from 142 meta-analyses reporting effects for 275 variables, which represent N > 1.9 million participants from k > 3,900 studies. Arranging variables by their content and type, we use an organizational framework of 16 conceptual categories that presents a detailed account of Agreeableness’ external relations. Overall, the trait has effects in a desirable direction for 93% of variables (grand mean [Formula: see text]). We also review lower order trait evidence for 42 variables from 20 meta-analyses. Using these empirical findings, in tandem with existing theory, we synthesize eight general themes that describe Agreeableness’ characteristic functioning across variables: self-transcendence, contentment, relational investment, teamworking, work investment, lower results emphasis, social norm orientation, and social integration. We conclude by discussing potential boundary conditions of findings, contributions and limitations of our review, and future research directions.
Collapse
Affiliation(s)
| | - Deniz S. Ones
- University of Minnesota Twin Cities, Minneapolis, USA
| |
Collapse
|
4
|
Hu C, Zhang S, Chen YY, Griggs TL. A meta-analytic study of subjective career plateaus. JOURNAL OF VOCATIONAL BEHAVIOR 2022. [DOI: 10.1016/j.jvb.2021.103649] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
5
|
Schanandore JV, Wolden M, Smart N. The accuracy and reliability of the Suchey-Brooks pubic symphysis age estimation method: Systematic review and meta-analysis. J Forensic Sci 2021; 67:56-67. [PMID: 34617597 DOI: 10.1111/1556-4029.14911] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Revised: 08/25/2021] [Accepted: 09/14/2021] [Indexed: 11/26/2022]
Abstract
Suchey-Brooks method has been studied many times with varying reports of accuracy and reliability. A systematic review and meta-analyses were utilized to quantitatively determine the accuracy and reliability of the Suchey-Brooks (S-B) method. A systematic search of PubMed and EBSCO health and medical databases was performed. Meta-analyses were performed to quantify the relationship between actual known age at death and the S-B method using Spearman's Rho and Pearson's r for (1) combined males and females estimates, (2) male-only estimates, and (3) female-only estimates. Overall correlation coefficient using Cohen's kappa, Spearman's Rho, and Pearson's r was also calculated to determine the interrater and intrarater reliability using the S-B method. Eighteen studies classified as moderate-to-high methodological quality met the inclusion criteria. The sample sizes were different for the combined male and female (n = 2620), male-only (n = 2602), and female-only (n = 1431) meta-analyses. The effect size of the age at death meta-analyses was large and significant for combined males and females (Spearman's Rho = 0.62; Pearson's r = 0.65), male-only (Spearman's Rho = 0.77; Pearson's r = 0.75), and female-only (Spearman's Rho = 0.71; Pearson's r = 0.71). The overall correlation coefficients of the interrater (Kappa = 0.76; Spearman's Rho = 0.73; Pearson's r = 0.80) and intrarater (Kappa = 0.81; Spearman's Rho = 0.91; Pearson's r = 0.83) reliability meta-analyses were large and significant. A significant degree of heterogeneity was present in all meta-analyses, with minimal evidence of publication bias. The meta-analyses results suggest the S-B method is highly reliable with a moderate-to-high degree of accuracy. Our results also identified sample size and accuracy differences between male and female individuals.
Collapse
Affiliation(s)
| | - Mitch Wolden
- Physical Therapy Program, University of Jamestown, Fargo, North Dakota, USA.,School of Science and Technology in the Faculty of Science, Agriculture, Business and Law, University of New England, Armidale, New South Wales, Australia
| | - Neil Smart
- School of Science and Technology in the Faculty of Science, Agriculture, Business and Law, University of New England, Armidale, New South Wales, Australia
| |
Collapse
|
6
|
Al Hosani KA, van Ewijk AR, Hussain M. Determinants of employee creative behavior in the UAE public sector. INTERNATIONAL JOURNAL OF PRODUCTIVITY AND PERFORMANCE MANAGEMENT 2021. [DOI: 10.1108/ijppm-11-2020-0601] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose
Service levels between public organizations in the United Arab Emirates (UAE) differ and fluctuate. Service levels were found to increase with the level of employee creative behavior (ECB), which is currently a focal point in organizational policies by the federal UAE government. While the literature presents an ample array of determinants of ECB, local context typically influences which determinants provide more leverage than others. Therefore, this research aims to present a customized ranking of determinants that enables managers in the UAE public sector to successfully stimulate ECB.
Design/methodology/approach
Through a systematic literature review, this study identified 21 determinants at four different levels and subsequently utilized the analytic hierarchical process (AHP) to map their relative importance. Data were attained from 33 experts: managers and senior employees from different organizations.
Findings
Findings indicate that, overall, the individual level is seen as the most important leverage point to stimulate ECB, followed by the team level. Comparing the determinants under each level, management support (organizational level) is perceived as the most impactful, followed by coworker support (team level), and enhancing creative self-efficacy (individual level).
Originality/value
This study compares obtained levels from other studies with new levels in building the AHP model. These insights guide managers in the UAE public sector who aim to enhance ECB, which will contribute to increased service quality. The study introduced a comprehensive ECB framework of 21 determinants. Even though many ECB studies applied in different countries and industries, but to researchers' knowledge, this is the first ECB study that applied at the UAE's public sector using the AHP model.
Collapse
|
7
|
Alwadani R, Ndubisi NO. Family business goal, sustainable supply chain management, and platform economy: a theory-based review & propositions for future research. INTERNATIONAL JOURNAL OF LOGISTICS-RESEARCH AND APPLICATIONS 2021. [DOI: 10.1080/13675567.2021.1944069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Rawa Alwadani
- Department of Management & Marketing, Qatar University, Doha, Qatar
| | | |
Collapse
|
8
|
Ružojčić M, Galić Z, Jerneić Ž. How does implicit aggressiveness translate into counterproductive work behaviors? The role of job satisfaction and workplace anger. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2021. [DOI: 10.1111/ijsa.12327] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Mitja Ružojčić
- Department of Psychology Faculty of Humanities and Social Sciences University of Zagreb Zagreb Croatia
| | - Zvonimir Galić
- Department of Psychology Faculty of Humanities and Social Sciences University of Zagreb Zagreb Croatia
| | - Željko Jerneić
- Department of Psychology Faculty of Humanities and Social Sciences University of Zagreb Zagreb Croatia
| |
Collapse
|
9
|
LeBreton JM, Reichin SL, Nijenhuis J, Cremers M, Heijden‐Lek K. Validity evidence and measurement equivalence for the Dutch translation of the conditional reasoning test for aggression. APPLIED PSYCHOLOGY-AN INTERNATIONAL REVIEW-PSYCHOLOGIE APPLIQUEE-REVUE INTERNATIONALE 2021. [DOI: 10.1111/apps.12309] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- James M. LeBreton
- Department of Psychology Pennsylvania State University University Park PA USA
| | - Sydney L. Reichin
- Department of Psychology Pennsylvania State University University Park PA USA
| | - Jan Nijenhuis
- Dutch Ministry of Defense Amsterdam The Netherlands
- Vrije Universiteit Amsterdam Amsterdam The Netherlands
- Biomedical Department Chosun University Gwangju South Korea
| | | | | |
Collapse
|
10
|
Critically evaluating and advancing research and statistical methods in vocational behavior research: Introduction to the 2020 special issue on research methodology. JOURNAL OF VOCATIONAL BEHAVIOR 2021. [DOI: 10.1016/j.jvb.2021.103529] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
11
|
DeSimone JA, Brannick MT, O’Boyle EH, Ryu JW. Recommendations for Reviewing Meta-Analyses in Organizational Research. ORGANIZATIONAL RESEARCH METHODS 2020. [DOI: 10.1177/1094428120967089] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
This article encourages transparency in the reporting of meta-analytic procedures. Specifically, we highlight aspects of meta-analytic search, coding, data presentation, and data analysis where published meta-analyses often fall short in presenting sufficient information to allow replication. We identify opportunities where reviewers can request additional information or analyses that will enhance transparent reporting practices and facilitate the evaluation of quality in meta-analytic reporting. We focus on concerns specific to (or prevalent in) meta-analyses conducted in organizational research. In doing so, we reference a number of existing and emerging techniques, highlighting their contribution to meta-analysis while emphasizing key information reviewers may request. Our focus is primarily on meta-analyses, but secondary uses of meta-analytic data are also considered. We conclude by providing a checklist for reviewers in an effort to facilitate the review process as it pertains to the goals of transparency and replicability.
Collapse
|
12
|
Steffens NK, Munt KA, van Knippenberg D, Platow MJ, Haslam SA. Advancing the social identity theory of leadership: A meta-analytic review of leader group prototypicality. ORGANIZATIONAL PSYCHOLOGY REVIEW 2020. [DOI: 10.1177/2041386620962569] [Citation(s) in RCA: 44] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
This research advances a social identity approach to leadership through a meta-analysis examining four novel hypotheses that clarify the nature and impact of leader group prototypicality (the extent to which a leader is perceived to embody shared social identity). A random-effects meta-analysis ( k = 128, N = 32,834) reveals a moderate-to-large effect of prototypicality that holds across evaluative and behavioral outcomes. The effect is stronger (a) when prototypicality is conceptualized as the ideal-type rather than the average group member, (b) for stronger prototypes (indexed by group longevity), and (c) for group members in formal rather than nonformal leadership roles. The effect is not contingent on group prototypicality entailing differentiation from other (out)groups. Additionally, results provide meta-analytic evidence of widely examined key factors: follower group identification (which enhances the relationship) and leader group-serving behavior (which attenuates the relationship). Building on these findings, we outline the implications for the next wave of theoretical and empirical work.
Collapse
|
13
|
Ajuogu PK, Wolden M, McFarlane JR, Hart RA, Carlson DJ, Van der Touw T, Smart NA. Effect of low- and high-protein maternal diets during gestation on reproductive outcomes in the rat: a systematic review and meta-analysis. J Anim Sci 2020; 98:5680668. [PMID: 31853549 DOI: 10.1093/jas/skz380] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2019] [Accepted: 12/17/2019] [Indexed: 01/17/2023] Open
Abstract
Studies with animal models have consistently demonstrated adverse health outcomes in offspring born following nutritional manipulation during gestation. However, the effects of gestational dietary protein modification on reproductive outcomes at birth are less clear. We, therefore, conducted a systematic review and meta-analysis of controlled trials to determine whether high- or low-protein diets are associated with altered reproductive outcomes in a commonly studied species, the rat. Included studies were identified through a systematic search using electronic databases and manual literature review to identify randomized studies published between June 1972 and March 2019. Thirty-two studies were identified and used to analyze the effects of low- and high-protein gestational diets on litter size, litter weight, gestational weight gain, and gestational feed intake. The results indicate that low-protein diets significantly reduced litter weight (P < 0.00001) and gestational weight gain (P < 0.0006), but did not influence litter size (P = 0.62) or gestational feed intake (P = 0.25). In contrast, high-protein diets were found to reduce gestational feed intake (P = 0.004) but did not influence litter size (P = 0.56), litter weight (P = 0.22), or gestational weight gain (P = 0.35). The results suggest that low but not high-protein gestational diets alter reproductive outcomes at birth in rats.
Collapse
Affiliation(s)
- Peter K Ajuogu
- School of Science and Technology, University of New England, Armidale, NSW, Australia
| | - Mitchell Wolden
- Physical Therapy Program, University of Jamestown, Fargo, ND
| | - James R McFarlane
- Centre for Bioactive Discovery in Health and Ageing, University of New England, Armidale, NSW, Australia
| | - Robert A Hart
- School of Science and Technology, University of New England, Armidale, NSW, Australia
| | - Debra J Carlson
- School of Health, Medical and Applied Sciences, Central Queensland University Australia, North Rockhampton, QLD, Australia
| | - Tom Van der Touw
- School of Science and Technology, University of New England, Armidale, NSW, Australia
| | - Neil A Smart
- School of Science and Technology, University of New England, Armidale, NSW, Australia
| |
Collapse
|
14
|
Wolden M, Hill B, Voorhees S. Predicting Success for Student Physical Therapists on the National Physical Therapy Examination: Systematic Review and Meta-Analysis. Phys Ther 2020; 100:73-89. [PMID: 31584670 DOI: 10.1093/ptj/pzz145] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/04/2019] [Revised: 03/07/2019] [Accepted: 06/10/2019] [Indexed: 11/12/2022]
Abstract
BACKGROUND In physical therapist education, the National Physical Therapy Examination (NPTE) is the predominant measure of student success. Because the NPTE is a high-stakes examination, predicting NPTE performance is important for physical therapist students and programs. PURPOSE The purpose of this study was to determine the relationships between first attempt NPTE performance, physical therapist applicant variables, and physical therapist student variables. The 4 identified physical therapist applicant variables were undergraduate cumulative grade point average (GPA), undergraduate GPA for prerequisite courses, Graduate Record Examination verbal and quantitative subscale scores, and admission scores. The 4 identified physical therapist student variables were first- and third-year physical therapist student GPA, clinical performance scores (first and final clinical experiences), noncognitive student variables, and comprehensive examination scores. DATA SOURCES A systematic search was performed using the databases PubMed and EBSCO (1966-2018). STUDY SELECTION Studies included in the review met the following criteria: the physical therapist education program offered an entry-level master's or doctorate degree, pertinent data were available for each independent variable, the relationship between the independent variable and the NPTE was investigated, the NPTE was a continuous variable, the program was a US entry-level therapist education program, and the study was peer reviewed. DATA EXTRACTION Two independent reviewers completed data extraction and quality appraisal using the McMaster Appraisal Tool. DATA SYNTHESIS Random-effects meta-analyses using the Pearson product-moment correlation coefficient as the measure of effect size was used. LIMITATIONS Limitations of the review included a lack of homogeneity, high NPTE pass rates, modifications to the McMaster Appraisal Tool, small sample sizes, and publication bias. CONCLUSIONS No one physical therapist applicant variable should be used as an independent predictor of first attempt NPTE performance. For physical therapist students, first- and third-year physical therapist student GPA had a strong relationship with first attempt NPTE performance; clinical performance had a weak and nonsignificant relationship with first attempt NPTE performance.
Collapse
Affiliation(s)
- Mitch Wolden
- Physical Therapy Program, University of Jamestown, 4190 26th Ave South, Fargo, ND 58104 (USA)
| | - Brent Hill
- School of Education, North Dakota State University, Fargo, North Dakota
| | | |
Collapse
|
15
|
Pietschnig J, Siegel M, Eder JSN, Gittler G. Effect Declines Are Systematic, Strong, and Ubiquitous: A Meta-Meta-Analysis of the Decline Effect in Intelligence Research. Front Psychol 2019; 10:2874. [PMID: 31920891 PMCID: PMC6930891 DOI: 10.3389/fpsyg.2019.02874] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2019] [Accepted: 12/04/2019] [Indexed: 12/14/2022] Open
Abstract
Empirical sciences in general and psychological science in particular are plagued by replicability problems and biased published effect sizes. Although dissemination bias-related phenomena such as publication bias, time-lag bias, or visibility bias are well-known and have been intensively studied, another variant of effect distorting mechanisms, so-called decline effects, have not. Conceptually, decline effects are rooted in low initial (exploratory) study power due to strategic researcher behavior and can be expected to yield overproportional effect declines. Although decline effects have been documented in individual meta-analytic investigations, systematic evidence for decline effects in the psychological literature remains to date unavailable. Therefore, we present in this meta-meta-analysis a systematic investigation of the decline effect in intelligence research. In all, data from 22 meta-analyses comprising 36 meta-analytical and 1,391 primary effect sizes (N = 697,000+) that have been published in the journal Intelligence were included in our analyses. Two different analytic approaches showed consistent evidence for a higher prevalence of cross-temporal effect declines compared to effect increases, yielding a ratio of about 2:1. Moreover, effect declines were considerably stronger when referenced to the initial primary study within a meta-analysis, yielding about twice the magnitude of effect increases. Effect misestimations were more substantial when initial studies had smaller sample sizes and reported larger effects, thus indicating suboptimal initial study power as the main driver of effect misestimations in initial studies. Post hoc study power comparisons of initial versus subsequent studies were consistent with this interpretation, showing substantially lower initial study power of declining, than of increasing effects. Our findings add another facet to the ever accumulating evidence about non-trivial effect misestimations in the scientific literature. We therefore stress the necessity for more rigorous protocols when it comes to designing and conducting primary research as well as reporting findings in exploratory and replication studies. Increasing transparency in scientific processes such as data sharing, (exploratory) study preregistration, but also self- (or independent) replication preceding the publication of exploratory findings may be suitable approaches to strengthen the credibility of empirical research in general and psychological science in particular.
Collapse
Affiliation(s)
- Jakob Pietschnig
- Department of Applied Psychology: Health, Development, Enhancement and Intervention, Faculty of Psychology, University of Vienna, Vienna, Austria
| | | | | | | |
Collapse
|
16
|
Erdfelder E, Heck DW. Detecting Evidential Value and p-Hacking With the p-Curve Tool. ZEITSCHRIFT FUR PSYCHOLOGIE-JOURNAL OF PSYCHOLOGY 2019. [DOI: 10.1027/2151-2604/a000383] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Abstract. Simonsohn, Nelson, and Simmons (2014a) proposed p-curve – the distribution of statistically significant p-values for a set of studies – as a tool to assess the evidential value of these studies. They argued that, whereas right-skewed p-curves indicate true underlying effects, left-skewed p-curves indicate selective reporting of significant results when there is no true effect (“ p-hacking”). We first review previous research showing that, in contrast to the first claim, null effects may produce right-skewed p-curves under some conditions. We then question the second claim by showing that not only selective reporting but also selective nonreporting of significant results due to a significant outcome of a more popular alternative test of the same hypothesis may produce left-skewed p-curves, even if all studies reflect true effects. Hence, just as right-skewed p-curves do not necessarily imply evidential value, left-skewed p-curves do not necessarily imply p-hacking and absence of true effects in the studies involved.
Collapse
Affiliation(s)
- Edgar Erdfelder
- Department of Psychology, School of Social Sciences, University of Mannheim, Germany
| | - Daniel W. Heck
- Department of Psychology, School of Social Sciences, University of Mannheim, Germany
- Department of Psychology, University of Marburg, Germany
| |
Collapse
|
17
|
van Aert RCM, Wicherts JM, van Assen MALM. Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS One 2019; 14:e0215052. [PMID: 30978228 PMCID: PMC6461282 DOI: 10.1371/journal.pone.0215052] [Citation(s) in RCA: 131] [Impact Index Per Article: 21.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2018] [Accepted: 03/26/2019] [Indexed: 01/08/2023] Open
Abstract
Publication bias is a substantial problem for the credibility of research in general and of meta-analyses in particular, as it yields overestimated effects and may suggest the existence of non-existing effects. Although there is consensus that publication bias exists, how strongly it affects different scientific literatures is currently less well-known. We examined evidence of publication bias in a large-scale data set of primary studies that were included in 83 meta-analyses published in Psychological Bulletin (representing meta-analyses from psychology) and 499 systematic reviews from the Cochrane Database of Systematic Reviews (CDSR; representing meta-analyses from medicine). Publication bias was assessed on all homogeneous subsets (3.8% of all subsets of meta-analyses published in Psychological Bulletin) of primary studies included in meta-analyses, because publication bias methods do not have good statistical properties if the true effect size is heterogeneous. Publication bias tests did not reveal evidence for bias in the homogeneous subsets. Overestimation was minimal but statistically significant, providing evidence of publication bias that appeared to be similar in both fields. However, a Monte-Carlo simulation study revealed that the creation of homogeneous subsets resulted in challenging conditions for publication bias methods since the number of effect sizes in a subset was rather small (median number of effect sizes equaled 6). Our findings are in line with, in its most extreme case, publication bias ranging from no bias until only 5% statistically nonsignificant effect sizes being published. These and other findings, in combination with the small percentages of statistically significant primary effect sizes (28.9% and 18.9% for subsets published in Psychological Bulletin and CDSR), led to the conclusion that evidence for publication bias in the studied homogeneous subsets is weak, but suggestive of mild publication bias in both psychology and medicine.
Collapse
Affiliation(s)
- Robbie C. M. van Aert
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
| | - Jelte M. Wicherts
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
| | - Marcel A. L. M. van Assen
- Department of Methodology and Statistics, Tilburg University, Tilburg, the Netherlands
- Department of Sociology, Utrecht University, Utrecht, the Netherlands
| |
Collapse
|
18
|
LeBreton JM, Grimaldi EM, Schoen JL. Conditional Reasoning: A Review and Suggestions for Future Test Development and Validation. ORGANIZATIONAL RESEARCH METHODS 2019. [DOI: 10.1177/1094428118816366] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Lawrence R. James spent the last 20 years of his 35-year career developing and validating a new theory of personality that he called conditional reasoning. This theory was focused on mapping and measuring core aspects of the implicit (i.e., unconscious) personality. In this article, we (a) review James’s seminal contributions to the theory and measurement of conditional reasoning, (b) discuss subsequent contributions made in the area of conditional reasoning, and (c) provide a brief “look under the hood” at James’s approach to test development and validation. This final section of our paper is designed to familiarize other researchers with the protocols that James and his colleagues have used over the past 20 years. Many of these protocols have gone unmentioned or only briefly acknowledged (e.g., in conference presentations or informal meetings); indeed, many of these validation protocols were “implicit” in the thinking of James and his approach to the study of personality. Having benefited from working closely with James, we were privy to many of these implicit assumptions and protocols that privately guided James’s early work on conditional reasoning.
Collapse
|
19
|
Zare M, Flinchbaugh C. Voice, creativity, and big five personality traits: A meta-analysis. HUMAN PERFORMANCE 2018. [DOI: 10.1080/08959285.2018.1550782] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
20
|
Gadelrab HF. An Investigation of Differential Relationships of Implicit and Explicit Aggression: Validation of an Arabic Version of the Conditional Reasoning Test for Aggression. J Pers Assess 2018; 101:609-620. [PMID: 30160525 DOI: 10.1080/00223891.2018.1501695] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Most personality research in the workplace relies on self-reports. Although self-report measures are believed to assess the explicit aspect of personality, a more recent approach for assessing implicit personality is based on conditional reasoning. The Conditional Reasoning Test for Aggression (CRT-A) was developed to assess implicit cognitions that justify aggressive behavior. The integrative model of personality proposes that both the explicit and implicit personality interact together to predict different behavioral outcomes. The results of this investigation are reported in two studies. The purpose of Study 1 was to adapt the CRT-A to the Arabic language and to examine its validity evidence using a large sample of Egyptian employees (n = 1,046). Study 2 aimed at examining the differential relationships of implicit and explicit components of aggressive personality in explaining different forms of aggressive behavior variance (n = 271). The results of Study 1 revealed that the psychometric properties of the adapted Arabic version of CRT-A were similar to those obtained for the United States. In addition, factor analysis results were consistent with the psychological theory used to build the conditional reasoning problems. The results of Study 2 provide substantial empirical support for the integrative model of assessment of aggression.
Collapse
Affiliation(s)
- Hesham F Gadelrab
- Department of Psychology, Kuwait University, Shuwaikh, Kuwait.,Department of Educational Psychology, Mansoura University, Mansoura, Egypt
| |
Collapse
|
21
|
Galić Z, Ružojčić M, Jerneić Ž, Tonković Grabovac M. Disentangling the Relationship Between Implicit Aggressiveness and Counterproductive Work Behaviors: The Role of Job Attitudes. HUMAN PERFORMANCE 2018. [DOI: 10.1080/08959285.2018.1455686] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
- Zvonimir Galić
- Department of Psychology, Faculty of Humanities and Social Sciences, University of Zagreb, Zagreb, Croatia
| | - Mitja Ružojčić
- Department of Psychology, Faculty of Humanities and Social Sciences, University of Zagreb, Zagreb, Croatia
| | - Željko Jerneić
- Department of Psychology, Faculty of Humanities and Social Sciences, University of Zagreb, Zagreb, Croatia
| | - Maša Tonković Grabovac
- Department of Psychology, University Department for Croatian Studies, University of Zagreb, Zagreb, Croatia
| |
Collapse
|
22
|
Schneck A. Examining publication bias-a simulation-based evaluation of statistical tests on publication bias. PeerJ 2017; 5:e4115. [PMID: 29204324 PMCID: PMC5712469 DOI: 10.7717/peerj.4115] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Accepted: 11/10/2017] [Indexed: 11/20/2022] Open
Abstract
Background Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Methods Four tests on publication bias, Egger’s test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer, meaning the repeated analysis of new datasets, or p-hacking, meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect (β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies (N = 100, 500), and the number of observations for the publication bias tests (K = 100, 1,000) were varied. Results All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. Discussion The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.
Collapse
Affiliation(s)
- Andreas Schneck
- Department of Sociology, Ludwig-Maximilians-Universität München, Munich, Germany
| |
Collapse
|
23
|
Abstract
Purpose
Information technology has been recognized as one of the keys to improved productivity in organizations. Yet, existing research has not paid sufficient attention to how information systems (ISs) influence the creative performance of individual users. The paper aims to discuss this issue.
Design/methodology/approach
This study draws on the theories of flow and cognitive load to establish a model of the predicted influences. The authors hypothesize that the information technology supports creativity by engaging individuals in a creative process and by lowering their cognitive load related to the process. To test these hypotheses, the authors employ a meta-analytical structural equation modeling approach using 24 previous studies on creativity and ISs use.
Findings
The results suggest that factors that help the user to maintain an interest in the performed task, immerse the user in a state of flow, and lower a person’s cognitive load during IS use can affect the user’s creative performance.
Research limitations/implications
The findings imply that a combination of the theories of flow and cognitive load complements the understanding of how ISs influence creativity.
Originality/value
This paper proposes an explanation on why ISs affect creativity, which can be used by scholars to position further research, and by practitioners to implement creativity support systems.
Collapse
|
24
|
Acute, Recent and Past HEV Infection among Voluntary Blood Donors in China: A Systematic Review and Meta-Analysis. PLoS One 2016; 11:e0161089. [PMID: 27597991 PMCID: PMC5012590 DOI: 10.1371/journal.pone.0161089] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2016] [Accepted: 07/31/2016] [Indexed: 12/12/2022] Open
Abstract
Introduction Hepatitis E virus is one of new threats to blood safety which was usually considered to be transmitted via fecal-oral route. China is one of the hyperendemic regions where frequent outbreaks of hepatitis E are noted. However, the overall prevalence of HEV infection among mainland Chinese blood donors is not clear until now. Method The peer-reviewed literatures reporting the prevalence of HEV in Chinese blood donors were identified by systematic searching of five electronic databases. The systematic review and meta-analysis were conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement issued in 2009. Data manipulation and statistical analyses were performed by Stata 12.0. Results Fourteen eligible articles involving 22 independent studies were included. Pooled prevalence of HEV infection biomarkers (anti-HEV IgG, anti-HEV IgM, RNA and antigen) among mainland Chinese blood donors were 29.2%, 1.1%, 0.1% and 0.1%, respectively which were higher than the data reported in other countries. The analysis of HEV genotypes indicated that the most prevalent strains in Chinese blood donors were genotype 1 and 4. Conclusions Mainland China is indicated with a relatively higher risk of transmission of hepatitis E through transfusion and the screening of blood donors for HEV RNA, especially in HEV-endemic areas, might reduce the potential risk of HEV infection via transfusion.
Collapse
|
25
|
Banks GC, McCauley KD, Gardner WL, Guler CE. A meta-analytic review of authentic and transformational leadership: A test for redundancy. LEADERSHIP QUARTERLY 2016. [DOI: 10.1016/j.leaqua.2016.02.006] [Citation(s) in RCA: 283] [Impact Index Per Article: 31.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
26
|
Kepes S, McDaniel MA. The Validity of Conscientiousness Is Overestimated in the Prediction of Job Performance. PLoS One 2015; 10:e0141468. [PMID: 26517553 PMCID: PMC4627756 DOI: 10.1371/journal.pone.0141468] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2015] [Accepted: 10/08/2015] [Indexed: 11/19/2022] Open
Abstract
Introduction Sensitivity analyses refer to investigations of the degree to which the results of a meta-analysis remain stable when conditions of the data or the analysis change. To the extent that results remain stable, one can refer to them as robust. Sensitivity analyses are rarely conducted in the organizational science literature. Despite conscientiousness being a valued predictor in employment selection, sensitivity analyses have not been conducted with respect to meta-analytic estimates of the correlation (i.e., validity) between conscientiousness and job performance. Methods To address this deficiency, we reanalyzed the largest collection of conscientiousness validity data in the personnel selection literature and conducted a variety of sensitivity analyses. Results Publication bias analyses demonstrated that the validity of conscientiousness is moderately overestimated (by around 30%; a correlation difference of about .06). The misestimation of the validity appears to be due primarily to suppression of small effects sizes in the journal literature. These inflated validity estimates result in an overestimate of the dollar utility of personnel selection by millions of dollars and should be of considerable concern for organizations. Conclusion The fields of management and applied psychology seldom conduct sensitivity analyses. Through the use of sensitivity analyses, this paper documents that the existing literature overestimates the validity of conscientiousness in the prediction of job performance. Our data show that effect sizes from journal articles are largely responsible for this overestimation.
Collapse
Affiliation(s)
- Sven Kepes
- Department of Management, School of Business, Virginia Commonwealth University, Richmond, Virginia, United States of America
- * E-mail:
| | - Michael A. McDaniel
- Department of Management, School of Business, Virginia Commonwealth University, Richmond, Virginia, United States of America
| |
Collapse
|
27
|
Kepes S, McDaniel MA. How Trustworthy Is the Scientific Literature in Industrial and Organizational Psychology? INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY-PERSPECTIVES ON SCIENCE AND PRACTICE 2015. [DOI: 10.1111/iops.12045] [Citation(s) in RCA: 95] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
The trustworthiness of research findings has been questioned in many domains of science. This article calls for a review of the trustworthiness of the scientific literature in industrial–organizational (I–O) psychology and a reconsideration of common practices that may harm the credibility of our literature. We note that most hypotheses in I–O psychology journals are confirmed. Thus, we are either approaching omniscience or our journals are publishing an unrepresentative sample of completed research. We view the latter explanation as more likely. We review structural problems in the publication process and in the conduct of research that is likely to promote a distortion of scientific knowledge. We then offer recommendations to make the I–O literature more accurate and trustworthy.
Collapse
|
28
|
Galić Z, Scherer KT, LeBreton JM. Validity Evidence for a Croatian Version of the Conditional Reasoning Test for Aggression. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2014. [DOI: 10.1111/ijsa.12082] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
- Zvonimir Galić
- Department of Psychology; University of Zagreb; Luciceva 3 10 000 Zagreb Croatia
| | | | | |
Collapse
|
29
|
Schewe AF, Hülsheger UR, Maier GW. Metaanalyse–praktische Schritte und Entscheidungen im Umsetzungsprozess. ZEITSCHRIFT FUR ARBEITS-UND ORGANISATIONSPSYCHOLOGIE 2014. [DOI: 10.1026/0932-4089/a000165] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
In diesem Beitrag wird der Durchführungsprozess metaanalytischer Techniken nach Hunter und Schmidt (2004) Schritt für Schritt beschrieben. In Form eines Tutoriums geben wir evidenzbasierte Empfehlungen, verweisen auf relevante Quellen und Hilfsmittel und bewerten alternative Vorgehensweisen in Bezug auf ihre Güte und Akzeptanz. Wir wenden uns an Forschende, die eine Metaanalyse mit der Besonderheit der Artefaktkorrekturen nach Hunter und Schmidt durchführen wollen und geben Anregungen für die Verbreitung metaanalytischer Ergebnisse sowohl im Wissenschaftskontext als auch im Austausch mit Praktikern.
Collapse
|
30
|
Affiliation(s)
- Ann Marie Ryan
- Department of Psychology, Michigan State University, East Lansing, Michigan 48824;
| | - Robert E. Ployhart
- Darla Moore School of Business, University of South Carolina, Columbia, South Carolina 29208;
| |
Collapse
|
31
|
Chiaburu DS, Smith TA, Wang J, Zimmerman RD. Relative Importance of Leader Influences for Subordinates’ Proactive Behaviors, Prosocial Behaviors, and Task Performance. JOURNAL OF PERSONNEL PSYCHOLOGY 2014. [DOI: 10.1027/1866-5888/a000105] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
We meta-analytically examine the relationships between three forms of leader influence, contingent reward (transactional), leader-member exchange (LMX; relational), and transformational (change-oriented) on subordinates’ proactive behaviors. Using non-self-reported data from a combined sample of more than 9,000 employees, we confirm positive relationships between leader influences and employee proactive outcomes. We examine the extent to which one leadership influence is stronger than the others in promoting subordinate proactivity. By combining our new meta-analytic data with existing meta-analytic correlations, we further investigate the extent to which various leadership predictors are differentially related to proactive and prosocial contextual performance, and to task performance. For all outcomes, there are only minimal differences between the contingent reward, LMX, and transformational leadership predictors. Using our results, we propose future research directions for the relationship between leader influences and subordinate work effectiveness.
Collapse
Affiliation(s)
- Dan S. Chiaburu
- Mays Business School, Texas A&M University, College Station, TX, USA
| | - Troy A. Smith
- Mays Business School, Texas A&M University, College Station, TX, USA
| | - Jiexin Wang
- Mays Business School, Texas A&M University, College Station, TX, USA
| | | |
Collapse
|
32
|
Chiaburu DS, Peng AC, Oh IS, Banks GC, Lomeli LC. Antecedents and consequences of employee organizational cynicism: A meta-analysis. JOURNAL OF VOCATIONAL BEHAVIOR 2013. [DOI: 10.1016/j.jvb.2013.03.007] [Citation(s) in RCA: 95] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
33
|
Abstract
Empirical research investigating the impact of top management team (TMT) diversity on executives’ decision making has produced inconclusive results. To synthesize and aggregate the results on the diversity-performance link, a meta-regression analysis (MRA) is conducted. It integrates more than 200 estimates from 53 empirical studies investigating TMT diversity and its impact on the quality of executives’ decision making as reflected in corporate performance. The analysis contributes to the literature by theoretically discussing and empirically examining the effects of TMT diversity on corporate performance. Our results do not show a link between TMT diversity and performance but provide evidence for publication bias. Thus, the findings raise doubts on the impact of TMT diversity on performance.
Collapse
Affiliation(s)
| | - Hong T. M. Bui
- Southampton University, School of Management, UK
- Vietnam National University, Hanoi, Vietnam
| |
Collapse
|
34
|
Chiaburu DS, Lorinkova NM, Van Dyne L. Employees’ Social Context and Change-Oriented Citizenship. GROUP & ORGANIZATION MANAGEMENT 2013. [DOI: 10.1177/1059601113476736] [Citation(s) in RCA: 92] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Change-oriented citizenship depends on support received from employees’ social context. Meta-analytic tests based on 131 independent samples and 38,409 employees confirmed positive relationships between leader, coworker, and organizational support and change-oriented citizenship, even after accounting for employees’ attitudes and intentions (i.e., job satisfaction, organizational commitment, and intention to quit). Moderator analyses indicated that specific coworker and organizational support had stronger relationships with change-oriented citizenship than generic support. In contrast, specific and generic leader support were equally important predictors of change-oriented citizenship.
Collapse
|
35
|
Kepes S, Banks GC, McDaniel M, Whetzel DL. Publication Bias in the Organizational Sciences. ORGANIZATIONAL RESEARCH METHODS 2012. [DOI: 10.1177/1094428112452760] [Citation(s) in RCA: 238] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Affiliation(s)
- Sven Kepes
- Virginia Commonwealth University, Richmond, VA, USA
| | | | | | | |
Collapse
|