What do meta-analysts need in primary studies? Guidelines and the SEMI checklist for facilitating cumulative knowledge

Meta-analysis is often recognized as the highest level of evidence due to its notable advantages. Therefore, ensuring the precision of its findings is of utmost importance. Insufficient reporting in primary studies poses challenges for meta-analysts, hindering study identification, effect size estimation, and meta-regression analyses. This manuscript provides concise guidelines for the comprehensive reporting of qualitative and quantitative aspects in primary studies. Adhering to these guidelines may help researchers enhance the quality of their studies and increase their eligibility for inclusion in future research syntheses, thereby enhancing research synthesis quality. Recommendations include incorporating relevant terms in titles and abstracts to facilitate study retrieval and reporting sufficient data for effect size calculation. Additionally, a new checklist is introduced to help applied researchers thoroughly report various aspects of their studies.

The perceived role of null results in school psychology research and publication

Researchers appear to assume that published research is limited to significant findings. If that is the case, it may be related to perceived or actual publication bias (i.e., journals publishing only significant findings) and/or the file-drawer problem (i.e., researchers not pursuing publication of null results). The lack of published null results can result in faulty decision-making based upon incomplete evidence. Thus, it is important to know the prevalence of, and the contributing factors to, researchers' failure to submit null results. Few studies have addressed this issue in psychology and none have targeted school psychology. Consequently, this study examined the file drawer problem and perception of publication bias among school psychologists. Survey data from 95 school psychology faculty indicated that participants published about half of the studies that they had conducted, suggesting that the file drawer problem is experienced by this population. While lack of time appeared to impact publication pursuit, participants' responses also suggested they believed in publication bias. Obtaining null results substantially impacted the decision to write up studies in pursuit of publication. Therefore, it seems that a sizeable percentage of school psychology research is not available for review by researchers or practitioners.

Leader Narcissism and Outcomes in Organizations: A Review at Multiple Levels of Analysis and Implications for Future Research

Narcissists often pursue leadership and are selected for leadership positions by others. At the same time, they act in their own best interest, putting the needs and interests of others at risk. While theoretical arguments clearly link narcissism and leadership, the question whether leader narcissism is good or bad for organizations and their members remains unanswered. Narcissism seems to have two sides, a bright and a dark one. This systematic literature review seeks to contribute to the ongoing academic discussion about the positive or negative impact of leader narcissism in organizations. Forty-five original research articles were categorized according to outcomes at three levels of analysis: the dyadic level (focusing on leader-follower relationships), the team level (focusing on work teams and small groups), and the organizational level. On this basis, we first summarized the current state of knowledge about the impact that leader narcissism has on outcomes at different levels of analysis. Next, we revealed similarities and contradictions between research findings within and across levels of analysis, highlighting persistent inconsistencies concerning the question whether leader narcissism has positive or negative consequences. Finally, we outlined theoretical and methodological implications for future studies of leader narcissism. This multi-level perspective ascertains a new, systematic view of leader narcissism and its consequences for organizations and their stakeholders. The article demonstrates the need for future research in the field of leader narcissism and opens up new avenues for inquiry.

Inference Based on the Best-Fitting Model can Contribute to the Replication Crisis: Assessing Model Selection Uncertainty Using a Bootstrap Approach

Inference and conclusions drawn from model fitting analyses are commonly based on a single "best-fitting" model. If model selection and inference are carried out using the same data model selection uncertainty is ignored. We illustrate the Type I error inflation that can result from using the same data for model selection and inference, and we then propose a simple bootstrap based approach to quantify model selection uncertainty in terms of model selection rates. A selection rate can be interpreted as an estimate of the replication probability of a fitted model. The benefits of bootstrapping model selection uncertainty is demonstrated in a growth mixture analyses of data from the National Longitudinal Study of Youth, and a 2-group measurement invariance analysis of the Holzinger-Swineford data.

PSYCHOLOGY. Estimating the reproducibility of psychological science

Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. We conducted replications of 100 experimental and correlational studies published in three psychology journals using high-powered designs and original materials when available. Replication effects were half the magnitude of original effects, representing a substantial decline. Ninety-seven percent of original studies had statistically significant results. Thirty-six percent of replications had statistically significant results; 47% of original effect sizes were in the 95% confidence interval of the replication effect size; 39% of effects were subjectively rated to have replicated the original result; and if no bias in original results is assumed, combining original and replication results left 68% with statistically significant effects. Correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.

Effectiveness of mission statements in organizations – A review

Even though mission statements are standard tools in organizations, their effectiveness is subject to substantial skepticism. This review integrates hitherto published research based on a broad range of objective and subjective effectiveness criteria above and beyond financial performance in for-profit as well as not-for-profit organizations. We conclude that the distal outcome effectiveness of mission statements depends on the following antecedents and intermediate outcomes: (1) the rationale underlying their development; (2) the process of their development and implementation; (3) their content and form; and (4) individual attitudes toward the mission statement. We thereby clarify preconditions of mission statement effectiveness in organizations, and reveal shortcomings in current research.

Effectiveness of mission statements in organizations – A review

Even though mission statements are standard tools in organizations, their effectiveness is subject to substantial skepticism. This review integrates hitherto published research based on a broad range of objective and subjective effectiveness criteria above and beyond financial performance in for-profit as well as not-for-profit organizations. We conclude that the distal outcome effectiveness of mission statements depends on the following antecedents and intermediate outcomes: (1) the rationale underlying their development; (2) the process of their development and implementation; (3) their content and form; and (4) individual attitudes toward the mission statement. We thereby clarify preconditions of mission statement effectiveness in organizations, and reveal shortcomings in current research.

The Decision Making Individual Differences Inventory and guidelines for the study of individual differences in judgment and decision-making research

Individual differences in decision making are a topic of longstanding interest, but often yield inconsistent and contradictory results. After providing an overview of individual difference measures that have commonly been used in judgment and decision-making (JDM) research, we suggest that our understanding of individual difference effects in JDM may be improved by amending our approach to studying them. We propose four recommendations for improving the pursuit of individual differences in JDM research: a more systematic approach; more theory-driven selection of measures; a reduced emphasis on main effects in favor of interactions between individual differences and decision features, situational factors, and other individual differences; and more extensive communication of results (whether significant or null, published or unpublished). As a first step, we offer our database—the Decision Making Individual Differences Inventory (DMIDI; http://html://www.sjdm.org/dmidi), a free, public resource that categorizes and describes the most common individual difference measures used in JDM research.

What's Wrong with Research Literatures? And how to Make Them Right

Meta-analysis is now the accepted procedure for summarizing research literatures in areas of applied psychology. Because of the bias for publishing statistically significant findings, while usually rejecting nonsignificant results, our research literatures yield misleading answers to important quantitative questions (e.g., How much better is the average psychotherapy patient relative to a comparable group of untreated controls? How much more aggressive are children who watch a great deal of violent TV than children who watch little or no violence on TV?). While all such research literatures provide overly optimistic meta-analytic estimates, exactly how practically important are these overestimates? Three studies testing the literature on implementation intentions finds only slightly elevated effectiveness estimates. Conversely, in three studies another growing research literature (the efficacy of remote intercessory prayer) is found to be misleading and is in all likelihood not a real effect (i.e., our three studies suggest the literature likely consists of Type I errors). Rules of thumb to predict which research literatures are likely invalid are offered. Finally, revised publication and data analysis procedures to generate unbiased research literatures in the future are examined.