Conversational assessment using artificial intelligence is as clinically useful as depression scales and preferred by users

BACKGROUND

Depression is prevalent, chronic, and burdensome. Due to limited screening access, depression often remains undiagnosed. Artificial intelligence (AI) models based on spoken responses to interview questions may offer an effective, efficient alternative to other screening methods.

OBJECTIVE

The primary aim was to use a demographically diverse sample to validate an AI model, previously trained on human-administered interviews, on novel bot-administered interviews, and to check for algorithmic biases related to age, sex, race, and ethnicity.

METHODS

Using the Aiberry app, adults recruited via social media (N = 393) completed a brief bot-administered interview and a depression self-report form. An AI model was used to predict form scores based on interview responses alone. For all meaningful discrepancies between model inference and form score, clinicians performed a masked review to determine which one they preferred.

RESULTS

There was strong concurrent validity between the model predictions and raw self-report scores (r = 0.73, MAE = 3.3). 90 % of AI predictions either agreed with self-report or with clinical expert opinion when AI contradicted self-report. There was no differential model performance across age, sex, race, or ethnicity.

LIMITATIONS

Limitations include access restrictions (English-speaking ability and access to smartphone or computer with broadband internet) and potential self-selection of participants more favorably predisposed toward AI technology.

CONCLUSION

The Aiberry model made accurate predictions of depression severity based on remotely collected spoken responses to a bot-administered interview. This study shows promising results for the use of AI as a mental health screening tool on par with self-report measures.

A guide for social science journal editors on easing into open science

Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).

How Can People Become Happier? A Systematic Review of Preregistered Experiments

Can happiness be reliably increased? Thousands of studies speak to this question. However, many of them were conducted during a period in which researchers commonly "p-hacked," creating uncertainty about how many discoveries might be false positives. To prevent p-hacking, happiness researchers increasingly preregister their studies, committing to analysis plans before analyzing data. We conducted a systematic literature search to identify preregistered experiments testing strategies for increasing happiness. We found surprisingly little support for many widely recommended strategies (e.g., performing random acts of kindness). However, our review suggests that other strategies-such as being more sociable-may reliably promote happiness. We also found strong evidence that governments and organizations can improve happiness by providing underprivileged individuals with financial support. We conclude that happiness research stands on the brink of an exciting new era, in which modern best practices will be applied to develop theoretically grounded strategies that can produce lasting gains in life satisfaction.

A comprehensive scan of psychological disciplines through self-identification on Google Scholar: Relative endorsement, topical coverage, and publication patterns

Psychological researchers often identify with psychological disciplines, such as social or clinical psychology. The current study analyzed Google Scholar profiles from 6,532 international scientists who attracted more than 100 citations in 2019 and self-identified with at least one of 10 common psychological disciplines (psychoanalysis; clinical psychology; (cognitive) neuroscience; developmental psychology; educational psychology; experimental psychology; biological psychology/psychophysiology; mathematical psychology/psychometrics; social psychology; personality psychology). Results indicated that almost half of all psychologists self-identified with either social psychology or cognitive neuroscience. There were 487 topics that were endorsed at least five times, ranging from highly discipline-specific topics to more integrative ones, such as emotion and personality. We also factor-analyzed frequencies of topical endorsement across disciplines and found two factors, which we interpreted as reflecting correlational and experimental research traditions (with social psychology being the largest discipline within the former tradition and cognitive neuroscience being the largest discipline within the latter tradition). Differences in productivity and impact were also found, with researchers identifying with psychometrics being the most productive and researchers identifying with personality psychology, cognitive neuroscience, and multidisciplinary psychology as the most impactful in terms of citation increases per additional output. Recommendations for promoting cross-fertilization across psychological disciplines are formulated.

Evaluation of Transparency and Openness Guidelines in Physical Therapist Journals

OBJECTIVE

The goals of this study were to evaluate the extent that physical therapist journals support open science research practices by adhering to the Transparency and Openness Promotion (TOP) guidelines and to assess the relationship between journal scores and their respective journal impact factor (JIF).

METHODS

Scimago, mapping studies, the National Library of Medicine, and journal author guidelines were searched to identify physical therapist journals for inclusion. Journals were graded on 10 standards (29 available total points) related to transparency with data, code, research materials, study design and analysis, preregistration of studies and statistical analyses, replication, and open science badges. The relationship between journal transparency and openness scores and their JIF was determined.

RESULTS

Thirty-five journals' author guidelines were assigned transparency and openness factor scores. The median score (interquartile range) across journals was 3.00 out of 29 (3.00) points (for all journals the scores ranged from 0 to 8). The 2 standards with the highest degree of implementation were design and analysis transparency (reporting guidelines) and study preregistration. No journals reported on code transparency, materials transparency, replication, and open science badges. TOP factor scores were a significant predictor of JIF scores.

CONCLUSION

There is low implementation of the TOP standards by physical therapist journals. TOP factor scores demonstrated predictive abilities for JIF scores. Policies from journals must improve to make open science practices the standard in research. Journals are in an influential position to guide practices that can improve the rigor of publication which, ultimately, enhances the evidence-based information used by physical therapists.

IMPACT

Transparent, open, and reproducible research will move the profession forward by improving the quality of research and increasing the confidence in results for implementation in clinical care.

Eleven strategies for making reproducible research and open science training the norm at research institutions

Reproducible research and open science practices have the potential to accelerate scientific progress by allowing others to reuse research outputs, and by promoting rigorous research that is more likely to yield trustworthy results. However, these practices are uncommon in many fields, so there is a clear need for training that helps and encourages researchers to integrate reproducible research and open science practices into their daily work. Here, we outline eleven strategies for making training in these practices the norm at research institutions. The strategies, which emerged from a virtual brainstorming event organized in collaboration with the German Reproducibility Network, are concentrated in three areas: (i) adapting research assessment criteria and program requirements; (ii) training; (iii) building communities. We provide a brief overview of each strategy, offer tips for implementation, and provide links to resources. We also highlight the importance of allocating resources and monitoring impact. Our goal is to encourage researchers - in their roles as scientists, supervisors, mentors, instructors, and members of curriculum, hiring or evaluation committees - to think creatively about the many ways they can promote reproducible research and open science practices in their institutions.

Open Science Standards at Journals that Inform Evidence-Based Policy

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

Research-Problem Validity in Primary Research: Precision and Transparency in Characterizing Past Knowledge

Four validity types evaluate the approximate truth of inferences communicated by primary research. However, current validity frameworks ignore the truthfulness of empirical inferences that are central to research-problem statements. Problem statements contrast a review of past research with other knowledge that extends, contradicts, or calls into question specific features of past research. Authors communicate empirical inferences, or quantitative judgments, about the frequency (e.g., "few," "most") and variability (e.g., "on the one hand," "on the other hand") in their reviews of existing theories, measures, samples, or results. We code a random sample of primary research articles and show that 83% of quantitative judgments in our sample are vague and do not have a transparent origin, making it difficult to assess their validity. We review validity threats of current practices. We propose that documenting the literature search, reporting how the search was coded, and quantifying the search results facilitates more precise judgments and makes their origin transparent. This practice enables research questions that are more closely tied to the existing body of knowledge and allows for more informed evaluations of the contribution of primary research articles, their design choices, and how they advance knowledge. We discuss potential limitations of our proposed framework.

Is Open Science Neoliberal?

The scientific-reform movement, frequently referred to as open science, has the potential to substantially reshape the nature of the scientific activity. For this reason, its sociopolitical antecedents and consequences deserve serious scholarly attention. In a recently formed literature that professes to meet this need, it has been widely argued that the movement is neoliberal. However, for two reasons it is hard to justify this widescale attribution: First, the critics mistakenly represent the movement as a monolithic structure, and second, the critics' arguments associating the movement with neoliberalism because of the movement's (a) preferential focus on methodological issues, (b) underlying philosophy of science, and (c) allegedly promarket ideological proclivities reflected in the methodology and science-policy proposals do not hold under closer scrutiny. These shortcomings show a lack of sufficient engagement with the reform literature. What is needed is more nuanced accounts of the sociopolitical underpinnings of scientific reform. To address this need, we propose a model for the analysis of reform proposals, which represents scientific methodology, axiology, science policy, and ideology as interconnected but relatively distinct domains, and thus allows for recognizing the divergent tendencies in the movement and the uniqueness of particular proposals.

1st ASCS: Expanding the ISCB Student Council Symposia to Asia

Since 2004, the ISCB Student Council (ISCB-SC) has successfully organized Student Council Symposia across several continents, including North America, Latin America, Europe, and Africa, as well as local events led by more than 25 Regional Student Groups (RSG) across the world. The ISCB-SC Symposia provide students and early career researchers the chance to showcase their work at an international venue in a format that includes keynote talks, round table discussions, workshops, and more. After several efforts spanning several years to build enough critical mass in the region, we have successfully organized the first Asian Student Council Symposium (1st ASCS). This article discusses the organizational details of this unprecedented event, the challenges faced, and the lessons learned.

Teaching open and reproducible scholarship: a critical review of the evidence base for current pedagogical methods and their outcomes

In recent years, the scientific community has called for improvements in the credibility, robustness and reproducibility of research, characterized by increased interest and promotion of open and transparent research practices. While progress has been positive, there is a lack of consideration about how this approach can be embedded into undergraduate and postgraduate research training. Specifically, a critical overview of the literature which investigates how integrating open and reproducible science may influence student outcomes is needed. In this paper, we provide the first critical review of literature surrounding the integration of open and reproducible scholarship into teaching and learning and its associated outcomes in students. Our review highlighted how embedding open and reproducible scholarship appears to be associated with (i) students' scientific literacies (i.e. students' understanding of open research, consumption of science and the development of transferable skills); (ii) student engagement (i.e. motivation and engagement with learning, collaboration and engagement in open research) and (iii) students' attitudes towards science (i.e. trust in science and confidence in research findings). However, our review also identified a need for more robust and rigorous methods within pedagogical research, including more interventional and experimental evaluations of teaching practice. We discuss implications for teaching and learning scholarship.

TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility

Lack of reproducibility of research results has become a major theme in recent years. As we emerge from the COVID-19 pandemic, economic pressures and exposed consequences of lack of societal trust in science make addressing reproducibility of urgent importance. TIER2 is a new international project funded by the European Commission under their Horizon Europe programme. Covering three broad research areas (social, life and computer sciences) and two cross-disciplinary stakeholder groups (research publishers and funders) to systematically investigate reproducibility across contexts, TIER2 will significantly boost knowledge on reproducibility, create tools, engage communities, implement interventions and policy across different contexts to increase re-use and overall quality of research results in the European Research Area and global R&I, and consequently increase trust, integrity and efficiency in research.

The Emotional Rewards of Prosocial Spending Are Robust and Replicable in Large Samples

Past studies show that spending money on other people— prosocial spending—increases a person’s happiness. However, foundational research on this topic was conducted prior to psychology’s credibility revolution (or “replication crisis”), so it is essential to ask whether the evidence supporting this claim is robust and replicable. Here, we consider all 15 published preregistered experiments on prosocial spending to evaluate whether there is causal evidence for the idea that spending money on other people promotes happiness. Although the evidence appears somewhat mixed, we argue that the emotional benefits of prosocial spending are robust and replicable in large samples. These benefits are particularly likely when people have some choice about whether or how to give and when they understand how their generosity makes a difference. This review provides renewed support for the idea that prosocial spending promotes happiness and offers a template for revisiting phenomena that were established prior to the credibility revolution.

Open science and public trust in science: Results from two studies

In two studies, we examined whether open science practices, such as making materials, data, and code of a study openly accessible, positively affect public trust in science. Furthermore, we investigated whether the potential trust-damaging effects of research being funded privately (e.g. by a commercial enterprise) may be buffered by such practices. After preregistering six hypotheses, we conducted a survey study (Study 1; N = 504) and an experimental study (Study 2; N = 588) in two German general population samples. In both studies, we found evidence for the positive effects of open science practices on trust, though it should be noted that in Study 2, results were more inconsistent. We did not however find evidence for the aforementioned buffering effect. We conclude that while open science practices may contribute to increasing trust in science, the importance of making use of open science practices visible should not be underestimated.

Reckoning With Our Crisis: An Agenda for the Field of Social and Personality Psychology

The replication crisis and credibility revolution in the 2010s brought a wave of doubts about the credibility of social and personality psychology. We argue that as a field, we must reckon with the concerns brought to light during this critical decade. How the field responds to this crisis will reveal our commitment to self-correction. If we do not take the steps necessary to address our problems and simply declare the crisis to be over or the problems to be fixed without evidence, we risk further undermining our credibility. To fully reckon with this crisis, we must empirically assess the state of the field to take stock of how credible our science actually is and whether it is improving. We propose an agenda for metascientific research, and we review approaches to empirically evaluate and track where we are as a field (e.g., analyzing the published literature, surveying researchers). We describe one such project (Surveying the Past and Present State of Published Studies in Social and Personality Psychology) underway in our research group. Empirical evidence about the state of our field is necessary if we are to take self-correction seriously and if we hope to avert future crises.

Journal N-Pact Factors From 2011 to 2019: Evaluating the Quality of Social/Personality Journals With Respect to Sample Size and Statistical Power

Scholars and institutions commonly use impact factors to evaluate the quality of empirical research. However, a number of findings published in journals with high impact factors have failed to replicate, suggesting that impact alone may not be an accurate indicator of quality. Fraley and Vazire proposed an alternative index, the N-pact factor, which indexes the median sample size of published studies, providing a narrow but relevant indicator of research quality. In the present research, we expand on the original report by examining the N-pact factor of social/personality-psychology journals between 2011 and 2019, incorporating additional journals and accounting for study design (i.e., between persons, repeated measures, and mixed). There was substantial variation in the sample sizes used in studies published in different journals. Journals that emphasized personality processes and individual differences had larger N-pact factors than journals that emphasized social-psychological processes. Moreover, N-pact factors were largely independent of traditional markers of impact. Although the majority of journals in 2011 published studies that were not well powered to detect an effect of ρ = .20, this situation had improved considerably by 2019. In 2019, eight of the nine journals we sampled published studies that were, on average, powered at 80% or higher to detect such an effect. After decades of unheeded warnings from methodologists about the dangers of small-sample designs, the field of social/personality psychology has begun to use larger samples. We hope the N-pact factor will be supplemented by other indices that can be used as alternatives to improve further the evaluation of research.

Do Open-Science Badges Increase Trust in Scientists Among Undergraduates, Scientists, and the Public?

In three experimental studies, we investigated whether badges for open-science practices have the potential to affect trust in scientists and topic-specific epistemic beliefs by student teachers (n = 270), social scientists (n = 250), or the public (n = 257), all of whom were at least 16 years old. Furthermore, we analyzed the moderating role of epistemic beliefs for badges and trust. Each participant was randomly assigned to two of three conditions: badges awarded, badges not awarded, and no badges (control). In all samples, our Bayesian analyses indicated that badges influence trust as expected, with one exception in the public sample: An additional positive effect of awarded badges (compared with no badges) was not supported. For students and scientists, we found evidence for the relation of badges and epistemic beliefs as well as epistemic beliefs and trust. Further, we found evidence for the absence of moderation by epistemic beliefs.

The Role of Mating-relevant Factors in the Perpetration of Digital Dating Abuse

Previous research has explored offline intimate partner violence from an evolutionary perspective, primarily focusing on the role of individual differences in perpetration and victimization. However, a current form of intimate partner violence is digital dating abuse, which involves abuse toward a romantic partner, occuring online through the use of electronic communication technology. This form of abuse differs from offline abuse, in that physical proximity is not required. Although research has focused on the effects digital dating abuse has on victims, little research has focused on the perpetration of digital dating abuse. This is important as research focused on perpetration can inform a wide range of initiatives geared toward understanding the factors which drive this behavior. Recent research has focused on evolutionary mating-relevant factors that drive the perpetration of digital dating abuse. Here, we extended and replicated previous work by reporting two studies (study 1, n = 114; study 2, n = 162) which explored the roles of mate value discrepancy, intrasexual competition, and relationship-contingent self-esteem in the perpetration of digital dating abuse. We found that mate value discrepancy (studies 1 and 2) and intrasexual competition (study 2) positively predicted the perpetration of digital dating abuse. To our knowledge, this article is the first to provide support that those who report high intrasexual competition, engage in greater levels of digital dating abuse, thus furthering theoretical advancements in this field by showing digital dating abuse is a mate retention tactic. Our findings further our understanding of online behavior in romantic relationships through an evolutionary psychological lens.

Shaping the Qualities, Values and Standards of Science. How Reporting Guidelines Improve the Transparency of Biomedical Research

What is scientific quality and how can it be achieved? Recent developments in clinical biomedicine gave prominence to transparency as a new core value for scientific research. Without transparency, other characteristics and values remain unknown. But how can abstract concepts and values be implemented in day-to-day scientific practices and what gets lost on the way? In order to answer this question, this study investigates the role of the PRISMA reporting guideline for writing systematic reviews and meta-analyses. By combining a document analysis and expert interviews with its developers, it attempts to bridge the gap between research practice and current modes of evaluation. Beside showing how the guideline was designed to be applicable and acceptable as a new standard, the analysis revealed crucial distinctions between transparency as an abstract quality goal and its practical implementation in the form of specifically formulated rules. Although PRISMA relies on transparency in order to be meaningful, it blurs the concept in order to circumvent some of its main disadvantages.

What does a personality science approach to post-traumatic growth reveal?

Recent research has advocated for the value of conceptualizing post-traumatic growth as positive personality change. However, most research continues to use both methodologically suspect assessment tools and unsupported theoretical assumptions. How can personality psychologists contribute to the pursuit of high-quality research on this topic? The current special issue, which grew out of the European Association for Personality Psychology (EAPP) expert meeting on “Integrating Post-Traumatic Growth and Personality Change”, held at the University of Nottingham, on September 16–17, 2019, includes 15 papers that (1) examine short-term change in multiple personality constructs after adversity, (2) highlight the limitations of only assessing post-traumatic growth with retrospective questionnaires, and (3) provide theoretical and methodological recommendations for the continued advancement of the study of post-traumatic growth through the lens of personality psychology. While these papers all address specific ways to advance the personality science of post-traumatic growth, collectively they highlight the many unanswered questions that future research should tackle.

Lessons learned from conducting a study of emotions and positive personality change in Syrian origin young adults who have recently resettled in the Netherlands

Post-traumatic growth is a compelling idea, yet extant research has often employed retrospective reports of change, rather than examining change over time. Research on samples of people that are traditionally seen as hard-to-reach are rare within personality psychology. In Karakter, we assessed a sample of Syrian origin young adults who recently resettled in the Netherlands (initial  N = 168) four times over a 13-month period to examine experiences of adversity, emotions, and positive personality change. Here, we provide a detailed narrative of the research process, beginning with a description of how we incorporated open science practices in Karakter. We then turn to a discussion of the changes, challenges, and opportunities we encountered in the research. In doing so, we discuss conceptual and methodological considerations when examining personality change. We close with suggestions for researchers who are interested in conducting similar studies with populations that are underrecruited in the future.

Credibility Beyond Replicability: Improving the Four Validities in Psychological Science

Psychological science’s “credibility revolution” has produced an explosion of metascientific work on improving research practices. Although much attention has been paid to replicability (reducing false positives), improving credibility depends on addressing a wide range of problems afflicting psychological science, beyond simply making psychology research more replicable. Here we focus on the “four validities” and highlight recent developments—many of which have been led by early-career researchers—aimed at improving these four validities in psychology research. We propose that the credibility revolution in psychology, which has its roots in replicability, can be harnessed to improve psychology’s validity more broadly.

That’s a Lot to Process! Pitfalls of Popular Path Models

Path models to test claims about mediation and moderation are a staple of psychology. But applied researchers may sometimes not understand the underlying causal inference problems and thus endorse conclusions that rest on unrealistic assumptions. In this article, we aim to provide a clear explanation for the limited conditions under which standard procedures for mediation and moderation analysis can succeed. We discuss why reversing arrows or comparing model fit indices cannot tell us which model is the right one and how tests of conditional independence can at least tell us where our model goes wrong. Causal modeling practices in psychology are far from optimal but may be kept alive by domain norms that demand every article makes some novel claim about processes and boundary conditions. We end with a vision for a different research culture in which causal inference is pursued in a much slower, more deliberate, and collaborative manner.

Personality Assessment in Legal Contexts: Introduction to the Special Issue

This special issue addresses a major gap in the literature by providing comprehensive, credible reviews of the psychometric evidence for and legal status of some of the most commonly-used psychological and personality assessment measures used in forensic evaluations. It responds to Neal and colleagues' (2019) call for research to improve the state of and access to knowledge about psychological assessments in legal contexts, and encourages critical thinking about forensic assessment in the spirit of improvement. These articles offer clarity about the strengths and weaknesses of a number of assessment instruments to inform psychologists' preparation for expert testimony, lawyers' preparation for direct and cross-examination, judges' evidence admissibility determinations, and scholars' future research. We assembled teams of authors with different perspectives and areas of expertise to review each tool fairly, including several adversarial collaborations. Articles on the Rorschach and R-PAS, MMPI-3, PCL-R, MCMI-IV and MACI-II, PAI and PAI-A, SIRS-2, HCR-20^V3, TSI and TSI-2, and the MacCAT-CA, ECST-R, and CAST*MR are included. To increase visibility, accessibility, and impact, this issue is published as free access, meaning the articles are available to download without charge. We anticipate these articles will be widely read and useful to scholars and practitioners in both psychology and law.

A systematic review of the flood vulnerability using geographic information system

The world has faced many disasters in recent years, but flood impacts have gained immense importance and attention due to their adverse effects. More than half of global flood destruction and damages occur in the Asia region, which causes losses of life, damage infrastructure, and creates panic conditions among the communities. To provide a better understanding of flood hazard management, flood vulnerability assessment is the primary objective. In this case, vulnerability is the central construct in flood analysis and assessment. Many researchers have defined different approaches and methods to understand vulnerability assessment and how geographic information systems assess the flood vulnerability and their associated risk. Geographic information systems track and predict the disaster trend and mitigate the risk and damages. This study systematically reviews the methodologies used to measure floods and their vulnerabilities by integrating geographic information system. Articles on flood vulnerability from 2010 to 2020 were selected and reviewed. Through the systematic review methodology of five research engines, the researchers discovered a difference in flood vulnerability assessment tools and techniques that can be bridged by integrating high-resolution data with a multidimensional vulnerability methodology. The study reviewed several vulnerability components and directly examined the shortcomings in flood vulnerability approaches at different levels. The research contributed that the indicator-based approach gives a better understanding of vulnerability assessment. The geographic information system provides an effective environment for mapping and precise analysis to mitigate the flood disaster.

Towards a culture of open scholarship: the role of pedagogical communities

The UK House of Commons Science and Technology Committee has called for evidence on the roles that different stakeholders play in reproducibility and research integrity. Of central priority are proposals for improving research integrity and quality, as well as guidance and support for researchers. In response to this, we argue that there is one important component of research integrity that is often absent from discussion: the pedagogical consequences of how we teach, mentor, and supervise students through open scholarship. We justify the need to integrate open scholarship principles into research training within higher education and argue that pedagogical communities play a key role in fostering an inclusive culture of open scholarship. We illustrate these benefits by presenting the Framework for Open and Reproducible Research Training (FORRT), an international grassroots community whose goal is to provide support, resources, visibility, and advocacy for the adoption of principled, open teaching and mentoring practices, whilst generating conversations about the ethics and social impact of higher-education pedagogy. Representing a diverse group of early-career researchers and students across specialisms, we advocate for greater recognition of and support for pedagogical communities, and encourage all research stakeholders to engage with these communities to enable long-term, sustainable change.

Transparent, Open, and Reproducible Prevention Science

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Coordinated data analysis: Knowledge accumulation in lifespan developmental psychology

Coordinated analysis is a powerful form of integrative analysis, and is well suited in its capacity to promote cumulative scientific knowledge, particularly in subfields of psychology that focus on the processes of lifespan development and aging. Coordinated analysis uses raw data from individual studies to create similar hypothesis tests for a given research question across multiple datasets, thereby making it less vulnerable to common criticisms of meta-analysis such as file drawer effects or publication bias. Coordinated analysis can sometimes use random effects meta-analysis to summarize results, which does not assume a single true effect size for a given statistical test. By fitting parallel models in separate datasets, coordinated analysis preserves the heterogeneity among studies, and provides a window into the generalizability and external validity of a set of results. The current article achieves three goals: First, it describes the phases of a coordinated analysis so that interested researchers can more easily adopt these methods in their labs. Second, it discusses the importance of coordinated analysis within the context of the credibility revolution in psychology. Third, it encourages the use of existing data networks and repositories for conducting coordinated analysis, in order to enhance accessibility and inclusivity. Subfields of research that require time- or resource- intensive data collection, such as longitudinal aging research, would benefit by adopting these methods. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

An examination of job embeddedness and organizational commitment in the context of HRD practices

Purpose

This paper aims to apply the theoretical perspective of job embeddedness to delineate how organizations could bundle and implement specific HRD practices that cater to fit, connections and the psychological costs of leaving to influence employees’ organizational commitment.

Design/methodology/approach

Using a dual-study approach, the current research uses survey responses collected from two samples of working adults to test the theorized framework using structural equation modelling.

Findings

Replicated results reveal that on-the-job embeddedness predicts affective commitment. There was no association between embeddedness at the community level and organizational commitment in either study.

Originality/value

This research offers a fresh perspective to explore the direct influence that embeddedness has on organizational commitment in the context of HRD practices.

Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014-2017)

Psychologists are navigating an unprecedented period of introspection about the credibility and utility of their discipline. Reform initiatives emphasize the benefits of transparency and reproducibility-related research practices; however, adoption across the psychology literature is unknown. Estimating the prevalence of such practices will help to gauge the collective impact of reform initiatives, track progress over time, and calibrate future efforts. To this end, we manually examined a random sample of 250 psychology articles published between 2014 and 2017. Over half of the articles were publicly available (154/237, 65%, 95% confidence interval [CI] = [59%, 71%]); however, sharing of research materials (26/183; 14%, 95% CI = [10%, 19%]), study protocols (0/188; 0%, 95% CI = [0%, 1%]), raw data (4/188; 2%, 95% CI = [1%, 4%]), and analysis scripts (1/188; 1%, 95% CI = [0%, 1%]) was rare. Preregistration was also uncommon (5/188; 3%, 95% CI = [1%, 5%]). Many articles included a funding disclosure statement (142/228; 62%, 95% CI = [56%, 69%]), but conflict-of-interest statements were less common (88/228; 39%, 95% CI = [32%, 45%]). Replication studies were rare (10/188; 5%, 95% CI = [3%, 8%]), and few studies were included in systematic reviews (21/183; 11%, 95% CI = [8%, 16%]) or meta-analyses (12/183; 7%, 95% CI = [4%, 10%]). Overall, the results suggest that transparency and reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility of psychology research can be compared.

When COVID-19 Is the Invader and Internal Communication Is the Hero: Understanding the Influence of Internal Communication on Individual Performance and Evaluating the Mediating Role of Perceived Support

Objective: The main objective of this study was to evaluate the impact of internal communication (IC) of organisations on the quality of the individual–organisation relationship, specifically with regard to the effects generated with respect to individual performance (IP) in a pandemic framework. In this sense, the study intends to evaluate the effects of IC on the employee’s IP, having as a mediator perceived organisational support (POS). Methodology: To achieve the aforementioned objectives, a cross-sectional quantitative study was prepared, data for which were collected during a period of confinement that took place between 9 February and 15 March 2021. A total of 340 individuals of both sexes participated in the study. Genders were 67.6% female and 32.4% male, with ages ranging from 25 years to over 61 years, from all districts of Portugal and the Islands. Results: The main results obtained showed that IC was positively and significantly correlated with POS and also with IP, and that there was also a total mediation effect of POS in the relationship between IC and IP. Practical implications: These results seem to support the need for organisations to invest in their internal communication practices as a way of stimulating strong and fruitful relationships between workers and the organisation. Internal communication seems to be a relevant indicator for the management of proximity relationships with workers, especially in adverse contexts, like the ones experienced during the pandemic crisis. Well-developed internal communication supports and practices seem to be a valid path towards developing bonds leading to improved performance.

The past, present and future of Registered Reports

Registered Reports are a form of empirical publication in which study proposals are peer reviewed and pre-accepted before research is undertaken. By deciding which articles are published based on the question, theory and methods, Registered Reports offer a remedy for a range of reporting and publication biases. Here, we reflect on the history, progress and future prospects of the Registered Reports initiative and offer practical guidance for authors, reviewers and editors. We review early evidence that Registered Reports are working as intended, while at the same time acknowledging that they are not a universal solution for irreproducibility. We also consider how the policies and practices surrounding Registered Reports are changing, or must change in the future, to address limitations and adapt to new challenges. We conclude that Registered Reports are promoting reproducibility, transparency and self-correction across disciplines and may help reshape how society evaluates research and researchers.

Assessing open science and citizen science in addictions and substance use research: A scoping review

BACKGROUND

The EU promotes 'Open Science' as a public good. Complementary to its implementation is Citizen Science, which redefines the relationship between the scientific community, civic society and the individual. Open Science and Citizen Science poses challenges for the substance use and addictions research community but may provide positive opportunities for future European addiction research. This paper explores both current barriers and potential facilitators for the implementation of Open Science and Citizen Science in substance use and addictions research.

METHODOLOGY

A scoping review was used to examine barriers and facilitators identified in the substance use and addiction research literature for the adoption of Open Science and Citizen Science.

RESULTS

'Technical' facilitators included the pre-registration of study protocols; publication of open-source datasets; open peer review and online tools. 'Motivational' facilitators included enhanced reputation; embracing co-creation; engaged citizenship and gamification. 'Economic' facilitators included the use of free tools and balanced remuneration of crowdworkers. 'Political' facilitators included better informed debates through the 'triple helix' approach and trust-generating transparency. 'Legal' facilitators included epidemiologically informed law enforcement; better policy surveillance and the validation of other datasets. 'Ethical' facilitators included the 'democratisation of science' and opportunities to explore new concepts of ethics in addiction research.

CONCLUSION

Open Science and Citizen Science in substance use and addictions research may provide a range of benefits in relation to the democratisation of science; transparency; efficiency and the reliability/validity of data. However, its implementation raises a range of research integrity and ethical issues that need be considered. These include issues related to participant recruitment; privacy; confidentiality; security; cost and industry involvement. Progressive journal policies to support Open Science practices; a shift in researcher norms; the use of free tools and the greater availability of methodological and ethical standards are likely to increase adoption in the field.

The Role of Human Fallibility in Psychological Research: A Survey of Mistakes in Data Management

Errors are an inevitable consequence of human fallibility, and researchers are no exception. Most researchers can recall major frustrations or serious time delays due to human errors while collecting, analyzing, or reporting data. The present study is an exploration of mistakes made during the data-management process in psychological research. We surveyed 488 researchers regarding the type, frequency, seriousness, and outcome of mistakes that have occurred in their research team during the last 5 years. The majority of respondents suggested that mistakes occurred with very low or low frequency. Most respondents reported that the most frequent mistakes led to insignificant or minor consequences, such as time loss or frustration. The most serious mistakes caused insignificant or minor consequences for about a third of respondents, moderate consequences for almost half of respondents, and major or extreme consequences for about one fifth of respondents. The most frequently reported types of mistakes were ambiguous naming/defining of data, version control error, and wrong data processing/analysis. Most mistakes were reportedly due to poor project preparation or management and/or personal difficulties (physical or cognitive constraints). With these initial exploratory findings, we do not aim to provide a description representative for psychological scientists but, rather, to lay the groundwork for a systematic investigation of human fallibility in research data management and the development of solutions to reduce errors and mitigate their impact.

Replicability, Robustness, and Reproducibility in Psychological Science

Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress. Expected final online publication date for the Annual Review of Psychology, Volume 73 is January 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Estimating the effects of trait knowledge on social perception

Research in social cognition has predominantly investigated perceptual and inferential processes separately; however, real-world social interactions usually involve integration between person inferences (e.g., generous, selfish) and the perception of physical appearance (e.g., thin, tall). Therefore, in the current work, we investigated the integration of different person-relevant signals, by estimating the extent to which bias in one social information processing system influences another. Following an initial stimulus-validation experiment (Experiment 1, N = 55), two further pre-registered experiments (Experiments 2, N = 55 and 3; N = 123) employed a priming paradigm to measure the effects of extraversion-diagnostic information on subsequent health and body-size judgements of a target body. The results were consistent across both priming experiments and supported our predictions: compared to trait-neutral control statements, extraversion-diagnostic statements increased judgements of health and decreased those of body size. As such, we show that trait-based knowledge does not only influence mappings towards similar types of person judgements, such as health judgements. Rather, even a brief re-configuration of trait-space alters mappings towards non-trait judgements, which are based on body size and shape. The results complement prior neuroimaging findings that showed functional interactions between the body-selective brain regions in the ventral visual stream and the theory of mind network when forming impressions of others. Therefore, we provide a functional signature of how distinct information processing units exchange signals and integrate information to form impressions. Overall, the current study underscores the value of behavioural work in complementing neuroscience research when investigating the role and properties of functional integration during impression formation. In addition, it stresses the potential limitations of an over-reliance on studying separate systems in isolation.

Functional specificity and neural integration in the aesthetic appreciation of artworks with implied motion

Although there is growing interest in the neural foundations of aesthetic experience, it remains unclear how particular mental subsystems (e.g. perceptual, affective and cognitive) are involved in different types of aesthetic judgements. Here, we use fMRI to investigate the involvement of different neural networks during aesthetic judgements of visual artworks with implied motion cues. First, a behavioural experiment (N = 45) confirmed a preference for paintings with implied motion over static cues. Subsequently, in a preregistered fMRI experiment (N = 27), participants made aesthetic and motion judgements towards paintings representing human bodies in dynamic and static postures. Using functional region-of-interest and Bayesian multilevel modelling approaches, we provide no compelling evidence for unique sensitivity within or between neural systems associated with body perception, motion and affective processing during the aesthetic evaluation of paintings with implied motion. However, we show suggestive evidence that motion and body-selective systems may integrate signals via functional connections with a separate neural network in dorsal parietal cortex, which may act as a relay or integration site. Our findings clarify the roles of basic visual and affective brain circuitry in evaluating a central aesthetic feature-implied motion-while also pointing towards promising future research directions, which involve modelling aesthetic preferences as hierarchical interplay between visual and affective circuits and integration processes in frontoparietal cortex.

Estimating the reproducibility of social learning research published between 1955 and 2018

Reproducibility is integral to science, but difficult to achieve. Previous research has quantified low rates of data availability and results reproducibility across the biological and behavioural sciences. Here, we surveyed 560 empirical publications, published between 1955 and 2018 in the social learning literature, a research topic that spans animal behaviour, behavioural ecology, cultural evolution and evolutionary psychology. Data were recoverable online or through direct data requests for 30% of this sample. Data recovery declines exponentially with time since publication, halving every 6 years, and up to every 9 years for human experimental data. When data for a publication can be recovered, we estimate a high probability of subsequent data usability (87%), analytical clarity (97%) and agreement of published results with reproduced findings (96%). This corresponds to an overall rate of recovering data and reproducing results of 23%, largely driven by the unavailability or incompleteness of data. We thus outline clear measures to improve the reproducibility of research on the ecology and evolution of social behaviour.

Prosociality in science

Science is unthinkable without collaboration between scientists. Yet, science is also unthinkable without competition (i.e., competing for the best and most solid arguments and limited, precious resources). In this review, we argue that scientific work routines represent social dilemmas and that two facets of prosociality help researchers solve these dilemmas: (i) sacrificing personal profit for the sake of collective profit (i.e., cooperation) and (ii) deciding to make oneself vulnerable to exploitation (i.e., trust). We use two contemporary developments in science to illustrate our reasoning: First, researchers' willingness to engage with the lay public (e.g., investing one's limited time to public engagement) and second, their commitment to 'open science' (e.g., sharing one's data and materials despite the risk of exploitation).

Behavioural science is unlikely to change the world without a heterogeneity revolution

In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.