1
|
Silverstein P, Elman C, Montoya A, McGillivray B, Pennington CR, Harrison CH, Steltenpohl CN, Röer JP, Corker KS, Charron LM, Elsherif M, Malicki M, Hayes-Harb R, Grinschgl S, Neal T, Evans TR, Karhulahti VM, Krenzer WLD, Belaus A, Moreau D, Burin DI, Chin E, Plomp E, Mayo-Wilson E, Lyle J, Adler JM, Bottesini JG, Lawson KM, Schmidt K, Reneau K, Vilhuber L, Waltman L, Gernsbacher MA, Plonski PE, Ghai S, Grant S, Christian TM, Ngiam W, Syed M. A guide for social science journal editors on easing into open science. Res Integr Peer Rev 2024; 9:2. [PMID: 38360805 PMCID: PMC10870631 DOI: 10.1186/s41073-023-00141-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 12/28/2023] [Indexed: 02/17/2024] Open
Abstract
Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).
Collapse
Affiliation(s)
- Priya Silverstein
- Department of Psychology, Ashland University, Ashland, USA.
- Institute for Globally Distributed Open Research and Education, Preston, UK.
| | - Colin Elman
- Maxwell School of Citizenship and Public Affairs, Syracuse University, Syracuse, USA
| | - Amanda Montoya
- Department of Psychology, University of California, Los Angeles, USA
| | | | - Charlotte R Pennington
- School of Psychology, College of Health & Life Sciences, Aston University, Birmingham, UK
| | | | | | - Jan Philipp Röer
- Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Germany
| | | | - Lisa M Charron
- American Family Insurance Data Science Institute, University of Wisconsin-Madison, Madison, USA
- Nelson Institute for Environmental Studies, University of Wisconsin-Madison, Madison, USA
| | - Mahmoud Elsherif
- Department of Psychology, University of Birmingham, Birmingham, UK
| | - Mario Malicki
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, USA
- Stanford Program On Research Rigor and Reproducibility, Stanford University, Stanford, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, USA
| | | | | | - Tess Neal
- Department of Psychology, Iowa State University, Ames, USA
- School of Social & Behavioral Sciences, Arizona State University, Tempe, USA
| | - Thomas Rhys Evans
- School of Human Sciences and Institute for Lifecourse Development, University of Greenwich, London, UK
| | - Veli-Matti Karhulahti
- Department of Music, Art and Culture Studies, University of Jyväskylä, Jyväskylä, Finland
| | | | - Anabel Belaus
- National Agency for Scientific and Technological Promotion, Córdoba, Argentina
| | - David Moreau
- School of Psychology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| | - Debora I Burin
- Facultad de Psicología, Universidad de Buenos Aires, Buenos Aires, Argentina
- CONICET, Buenos Aires, Argentina
| | | | - Esther Plomp
- Faculty of Applied Sciences, Delft University of Technology, Delft, Netherlands
- The, The Alan Turing Institute, Turing Way, London, UK
| | - Evan Mayo-Wilson
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, USA
| | - Jared Lyle
- Inter-University Consortium for Political and Social Research (ICPSR), University of Michigan, Ann Arbor, USA
| | | | - Julia G Bottesini
- Maxwell School of Citizenship and Public Affairs, Syracuse University, Syracuse, USA
| | | | | | - Kyrani Reneau
- Inter-University Consortium for Political and Social Research (ICPSR), University of Michigan, Ann Arbor, USA
| | - Lars Vilhuber
- Economics Department, Cornell University, Ithaca, USA
| | - Ludo Waltman
- Centre for Science and Technology Studies, Leiden University, Leiden, Netherlands
| | | | - Paul E Plonski
- Department of Psychology, Tufts University, Medford, USA
| | - Sakshi Ghai
- Department of Psychology, University of Cambridge, Cambridge, USA
| | - Sean Grant
- HEDCO Institute for Evidence-Based Practice, College of Education, University of Oregon, Eugene, USA
| | - Thu-Mai Christian
- Odum Institute for Research in Social Science, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - William Ngiam
- Institute of Mind and Biology, University of Chicago, Chicago, USA
- Department of Psychology, University of Chicago, Chicago, USA
| | - Moin Syed
- Department of Psychology, University of Minnesota, Minneapolis, USA
| |
Collapse
|
2
|
Elmer T. Computational social science is growing up: why puberty consists of embracing measurement validation, theory development, and open science practices. EPJ DATA SCIENCE 2023; 12:58. [PMID: 38098785 PMCID: PMC10716103 DOI: 10.1140/epjds/s13688-023-00434-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Accepted: 11/30/2023] [Indexed: 12/17/2023]
Abstract
Puberty is a phase in which individuals often test the boundaries of themselves and surrounding others and further define their identity - and thus their uniqueness compared to other individuals. Similarly, as Computational Social Science (CSS) grows up, it must strike a balance between its own practices and those of neighboring disciplines to achieve scientific rigor and refine its identity. However, there are certain areas within CSS that are reluctant to adopt rigorous scientific practices from other fields, which can be observed through an overreliance on passively collected data (e.g., through digital traces, wearables) without questioning the validity of such data. This paper argues that CSS should embrace the potential of combining both passive and active measurement practices to capitalize on the strengths of each approach, including objectivity and psychological quality. Additionally, the paper suggests that CSS would benefit from integrating practices and knowledge from other established disciplines, such as measurement validation, theoretical embedding, and open science practices. Based on this argument, the paper provides ten recommendations for CSS to mature as an interdisciplinary field of research.
Collapse
Affiliation(s)
- Timon Elmer
- Department of Psychology, Applied Social and Health Psychology, University of Zurich, Binzmühlestrasse 14/14, 8050 Zurich, Switzerland
| |
Collapse
|
3
|
Liu B, Wei L. Unintended effects of open data policy in online behavioral research: An experimental investigation of participants’ privacy concerns and research validity. COMPUTERS IN HUMAN BEHAVIOR 2023. [DOI: 10.1016/j.chb.2022.107537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
4
|
Jeng W, He D. Surveying research data-sharing practices in US social sciences: a knowledge infrastructure-inspired conceptual framework. ONLINE INFORMATION REVIEW 2022. [DOI: 10.1108/oir-03-2020-0079] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
PurposeThis study develops a conceptual framework and a series of instruments for capturing researchers' data-sharing practices in the social sciences, by synergizing the theory of knowledge infrastructure and the theory of remote scientific collaboration.Design/methodology/approachThis paper triangulates the results of three studies of data sharing across the social sciences, with 144 participants in total, and classifies the confusion, “frictions” and opportunities arising from such sharing into four overarching dimensions: data characteristics, technological infrastructure, research culture and individual drivers.FindingsBased on the sample, the findings suggest that the majority of faculty and students in social science research do not share their data because many of them are unaware of the benefits and methods of doing so. Additional findings regarding social scientists' data-sharing behaviors include: (1) those who do share qualitative data in data repositories are more likely to share their research tools than their raw data; and (2) perceived technical support and extrinsic motivation are both strong predictors of qualitative data sharing (a previously underresearched subtype of social science data sharing).Originality/valueThe study confirms the previously hypothesized nature of “friction” in qualitative data sharing in the social sciences, arising chiefly from the time and labor intensiveness of ensuring data privacy.Peer reviewThe peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-03-2020-0079.
Collapse
|
5
|
Reitz AK, Weidmann R, Wünsche J, Bühler JL, Burriss RP, Grob A. In good times and in bad: A longitudinal analysis of the impact of bereavement on self-esteem and life satisfaction in couples. EUROPEAN JOURNAL OF PERSONALITY 2022. [DOI: 10.1177/08902070211054896] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
This study examines the impact of bereavement on self-esteem and life satisfaction in both partners of a romantic couple. We investigate the moderating effects of the type of the lost relationship (close family, close friends/others) and romantic relationship characteristics (daily social support, responsiveness-closeness, self-disclosure). We examined 1238 individuals in 619 male–female couples from the ages 18 to 81 ( M [ SD] = 31.97 years [13.26]). Both partners completed questionnaires at two assessments that were 20 months ( SD = 2.02 months) apart, in between which n = 216 individuals were bereaved. Actor–partner interdependence models showed that bereavement did not predict later self-esteem or life satisfaction in either of the partners. The relationship characteristics and the type of lost relationship did not moderate the effects. The subjective meaning and distress of the loss predicted later self-esteem and life satisfaction. The self-esteem increase was larger for bereaved with a positive/neutral than for bereaved with a negative meaning of the bereavement. We found a partner effect on self-esteem for the group of bereaved who reported a negative meaning of the bereavement. The findings demonstrate that bereavement can impact romantic partners' self-esteem and that the subjective experience of bereavement helps understand individual differences in the effect of bereavement on self-esteem and life satisfaction.
Collapse
Affiliation(s)
| | - Rebekka Weidmann
- Michigan State University, East Lansing, MI, USA
- University of Basel, Basel, Switzerland
| | | | - Janina L. Bühler
- University of Basel, Basel, Switzerland
- Heidelberg University, Heidelberg, Germany
| | | | | |
Collapse
|
6
|
Towse AS, Ellis DA, Towse JN. Making data meaningful: guidelines for good quality open data. THE JOURNAL OF SOCIAL PSYCHOLOGY 2021; 161:395-402. [PMID: 34292132 DOI: 10.1080/00224545.2021.1938811] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
7
|
Baker ZG, Gentzis EA, Watlington EM, Castejon S, Petit WE, Britton M, Haddad S, DiBello AM, Rodriguez LM, Derrick JL, Knee CR. Reflections on a registered report replicating a body of dyadic cross-sectional research. PERSONAL RELATIONSHIPS 2020; 27:907-938. [PMID: 36419735 PMCID: PMC9681012 DOI: 10.1111/pere.12343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
This article reflects on a new kind of registered report (RR) that replicated the work of an early career researcher. The research items targeted in this RR were peer-reviewed, cross-sectional, dyadic studies to which the first author of this RR had contributed. The findings being replicated are not noteworthy for their prestige or representativeness of the wider field. Instead, this method of replication may have several benefits and less desirable qualities for the researcher and research team whose work is being replicated, for science more broadly, and for relationship science specifically, reviewed herein. The authors hope that this reflection inspires researchers to improve upon their methodology by incorporating replication of their work early and often into their own research process.
Collapse
Affiliation(s)
| | | | | | | | | | - Maggie Britton
- Department of Psychology, University of Houston, Houston, Texas
| | - Sana Haddad
- Department of Psychology, University of Houston, Houston, Texas
| | - Angelo M. DiBello
- Department of Psychology, City University of New York, Brooklyn College, Brooklyn, New York
- Center for Alcohol and Addiction Studies, Department of Behavioral and Social Sciences, Brown University, Providence, Rhode Island
| | - Lindsey M. Rodriguez
- Department of Psychology, University of South Florida St Petersburg, St. Petersburg, Florida
| | - Jaye L. Derrick
- Department of Psychology, University of Houston, Houston, Texas
| | - C. Raymond Knee
- Department of Psychology, University of Houston, Houston, Texas
| |
Collapse
|
8
|
Spielmann SS, Gere J, Cantarella IA, Roberson J, Brindley SL. Relationship quality and accuracy of detecting a romantic partner's fear of being single. JOURNAL OF RESEARCH IN PERSONALITY 2020. [DOI: 10.1016/j.jrp.2020.103967] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
9
|
Quantity Over Quality? Reproducible Psychological Science from a Mixed Methods Perspective. COLLABRA: PSYCHOLOGY 2020. [DOI: 10.1525/collabra.284] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
A robust dialogue about the (un)reliability of psychological science findings has emerged in recent years. In response, metascience researchers have developed innovative tools to increase rigor, transparency, and reproducibility, stimulating rapid improvement and adoption of open science practices. However, existing reproducibility guidelines are geared toward purely quantitative study designs. This leaves some ambiguity as to how such guidelines should be implemented in mixed methods (MM) studies, which combine quantitative and qualitative research. Drawing on extant literature, our own experiences, and feedback from 79 self-identified MM researchers, the current paper addresses two main questions: (a) how and to what extent do existing reproducibility guidelines apply to MM study designs; and (b) can existing reproducibility guidelines be improved by incorporating best practices from qualitative research and epistemology? In answer, we offer 10 key recommendations for use within and outside of MM research. Finally, we argue that good science and good ethical practice are mutually reinforcing and lead to meaningful, credible science.
Collapse
|
10
|
Campbell R, Goodman-Williams R, Javorka M. A Trauma-Informed Approach to Sexual Violence Research Ethics and Open Science. JOURNAL OF INTERPERSONAL VIOLENCE 2019; 34:4765-4793. [PMID: 31514606 DOI: 10.1177/0886260519871530] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The practice of ethics in social science research is a reflexive process of self-review to define a profession's collective responsibility in the face of changing norms and expectations. In recent years, we have seen transformative changes in how society thinks about supporting sexual assault survivors, and how the scientific community thinks about our obligations to society. Decades of research on trauma and its impact has raised awareness about the needs of victimized individuals, giving rise to the trauma-informed practice movement, which emphasizes that service providers must center survivors' well-being in all interactions, decisions, and program practices. The field of sexual assault research helped give rise to this movement and provides empirical support for its guiding tenets, and in this article, we explore how to bring these ideas full circle to begin articulating trauma-informed principles for research. A trauma-informed perspective on research challenges scientists to go beyond the requirements of the Belmont Report (1979) and institutional review boards' (IRB) regulations to develop research procedures that fully support survivors' choice, control, and empowerment. Such reflection on participants' rights is particularly important given the open science movement sweeping academia, which calls on scientists to share their data publicly to promote transparency, replication, and new discoveries. Disseminating data could pose significant safety, privacy, and confidentiality risks for victims of sexual assault, so we need to evaluate what open science means within a trauma-informed framework. In this article, we examine three key stages of the research process-participant recruitment, data collection, and dissemination-and consider how trauma-informed principles could help, but also could complicate, research practices. We explore these tensions and offer potential solutions so that research on sexual trauma embodies trauma-informed practice.
Collapse
|
11
|
Jekel M, Fiedler S, Allstadt Torras R, Mischkowski D, Dorrough AR, Glöckner A. How to Teach Open Science Principles in the Undergraduate Curriculum—The Hagen Cumulative Science Project. PSYCHOLOGY LEARNING AND TEACHING-PLAT 2019. [DOI: 10.1177/1475725719868149] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
The Hagen Cumulative Science Project is a large-scale replication project based on students’ thesis work. In the project, we aim to (a) teach students to conduct the entire research process for conducting a replication according to open science standards and (b) contribute to cumulative science by increasing the number of direct replications. We describe the procedural steps of the project from choosing suitable replication studies to guiding students through the process of conducting a replication, and processing results in a meta-analysis. Based on the experience of more than 80 replications, we summarize how such a project can be implemented. We present practical solutions that have been shown to be successful as well as discuss typical obstacles and how they can be solved. We argue that replication projects are beneficial for all groups involved: Students benefit by being guided through a highly structured protocol and making actual contributions to science. Instructors benefit by using time resources effectively for cumulative science and fulfilling teaching obligations in a meaningful way. The scientific community benefits from the resulting greater number of replications and teaching state-of-the-art methodology. We encourage the use of student thesis-based replication projects for thesis work in academic bachelor and master curricula.
Collapse
Affiliation(s)
| | | | | | | | | | - Andreas Glöckner
- University of Cologne, Germany
- Max Planck Institute for Collective Goods, Germany
| |
Collapse
|
12
|
Tackett JL, Brandes CM, King KM, Markon KE. Psychology's Replication Crisis and Clinical Psychological Science. Annu Rev Clin Psychol 2019; 15:579-604. [PMID: 30673512 DOI: 10.1146/annurev-clinpsy-050718-095710] [Citation(s) in RCA: 61] [Impact Index Per Article: 12.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Despite psychological scientists' increasing interest in replicability, open science, research transparency, and the improvement of methods and practices, the clinical psychology community has been slow to engage. This has been shifting more recently, and with this review, we hope to facilitate this emerging dialogue. We begin by examining some potential areas of weakness in clinical psychology in terms of methods, practices, and evidentiary base. We then discuss a select overview of solutions, tools, and current concerns of the reform movement from a clinical psychological science perspective. We examine areas of clinical science expertise (e.g., implementation science) that should be leveraged to inform open science and reform efforts. Finally, we reiterate the call to clinical psychologists to increase their efforts toward reform that can further improve the credibility of clinical psychological science.
Collapse
Affiliation(s)
- Jennifer L Tackett
- Department of Psychology, Northwestern University, Evanston, Illinois 60208, USA;
| | - Cassandra M Brandes
- Department of Psychology, Northwestern University, Evanston, Illinois 60208, USA;
| | - Kevin M King
- Department of Psychology, University of Washington, Seattle, Washington 98195, USA
| | - Kristian E Markon
- Department of Psychological and Brain Sciences, University of Iowa, Iowa City, Iowa 52242, USA
| |
Collapse
|