1
|
Himanen L, Conte E, Gauffriau M, Strøm T, Wolf B, Gadd E. The SCOPE framework - implementing ideals of responsible research assessment. F1000Res 2024; 12:1241. [PMID: 38813348 PMCID: PMC11134161 DOI: 10.12688/f1000research.140810.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/08/2024] [Indexed: 05/31/2024] Open
Abstract
Background Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated. Methods SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation. Results This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings. Conclusions The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.
Collapse
Affiliation(s)
- Laura Himanen
- Faculty of Management and Business, Tampere University, Kanslerinrinne 1, FI-33014, Tampere, Finland
- CSC – IT Center for Science, Keilaranta 14, Espoo, 02101, Finland
| | - Erica Conte
- Unity Health Toronto, 30 Bond St, Toronto, Ontario, L1Z 1P3, Canada
| | - Marianne Gauffriau
- IT University of Copenhagen, Rued Langgaards vej 7, Copenhagen, DK-2300, Denmark
| | - Tanja Strøm
- Oslo Metropolitan University, (OsloMet), Pilestredet 46, Oslo, 0167, Norway
| | - Baron Wolf
- University of Kentucky, 311 Main Building, Lexington, Kentucky, 40502, USA
| | - Elizabeth Gadd
- Loughborough University, Epinal Way, Loughborough, England, LE11 3TU, UK
| |
Collapse
|
2
|
Silverstein P, Elman C, Montoya A, McGillivray B, Pennington CR, Harrison CH, Steltenpohl CN, Röer JP, Corker KS, Charron LM, Elsherif M, Malicki M, Hayes-Harb R, Grinschgl S, Neal T, Evans TR, Karhulahti VM, Krenzer WLD, Belaus A, Moreau D, Burin DI, Chin E, Plomp E, Mayo-Wilson E, Lyle J, Adler JM, Bottesini JG, Lawson KM, Schmidt K, Reneau K, Vilhuber L, Waltman L, Gernsbacher MA, Plonski PE, Ghai S, Grant S, Christian TM, Ngiam W, Syed M. A guide for social science journal editors on easing into open science. Res Integr Peer Rev 2024; 9:2. [PMID: 38360805 PMCID: PMC10870631 DOI: 10.1186/s41073-023-00141-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 12/28/2023] [Indexed: 02/17/2024] Open
Abstract
Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).
Collapse
Affiliation(s)
- Priya Silverstein
- Department of Psychology, Ashland University, Ashland, USA.
- Institute for Globally Distributed Open Research and Education, Preston, UK.
| | - Colin Elman
- Maxwell School of Citizenship and Public Affairs, Syracuse University, Syracuse, USA
| | - Amanda Montoya
- Department of Psychology, University of California, Los Angeles, USA
| | | | - Charlotte R Pennington
- School of Psychology, College of Health & Life Sciences, Aston University, Birmingham, UK
| | | | | | - Jan Philipp Röer
- Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Germany
| | | | - Lisa M Charron
- American Family Insurance Data Science Institute, University of Wisconsin-Madison, Madison, USA
- Nelson Institute for Environmental Studies, University of Wisconsin-Madison, Madison, USA
| | - Mahmoud Elsherif
- Department of Psychology, University of Birmingham, Birmingham, UK
| | - Mario Malicki
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, USA
- Stanford Program On Research Rigor and Reproducibility, Stanford University, Stanford, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, USA
| | | | | | - Tess Neal
- Department of Psychology, Iowa State University, Ames, USA
- School of Social & Behavioral Sciences, Arizona State University, Tempe, USA
| | - Thomas Rhys Evans
- School of Human Sciences and Institute for Lifecourse Development, University of Greenwich, London, UK
| | - Veli-Matti Karhulahti
- Department of Music, Art and Culture Studies, University of Jyväskylä, Jyväskylä, Finland
| | | | - Anabel Belaus
- National Agency for Scientific and Technological Promotion, Córdoba, Argentina
| | - David Moreau
- School of Psychology and Centre for Brain Research, University of Auckland, Auckland, New Zealand
| | - Debora I Burin
- Facultad de Psicología, Universidad de Buenos Aires, Buenos Aires, Argentina
- CONICET, Buenos Aires, Argentina
| | | | - Esther Plomp
- Faculty of Applied Sciences, Delft University of Technology, Delft, Netherlands
- The, The Alan Turing Institute, Turing Way, London, UK
| | - Evan Mayo-Wilson
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, USA
| | - Jared Lyle
- Inter-University Consortium for Political and Social Research (ICPSR), University of Michigan, Ann Arbor, USA
| | | | - Julia G Bottesini
- Maxwell School of Citizenship and Public Affairs, Syracuse University, Syracuse, USA
| | | | | | - Kyrani Reneau
- Inter-University Consortium for Political and Social Research (ICPSR), University of Michigan, Ann Arbor, USA
| | - Lars Vilhuber
- Economics Department, Cornell University, Ithaca, USA
| | - Ludo Waltman
- Centre for Science and Technology Studies, Leiden University, Leiden, Netherlands
| | | | - Paul E Plonski
- Department of Psychology, Tufts University, Medford, USA
| | - Sakshi Ghai
- Department of Psychology, University of Cambridge, Cambridge, USA
| | - Sean Grant
- HEDCO Institute for Evidence-Based Practice, College of Education, University of Oregon, Eugene, USA
| | - Thu-Mai Christian
- Odum Institute for Research in Social Science, University of North Carolina at Chapel Hill, Chapel Hill, USA
| | - William Ngiam
- Institute of Mind and Biology, University of Chicago, Chicago, USA
- Department of Psychology, University of Chicago, Chicago, USA
| | - Moin Syed
- Department of Psychology, University of Minnesota, Minneapolis, USA
| |
Collapse
|