1
|
Johnson M, Maggio LA, Konopasky A. Putting on Academic Armor: How Black Physicians and Trainees Take Stances to Make Racism Visible Amid Publishing Constraints. TEACHING AND LEARNING IN MEDICINE 2024; 36:337-347. [PMID: 37293803 DOI: 10.1080/10401334.2023.2215744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Accepted: 05/10/2023] [Indexed: 06/10/2023]
Abstract
Starting with reflexivity: As a Black woman medical student at a predominately white institution, a white woman full professor and deputy editor-in-chief of a journal, and a white woman associate professor with a deep interest in language, we understand that medicine and medical education interpellate each of us as a particular kind of subject. As such, we begin with a narrative grounding in our personal stances. Phenomenon: While there are a growing number of empirical studies of Black physicians' and trainees' experiences of racism, there are still few accounts from a first-person perspective. Black authors of these personal commentaries or editorials, who already experience microaggressions and racial trauma in their work spaces, must put on their academic armor to further experience them in publishing spaces. This study seeks to understand the stances Black physicians and trainees take as they share their personal experiences of racism. Approach: We searched four databases, identifying 29 articles authored by Black physicians and trainees describing their experiences. During initial analysis, we identified and coded for three sets of discursive strategies: identification, intertextuality, and space-time. Throughout the study, we reflected on our own stances in relation to the experience of conducting the study and its findings. Findings: Authors engaged in stance-taking, which aligned with the concept of donning academic armor, by evaluating and positioning themselves with respect to racism and the norms of academic discourse in response to ongoing conversations both within medicine and in the broader U.S. culture. They did this by (a) positioning themselves as being Black and, therefore, qualified to notice and name personal racist experiences while also aligning themselves with the reader through shared professional experiences and goals; (b) intertextual connections to other related events, people, and institutions that they-and their readers-value; and (c) aligning themselves with a hoped-for future rather than a racist present. Personal insights: Because the discourses of medicine and medical publishing interpellate Black authors as Others they must carefully consider the stances they take, particularly when naming racism. The academic armor they put on must be able to not only defend them from attack but also help them slip unseen through institutional bodies replete with mechanisms to eject them. In addition to analyzing our own personal stance, we leave readers with thought-provoking questions regarding this armor as we return to narrative grounding.
Collapse
Affiliation(s)
- Monnique Johnson
- Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Lauren A Maggio
- Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA
| | - Abigail Konopasky
- Department of Medical Education, Dartmouth's Geisel School of Medicine, Hanover, New Hampshire, USA
| |
Collapse
|
2
|
Chien PFW, Elsuity MA, Rashwan MM, Núñez-Núñez M, Khan KS, Zamora-Romero J, Bueno-Cavanillas A, Fawzy M. Post-publication research integrity concerns in randomized clinical trials: A scoping review of the literature. Int J Gynaecol Obstet 2024. [PMID: 38571333 DOI: 10.1002/ijgo.15488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2023] [Revised: 03/08/2024] [Accepted: 03/10/2024] [Indexed: 04/05/2024]
Abstract
BACKGROUND Post-publication handling of integrity concerns in randomized clinical trials (RCTs) is a contentious matter. OBJECTIVES We undertook a scoping systematic review to map the literature regarding post-publication integrity issues in RCTs. SEARCH STRATEGY AND SELECTION CRITERIA Following prospective registration (https://osf.io/pgxd8) we initially searched PubMed and Scopus but subsequently extended it to include the Cochrane Library, and Google Scholar databases without language, article type or publication time restriction until November 2022. Reviewers independently selected published articles covering any aspect of post-publication research integrity concerns in RCTs. DATA COLLECTION AND ANALYSIS The study findings grouped within domains relating to issues concerning post-publication integrity were extracted in duplicate, verified by a third reviewer, and then tabulated. MAIN RESULTS The initial search captured 3159 citations, of which 89 studies were included in the review. Cross-sectional studies constituted the majority of included studies (n = 34, 38.2%), followed by systematic reviews (n = 10, 11.2%), methodology reviews/studies (n = 9, 10.1%) and other types of descriptive studies (n = 8, 9.0%). A total of 21 articles (23.6%) covered the domain on general issues, 25 (28.1%) in the journal's instructions and policies domain, eight (9.0%) in the editorial and peer review domain, one (1.1%) in the correspondence and complaints (post-publication peer review) domain, 12 (13.5%) in the investigation for concerns domain, six (6.7%) in the post-investigation decisions and sanctions domain, none in the critical appraisal guidance domain, five (5.6%) in the integrity assessment in systematic reviews domain, and 26 (29.2%) in the recommendations for future research domain. A total of 12 of the selected articles (13.5%) covered two (n = 9) or three (n = 3) different domains. CONCLUSIONS Various research integrity domains and issues covering post-publication aspects of RCT integrity were captured and gaps were identified, mostly related with the necessary implications for all stakeholders to improve research transparency. There is an urgent need for a multistakeholder consensus towards creating specific statements for addressing post-publication integrity concerns in RCTs.
Collapse
Affiliation(s)
- Patrick F W Chien
- Department of Obstetrics and Gynecology, RCSI and UCD Malaysia Campus, Penang, Malaysia
| | - Mohamad A Elsuity
- Department of Dermatology, Venereology and Andrology, Sohag University, Sohag, Egypt
- Ibnsina, Amshaj & Ajyal IVF Centers, Sohag, Egypt
| | - Mosab M Rashwan
- Department of Forensic Medicine & Clinical Toxocology, Faculty of Medine, Sohag University, Sohag, Egypt
| | - María Núñez-Núñez
- Pharmacy Department, University, Hospital Clínico San Cecilio, Granada, Spain
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
| | - Khalid S Khan
- Department of Preventive Medicine and Public Health, Faculty of Medicine, University of Granada, Granada, Spain
- CIBER Epidemiology and Public Health, Madrid, Spain
| | - Javier Zamora-Romero
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
- Clinical Biostatistics Unit, Hospital Ramón y Cajal (IRYCIS), Madrid, Spain
| | - Aurora Bueno-Cavanillas
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
- Preventive Medicine and Public Health, University of Granada Faculty of Medicine, Granada, Spain
| | - Mohamed Fawzy
- IbnSina (Sohag), Banon (Assiut), Qena (Qena), Amshag (Sohag) IVF Facilities, Sohag, Assiut, Qena, Egypt
| |
Collapse
|
3
|
Khan KS, Fawzy M, Chien PFW. Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment. Int J Gynaecol Obstet 2023; 163:733-743. [PMID: 37184087 DOI: 10.1002/ijgo.14837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 04/06/2023] [Accepted: 04/16/2023] [Indexed: 05/16/2023]
Abstract
The integrity of randomized clinical trials (RCT) has become a concern owing to a recent rise in the number of retractions and the repercussions this has for evidence-based patient care. However, there is little research on the subject of RCT integrity assessment. Recent literature reviews have revealed that journals' authors' instructions concerning integrity and their investigation policies concerning allegations of misconduct are heterogeneous. The judicious use of integrity tests applied to RCT manuscripts is hampered by an absence of data concerning misconduct prevalence (pre-test probability), a failure to evaluate test performance (validity) and a lack of consensus over a gold standard (against which test accuracy can be evaluated). These deficiencies hinder the post-publication correction of RCT records, the integrity evaluations in systematic reviews of RCTs and the prospective application of preventive solutions in RCT peer-review and preprint assessment. Dealing with the current controversy about trustworthiness of RCT evidence requires a strong investment in research, reform and education concerning research integrity. The purpose of this review article is to highlight the current limitations in dealing with trial integrity-related concerns and to propose solutions to some of these issues.
Collapse
Affiliation(s)
- Khalid S Khan
- Department of Preventive Medicine and Public Health, Faculty of Medicine, University of Granada, Granada, Spain
- CIBER Epidemiology and Public Health, Madrid, Spain
| | - Mohamed Fawzy
- IbnSina (Sohag), Banon (Assiut), Qena (Qena), Amshag (Sohag) IVF Facilities, Cairo, Egypt
| | - Patrick F W Chien
- Department of Obstetrics & Gynecology, RCSI & UCD Malaysia Campus, Penang, Malaysia
| |
Collapse
|
4
|
Moon T, Bahadur A, Aalberg J, Jonczyk M, Chen L, Margenthaler JA, Salehi P, Chatterjee A. Assessment of Conflicts of Interest in the Transcarotid Artery Revascularization Literature. J Surg Res 2023; 291:133-138. [PMID: 37390592 DOI: 10.1016/j.jss.2023.05.032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2023] [Revised: 04/04/2023] [Accepted: 05/16/2023] [Indexed: 07/02/2023]
Abstract
INTRODUCTION To systematically review the accuracy of self-reported conflicts of interest (COIs) among transcarotid artery revascularization (TCAR) studies and evaluate factors associated with increased discrepancies. MATERIALS AND METHODS A literature search identified all TCAR-related studies with at least one American author published between January 2017 and December 2020. Industry payments from Silk Road Medical, Inc. were collected using the Centers for Medicare and Medicaid Open Payments database. COI discrepancies were identified by comparing author declaration statements with payments found for the year of publication and year prior (24-mo period). Risk factors for COI discrepancy were evaluated at both the study and author level. RESULTS A total of 79 studies (472 authors) were identified. Sixty four studies (81%) had at least one author who received payments from Silk Road Medical, Inc. Fifty eight (73%) studies had at least one author who received an undeclared payment. Consulting fees represented the majority of general payment subtype (60%). Authors who accurately disclosed payments received significantly higher median payments compared to authors who did not accurately disclose payments ($37,222 [interquartile range: $28,203-$132,589] versus $1748 [interquartile range $257-$35,041], P < 0.0001). Senior authors were significantly more likely to have a COI discrepancy compared to first authors (P = 0.0219). CONCLUSIONS The majority of TCAR-related studies did not accurately declare COI. A multivariate analysis demonstrated no effect of sponsorship on study recommendations or impact factor. This study highlights the need for increased efforts in accountability to improve the transparency of industry sponsorship, especially when consulting authors are reporting their results on patient outcomes.
Collapse
Affiliation(s)
- Tina Moon
- Department of Surgery, Tufts Medical Center, Boston, Massachusetts.
| | - Aneesh Bahadur
- Tufts University School of Medicine, Boston, Massachusetts
| | - Jeffrey Aalberg
- Department of Surgery, Tufts Medical Center, Boston, Massachusetts
| | - Michael Jonczyk
- Department of Surgery, Lahey Clinic, Burlington, Massachusetts
| | - Lilian Chen
- Division of Colorectal Surgery, Department of Surgery, Tufts Medical Center, Boston, Massachusetts
| | - Julie A Margenthaler
- Section of Endocrine and Oncologic Surgery, Department of Surgery, Washington University School of Medicine, St. Louis, Missouri
| | - Payam Salehi
- Division of Vascular Surgery, Cardiovascular Center, Tufts Medical Center, Boston, Massachusetts
| | - Abhishek Chatterjee
- Division of Plastic and Reconstructive Surgery, Department of Surgery, Tufts Medical Center, Boston, Massachusetts
| |
Collapse
|
5
|
Núñez-Núñez M, Maes-Carballo M, Mignini LE, Chien PFW, Khalaf Y, Fawzy M, Zamora J, Khan KS, Bueno-Cavanillas A. Research integrity in randomized clinical trials: A scoping umbrella review. Int J Gynaecol Obstet 2023; 162:860-876. [PMID: 37062861 DOI: 10.1002/ijgo.14762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Revised: 03/12/2023] [Accepted: 03/16/2023] [Indexed: 04/18/2023]
Abstract
BACKGROUND Randomized clinical trials (RCTs) are experiencing a crisis of confidence in their trustworthiness. Although a comprehensive literature search yielded several reviews on RCT integrity, an overarching overview is lacking. OBJECTIVES The authors undertook a scoping umbrella review of the research integrity literature concerning RCTs. SEARCH STRATEGY AND SELECTION CRITERIA Following prospective registration (https://osf.io/3ursn), two reviewers independently searched PubMed, Scopus, The Cochrane Library, and Google Scholar, without language or time restrictions, until November 2021. The authors included systematic reviews covering any aspect of research integrity throughout the RCT lifecycle. DATA COLLECTION AND ANALYSIS The authors assessed methodological quality using a modified AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) tool and collated the main findings. MAIN RESULTS A total of 55 relevant reviews, summarizing 6001 studies (median per review, 63; range, 8-1106) from 1964 to 2021, had an overall critically low quality of 96% (53 reviews). Topics covered included general aspects (15%), design and approval (22%), conduct and monitoring (11%), reporting (38%), postpublication concerns (2%), and future research (13%). The most common integrity issues covered were ethics (18%) and transparency (18%). CONCLUSIONS Low-quality reviews identified various integrity issues across the RCT lifecycle, emphasizing the importance of high ethical standards and professionalism while highlighting gaps in the integrity landscape. Multistakeholder consensus is needed to develop specific RCT integrity standards.
Collapse
Affiliation(s)
- María Núñez-Núñez
- Pharmacy Department, University Hospital Clínico San Cecilio, Granada, Spain
- Biomedical research institute of Granada (IBS-Granada), Granada, Spain
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
| | - Marta Maes-Carballo
- General Surgery Department. Breast Cancer Unit, Complexo Hospitalario de Ourense, Ourense, Spain
- General Surgery Department, Hospital Público Verín, Ourense, Spain
| | | | | | - Yacoub Khalaf
- Guy's & St Thomas' Hospital Foundation Trust, London, UK
| | - Mohamed Fawzy
- IbnSina (Sohag), Banon (Assiut), Qena (Qena), Amshag (Sohag) IVF Facilities, Cairo, Egypt
| | - Javier Zamora
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
- Clinical Biostatistics Unit, Hospital Ramón y Cajal (IRYCIS), Madrid, Spain
| | - Khalid S Khan
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
- Preventive Medicine and Public Health, University of Granada Faculty of Medicine, Granada, Spain
| | - Aurora Bueno-Cavanillas
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP-Spain), Madrid, Spain
- Preventive Medicine and Public Health, University of Granada Faculty of Medicine, Granada, Spain
| |
Collapse
|
6
|
Núñez-Núñez M, Cano-Ibáñez N, Zamora J, Bueno-Cavanillas A, Khan KS. Assessing the Integrity of Clinical Trials Included in Evidence Syntheses. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:6138. [PMID: 37372725 DOI: 10.3390/ijerph20126138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 06/05/2023] [Accepted: 06/12/2023] [Indexed: 06/29/2023]
Abstract
Evidence syntheses of randomized clinical trials (RCTs) offer the highest level of scientific evidence for informing clinical practice and policy. The value of evidence synthesis itself depends on the trustworthiness of the included RCTs. The rising number of retractions and expressions of concern about the authenticity of RCTs has raised awareness about the existence of problematic studies, sometimes called "zombie" trials. Research integrity, i.e., adherence to ethical and professional standards, is a multi-dimensional concept that is incompletely evaluated for the RCTs included in current evidence syntheses. Systematic reviewers tend to rely on the editorial and peer-review system established by journals as custodians of integrity of the RCTs they synthesize. It is now well established that falsified and fabricated RCTs are slipping through. Thus, RCT integrity assessment becomes a necessary step in systematic reviews going forward, in particular because RCTs with data-related integrity concerns remain available for use in evidence syntheses. There is a need for validated tools for systematic reviewers to proactively deploy in the assessment of integrity deviations without having to wait for RCTs to be retracted by journals or expressions of concern issued. This article analyzes the issues and challenges in conducting evidence syntheses where the literature contains RCTs with possible integrity deficits. The way forward in the form of formal RCT integrity assessments in systematic reviews is proposed, and implications of this new initiative are discussed. Future directions include emphasizing ethical and professional standards, providing tailored integrity-specific training, and creating systems to promote research integrity, as improvements in RCT integrity will benefit evidence syntheses.
Collapse
Affiliation(s)
- María Núñez-Núñez
- Pharmacy Department, Clínico San Cecilio University Hospital, 18016 Granada, Spain
- Biosanitary Research Institute (Ibs. Granada), 18012 Granada, Spain
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
| | - Naomi Cano-Ibáñez
- Biosanitary Research Institute (Ibs. Granada), 18012 Granada, Spain
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
- Department of Preventive Medicine and Public Health, University of Granada, 18016 Granada, Spain
| | - Javier Zamora
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
- Department of Biostatistics, Ramón y Cajal University Hospital (IRYCIS), 28034 Madrid, Spain
- Institute of Metabolism and Systems Research, University of Birmingham, Birmingham B15 2TT, UK
| | - Aurora Bueno-Cavanillas
- Biosanitary Research Institute (Ibs. Granada), 18012 Granada, Spain
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
- Department of Preventive Medicine and Public Health, University of Granada, 18016 Granada, Spain
| | - Khalid Saeed Khan
- Consortium for Biomedical Research in Epidemiology and Public Health (CIBERESP), 28029 Madrid, Spain
- Department of Preventive Medicine and Public Health, University of Granada, 18016 Granada, Spain
| |
Collapse
|
7
|
Barnett A. Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. F1000Res 2023; 11:783. [PMID: 37360941 PMCID: PMC10285343 DOI: 10.12688/f1000research.123002.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/24/2023] [Indexed: 06/28/2023] Open
Abstract
Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.
Collapse
Affiliation(s)
- Adrian Barnett
- Australian Centre for Health Services Innovation & Centre for Healthcare Transformation, Queensland University of Technology, Kelvin Grove, Queensland, 4059, Australia
| |
Collapse
|
8
|
Barnett A. Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. F1000Res 2023; 11:783. [PMID: 37360941 PMCID: PMC10285343 DOI: 10.12688/f1000research.123002.1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/24/2023] [Indexed: 10/12/2023] Open
Abstract
Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.
Collapse
Affiliation(s)
- Adrian Barnett
- Australian Centre for Health Services Innovation & Centre for Healthcare Transformation, Queensland University of Technology, Kelvin Grove, Queensland, 4059, Australia
| |
Collapse
|
9
|
Khan KS. International multi-stakeholder consensus statement on clinical trial integrity. BJOG 2023. [PMID: 37161843 DOI: 10.1111/1471-0528.17451] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 02/06/2023] [Accepted: 03/03/2023] [Indexed: 05/11/2023]
Abstract
OBJECTIVE To prepare a set of statements for randomised clinical trials (RCT) integrity through an international multi-stakeholder consensus. METHODS The consensus was developed via: multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percentage of majority opinions; and, a final consensus development meeting. Prospective registrations: (https://osf.io/bhncy, https://osf.io/3ursn). RESULTS There were 30 stakeholders representing 15 countries from five continents including triallists, ethicists, methodologists, statisticians, consumer representatives, industry representatives, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with eight additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n = 6), design and approval (n = 11), conduct and monitoring (n = 19), reporting of protocols and findings (n = 20), post-publication concerns (n = 12), and future research and development (n = 13). CONCLUSION Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.
Collapse
|
10
|
Abstract
Scholars need to be able to trust each other, because otherwise they cannot collaborate and use each other's findings. Similarly trust is essential for research to be applied for individuals, society or the natural environment. The trustworthiness is threatened when researchers engage in questionable research practices or worse. By adopting open science practices, research becomes transparent and accountable. Only then it is possible to verify whether trust in research findings is justified. The magnitude of the issue is substantial with a prevalence of four percent for both fabrication and falsification, and more than 50% for questionable research practices. This implies that researchers regularly engage in behaviors that harm the validity and trustworthiness of their work. What is good for the quality and reliability of research is not always good for a scholarly career. Navigating this dilemma depends on how virtuous the researcher at issue is, but also on the local research climate and the perverse incentives in the way the research system functions. Research institutes, funding agencies and scholarly journals can do a lot to foster research integrity, first and foremost by improving the quality of peer review and reforming researcher assessment.
Collapse
Affiliation(s)
- Lex Bouter
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands.,Department of Philosophy, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
11
|
Malički M, Aalbersberg IJJ, Bouter L, Mulligan A, ter Riet G. Transparency in conducting and reporting research: A survey of authors, reviewers, and editors across scholarly disciplines. PLoS One 2023; 18:e0270054. [PMID: 36888682 PMCID: PMC9994678 DOI: 10.1371/journal.pone.0270054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 06/03/2022] [Indexed: 03/09/2023] Open
Abstract
Calls have been made for improving transparency in conducting and reporting research, improving work climates, and preventing detrimental research practices. To assess attitudes and practices regarding these topics, we sent a survey to authors, reviewers, and editors. We received 3,659 (4.9%) responses out of 74,749 delivered emails. We found no significant differences between authors', reviewers', and editors' attitudes towards transparency in conducting and reporting research, or towards their perceptions of work climates. Undeserved authorship was perceived by all groups as the most prevalent detrimental research practice, while fabrication, falsification, plagiarism, and not citing prior relevant research, were seen as more prevalent by editors than authors or reviewers. Overall, 20% of respondents admitted sacrificing the quality of their publications for quantity, and 14% reported that funders interfered in their study design or reporting. While survey respondents came from 126 different countries, due to the survey's overall low response rate our results might not necessarily be generalizable. Nevertheless, results indicate that greater involvement of all stakeholders is needed to align actual practices with current recommendations.
Collapse
Affiliation(s)
- Mario Malički
- Urban Vitality Centre of Expertise, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands
- * E-mail:
| | | | - Lex Bouter
- Faculty of Humanities, Department of Philosophy, Vrije Universiteit, Amsterdam, The Netherlands
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | | | - Gerben ter Riet
- Urban Vitality Centre of Expertise, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands
- Department of Cardiology, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| |
Collapse
|
12
|
Ross-Hellauer T, Klebel T, Bannach-Brown A, Horbach SP, Jabeen H, Manola N, Metodiev T, Papageorgiou H, Reczko M, Sansone SA, Schneider J, Tijdink J, Vergoulis T. TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility. RESEARCH IDEAS AND OUTCOMES 2022. [DOI: 10.3897/rio.8.e98457] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
Lack of reproducibility of research results has become a major theme in recent years. As we emerge from the COVID-19 pandemic, economic pressures and exposed consequences of lack of societal trust in science make addressing reproducibility of urgent importance. TIER2 is a new international project funded by the European Commission under their Horizon Europe programme. Covering three broad research areas (social, life and computer sciences) and two cross-disciplinary stakeholder groups (research publishers and funders) to systematically investigate reproducibility across contexts, TIER2 will significantly boost knowledge on reproducibility, create tools, engage communities, implement interventions and policy across different contexts to increase re-use and overall quality of research results in the European Research Area and global R&I, and consequently increase trust, integrity and efficiency in research.
Collapse
|
13
|
Hardwicke TE, Salholz-Hillel M, Malički M, Szűcs D, Bendixen T, Ioannidis JPA. Statistical guidance to authors at top-ranked journals across scientific disciplines. AM STAT 2022. [DOI: 10.1080/00031305.2022.2143897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Tom E. Hardwicke
- Department of Psychology, University of Amsterdam.
- Melbourne School of Psychological Sciences, University of Melbourne.
| | - Maia Salholz-Hillel
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH) at Charité Universitätsmedizin Berlin.
| | - Mario Malički
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University.
| | - Dénes Szűcs
- Department of Psychology, University of Cambridge.
| | | | - John P. A. Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University.
- Meta-Research Innovation Center Berlin (METRIC-B), QUEST Center for Transforming Biomedical Research, Berlin Institute of Health at Charité – Universitätsmedizin Berlin.
- Departments of Medicine, Epidemiology and Population Health, Biomedical Data Science, and Statistics, Stanford University
| |
Collapse
|
14
|
Khan KS, Chien PFW. Letter to the Editor: Research and consensus is required concerning integrity of randomized clinical trials. Int J Gynaecol Obstet 2022; 159:613. [PMID: 36030408 DOI: 10.1002/ijgo.14394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 08/04/2022] [Indexed: 11/11/2022]
|
15
|
Factors Associated with the Quality and Transparency of National Guidelines: A Mixed-Methods Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19159515. [PMID: 35954872 PMCID: PMC9367745 DOI: 10.3390/ijerph19159515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 07/28/2022] [Accepted: 07/29/2022] [Indexed: 02/04/2023]
Abstract
We assessed the methodological quality and transparency of all the national clinical practice guidelines that were published in Croatia up until 2017 and explored the factors associated with their quality rating. An in-depth quantitative and qualitative analysis was performed using rigorous methodology. We evaluated the guidelines using a validated AGREE II instrument with four raters; we used multiple linear regressions to identify the predictors of quality; and two focus groups, including guideline developers, to further explore the guideline development process. The majority of the guidelines (N = 74) were developed by medical societies. The guidelines’ quality was rated low: the median standardized AGREE II score was low, 36% (IQR 28–42), and so were the overall-assessments. The aspects of the guidelines that were rated best were the “clarity of presentation” and the “scope and purpose” (median ≥ 59%); however, the other four domains received very low scores (15–33%). Overall, the guideline quality did not improve over time. The guidelines that were developed by medical societies scored significantly worse than those developed by governmental, or unofficial working groups (12–43% per domain). In focus group discussions, inadequate methodology, a lack of implementation systems in place, a lack of awareness about editorial independence, and broader expertise/perspectives in working groups were identified as factors behind the low scores. The factors identified as affecting the quality of the national guidelines may help stakeholders who are developing interventions and education programs aimed at improving guideline quality worldwide.
Collapse
|
16
|
Integrity culture is underpinned by education, not post-submission dishonesty assessments. Reprod Biomed Online 2022; 45:181. [DOI: 10.1016/j.rbmo.2022.02.020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Accepted: 02/18/2022] [Indexed: 11/24/2022]
|