Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017

International multistakeholder consensus statement on post-publication integrity issues in randomized clinical trials by Cairo Consensus Group

The number of retractions of randomized clinical trials (RCTs) following post-publication allegations of misconduct is increasing. To address this issue, we aimed to establish an international multistakeholder consensus on post-publication integrity concerns related to RCTs. After prospective registration (https://osf.io/njksm), we assembled a multidisciplinary stakeholder group comprising 48 participants from 18 countries across six continents, recruited using a curated list of journal editors and snowballing. An underpinning evidence synthesis collated 89 articles related to post-publication integrity concerns. Integrity statements related to RCTs created were subjected to anonymized two-round Delphi survey. A hybrid face-to-face-online consensus development meeting was convened to consolidate the consensus. The response rates of the two Delphi survey rounds were 65% (31/48) and 67% (32/ 48), respectively. There were 101 and 41 statements in the first and second Delphi rounds, respectively. After the two Delphi rounds and the consensus development meeting, consensus was achieved on 104 statements consolidated to 84 after merging, editing, and removing duplicates. This set of statements included general aspects (n = 9), journal instructions (n = 14), editorial and peer review (n = 7), correspondence and complaints (n = 4), investigations for integrity concerns (n = 16), decisions and sanctions (n = 9), critical appraisal guidance (n = 1), systematic reviews of RCTs (n = 8), and research recommendations (n = 16). In conclusion, this international multistakeholder consensus statement aimed to underpin policies for preventing post-publication integrity concerns in RCT publications and assist in improving investigations of misconduct allegations.

A systematic review and meta-analysis of the comparison of laparoscopic radiofrequency ablation to percutaneous radiofrequency ablation for hepatocellular carcinoma

Background

The comparative evaluation of laparoscopic and percutaneous techniques for liver radiofrequency ablation remains unexplored. This study aims to determine the most effective ablation technique and patient selection for hepatocellular carcinoma (HCC) by analyzing the efficacy and safety of laparoscopic radiofrequency ablation (LRFA) versus percutaneous radiofrequency ablation (PRFA).

Methods

Two investigators (Y-QW and PZ) independently performed a literature search in the Cochrane Library, PubMed, Web of Science and Embase databases. Study quality was assessed using the Newcastle-Ottawa Scale or Cochrane risk-of-bias tool. Meta-analysis was conducted with Review Manager 5.4, applying either fixed- or random-effects models depending on study heterogeneity. The chi-square test (χ²) and I² statistics were employed for heterogeneity analysis.

Results

Eight publications involving 1059 patients were included. Among them, 456 underwent LRFA and 603 underwent PRFA. LRFA showed a significantly better 3-year RFS than PRFA (OR: 1.89, 95% CI: 1.27-2.83, p = 0.002), the incidence rate of local recurrence was significantly fewer in the LRFA group (OR: 0.40, 95% CI: 0.23-0.69, p = 0.0010), but the postoperative hospital stay time was slightly shorter in the PFRA group (MD = 1.30; 95% CI 0.26 to 2.35; p=0. 01). Patients in the LRFA group had no significant difference in total postoperative complications, ablation success rates, overall survival (OS) and 1,5-year disease-free survival (DFS).

Conclusion

Both LRFA and PRFA are effective treatments for HCC. LRFA shows better oncologic outcomes, including lower local recurrence and improved mid-term DFS. PRFA is simpler, less invasive and shorter hospital stays. The choice should be tailored to individual patient needs, considering tumor characteristics, comorbidities, and available expertise. Future research should focus on large-scale, prospective trials to validate these findings.

Systematic review and registration

https://www.crd.york.ac.uk/PROSPERO/, identifier CRD42024601797.

Global environmental change mediated response of wetland plants: Evidence from past decades

Wetland ecosystems are critically affected by global environmental changes, yet understanding the impact of these changes on wetland plants remains a challenge. This review article employs a comprehensive approach, including bibliographic analysis, utilization of various climate models for historical data retrieval, and extensive literature survey, to investigate the response of wetland plants to environmental shifts over the past decades. The analysis conducted in this study uncovers a multitude of climatic parameters that exhibit an influence on the dynamics of wetland vegetation. Results indicated a significant positive trend in atmospheric CO2 concentration, leading to increased water use efficiency in some plant species, particularly C3 plants. However, C4 plants did not show the same positive response. Nitrous oxide growth rate showed a weaker, less consistent trend than CO2, highlighting the need for further investigation into the complex factors influencing Nitrous oxide emissions from wetlands. Methane growth rate and global mean sea level demonstrated a strong positive linear trend. Ocean pH exhibited a statistically significant downward trend (acidification), while sea surface temperature showed a moderate but statistically significant upward trend. Glacier mass balance revealed a significant negative trend. Although some plants may benefit from increased CO2 initially, but the combined effects of rising sea levels, ocean acidification, and temperature changes pose substantial threats to the overall health and diversity of wetland plant life.

Generative artificial intelligence (GAI) usage guidelines for scholarly publishing: a cross-sectional study of medical journals

BACKGROUND

Generative artificial intelligence (GAI) has developed rapidly and been increasingly used in scholarly publishing, so it is urgent to examine guidelines for its usage. This cross-sectional study aims to examine the coverage and type of recommendations of GAI usage guidelines among medical journals and how these factors relate to journal characteristics.

METHODS

From the SCImago Journal Rank (SJR) list for medicine in 2022, we generated two groups of journals: top SJR ranked journals (N = 200) and random sample of non-top SJR ranked journals (N = 140). For each group, we examined the coverage of author and reviewer guidelines across four categories: no guidelines, external guidelines only, own guidelines only, and own and external guidelines. We then calculated the number of recommendations by counting the number of usage recommendations for author and reviewer guidelines separately. Regression models examined the relationship of journal characteristics with the coverage and type of recommendations of GAI usage guidelines.

RESULTS

A higher proportion of top SJR ranked journals provided author guidelines compared to the random sample of non-top SJR ranked journals (95.0% vs. 86.7%, P < 0.01). The two groups of journals had the same median of 5 on a scale of 0 to 7 for author guidelines and a median of 1 on a scale of 0 to 2 for reviewer guidelines. However, both groups had lower percentages of journals providing recommendations for data analysis and interpretation, with the random sample of non-top SJR ranked journals having a significantly lower percentage (32.5% vs. 16.7%, P < 0.05). A higher SJR score was positively associated with providing GAI usage guidelines for both authors (all P < 0.01) and reviewers (all P < 0.01) among the random sample of non-top SJR ranked journals.

CONCLUSIONS

Although most medical journals provided their own GAI usage guidelines or referenced external guidelines, some recommendations remained unspecified (e.g., whether AI can be used for data analysis and interpretation). Additionally, journals with lower SJR scores were less likely to provide guidelines, indicating a potential gap that warrants attention. Collaborative efforts are needed to develop specific recommendations that better guide authors and reviewers.

Compliance with research participant protection guidelines by Nigerian medical journals

BACKGROUND

Stakeholders in medical research have roles in ensuring that research participants are protected. Medical journals play gatekeeping roles in the responsible conduct of research. They help guard against the publication of findings of unethical research, such as those with compromised participant welfare. Nigerian medical journals are being created to support the growing number of research enterprises. In this study, we aimed to determine the compliance of Nigerian medical journals with guidelines on research participant protection.

METHODS

This was a descriptive cross-sectional study of Nigerian medical journals and articles. We used a checklist to obtain information on journal characteristics and the presence of recommendations from the International Committee of Medical Journal Editors (ICMJE) on the protection of research participants in the journal instructions to authors and articles. The data were analysed via IBM SPSS version 23.

RESULTS

We studied 40 journals and 350 journal articles. Thirty-one (77.5%) journals required ethical approval and the Declaration of Helsinki statement in their instructions to the authors, while informed consent was present in 26 (65.0%) journals; 6 (15.0%) journals had no participant protection guidelines. Forty-one (11.7%) articles complied with all three recommendations on research participant protection, whereas 60 (17.1%) articles did not. Ethical approval was most common in 268 (76.6%) articles, whereas it was least common in statements on the Declaration of Helsinki in 50 (14.3%) articles. The presence of participant protection recommendations in instructions to authors was not associated with compliance with these recommendations in published articles (p > 0.05).

CONCLUSION

Although there is fairly good compliance of Nigerian medical journals with research participant protection recommendations, there are still gaps, which highlight the need for remedial measures.

Releasing synthetic data from the Avon Longitudinal Study of Parents and Children (ALSPAC): Guidelines and applied examples

The Avon Longitudinal Study of Parents and Children (ALSPAC) is a prospective birth cohort. Since its inception in the early 1990s, the study has collected over thirty years of data on approximately 15,000 mothers, their partners, and their offspring, resulting in over 100,000 phenotype variables to date. Maintaining data security and participant anonymity and confidentiality are key principles for the study, meaning that data access is restricted to bona fide researchers who must apply to use data, which is then shared on a project-by-project basis. Despite these legitimate reasons for restricting data access, this does run counter to emerging best scientific practices encouraging making data openly available to facilitate transparent and reproducible research. Given the rich nature of the resource, ALSPAC data are also a valuable educational tool, used for teaching a variety of methods, such as longitudinal modelling and approaches to modelling missing data. To support these efforts and to overcome the restrictions in place with the study's data sharing policy, we discuss methods for generating and making openly available synthesised ALSPAC datasets; these synthesised datasets are modelled on the original ALSPAC data, thus maintaining variable distributions and relations among variables (including missing data) as closely as possible, while at the same time preserving participant anonymity and confidentiality. We discuss how ALSPAC data can be synthesised using the 'synthpop' package in the R statistical programming language (including an applied example), present a list of guidelines for researchers wishing to release such synthesised ALSPAC data to follow, and demonstrate how this approach can be used as an educational tool to illustrate longitudinal modelling methods.

Are plain language summaries published in health journals written according to instructions and health literacy principles? A systematic environmental scan

BACKGROUND

Plain language summaries (PLSs) are easy-to-understand summaries of research articles that should follow principles of plain language and health literacy. PLS author instructions from health journals help guide authors on word count/PLS length, structure and the use of jargon. However, it is unclear whether published PLSs currently adhere to author instructions.

OBJECTIVES

This study aims to determine (1) the degree of compliance of published PLSs against the PLS author instructions in health journals and (2) the extent to which PLSs meet health literacy principles.

STUDY DESIGN

We conducted a three-part systematic environmental scan.

METHODS

We examined 26 health journals identified from a previous review. In part 1, we assessed the inclusion frequency of PLSs in the 26 journals; in part 2, we assessed the level of compliance of PLSs with PLS author instructions; and in part 3, we conducted a health literacy assessment of the PLSs.

RESULTS

Part 1: we found PLSs for 20/26 (76.9%) included journals. Part 2: no journal achieved 100% compliance with PLS author instructions. The highest level was 86% and the lowest was 0%. Part 3: no PLS was written at a readability level suitable for a general audience. The mean reading level was grade 15.8 (range 10.2-21.2and mean percentage of complex words, 31% (range 8.5%-49.8%).

DISCUSSION

PLSs are an important means through which consumers can access research findings. We found a lack of compliance between PLS author instructions and PLSs published in health journals that may impede access and use by consumers. This study highlights the need for better ways to support authors adhere to PLS instructions and improved monitoring by journals.

Why research integrity matters and how it can be improved

Scholars need to be able to trust each other, because otherwise they cannot collaborate and use each other's findings. Similarly trust is essential for research to be applied for individuals, society or the natural environment. The trustworthiness is threatened when researchers engage in questionable research practices or worse. By adopting open science practices, research becomes transparent and accountable. Only then it is possible to verify whether trust in research findings is justified. The magnitude of the issue is substantial with a prevalence of four percent for both fabrication and falsification, and more than 50% for questionable research practices. This implies that researchers regularly engage in behaviors that harm the validity and trustworthiness of their work. What is good for the quality and reliability of research is not always good for a scholarly career. Navigating this dilemma depends on how virtuous the researcher at issue is, but also on the local research climate and the perverse incentives in the way the research system functions. Research institutes, funding agencies and scholarly journals can do a lot to foster research integrity, first and foremost by improving the quality of peer review and reforming researcher assessment.

[International Multi-stakeholder Consensus Statement on Clinical Trial Integrity]

BACKGROUND

Science integrity initiatives require specific recommendations for randomised clinical trials (RCT).

OBJECTIVE

To prepare a set of statements for RCT integrity through an international multi-stakeholder consensus.

METHODS

The consensus was developed via multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percent of majority opinions; and, a final consensus development meeting.

RESULTS

There were 30 stakeholders representing 15 countries from 5 continents including trialists, ethicists, methodologists, statisticians, consumer representative, industry representative, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with 8 additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n=6), design and approval (n=11), conduct and monitoring (n=19), reporting of protocols and findings (n=20), post-publication concerns (n=12), and future research and development (n=13).

CONCLUSION

Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.

Post-publication research integrity concerns in randomized clinical trials: A scoping review of the literature

BACKGROUND

Post-publication handling of integrity concerns in randomized clinical trials (RCTs) is a contentious matter.

OBJECTIVES

We undertook a scoping systematic review to map the literature regarding post-publication integrity issues in RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/pgxd8) we initially searched PubMed and Scopus but subsequently extended it to include the Cochrane Library, and Google Scholar databases without language, article type or publication time restriction until November 2022. Reviewers independently selected published articles covering any aspect of post-publication research integrity concerns in RCTs.

DATA COLLECTION AND ANALYSIS

The study findings grouped within domains relating to issues concerning post-publication integrity were extracted in duplicate, verified by a third reviewer, and then tabulated.

MAIN RESULTS

The initial search captured 3159 citations, of which 89 studies were included in the review. Cross-sectional studies constituted the majority of included studies (n = 34, 38.2%), followed by systematic reviews (n = 10, 11.2%), methodology reviews/studies (n = 9, 10.1%) and other types of descriptive studies (n = 8, 9.0%). A total of 21 articles (23.6%) covered the domain on general issues, 25 (28.1%) in the journal's instructions and policies domain, eight (9.0%) in the editorial and peer review domain, one (1.1%) in the correspondence and complaints (post-publication peer review) domain, 12 (13.5%) in the investigation for concerns domain, six (6.7%) in the post-investigation decisions and sanctions domain, none in the critical appraisal guidance domain, five (5.6%) in the integrity assessment in systematic reviews domain, and 26 (29.2%) in the recommendations for future research domain. A total of 12 of the selected articles (13.5%) covered two (n = 9) or three (n = 3) different domains.

CONCLUSIONS

Various research integrity domains and issues covering post-publication aspects of RCT integrity were captured and gaps were identified, mostly related with the necessary implications for all stakeholders to improve research transparency. There is an urgent need for a multistakeholder consensus towards creating specific statements for addressing post-publication integrity concerns in RCTs.

Research integrity in Instructions for Authors in Japanese medical journals using ICMJE Recommendations: A descriptive literature study

BACKGROUND

The International Committee of Medical Journal Editors (ICMJE) has published Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals. These provide a global standard for writing and editing medical articles, including research integrity. However, no study has examined the research integrity-related content of Japanese medical journals' Instructions for Authors. We therefore compared research integrity content in ICMJE member journals with those in the English- and Japanese-language journals of the Japanese Association of Medical Sciences (JAMS).

MATERIALS AND METHODS

This was a descriptive literature study. We obtained Instructions for Authors from English- and Japanese-language journals listed on the JAMS website and the ICMJE member journals listed on the ICMJE website as of September 1, 2021. We compared the presence of 20 topics (19 in the ICMJE Recommendations plus compliance with ICMJE) in the Instructions for Authors, and analyzed the content of the conflict of interest disclosure.

RESULTS

We evaluated 12 ICMJE member journals, and 82 English-language and 99 Japanese-language subcommittee journals. The median number of topics covered was 10.5 for ICMJE member journals, 10 for English-language journals, and three for Japanese-language journals. Compliance with ICMJE was mentioned by 10 (83%) ICMJE member journals, 75 (91%) English-language journals, and 29 (29%) Japanese-language journals. The ICMJE Conflicts of Interest Disclosure Form was requested by seven (64%) ICMJE member journals, 15 (18%) English-language journals, and one (1%) Japanese-language journal.

CONCLUSIONS

Although the topics in the JAMS English-language journals resembled those in the ICMJE member journals, the median value of ICMJE-related topic inclusion was approximately one-third lower in JAMS Japanese-language journals than in ICMJE member journals. It is hoped that Japanese-language journals whose conflict of interest disclosure policies differ from ICMJE standards will adopt international standards to deter misconduct and ensure publication quality.

Putting on Academic Armor: How Black Physicians and Trainees Take Stances to Make Racism Visible Amid Publishing Constraints

Starting with reflexivity: As a Black woman medical student at a predominately white institution, a white woman full professor and deputy editor-in-chief of a journal, and a white woman associate professor with a deep interest in language, we understand that medicine and medical education interpellate each of us as a particular kind of subject. As such, we begin with a narrative grounding in our personal stances. Phenomenon: While there are a growing number of empirical studies of Black physicians' and trainees' experiences of racism, there are still few accounts from a first-person perspective. Black authors of these personal commentaries or editorials, who already experience microaggressions and racial trauma in their work spaces, must put on their academic armor to further experience them in publishing spaces. This study seeks to understand the stances Black physicians and trainees take as they share their personal experiences of racism. Approach: We searched four databases, identifying 29 articles authored by Black physicians and trainees describing their experiences. During initial analysis, we identified and coded for three sets of discursive strategies: identification, intertextuality, and space-time. Throughout the study, we reflected on our own stances in relation to the experience of conducting the study and its findings. Findings: Authors engaged in stance-taking, which aligned with the concept of donning academic armor, by evaluating and positioning themselves with respect to racism and the norms of academic discourse in response to ongoing conversations both within medicine and in the broader U.S. culture. They did this by (a) positioning themselves as being Black and, therefore, qualified to notice and name personal racist experiences while also aligning themselves with the reader through shared professional experiences and goals; (b) intertextual connections to other related events, people, and institutions that they-and their readers-value; and (c) aligning themselves with a hoped-for future rather than a racist present. Personal insights: Because the discourses of medicine and medical publishing interpellate Black authors as Others they must carefully consider the stances they take, particularly when naming racism. The academic armor they put on must be able to not only defend them from attack but also help them slip unseen through institutional bodies replete with mechanisms to eject them. In addition to analyzing our own personal stance, we leave readers with thought-provoking questions regarding this armor as we return to narrative grounding.

Integrity of randomized clinical trials: Performance of integrity tests and checklists requires assessment

The integrity of randomized clinical trials (RCT) has become a concern owing to a recent rise in the number of retractions and the repercussions this has for evidence-based patient care. However, there is little research on the subject of RCT integrity assessment. Recent literature reviews have revealed that journals' authors' instructions concerning integrity and their investigation policies concerning allegations of misconduct are heterogeneous. The judicious use of integrity tests applied to RCT manuscripts is hampered by an absence of data concerning misconduct prevalence (pre-test probability), a failure to evaluate test performance (validity) and a lack of consensus over a gold standard (against which test accuracy can be evaluated). These deficiencies hinder the post-publication correction of RCT records, the integrity evaluations in systematic reviews of RCTs and the prospective application of preventive solutions in RCT peer-review and preprint assessment. Dealing with the current controversy about trustworthiness of RCT evidence requires a strong investment in research, reform and education concerning research integrity. The purpose of this review article is to highlight the current limitations in dealing with trial integrity-related concerns and to propose solutions to some of these issues.

Assessment of Conflicts of Interest in the Transcarotid Artery Revascularization Literature

INTRODUCTION

To systematically review the accuracy of self-reported conflicts of interest (COIs) among transcarotid artery revascularization (TCAR) studies and evaluate factors associated with increased discrepancies.

MATERIALS AND METHODS

A literature search identified all TCAR-related studies with at least one American author published between January 2017 and December 2020. Industry payments from Silk Road Medical, Inc. were collected using the Centers for Medicare and Medicaid Open Payments database. COI discrepancies were identified by comparing author declaration statements with payments found for the year of publication and year prior (24-mo period). Risk factors for COI discrepancy were evaluated at both the study and author level.

RESULTS

A total of 79 studies (472 authors) were identified. Sixty four studies (81%) had at least one author who received payments from Silk Road Medical, Inc. Fifty eight (73%) studies had at least one author who received an undeclared payment. Consulting fees represented the majority of general payment subtype (60%). Authors who accurately disclosed payments received significantly higher median payments compared to authors who did not accurately disclose payments ($37,222 [interquartile range: $28,203-$132,589] versus $1748 [interquartile range $257-$35,041], P < 0.0001). Senior authors were significantly more likely to have a COI discrepancy compared to first authors (P = 0.0219).

CONCLUSIONS

The majority of TCAR-related studies did not accurately declare COI. A multivariate analysis demonstrated no effect of sponsorship on study recommendations or impact factor. This study highlights the need for increased efforts in accountability to improve the transparency of industry sponsorship, especially when consulting authors are reporting their results on patient outcomes.

Research integrity in randomized clinical trials: A scoping umbrella review

BACKGROUND

Randomized clinical trials (RCTs) are experiencing a crisis of confidence in their trustworthiness. Although a comprehensive literature search yielded several reviews on RCT integrity, an overarching overview is lacking.

OBJECTIVES

The authors undertook a scoping umbrella review of the research integrity literature concerning RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/3ursn), two reviewers independently searched PubMed, Scopus, The Cochrane Library, and Google Scholar, without language or time restrictions, until November 2021. The authors included systematic reviews covering any aspect of research integrity throughout the RCT lifecycle.

DATA COLLECTION AND ANALYSIS

The authors assessed methodological quality using a modified AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) tool and collated the main findings.

MAIN RESULTS

A total of 55 relevant reviews, summarizing 6001 studies (median per review, 63; range, 8-1106) from 1964 to 2021, had an overall critically low quality of 96% (53 reviews). Topics covered included general aspects (15%), design and approval (22%), conduct and monitoring (11%), reporting (38%), postpublication concerns (2%), and future research (13%). The most common integrity issues covered were ethics (18%) and transparency (18%).

CONCLUSIONS

Low-quality reviews identified various integrity issues across the RCT lifecycle, emphasizing the importance of high ethical standards and professionalism while highlighting gaps in the integrity landscape. Multistakeholder consensus is needed to develop specific RCT integrity standards.

Assessing the Integrity of Clinical Trials Included in Evidence Syntheses

Evidence syntheses of randomized clinical trials (RCTs) offer the highest level of scientific evidence for informing clinical practice and policy. The value of evidence synthesis itself depends on the trustworthiness of the included RCTs. The rising number of retractions and expressions of concern about the authenticity of RCTs has raised awareness about the existence of problematic studies, sometimes called "zombie" trials. Research integrity, i.e., adherence to ethical and professional standards, is a multi-dimensional concept that is incompletely evaluated for the RCTs included in current evidence syntheses. Systematic reviewers tend to rely on the editorial and peer-review system established by journals as custodians of integrity of the RCTs they synthesize. It is now well established that falsified and fabricated RCTs are slipping through. Thus, RCT integrity assessment becomes a necessary step in systematic reviews going forward, in particular because RCTs with data-related integrity concerns remain available for use in evidence syntheses. There is a need for validated tools for systematic reviewers to proactively deploy in the assessment of integrity deviations without having to wait for RCTs to be retracted by journals or expressions of concern issued. This article analyzes the issues and challenges in conducting evidence syntheses where the literature contains RCTs with possible integrity deficits. The way forward in the form of formal RCT integrity assessments in systematic reviews is proposed, and implications of this new initiative are discussed. Future directions include emphasizing ethical and professional standards, providing tailored integrity-specific training, and creating systems to promote research integrity, as improvements in RCT integrity will benefit evidence syntheses.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials

Background: Papers describing the results of a randomised trial should include a baseline table that compares the characteristics of randomised groups. Researchers who fraudulently generate trials often unwittingly create baseline tables that are implausibly similar (under-dispersed) or have large differences between groups (over-dispersed). I aimed to create an automated algorithm to screen for under- and over-dispersion in the baseline tables of randomised trials. Methods: Using a cross-sectional study I examined 2,245 randomised controlled trials published in health and medical journals on PubMed Central. I estimated the probability that a trial's baseline summary statistics were under- or over-dispersed using a Bayesian model that examined the distribution of t-statistics for the between-group differences, and compared this with an expected distribution without dispersion. I used a simulation study to test the ability of the model to find under- or over-dispersion and compared its performance with an existing test of dispersion based on a uniform test of p-values. My model combined categorical and continuous summary statistics, whereas the uniform test used only continuous statistics. Results: The algorithm had a relatively good accuracy for extracting the data from baseline tables, matching well on the size of the tables and sample size. Using t-statistics in the Bayesian model out-performed the uniform test of p-values, which had many false positives for skewed, categorical and rounded data that were not under- or over-dispersed. For trials published on PubMed Central, some tables appeared under- or over-dispersed because they had an atypical presentation or had reporting errors. Some trials flagged as under-dispersed had groups with strikingly similar summary statistics. Conclusions: Automated screening for fraud of all submitted trials is challenging due to the widely varying presentation of baseline tables. The Bayesian model could be useful in targeted checks of suspected trials or authors.

International multi-stakeholder consensus statement on clinical trial integrity

OBJECTIVE

To prepare a set of statements for randomised clinical trials (RCT) integrity through an international multi-stakeholder consensus.

METHODS

The consensus was developed via: multi-country multidisciplinary stakeholder group composition and engagement; evidence synthesis of 55 systematic reviews concerning RCT integrity; anonymised two-round modified Delphi survey with consensus threshold based on the average percentage of majority opinions; and, a final consensus development meeting. Prospective registrations: (https://osf.io/bhncy, https://osf.io/3ursn).

RESULTS

There were 30 stakeholders representing 15 countries from five continents including triallists, ethicists, methodologists, statisticians, consumer representatives, industry representatives, systematic reviewers, funding body panel members, regulatory experts, authors, journal editors, peer-reviewers and advisors for resolving integrity concerns. Delphi survey response rate was 86.7% (26/30 stakeholders). There were 111 statements (73 stakeholder-provided, 46 systematic review-generated, 8 supported by both) in the initial long list, with eight additional statements provided during the consensus rounds. Through consensus the final set consolidated 81 statements (49 stakeholder-provided, 41 systematic review-generated, 9 supported by both). The entire RCT life cycle was covered by the set of statements including general aspects (n = 6), design and approval (n = 11), conduct and monitoring (n = 19), reporting of protocols and findings (n = 20), post-publication concerns (n = 12), and future research and development (n = 13).

CONCLUSION

Implementation of this multi-stakeholder consensus statement is expected to enhance RCT integrity.

Transparency in conducting and reporting research: A survey of authors, reviewers, and editors across scholarly disciplines

Calls have been made for improving transparency in conducting and reporting research, improving work climates, and preventing detrimental research practices. To assess attitudes and practices regarding these topics, we sent a survey to authors, reviewers, and editors. We received 3,659 (4.9%) responses out of 74,749 delivered emails. We found no significant differences between authors', reviewers', and editors' attitudes towards transparency in conducting and reporting research, or towards their perceptions of work climates. Undeserved authorship was perceived by all groups as the most prevalent detrimental research practice, while fabrication, falsification, plagiarism, and not citing prior relevant research, were seen as more prevalent by editors than authors or reviewers. Overall, 20% of respondents admitted sacrificing the quality of their publications for quantity, and 14% reported that funders interfered in their study design or reporting. While survey respondents came from 126 different countries, due to the survey's overall low response rate our results might not necessarily be generalizable. Nevertheless, results indicate that greater involvement of all stakeholders is needed to align actual practices with current recommendations.

TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility

Lack of reproducibility of research results has become a major theme in recent years. As we emerge from the COVID-19 pandemic, economic pressures and exposed consequences of lack of societal trust in science make addressing reproducibility of urgent importance. TIER2 is a new international project funded by the European Commission under their Horizon Europe programme. Covering three broad research areas (social, life and computer sciences) and two cross-disciplinary stakeholder groups (research publishers and funders) to systematically investigate reproducibility across contexts, TIER2 will significantly boost knowledge on reproducibility, create tools, engage communities, implement interventions and policy across different contexts to increase re-use and overall quality of research results in the European Research Area and global R&I, and consequently increase trust, integrity and efficiency in research.

Factors Associated with the Quality and Transparency of National Guidelines: A Mixed-Methods Study

We assessed the methodological quality and transparency of all the national clinical practice guidelines that were published in Croatia up until 2017 and explored the factors associated with their quality rating. An in-depth quantitative and qualitative analysis was performed using rigorous methodology. We evaluated the guidelines using a validated AGREE II instrument with four raters; we used multiple linear regressions to identify the predictors of quality; and two focus groups, including guideline developers, to further explore the guideline development process. The majority of the guidelines (N = 74) were developed by medical societies. The guidelines’ quality was rated low: the median standardized AGREE II score was low, 36% (IQR 28–42), and so were the overall-assessments. The aspects of the guidelines that were rated best were the “clarity of presentation” and the “scope and purpose” (median ≥ 59%); however, the other four domains received very low scores (15–33%). Overall, the guideline quality did not improve over time. The guidelines that were developed by medical societies scored significantly worse than those developed by governmental, or unofficial working groups (12–43% per domain). In focus group discussions, inadequate methodology, a lack of implementation systems in place, a lack of awareness about editorial independence, and broader expertise/perspectives in working groups were identified as factors behind the low scores. The factors identified as affecting the quality of the national guidelines may help stakeholders who are developing interventions and education programs aimed at improving guideline quality worldwide.