A systematic scoping review of adherence to reporting guidelines in health care literature

Reporting guidelines in medical artificial intelligence: a systematic review and meta-analysis

BACKGROUND

The field of Artificial Intelligence (AI) holds transformative potential in medicine. However, the lack of universal reporting guidelines poses challenges in ensuring the validity and reproducibility of published research studies in this field.

METHODS

Based on a systematic review of academic publications and reporting standards demanded by both international consortia and regulatory stakeholders as well as leading journals in the fields of medicine and medical informatics, 26 reporting guidelines published between 2009 and 2023 were included in this analysis. Guidelines were stratified by breadth (general or specific to medical fields), underlying consensus quality, and target research phase (preclinical, translational, clinical) and subsequently analyzed regarding the overlap and variations in guideline items.

RESULTS

AI reporting guidelines for medical research vary with respect to the quality of the underlying consensus process, breadth, and target research phase. Some guideline items such as reporting of study design and model performance recur across guidelines, whereas other items are specific to particular fields and research stages.

CONCLUSIONS

Our analysis highlights the importance of reporting guidelines in clinical AI research and underscores the need for common standards that address the identified variations and gaps in current guidelines. Overall, this comprehensive overview could help researchers and public stakeholders reinforce quality standards for increased reliability, reproducibility, clinical validity, and public trust in AI research in healthcare. This could facilitate the safe, effective, and ethical translation of AI methods into clinical applications that will ultimately improve patient outcomes.

CONSORT-TM: Text classification models for assessing the completeness of randomized controlled trial publications

Objective

To develop text classification models for determining whether the checklist items in the CONSORT reporting guidelines are reported in randomized controlled trial publications.

Materials and Methods

Using a corpus annotated at the sentence level with 37 fine-grained CONSORT items, we trained several sentence classification models (PubMedBERT fine-tuning, BioGPT fine-tuning, and in-context learning with GPT-4) and compared their performance. To address the problem of small training dataset, we used several data augmentation methods (EDA, UMLS-EDA, text generation and rephrasing with GPT-4) and assessed their impact on the fine-tuned PubMedBERT model. We also fine-tuned PubMedBERT models limited to checklist items associated with specific sections (e.g., Methods) to evaluate whether such models could improve performance compared to the single full model. We performed 5-fold cross-validation and report precision, recall, F1 score, and area under curve (AUC).

Results

Fine-tuned PubMedBERT model that takes as input the sentence and the surrounding sentence representations and uses section headers yielded the best overall performance (0.71 micro-F1, 0.64 macro-F1). Data augmentation had limited positive effect, UMLS-EDA yielding slightly better results than data augmentation using GPT-4. BioGPT fine-tuning and GPT-4 in-context learning exhibited suboptimal results. Methods-specific model yielded higher performance for methodology items, other section-specific models did not have significant impact.

Conclusion

Most CONSORT checklist items can be recognized reasonably well with the fine-tuned PubMedBERT model but there is room for improvement. Improved models can underpin the journal editorial workflows and CONSORT adherence checks and can help authors in improving the reporting quality and completeness of their manuscripts.

Guidelines for reporting pediatric and child health clinical trial protocols and reports: study protocol for SPIRIT-Children and CONSORT-Children

BACKGROUND

Despite the critical importance of clinical trials to provide evidence about the effects of intervention for children and youth, a paucity of published high-quality pediatric clinical trials persists. Sub-optimal reporting of key trial elements necessary to critically appraise and synthesize findings is prevalent. To harmonize and provide guidance for reporting in pediatric controlled clinical trial protocols and reports, reporting guideline extensions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Consolidated Standards of Reporting Trials (CONSORT) guidelines specific to pediatrics are being developed: SPIRIT-Children (SPIRIT-C) and CONSORT-Children (CONSORT-C).

METHODS

The development of SPIRIT-C/CONSORT-C will be informed by the Enhancing the Quality and Transparency of Health Research Quality (EQUATOR) method for reporting guideline development in the following stages: (1) generation of a preliminary list of candidate items, informed by (a) items developed during initial development efforts and child relevant items from recent published SPIRIT and CONSORT extensions; (b) two systematic reviews and environmental scan of the literature; (c) workshops with young people; (2) an international Delphi study, where a wide range of panelists will vote on the inclusion or exclusion of candidate items on a nine-point Likert scale; (3) a consensus meeting to discuss items that have not reached consensus in the Delphi study and to "lock" the checklist items; (4) pilot testing of items and definitions to ensure that they are understandable, useful, and applicable; and (5) a final project meeting to discuss each item in the context of pilot test results. Key partners, including young people (ages 12-24 years) and family caregivers (e.g., parents) with lived experiences with pediatric clinical trials, and individuals with expertise and involvement in pediatric trials will be involved throughout the project. SPIRIT-C/CONSORT-C will be disseminated through publications, academic conferences, and endorsement by pediatric journals and relevant research networks and organizations.

DISCUSSION

SPIRIT/CONSORT-C may serve as resources to facilitate comprehensive reporting needed to understand pediatric clinical trial protocols and reports, which may improve transparency within pediatric clinical trials and reduce research waste.

TRIAL REGISTRATION

The development of these reporting guidelines is registered with the EQUATOR Network: SPIRIT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials-protocols/#35 ) and CONSORT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials/#CHILD ).

Studying Adherence to Reporting Standards in Kinesiology: A Post-publication Peer Review Brief Report

To demonstrate how post-publication peer reviews-using journal article reporting standards-could improve the design and write-up of kinesiology research, the authors performed a post-publication peer review on one systematic literature review published in 2020. Two raters (1st & 2nd authors) critically appraised the case article between April and May 2021. The latest Journal Article Reporting Standards by the American Psychological Association relevant to the review were used: i.e., Table 1 (quantitative research standards) and Table 9 (research synthesis standards). A standard fully met was deemed satisfactory. Per Krippendorff's alpha-coefficient, inter-rater agreement was moderate for Table 1 (k-alpha = .57, raw-agreement = 72.2%) and poor for Table 9 (k-alpha = .09, raw-agreement = 53.6%). A 100% consensus was reached on all discrepancies. Results suggest the case article's Abstract, Methods, and Discussion sections required clarification or more detail. Per Table 9 standards, four sections were largely incomplete: i.e., Abstract (100%-incomplete), Introduction (66%-incomplete), Methods (75%-incomplete), and Discussion (66%-incomplete). Case article strengths included tabular summary of studies analyzed in the systematic review and a cautionary comment about the review's generalizability. The article's write-up gave detail to help the reader understand the scope of the study and decisions made by the authors. However, adequate detail was not provided to assess the credibility of all claims made in the article. This could affect readers' ability to obtain critical and nuanced understanding of the article's topics. The results of this critique should encourage (continuing) education on journal article reporting standards for diverse stakeholders (e.g., authors, reviewers).

Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles: Primary Results of 2 Randomized Trials

Importance

Numerous studies have shown that adherence to reporting guidelines is suboptimal.

Objective

To evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles.

Design, Setting, and Participants

Two parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group.

Interventions

The first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded.

Main Outcomes and Measures

The difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles.

Results

In the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%).

Conclusions and Relevance

These 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future.

Trial Registration

ClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

Reporting quality of randomized controlled trials evaluating non-vitamin K oral anticoagulants in atrial fibrillation: a systematic review

BACKGROUND

Randomized controlled trials (RCTs) are subject to bias if they lack methodological quality. Furthermore, optimal and transparent reporting of RCT findings aids their critical appraisal and interpretation. This study aimed to comprehensively evaluate the report quality of RCTs of non-vitamin K oral anticoagulants (NOACs) for the treatment of atrial fibrillation (AF) and to analyze the factors influencing the quality.

METHODS

By searching PubMed, Embase, Web of Science, and Cochrane Library databases RCTs published from inception to 2022 evaluating the efficacy of NOACs on AF were collected. By using the 2010 Consolidated Standards for Reporting Tests (CONSORT) statement, the overall quality of each report was assessed.

RESULTS

Sixty-two RCTs were retrieved in this study. The median of overall quality score in 2010 was 14 (range: 8.5-20). The extent of compliance with the Consolidated Standards of Reporting Trials reporting guideline differed substantially across items: 9 items were reported adequately (more than 90%), and 3 were reported adequately in less than 10% of trials. Multivariate linear regression analysis showed that the higher reporting scores were associated with higher journal impact factor (P = 0.01), international collaboration (P < 0.01), and Sources of trial funding (P = 0.02).

CONCLUSIONS

Although a large number of randomized controlled trials of NOACs for the treatment of AF were published after the CONSORT statement in 2010, the overall quality is still not satisfactory, thus weakening their potential utility and may mislead clinical decisions. This survey provides the first hint for researchers conducting trials of NOACs for AF to improve the quality of reports and to actively apply the CONSORT statement.

Data source profile reporting by studies that use routinely collected health data to explore the effects of drug treatment

BACKGROUND

Routinely collected health data (RCD) are important resource for exploring drug treatment effects. Adequate reporting of data source profiles may increase the credibility of evidence generated from these data. This study conducted a systematic literature review to evaluate the reporting characteristics of databases used by RCD studies to explore the effects of drug treatment.

METHODS

Observational studies published in 2018 that used RCD to explore the effects of drug treatment were identified by searching PubMed. We categorized eligible reports into two groups by journal impact factor (IF), including the top 5 general medical journals (NEJM, Lancet, JAMA, BMJ and JAMA Internal Medicine) and the other journals. The reporting characteristics of the databases used were described and compared between the two groups and between studies citing and not citing database references.

RESULTS

A total of 222 studies were included, of which 53 (23.9%) reported that they applied data linkage, 202 (91.0%) reported the type of database, and 211 (95.0%) reported the coverage of the data source. Only 81 (36.5%) studies reported the timeframe of the database. Studies in high-impact journals were more likely to report that they applied data linkage (65.1% vs. 20.2%) and used electronic medical records (EMR) (73.7% vs. 30.0%) and national data sources (77.8% vs. 51.3%) than those published in other medical journals. There were 137/222 (61.7%) cited database references. Studies with database-specific citations had better reporting of the data sources and were more likely to publish in high-impact journals than those without (mean IF, 6.08 vs. 4.09).

CONCLUSIONS

Some deficits were found in the reporting quality of databases in studies that used RCD to explore the effects of drug treatment. Studies citing database-specific references may provide detailed information regarding data source characteristics. The adoption of reporting guidelines and education on their use is urgently needed to promote transparency by research groups.

Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines

Reporting of intervention research has been inadequate for many years. The development and promotion of freely available checklists aims to address this problem by providing researchers with a list of items that require reporting to enable study interpretation and replication. In this commentary, we present evidence from a recent systematic review of 51 randomised controlled trials published 2015-2020 that inadequate intervention reporting remains a widespread issue and that checklists are not being used to describe all intervention components. In 2022, we assessed the submission guidelines of 33 journals that published articles included in our review and found that just one at the time encouraged the use of reporting checklists for all intervention components. To drive progress, we contacted the editors of the other 32 journals and requested that they update their submission guidelines in response. We conclude by highlighting the waste associated with current practices and encourage journals from all fields to urgently review their submission guidelines. Only through collective action can we build an evidence base that is fit for purpose.

What Are the Determinants of the Quality of Systematic Reviews in the International Journals of Occupational Medicine? A Methodological Study Review of Published Literature

Objective: The aim of this study was to evaluate the methodological quality of systematic reviews published in occupational medicine journals from 2014 to 2021. Methods: Papers edited between 2014 and 2021 in the 14 open access journals with the highest impact were assessed for their quality. Studies were included if they were systematic reviews and meta-analyses, and if they were published in English. Results: The study included 335 studies. Among these, 149 were meta-analyses and 186 were systematic reviews. The values of the AMSTAR-2 score range between three and fourteen with a mean value of 9.85 (SD = 2.37). The factors that significantly and directly associate to a higher AMSTAR-2 score were impact factor (p = 0.003), number of consulted research databases (p = 0.011), declaration of PRISMA statement (p = 0.003), year of publication (p < 0.001) and performing a meta-analysis (p < 0.001).The R² values from the multivariate analysis showed that the AMSTAR-2 score could be predicted by the inclusion of these parameters by up to 23%. Conclusions: This study suggests a quality assessment methodology that could help readers in a fast identification of good systematic reviews or meta-analyses. Future studies should analyze more journals without applying language restrictions and consider a wider range of years of publication in order to give a more robust evidence for results.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

The reporting and handling of missing data in longitudinal studies of older adults is suboptimal: a methodological survey of geriatric journals

Background

Missing data are common in longitudinal studies, and more so, in studies of older adults, who are susceptible to health and functional decline that limit completion of assessments. We assessed the extent, current reporting, and handling of missing data in longitudinal studies of older adults.

Methods

Medline and Embase databases were searched from 2015 to 2019 for publications on longitudinal observational studies conducted among persons ≥55 years old. The search was restricted to 10 general geriatric journals published in English. Reporting and handling of missing data were assessed using questions developed from the recommended standards. Data were summarised descriptively as frequencies and proportions.

Results

A total of 165 studies were included in the review from 7032 identified records. In approximately half of the studies 97 (62.5%), there was either no comment on missing data or unclear descriptions. The percentage of missing data varied from 0.1 to 55%, with a 14% average among the studies that reported having missing data. Complete case analysis was the most common method for handling missing data with nearly 75% of the studies (n = 52) excluding individual observations due to missing data, at the initial phase of study inclusion or at the analysis stage. Of the 10 studies where multiple imputation was used, only 1 (10.0%) study followed the guideline for reporting the procedure fully using online supplementary documents.

Conclusion

The current reporting and handling of missing data in longitudinal observational studies of older adults are inadequate. Journal endorsement and implementation of guidelines may potentially improve the quality of missing data reporting. Further, authors should be encouraged to use online supplementary files to provide additional details on how missing data were addressed, to allow for more transparency and comprehensive appraisal of studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01605-w.

Reporting of abstracts in studies that used routinely collected data for exploring drug treatment effects: a cross-sectional survey

Background

In recent years, studies that used routinely collected data (RCD), such as electronic medical records and administrative claims, for exploring drug treatment effects, including effectiveness and safety, have been increasingly published. Abstracts of such studies represent a highly attended source for busy clinicians or policy-makers, and are important for indexing by literature database. If less clearly presented, they may mislead decisions or indexing. We thus conducted a cross-sectional survey to systematically examine how the abstracts of such studies were reported.

Methods

We searched PubMed to identify all observational studies published in 2018 that used RCD for assessing drug treatment effects. Teams of methods-trained collected data from eligible studies using pilot-tested, standardized forms that were developed and expanded from “The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology” (RECORD-PE) statement. We used descriptive analyses to examine how authors reported data source, study design, data analysis, and interpretation of findings.

Results

A total of 222 studies were included, of which 118 (53.2%) reported type of database used, 17 (7.7%) clearly reported database linkage, and 140 (63.1%) reported coverage of data source. Only 44 (19.8%) studies stated a predefined hypothesis, 127 (57.2%) reported study design, 140 (63.1%) reported statistical models used, 142 (77.6%) reported adjusted estimates, 33 (14.9%) mentioned sensitivity analyses, and 39 (17.6%) made a strong claim about treatment effect. Studies published in top 5 general medicine journals were more likely to report the name of data source (94.7% vs. 67.0%) and study design (100% vs. 53.2%) than those in other journals.

Conclusions

The under-reporting of key methodological features in abstracts of RCD studies was common, which would substantially compromise the indexing of this type of literature and prevent the effective use of study findings. Substantial efforts to improve the reporting of abstracts in these studies are highly warranted.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01482-9.

Comparing the reporting and conduct quality of exercise and pharmacological randomised controlled trials: a systematic review

OBJECTIVE

Evaluate the quality of exercise randomised controlled trial (RCT) reporting and conduct in clinical populations (ie, adults with or at risk of chronic conditions) and compare with matched pharmacological RCTs.

DESIGN

Systematic review.

DATA SOURCES

Embase (Elsevier), PubMed (NLM) and CINAHL (EBSCO).

STUDY SELECTION

RCTs of exercise in clinical populations with matching pharmacological RCTs published in leading clinical, medical and specialist journals with impact factors ≥15.

REVIEW METHODS

Overall RCT quality was evaluated by two independent reviewers using three research reporting guidelines (ie, Consolidated Standards of Reporting Trials (CONSORT; pharmacological RCTs)/CONSORT for non-pharmacological treatments; exercise RCTs), CONSORT-Harms, Template for Intervention Description and Replication) and two risk of bias assessment (research conduct) tools (ie, Cochrane Risk of Bias, Jadad Scale). We compared research reporting and conduct quality within exercise RCTs with matched pharmacological RCTs, and examined factors associated with quality in exercise and pharmacological RCTs, separately.

FINDINGS

Forty-eight exercise RCTs (11 658 patients; median sample n=138) and 48 matched pharmacological RCTs were evaluated (18 501 patients; median sample n=160). RCTs were conducted primarily in cardiovascular medicine (43%) or oncology (31%). Overall quality score (composite of all research reporting and conduct quality scores; primary endpoint) for exercise RCTs was 58% (median score 46 of 80; IQR: 39-51) compared with 77% (53 of 68; IQR: 47-58) in the matched pharmacological RCTs (p≤0.001). Individual quality scores for trial reporting and conduct were lower in exercise RCTs compared with matched pharmacological RCTs (p≤0.03). Factors associated with higher overall quality scores for exercise RCTs were journal impact factor (≥25), sample size (≥152) and publication year (≥2013).

CONCLUSIONS AND RELEVANCE

Research reporting and conduct quality within exercise RCTs is inferior to matched pharmacological RCTs. Suboptimal RCT reporting and conduct impact the fidelity, interpretation, and reproducibility of exercise trials and, ultimately, implementation of exercise in clinical populations.

PROSPERO REGISTRATION NUMBER

CRD42018095033.

Determinants of clinical practice guidelines' utilization for the management of musculoskeletal disorders: a scoping review

Context

Many clinical practice guidelines have been developed for the management of musculoskeletal disorders (MSDs). However, there is a gap between evidence-based knowledge and clinical practice, and reasons are poorly understood. Understanding why healthcare providers use clinical practice guidelines is essential to improve their implementation, dissemination, and adherence.

Aim

To identify determinants of clinical practice guidelines’ utilization by health care providers involved in the assessment and management of MSDs.

Method

A scoping review of the literature was conducted. Three databases were searched from inception to March 2021. Article identification, study design, methodological quality, type of healthcare providers, MSDs, barriers and facilitators associated with guidelines’ utilization were extracted from selected articles.

RESULTS: 8671 citations were retrieved, and 43 articles were selected. 51% of studies were from Europe, and most were quantitative studies (64%) following a cross-sectional design (88%). Almost 80% of articles dealt with low back pain guidelines, and the most studied healthcare providers were general practitioners or physiotherapists. Five main barriers to guideline utilization were expressed by providers: 1) disagreement between recommendations and patient expectations; 2) guidelines not specific to individual patients; 3) unfamiliarity with “non-specific” term, or with the bio psychosocial model of MSDs; 4) time consuming; and 5) heterogeneity in guideline methods. Four main facilitators to guideline utilization were cited: 1) clinician’s interest in evidence-based practice; 2) perception from clinicians that the guideline will improve triage, diagnosis and management; 3) time efficiency; and 4) standardized language.

Conclusion

Identifying modifiable determinants is the first step in developing implementation strategies to improve guideline utilization in clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12891-021-04204-w.

Intervention Reporting of Published Trials Is Insufficient in Orthopaedic Surgery Journals: Application of the Template for Intervention Description and Replication Checklist

Purpose

Outcomes of randomized controlled trials (RCTs) have been shown to influence clinical decision making. Thus, the quality and reliability of these outcomes are essential for both patients and medical care providers. To date, no study has assessed the quality of intervention reporting of RCTs in orthopaedics. The aim of this study was to evaluate the quality of intervention reporting of published RCTs in the field of orthopaedics using the Template for Intervention Description and Replication (TIDieR) checklist.

Methods

In this cross-sectional analysis, we applied the TIDieR checklist to assess the quality of intervention reporting in orthopaedic RCTs. Additionally, we evaluated the TIDieR checklist’s influence on intervention reporting by comparing overall adherence to checklist items in trials published before the TIDieR checklist’s release versus trials published after its release. Finally, we assessed whether certain factors were associated with the quality of intervention reporting.

Results

From a random sample of 300 publications in orthopaedic journals, 175 parallel-arm and cluster RCTs were identified. The overall rate of adherence to TIDieR items was 58.4%. Only 31.4% of orthopaedic RCTs adhered to at least 6 of the 12 TIDieR checklist items, whereas 0% adhered to all 12 items. We found no significant improvement in the quality of intervention reporting in studies published after the TIDieR checklist’s release compared with studies published before its release (P = .97). Additionally, preregistered trials were associated with more complete intervention reporting.

Conclusions

Our results suggest suboptimal reporting of orthopaedic RCT interventions. In addition, the TIDieR checklist’s intended effect—to better the quality of RCT intervention reporting—appears to have fallen short of its goal.

Clinical Relevance

Because outcomes of RCTs are used to guide clinical decision making, it is essential that orthopaedic surgeons and clinical practice guideline panels are equipped with high-quality published research. Increasing the accuracy of intervention reporting may lead to more accurate clinical application. Thus, adoption of more stringent reporting of trial interventions by researchers, authors, and journal editors may improve the quality of orthopaedic research, as well as improve patient outcomes.

Guidelines are needed for studies of pre-treatment HIV drug resistance: a methodological study

BACKGROUND

The expansion of access to antiretroviral therapy (ART) has been accompanied by an increase in pre-treatment drug resistance (PDR). While it is critical to monitor the increasing prevalence of PDR across countries and populations to inform optimal regimen selection, the completeness of reporting is often suboptimal, limiting the interpretation and generalizability of the results. Indeed, there is no formal guidance on how studies investigating the prevalence of drug resistance should be reported. Thus, we sought to determine the completeness of reporting in studies of PDR and the factors associated with sub-optimal reporting to ascertain the need for guidelines.

METHODS

As part of a systematic review on the global prevalence of PDR in key populations (men who have sex with men, sex workers, transgender people, people who inject drugs and people in prisons), we searched 10 electronic databases until January 2019. We extracted information on selected study characteristics useful for interpreting prevalence data. Data were extracted in duplicate. Analyses of variance and correlation were used to explore factors that may explain the number of items reported.

RESULTS

We found 650 studies of which 387 were screened as full text and 234 were deemed eligible. The included studies were published between 1997 and 2019 and included a median of 239 (quartile 1 = 101; quartile 3 = 778) participants. Most studies originated from high-income countries (125/234; 53.0%). Of 23 relevant data items, including study design, setting, participant sociodemographic characteristics, HIV risk factors, type of resistance test conducted, definition of resistance, the mean (standard deviation) number of items reported was 13 (2.2). We found that more items were reported in studies published more recently (r = 0.20; p < 0.002) and in studies at low risk of bias (F [2231] = 8.142; p < 0.001).

CONCLUSIONS

Incomplete reporting in studies on PDR makes characterising levels of PDR in subpopulations across countries challenging. Hence, guidelines are needed to define a minimum set of variables to be included in such studies.

Toward assessing clinical trial publications for reporting transparency

OBJECTIVE

To annotate a corpus of randomized controlled trial (RCT) publications with the checklist items of CONSORT reporting guidelines and using the corpus to develop text mining methods for RCT appraisal.

METHODS

We annotated a corpus of 50 RCT articles at the sentence level using 37 fine-grained CONSORT checklist items. A subset (31 articles) was double-annotated and adjudicated, while 19 were annotated by a single annotator and reconciled by another. We calculated inter-annotator agreement at the article and section level using MASI (Measuring Agreement on Set-Valued Items) and at the CONSORT item level using Krippendorff's α. We experimented with two rule-based methods (phrase-based and section header-based) and two supervised learning approaches (support vector machine and BioBERT-based neural network classifiers), for recognizing 17 methodology-related items in the RCT Methods sections.

RESULTS

We created CONSORT-TM consisting of 10,709 sentences, 4,845 (45%) of which were annotated with 5,246 labels. A median of 28 CONSORT items (out of possible 37) were annotated per article. Agreement was moderate at the article and section levels (average MASI: 0.60 and 0.64, respectively). Agreement varied considerably among individual checklist items (Krippendorff's α= 0.06-0.96). The model based on BioBERT performed best overall for recognizing methodology-related items (micro-precision: 0.82, micro-recall: 0.63, micro-F1: 0.71). Combining models using majority vote and label aggregation further improved precision and recall, respectively.

CONCLUSION

Our annotated corpus, CONSORT-TM, contains more fine-grained information than earlier RCT corpora. Low frequency of some CONSORT items made it difficult to train effective text mining models to recognize them. For the items commonly reported, CONSORT-TM can serve as a testbed for text mining methods that assess RCT transparency, rigor, and reliability, and support methods for peer review and authoring assistance. Minor modifications to the annotation scheme and a larger corpus could facilitate improved text mining models. CONSORT-TM is publicly available at https://github.com/kilicogluh/CONSORT-TM.

Assessing Implementation Strategy Reporting in the Mental Health Literature: A Narrative Review

Inadequate implementation strategy reporting restricts research synthesis and replicability. We explored the implementation strategy reporting quality of a sample of mental health articles using Proctor et al.'s (Implement Sci 8:139, 2013) reporting recommendations. We conducted a narrative review to generate the sample of articles and assigned a reporting quality score to each article. The mean article reporting score was 54% (range 17-100%). The most reported domains were: name (100%), action (82%), target (80%), and actor (67%). The least reported domains included definition (6%), temporality (26%), justification (34%), and outcome (37%). We discuss limitations and provide recommendations to improve reporting.

Inexpensive Systemic Inflammatory Biomarkers in Ovarian Cancer: An Umbrella Systematic Review of 17 Prognostic Meta-Analyses

The association of several inflammation-based biomarkers [lymphocyte-to-monocyte, neutrophil-to-lymphocyte, and platelet-to-lymphocyte ratios (LMR, NLR, and PLR, respectively)] with the survival of epithelial ovarian cancer (EOC) patients has been extensively investigated in several systematic reviews and meta-analyses (MAs) of observational studies. The aim of this umbrella systematic review is to appraise all available results in published MAs that explored the association between these biomarkers and EOC outcomes. An umbrella systematic review of the current evidence for systemic inflammatory biomarkers in the peripheral blood of EOC patients was performed by searching several databases including PubMed/Medline and Web of Science. The quality of the MAs was appraised using the AMSTAR-2 tool as well as other qualitative criteria. The evidence was graded from convincing (Class I) to weak (Class IV). Our umbrella review appraised 17 MAs of retrospective studies (range: 7-16) with a number of enrolled patients ranging from 1,636 to 4,910 patients in each MA. All these MAs demonstrated that pretreatment high NLR and PLR, as well as low LMR, were independent predictors of poor overall survival and progression-free survival in EOC. Nearly all published MAs were conducted by Chinese researchers (16/17) and were redundant in their character. Another issue in these MAs is the absence of prior PROSPERO database registration as well as the earlier exclusion of the gray literature. On the other hand, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Meta-analyses Of Observational Studies in Epidemiology (MOOSE)-based reporting guidelines were used in nine out of the 17 MAs. A good number of MAs have transparently provided funding acknowledgment. The AMSTAR-2-based assessment showed low quality in 11 out of the 17 reviewed MAs. This negative rating was largely due to the absence of critical domains. Finally, all evaluated MAs were rated as Class III or IV (suggestive and weak, respectively). Despite the power of MAs in increasing sampling and precision, the quality of the current non-randomized evidence on this topic is still weak.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO, identifier CRD42020201493.

Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC)

INTRODUCTION

Methodological studies (ie, studies that evaluate the design, conduct, analysis or reporting of other studies in health research) address various facets of health research including, for instance, data collection techniques, differences in approaches to analyses, reporting quality, adherence to guidelines or publication bias. As a result, methodological studies can help to identify knowledge gaps in the methodology of health research and strategies for improvement in research practices. Differences in methodological study names and a lack of reporting guidance contribute to lack of comparability across studies and difficulties in identifying relevant previous methodological studies. This paper outlines the methods we will use to develop an evidence-based tool-the MethodologIcal STudy reportIng Checklist-to harmonise naming conventions and improve the reporting of methodological studies.

METHODS AND ANALYSIS

We will search for methodological studies in the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Embase, MEDLINE, Web of Science, check reference lists and contact experts in the field. We will extract and summarise data on the study names, design and reporting features of the included methodological studies. Consensus on study terms and recommended reporting items will be achieved via video conference meetings with a panel of experts including researchers who have published methodological studies.

ETHICS AND DISSEMINATION

The consensus study has been exempt from ethics review by the Hamilton Integrated Research Ethics Board. The results of the review and the reporting guideline will be disseminated in stakeholder meetings, conferences, peer-reviewed publications, in requests to journal editors (to endorse or make the guideline a requirement for authors), and on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network and reporting guideline websites.

REGISTRATION

We have registered the development of the reporting guideline with the EQUATOR Network and publicly posted this project on the Open Science Framework (www.osf.io/9hgbq).

Quality of reporting of outcomes in trials of therapeutic interventions for pressure injuries in adults: a systematic methodological survey

Randomised controlled trials of therapeutic interventions for pressure injuries should include a clear description of outcomes to increase transparency and replicability and improve the construction of scientific evidence. The objective of this study was to assess the completeness of the descriptions of the outcomes of therapeutic interventions in adults with pressure injury (PI) and factors associated with completeness. This was a systematic methodological survey. The completeness of the outcome was assessed according to five criteria: domain (title), specific measure (technique/instrument used), specific metric, or format of the outcome data of each participant that was used for analysis, aggregation (method data from each group were summarised), and time that was used for analysis. Sixty-eight studies were included for analysis. A total of 265 outcomes were reported, and 46 trials (67.6%) had 73 primary outcomes, which were mainly intermediates/substitutes (78.8%). The main outcome evaluated was the ulcer area reduction (36.6%). Approximately 37.2% of the outcomes were incompletely reported, and the least described element was the data aggregation method (72.8%). Only 48.4% of the outcomes with the specified technique had the same reference or validation. Poor quality of reporting outcomes was associated with studies with an older year of publication and a small sample size, single-center studies, and those sponsored by industry. PI studies use many outcomes, mostly surrogates or intermediates, and some of them are incompletely described.

Systematic Review and STARD Scoring of Renal Cell Carcinoma Circulating Diagnostic Biomarker Manuscripts

Background

No validated molecular biomarkers exist to help guide diagnosis of renal cell carcinoma (RCC) patients. We seek to evaluate the quality of published RCC circulating diagnostic biomarker manuscripts using the Standards for Reporting of Diagnostic Accuracy Studies (STARD) guidelines.

Methods

The phrase “(renal cell carcinoma OR renal cancer OR kidney cancer OR kidney carcinoma) AND circulating AND (biomarkers OR cell free DNA OR tumor DNA OR methylated cell free DNA OR methylated tumor DNA)” was searched in Embase, MEDLINE, and PubMed in March 2018. Relevant manuscripts were scored using 41 STARD subcriteria for a maximal score of 26 points. All tests of statistical significance were 2 sided.

Results

The search identified 535 publications: 27 manuscripts of primary research were analyzed. The median STARD score was 11.5 (range = 7-16.75). All manuscripts had appropriate abstracts, introductions, and distribution of alternative diagnoses. None of the manuscripts stated how indeterminant data were handled or if adverse events occurred from performing the index test or reference standard. Statistically significantly higher STARD scores were present in manuscripts reporting receiver operator characteristic curves (P < .001), larger sample sizes (P = .007), and after release of the original STARD statement (P = .005).

Conclusions

Most RCC circulating diagnostic biomarker manuscripts poorly adhere to the STARD guidelines. Future studies adhering to STARD guidelines may address this unmet need.

More pilot trials could plan to use qualitative data: a meta-epidemiological study

Background

Pilot trials often use quantitative data such as recruitment rate and retention rate to inform the design and feasibility of a larger trial. However, qualitative data such as patient, healthcare provider, and research staff perceptions of an intervention may also provide insights for a larger trial.

Methods

As part of a larger study investigating the reporting of progression criteria in pilot studies, we sought to determine how often pilot studies planned to use qualitative data to inform the design and feasibility of a larger trial and the factors associated with plans to use qualitative data. We searched for protocols of pilot studies of randomized trials in PubMed between 2013 and 2017.

Results

We included 227 articles. Only 92 (40.5%; 95% confidence interval [CI] 34.1–47.2) reported plans to collect qualitative data. The factors associated with collecting qualitative data were large studies (defined as sample size ≥ 60; adjusted odds ratio [aOR] 2.77; 95% CI 1.47–5.23; p = 0.002) and studies from Europe (aOR 3.86; 95% CI 1.68–8.88; p = 0.001) compared to North America and the rest of the world. Pilot trials with pharmacological interventions were less likely to plan to collect qualitative data (aOR 0.20; 95% CI 0.07–0.58; p = 0.003).

Conclusions

Qualitative data is not used enough in pilot trials. Large pilot trials, pilot trials from Europe, and pilot trials of non-pharmacological interventions are more likely to plan for qualitative data.

A tutorial on methodological studies: the what, when, how and why

BACKGROUND

Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

MAIN BODY

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

CONCLUSION

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Has the reporting quality of published randomised controlled trial protocols improved since the SPIRIT statement? A methodological study

OBJECTIVES

To determine the reporting quality of published randomised controlled trial (RCT) protocols before and after the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement (2013), and any association with author, trial or journal factors.

DESIGN

Methodological study.

DATA SOURCES

MEDLINE, Embase and CENTRAL were electronically searched using optimised search strategies.

ELIGIBILITY CRITERIA

Protocols written for an RCT of living humans, published in full text in a peer-reviewed journal and published in the English language.

MAIN OUTCOME

Primary outcome was the overall proportion of checklist items which were adequately reported in RCT protocols published before and after the SPIRIT statement.

RESULTS

300 RCT protocols were retrieved; 150 from the period immediately before the SPIRIT statement (9 July 2012 to 28 December 2012) and 150 from a recent period after the SPIRIT statement (25 January 2019 to 20 March 2019). 47.9% (95% CI, 46.5% to 49.3%) of checklist items were adequately reported in RCT protocols before the SPIRIT statement and 56.7% (95% CI, 54.9% to 58.5%) after the SPIRIT statement. This represents an 8.8% (95% CI, 6.6% to 11.1%; p<0.0001) mean improvement in the overall proportion of checklist items adequately reported since the SPIRIT statement. While 40% of individual checklist items had a significant improvement in adequate reporting after the SPIRIT statement, 11.3% had a significant deterioration and there were no RCT protocols in which all individual checklist items were complete. The factors associated with higher reporting quality of RCT protocols in multiple regression analysis were author expertise or experience in epidemiology or statistics, multicentre trials, longer protocol word length and publicly reported journal policy of compliance with the SPIRIT statement.

CONCLUSION

The overall reporting quality of RCT protocols has significantly improved since the SPIRIT statement, although a substantial proportion of individual checklist items remain poorly reported. Continued and concerted efforts are required by journals, editors, reviewers and investigators to improve the completeness and transparency of RCT protocols.

Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial

OBJECTIVE

To evaluate the impact of an editorial intervention to improve completeness of reporting of reports of randomised trials.

DESIGN

Randomised controlled trial (RCT).

SETTING

BMJ Open's quality improvement programme.

PARTICIPANTS

24 manuscripts describing RCTs.

INTERVENTIONS

We used an R Shiny application to randomise manuscripts (1:1 allocation ratio, blocks of 4) to the intervention (n=12) or control (n=12) group. The intervention was performed by a researcher with expertise in the content of the Consolidated Standards of Reporting Trials (CONSORT) and consisted of an evaluation of completeness of reporting of eight core CONSORT items using the submitted checklist to locate information, and the production of a report containing specific requests for authors based on the reporting issues found, provided alongside the peer review reports. The control group underwent the usual peer review.

OUTCOMES

The primary outcome is the number of adequately reported items (0-8 scale) in the revised manuscript after the first round of peer review. The main analysis was intention-to-treat (n=24), and we imputed the scores of lost to follow-up manuscripts (rejected after peer review and not resubmitted). The secondary outcome is the proportion of manuscripts where each item was adequately reported. Two blinded reviewers assessed the outcomes independently and in duplicate and solved disagreements by consensus. We also recorded the amount of time to perform the intervention.

RESULTS

Manuscripts in the intervention group (mean: 7.01; SD: 1.47) were more completely reported than those in the control group (mean: 5.68; SD: 1.43) (mean difference 1.43, 95% CI 0.31 to 2.58). We observed the main differences in items 6a (outcomes), 9 (allocation concealment mechanism), 11a (blinding) and 17a (outcomes and estimation). The mean time to perform the intervention was 87 (SD 42) min.

CONCLUSIONS

We demonstrated the benefit of involving a reporting guideline expert in the editorial process. Improving the completeness of RCTs is essential to enhance their usability.

TRIAL REGISTRATION NUMBER

NCT03751878.

Community health workers involvement in preventative care in primary healthcare: a systematic scoping review

OBJECTIVES

To review effective models of community health worker (CHW) involvement in preventive care for disadvantaged culturally and linguistically diverse (CALD) patients in primary healthcare (PHC) that may be applicable to the Australian context.

DESIGN

Systematic scoping review.

DATA SOURCES

The studies were gathered through searching Medline, EMBASE, EMCARE, PsycINFO, CINAHL and online portals of relevant organisations.

ELIGIBILITY CRITERIA

All selected studies were original research studies which essentially evaluated preventive intervention undertake by CHWs in PHC. The intervened population were adults with or without diagnosed chronic health disease, culturally and linguistically diverse, or vulnerable due to geographic, economic and/or cultural characteristics that impede or compromise their access to healthcare.

DATA EXTRACTION AND SYNTHESIS

Data extraction was undertaken systematically in an excel spreadsheet while the findings were synthesised in a narrative manner. The quality appraisal of the selected studies was performed using effective public health practice project quality assessment tool.

RESULTS

A total of 1066 articles were identified during the initial search of six bibliographic databases. After screening the title, abstract and full text, 37 articles met the selection and methodological criteria and underwent data extraction. A high-quality evidence-base supporting the positive impact of CHWs supporting patients' access to healthcare and influencing positive behaviour change was found. Positive impacts of CHW interventions included improvements in clinical disease indicators, screening rates and behavioural change. Education-focused interventions were more effective in improving patient behaviour, whereas navigation interventions were most effective in improving access to services. Implementation was enhanced by cultural and linguistic congruence and specific training of CHWs in the intervention but reduced by short duration interventions, dropouts and poor adherence of patients.

CONCLUSION

The evidence generated from this systematic scoping review demonstrates the contribution of CHWs to improving access to preventive care for patients from CALD and disadvantaged backgrounds by providing both education and navigational interventions. More research is needed on CHW training and the incorporation of CHWs into primary health care (PHC) teams.

How about the reporting quality of case reports in nursing field?

BACKGROUND

As a significantly important part of clinical practice, the professional nursing process can be advanced in many ways. Despite the fact that case reports are regarded to be of a lower quality grade in the hierarchy of evidence, one of the principles of evidence-based medicine is that decision-making should be based on a systematic summary of evidence. However, the evidence on the reporting characteristics of case reports in the nursing field is deficient.

AIM

To use the CARE guidelines to assess reporting quality and factors influencing the quality of case reports in the nursing field.

METHODS

Nursing science citation indexed (SCI-indexed) journals were identified from the professional website. Each of the identified journals was searched on their website for articles published before December 2017. Twenty-one sub-items on the CARE checklist were recorded as “YES”, “PARTLY”, or “NO” according to information reported by the included studies. The responses were assigned corresponding scores of 1, 0.5, and 0, respectively. The overall score was the sum of the 21 sub-items and was defined as “high” (more than 15), “medium” (10.5 to 14.5), and “low” (less than 10). The means, standard deviations, odds ratios (OR), and the associated 95% confidence interval (CI) were determined using Stata 12.0 software.

RESULTS

Ultimately, 184 case reports from 16 SCI-indexed journals were identified, with overall scores ranging from 6.5 to 18 (mean = 13.6 ± 2.3). Of the included case reports, 10.3% were regarded low-quality, 52.7% were considered middle-quality, and 37% were regarded high-quality. There were statistical differences in the mean overall scores of the included case reports with funding versus those without funding (14.2 ± 1.7 vs 13.6 ± 2.4, respectively; P = 0.4456) and journal impact factor < 1.8 versus impact factor ≥ 1.8 (13.3 ± 2.3 vs 13.6 ± 2.4, respectively; P = 0.4977). Five items from the CARE guidelines, 5a (Patient), 6 (Clinical findings), 8c (Diagnostic reasoning), 9 (Therapeutic intervention), and 11d (The main take-away lessons) were well-reported (Reporting rate more than 90%) in most of the included case reports. However, only three items, 2 (Keywords, OR = 0.42, 95%CI: 0.19-0.92, P = 0.03), 4 (Introduction, OR = 0.35, 95%CI: 0.15-0.83, P = 0.017), and 11b (The relevant medical literature, OR = 0.19, 95%CI: 0.06-0.56, P = 0.003) were considered better-reported after the CARE guidelines published in 2013.

CONCLUSION

The reporting quality of case reports in the nursing field apparently has not improved since the publication of the CARE guidelines.

The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study

Introduction

Pilot and feasibility trials are conducted to determine feasibility or to collect information that would inform the design of a larger definitive trial. Clear progression criteria are required to determine if a definitive or main trial is feasible and how it should be designed. We sought to determine how often progression criteria are reported and the associated factors.

Methods

We conducted a methodological review of protocols for pilot randomised trials published in three journals that publish research protocols (BMJ Open, Trials, Pilot and Feasibility Studies), using a PubMed search (2013–2017). We extracted bibliometric information including the country in which the study was conducted, source of funding, type of intervention, use of a primary feasibility outcome, sample size reporting, and justification. We used generalised linear models to determine the factors associated with reporting progression criteria.

Results

Our search retrieved 276 articles, of which 49 were not eligible. We included 227 articles. Overall, 45/227 (19.8%; 95% confidence interval [CI] 14.8–25.6) reported progression criteria. Protocols published in more recent years were significantly associated with higher odds of reporting progression criteria (adjusted odds ratio [aOR] 1.40; 95% CI 1.03–1.92; p = 0.034). Pilot trials from Europe (aOR 0.19; 95% CI 0.08–0.48; p < 0.001) and the rest of the world (aOR 0.05; 95% CI 0.01–0.18; p < 0.003) compared to North America were significantly associated with lower odds of reporting progression criteria. Journal, source of funding, sample size, intervention type, and having a primary outcome related to feasibility were not significantly associated with reporting progression criteria.

Conclusion

Progression criteria are not often explicitly stated in protocols of pilot trials leaving room for varied interpretation of findings. The development of formal guidance for progression criteria in protocols of pilot trials is warranted.

Inadequate Reporting of Analytical Characteristics of Biomarkers Used in Clinical Research: A Threat to Interpretation and Replication of Study Findings

BACKGROUND

Analytical characteristics of methods to measure biomarkers determine how well the methods measure what they claim to measure. Transparent reporting of analytical characteristics allows readers to assess the validity and generalizability of clinical studies in which biomarkers are used. Our aims were to assess the reporting of analytical characteristics of biomarkers used in clinical research and to evaluate the extent of reported characterization procedures for assay precision.

METHODS

We searched 5 medical journals (Annals of Internal Medicine, JAMA: The Journal of the American Medical Association, The Lancet, The New England Journal of Medicine, and PLOS Medicine) over a 10-year period for the term "biomarker" in the full-text field. We included studies in which biomarkers were used for inclusion/exclusion of study participants, for patient classification, or as a study outcome. We tabulated the frequencies of reporting of 11 key analytical characteristics (such as analytical accuracy of test results) in the included studies.

RESULTS

A total of 544 studies and 1299 biomarker uses met the inclusion criteria. No information on analytical characteristics was reported for 67% of the biomarkers. For 65 biomarkers (3%), ≥4 characteristics were reported (range, 4-8). The manufacturer of the measurement procedure could not be determined for 688 (53%) of the 1299 biomarkers. The extent of assessments of assay imprecision, when reported, did not meet expectations for clinical use of biomarkers.

CONCLUSIONS

Reporting of the analytical performance of biomarker measurements is variable and often absent from published clinical studies. We suggest that readers need fuller reporting of analytical characteristics to interpret study results, assess generalizability of conclusions, and compare results among clinical studies.

A survey exploring biomedical editors' perceptions of editorial interventions to improve adherence to reporting guidelines

Background: Improving the completeness of reporting of biomedical research is essential for improving its usability. For this reason, hundreds of reporting guidelines have been created in the last few decades but adherence to these remains suboptimal. This survey aims to inform future evaluations of interventions to improve adherence to reporting guidelines. In particular, it gathers editors' perceptions of a range of interventions at various stages in the editorial process.   Methods: We surveyed biomedical journal editors that were knowledgeable about this topic. The questionnaire included open and closed questions that explored (i) the current practice of their journals, (ii) their perceptions of the ease of implementation and the potential effectiveness of different interventions, (iii) the barriers and facilitators associated with these interventions, and (iv) suggestions for future interventions and incentives. Results: Of the 99 editors invited, 24 (24%) completed the survey. Involving trained editors or administrative staff was deemed the potentially most effective intervention but, at the same time, it was considered moderately difficult to implement due to logistic and resource issues. Participants believed that checking adherence to guidelines goes beyond the role of peer reviewers and could decrease the overall quality of reviews. Journals incentivising adherence, and publishers and medical institutions encouraging journals to adopt strategies to boost adherence were two recurrent themes. Conclusions: Further evaluation of interventions are required. These evaluations could take into account the points raised in this survey.

A survey exploring biomedical editors' perceptions of editorial interventions to improve adherence to reporting guidelines

Background: Improving the completeness of reporting of biomedical research is essential for improving its usability. For this reason, hundreds of reporting guidelines have been created in the last few decades but adherence to these remains suboptimal. This survey aims to inform future evaluations of interventions to improve adherence to reporting guidelines. In particular, it gathers editors' perceptions of a range of interventions at various stages in the editorial process.   Methods: We surveyed biomedical journal editors that were knowledgeable about this topic. The questionnaire included open and closed questions that explored (i) the current practice of their journals, (ii) their perceptions of the ease of implementation and the potential effectiveness of different interventions, (iii) the barriers and facilitators associated with these interventions, and (iv) suggestions for future interventions and incentives. Results: Of the 99 editors invited, 24 (24%) completed the survey. Involving trained editors or administrative staff was deemed the potentially most effective intervention but, at the same time, it was considered moderately difficult to implement due to logistic and resource issues. Participants believed that checking adherence to guidelines goes beyond the role of peer reviewers and could decrease the overall quality of reviews. Journals incentivising adherence, and publishers and medical institutions encouraging journals to adopt strategies to boost adherence were two recurrent themes. Conclusions: Further evaluation of interventions are required. These evaluations could take into account the points raised in this survey.

Assessing Implementation Strategy Reporting in the Mental Health Literature: A Narrative Review

Inadequate implementation strategy reporting restricts research synthesis and replicability. We explored the implementation strategy reporting quality of a sample of mental health articles using Proctor et al.'s (Implement Sci 8:139, 2013) reporting recommendations. We conducted a narrative review to generate the sample of articles and assigned a reporting quality score to each article. The mean article reporting score was 54% (range 17-100%). The most reported domains were: name (100%), action (82%), target (80%), and actor (67%). The least reported domains included definition (6%), temporality (26%), justification (34%), and outcome (37%). We discuss limitations and provide recommendations to improve reporting.

A systematic methodological review of non-randomised interventional studies of elective ventral hernia repair: clear definitions and a standardised minimum dataset are needed

BACKGROUND

Ventral hernias (VHs) often recur after surgical repair and subsequent attempts at repair are especially challenging. Rigorous research to reduce recurrence is required but such studies must be well-designed and report representative and comprehensive outcomes.

OBJECTIVE

We aimed to assesses methodological quality of non-randomised interventional studies of VH repair by systematic review.

METHODS

We searched the indexed literature for non-randomised studies of interventions for VH repair, January 1995 to December 2017 inclusive. Each prospective study was coupled with a corresponding retrospective study using pre-specified criteria to provide matched, comparable groups. We applied a bespoke methodological tool for hernia trials by combining relevant items from existing published tools. Study introduction and rationale, design, participant inclusion criteria, reported outcomes, and statistical methods were assessed.

RESULTS

Fifty studies (17,608 patients) were identified: 25 prospective and 25 retrospective. Overall, prospective studies scored marginally higher than retrospective studies for methodological quality, median score 17 (IQR: 14-18) versus 15 (IQR 12-18), respectively. For the sub-categories investigated, prospective studies achieved higher median scores for their, 'introduction', 'study design' and 'participants'. Surprisingly, no study stated that a protocol had been written in advance. Only 18 (36%) studies defined a primary outcome, and only 2 studies (4%) described a power calculation. No study referenced a standardised definition for VH recurrence and detection methods for recurrence varied widely. Methodological quality did not improve with publication year or increasing journal impact factor.

CONCLUSION

Currently, non-randomised interventional studies of VH repair are methodologically poor. Clear outcome definitions and a standardised minimum dataset are needed.

A methodological survey on reporting of pilot and feasibility trials for physiotherapy interventions: a study protocol

INTRODUCTION

Pilot and feasibility trials aim to test whether a full trial can be conducted or if any procedures must be changed for the full trial. Pilot trials must be reported in a transparent, accurate and complete way. In this report, we present a protocol for a methodological survey with the following aims: (1) to determine the percentage of physiotherapy trial reports which claim to be pilot or feasibility trials that evaluate feasibility, (2) to determine the aspect of feasibility evaluated in the primary objectives of the pilot or feasibility trials, (3) to describe the completeness of reporting of abstracts and full articles of pilot or feasibility trials using the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials and (4) to investigate factors associated with completeness of reporting of pilot or feasibility trials.

METHODS AND ANALYSIS

Reports of randomised controlled trials indexed in the Physiotherapy Evidence Database (PEDro) that claim to be pilot or feasibility trials and published in 2011-2017 will be included. Two independent reviewers will confirm eligibility and classify the aspect of feasibility being evaluated in the objectives of the included pilot or feasibility trials. Completeness of reporting of both the abstract and the full article will be evaluated using the CONSORT extension to randomised pilot and feasibility trials. The primary analysis will be a descriptive analysis about the reporting quality of abstracts and full texts of pilot and feasibility trials. We will use generalised estimating equation analysis to explore factors associated with completeness of reporting.

ETHICS AND DISSEMINATION

The results of this study will be disseminated by presentation at conferences and will be submitted for publication in a peer-reviewed journal. Ethical approval is not necessary for this study.

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

Uniformity in measuring adherence to reporting guidelines: the example of TRIPOD for assessing completeness of reporting of prediction model studies

To promote uniformity in measuring adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, a reporting guideline for diagnostic and prognostic prediction model studies, and thereby facilitate comparability of future studies assessing its impact, we transformed the original 22 TRIPOD items into an adherence assessment form and defined adherence scoring rules. TRIPOD specific challenges encountered were the existence of different types of prediction model studies and possible combinations of these within publications. More general issues included dealing with multiple reporting elements, reference to information in another publication, and non-applicability of items. We recommend our adherence assessment form to be used by anyone (eg, researchers, reviewers, editors) evaluating adherence to TRIPOD, to make these assessments comparable. In general, when developing a form to assess adherence to a reporting guideline, we recommend formulating specific adherence elements (if needed multiple per reporting guideline item) using unambiguous wording and the consideration of issues of applicability in advance.

Quality of reporting of outcomes in trials of therapeutic interventions for pressure ulcers in adults: a protocol for a systematic survey

Pressure ulcers (PUs) have a high incidence, especially in hospital units. Randomised clinical trials (RCTs) of therapeutic interventions for PU should include a clear description of the outcomes and results to enhance transparency and replicability.

OBJECTIVES

The primary objective of this study is to assess the completeness of the descriptions of the outcomes of therapeutic interventions in RCTs in adult patients with PU. The secondary objectives are to evaluate the types of reported primary outcomes, measurement methods or tools used to evaluate the outcomes and the results of reported outcomes.

METHODS

We will conduct a systematic survey of RCTs published from January 2006 to April 2018. The selection process of the studies will be done in two stages of screening: title and abstract, and full text revision, always by two researchers independently. The completeness of the outcome will be assessed according to five criteria: domain (outcome title), specific measurement or technique/instrument used, specific metric or format of the outcome data that will be used for analysis, method of aggregation (how data from each group will be summarised) and time-points that will be used for analysis. The quality of the results of the outcome will be classified as either complete, incomplete or unreported. We will conduct a descriptive analysis of the number, type and degrees of outcome specification in the included RCTs. The frequency of categories in each domain of the outcomes will also be reported. The median and IQR will be estimated for each element of the specified outcome (out of five).

ETHICS AND DISSEMINATION

This will be the first systematic assessment of the outcomes of therapeutic interventions used for pressure ulcers. After completion, this review will be submitted to a peer-reviewed journals.

Primary Health Care Models Addressing Health Equity for Immigrants: A Systematic Scoping Review

To examine two healthcare models, specifically "Primary Medical Care" (PMC) and "Primary Health Care" (PHC) in the context of immigrant populations' health needs. We conducted a systematic scoping review of studies that examined primary care provided to immigrants. We categorized studies into two models, PMC and PHC. We used subjects of access barriers and preventive interventions to analyze the potential of PMC/PHC to address healthcare inequities. From 1385 articles, 39 relevant studies were identified. In the context of immigrant populations, the PMC model was found to be more oriented to implement strategies that improve quality of care of the acute and chronically ill, while PHC models focused more on health promotion and strategies to address cultural and access barriers to care, and preventive strategies to address social determinants of health. Primary Health Care models may be better equipped to address social determinants of health, and thus have more potential to reduce immigrant populations' health inequities.

The assessment of the quality of randomized controlled trials published in Indian medical journals

Aim:

In this retrospective cross-sectional study, we sought to evaluate if the published randomized controlled trials (RCTs) reported in the year 2017 among the Indian medical journals (IMJs) complied with the Consolidated Standards of Reporting Trials (CONSORT) guidelines and identify domains where reporting could be improved.

Methods:

A literature search was performed using PubMed and Google Scholar to identify all the IMJs that published RCTs in the year 2017. In the archives of the identified journals, the number of published RCTs was identified and the full text was obtained. We selected articles that stated RCT in abstract and title and that evaluated the safety and efficacy of all therapeutic and preventive interventions.

Results:

A total of seven IMJs comprising of the Indian Journal of Anesthesia, Indian Journal of Dermatology, Venereology and Leprology, Indian Journal of Pharmacology, Indian Journal of Ophthalmology, Journal of Obstetrics and Gynaecology, Journal of Pharmacology and Pharmacotherapeutics, and Indian Journal of Medical and Pediatric Oncology that published a total of 84 RCTs were included. The mean compliance score of all the RCTs was 13.7 ± 2.66 (57%). Most RCTs had serious reporting deficiencies in the methodology and result sections.

Discussion:

In spite of journals making it mandatory for prospective authors to comply with the CONSORT guidelines, it is intriguing that there continues to be significant lacunae in reporting RCTs adequately in most IMJs.

Conclusion:

There is an urgent need to impart training to the medical community of our country in clinical research methods and reporting of RCTs.

A descriptive analysis of non-Cochrane child-relevant systematic reviews published in 2014

BACKGROUND

Consumers, clinicians, policymakers and researchers require high quality evidence to guide decision-making in child health. Though Cochrane systematic reviews (SRs) are a well-established source of evidence, little is known about the characteristics of non-Cochrane child-relevant SRs. To complement published descriptions of Cochrane SRs, we aimed to characterize the epidemiologic, methodological, and reporting qualities of non-Cochrane child-relevant SRs published in 2014.

METHODS

English-language child-relevant SRs of quantitative primary research published outside the Cochrane Library in 2014 were eligible for this descriptive analysis. A research librarian searched MEDLINE, CINAHL, Web of Science, and PubMed in August 2015. A single reviewer screened articles for inclusion; a second verified the excluded studies. Reviewers extracted: general characteristics of the review; included study characteristics; methodological approaches. We performed univariate analyses and presented the findings narratively.

RESULTS

We identified 1598 child-relevant SRs containing a median (IQR) 19 (11, 33) studies. These originated primarily from high-income countries (n = 1247, 78.0%) and spanned 47 of the 53 Cochrane Review Groups. Most synthesized therapeutic (n = 753, 47.1%) or epidemiologic (n = 701, 43.8%) evidence. Though 39.3% (n = 628) of SRs included evidence related to children only, few were published in pediatric-specific journals (n = 283, 17.7%). Reporting quality seemed poor based on the items we assessed; few reviews mentioned an a-priori protocol (n = 246, 15.4%) or registration (n = 111, 6.9%), and only 23.4% (n = 374) specified a primary outcome. Many SRs relied solely on evidence from non-RCTs (n = 796, 49.8%). Less than two-thirds (n = 953, 59.6%) appraised the quality of included studies and assessments of the certainty of the body of evidence were rare (n = 102, 6.4%).

CONCLUSIONS

Child-relevant Cochrane SRs are a known source of high quality evidence in pediatrics. There exists, however, an abundance of evidence from non-Cochrane SRs that may be complementary. Our findings show that high-quality non-Cochrane SRs may not be practical nor easy for knowledge users to find. Improvements are needed to ensure that evidence syntheses published outside of the Cochrane Library adhere to the high standard of conduct and reporting characteristic of Cochrane SRs.

Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update

PURPOSE

Reporting guidelines (eg, Consolidated Standards of Reporting Trials [CONSORT] statement) are intended to improve reporting standards and enhance the transparency and reproducibility of research findings. Despite accessibility of such guidelines, researchers are not required to adhere to them. Our goal was to determine the current status of reporting quality in the medical literature and examine whether adherence of reporting guidelines has improved since the inception of reporting guidelines.

MATERIALS AND METHODS

Eight reporting guidelines, such as CONSORT, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), Quality of Reporting of Meta-analysis (QUOROM), STAndards for Reporting of Diagnostic accuracy (STARD), Animal Research: Reporting In Vivo Experiments (ARRIVE), Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) were examined. Our inclusion criteria included reviews published between January 1996 to September 2016 which investigated the adherence to reporting guidelines in the literature that addressed clinical trials, systematic reviews, observational studies, meta-analysis, diagnostic accuracy, economic evaluations, and preclinical animal studies that were in English. All reviews were found on Web of Science, Excerpta Medical Database (EMBASE), MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL).

RESULTS

Among the general searching of 26,819 studies by using the designed searching method, 124 studies were included post screening. We found that 87.9% of the included studies reported suboptimal adherence to reporting guidelines. Factors associated with poor adherence included non-pharmacological interventions, year of publication, and trials concluding with significant results. Improved adherence was associated with better study designs such as allocation concealment, random sequence, large sample sizes, adequately powered studies, multiple authorships, and being published in journals endorsing guidelines.

CONCLUSION

We conclude that the level of adherence to reporting guidelines remains suboptimal. Endorsement of reporting guidelines by journals is important and recommended.

Quality of pilot trial abstracts in heart failure is suboptimal: a systematic survey

BACKGROUND

Pilot trials are miniature researches carried out with the sole aim of acting as the precursor for larger more definitive studies. Abstracts are used to summarize and introduce the findings to the reading audience. There is substantive empirical evidence showing that abstracts, despite their important roles, are not informative enough, lacking the necessary details. This systematic survey was designed to assess the quality of reporting of heart failure pilot trial abstracts. The quality of reporting was defined as the completeness of reporting based on adherence to the CONSORT extension for reporting of pilot trial abstracts. We also identified factors associated with reporting quality.

METHODS

We searched MEDLINE (PubMed), Cochrane Controlled Trials Register, Scopus, and African-wide information databases for abstracts from heart failure pilot trials in humans published from 1 January 1990 to 30 November 2016. These were assessed to determine the extent of adherence to CONSORT extension checklist for reporting of abstracts of pilot trials. We screened identified studies for inclusion based on title and abstract. Data were independently extracted by two reviewers using the checklist. We used regression analysis to assess the association between completeness of reporting (measured as the number of items in the CONSORT extension checklist for reporting of abstracts in pilot trials contained in each abstract) and factors influencing the quality of the reports.

RESULTS

Two hundred and twenty-eight (228) articles were retrieved, of which 92 met the inclusion criteria. The mean CONSORT extension score was 8.3/16 (standard deviation 1.7); the least reported items were the source of funding (1% [1/92]), trial registration (13% [12/92]), randomization sequence (13% [12/92]), number randomized to each arm (16% [15/92]), and number analyzed in each arm (16% [15/92]). Multivariable regression analysis showed that pharmacological intervention pilot trials [incidence rate ratio (IRR) = 0.88; 95% confidence interval (CI), 0.81-0.97] were significantly associated with better reporting. Other factors such as structured abstract (IRR = 1.10; 95% CI, 0.99-1.23) and CONSORT endorsement (IRR = 1.10; 95% CI, 0.99-1.23) only showed minimal relationship with better reporting quality.

CONCLUSION

The quality of reporting of abstracts of heart failure pilot trials was suboptimal. Pharmacological intervention was significantly associated with better reporting. These findings are consistent with previous research on reporting of trials.

Reporting quality of randomised controlled trial abstracts on age-related macular degeneration health care: a cross-sectional quantification of the adherence to CONSORT abstract reporting recommendations

OBJECTIVE

To assess the reporting quality of randomised controlled trial (RCT) abstracts on age-related macular degeneration (AMD) healthcare, to evaluate the adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement's recommendations on minimum abstract information and to identify journal characteristics associated with abstract reporting quality.

DESIGN

Cross-sectional evaluation of RCT abstracts on AMD healthcare.

METHODS

A PubMed search was implemented to identify RCT abstracts on AMD healthcare published in the English language between January 2004 and December 2013. Data extraction was performed by two parallel readers independently by means of a documentation format in accordance with the 16 items of the CONSORT checklist for abstracts. The total number of criteria fulfilled by an abstract was derived as primary endpoint of the investigation; incidence rate ratios (IRRs) with unadjusted 95% CI were estimated by means of multiple Poisson regression to identify journal and article characteristics (publication year, multicentre design, structured abstract recommendations, effective sample size, effective abstract word counts and journal impact factor) possibly associated with the total number of fulfilled items.

STUDY CHARACTERISTICS

136 of 673 identified abstracts (published in 36 different journals) fulfilled all eligibility criteria.

RESULTS

The median number of fulfilled items was 7 (95% CI 7 to 8). No abstract reported all 16 recommended items; the maximum total number was 14, the minimum 3 of 16 items. Multivariate analysis only demonstrated the abstracts' word counts as being significantly associated with a better reporting of abstracts (Poisson regression-based IRR 1.002, 95% CI 1.001 to 1.003).

CONCLUSIONS

Reporting quality of RCT abstracts on AMD investigations showed a considerable potential for improvement to meet the CONSORT abstract reporting recommendations. Furthermore, word counts of abstracts were identified as significantly associated with the overall abstract reporting quality.

Enhancing research publications and advancing scientific writing in health research collaborations: sharing lessons learnt from the trenches

BACKGROUND

Disseminating research protocols, processes, methods or findings via peer-reviewed publications has substantive merits and benefits to various stakeholders.

PURPOSE

In this article, we share strategies to enhance research publication contents (ie, what to write about) and to facilitate scientific writing (ie, how to write) in health research collaborations.

METHODS

Empirical experience sharing.

RESULTS

To enhance research publication contents, we encourage identifying appropriate opportunities for publications, publishing protocols ahead of results papers, seeking publications related to methodological issues, considering justified secondary analyses, and sharing academic process or experience. To advance writing, we suggest setting up scientific writing as a goal, seeking an appropriate mentorship, making full use of scientific meetings and presentations, taking some necessary formal training in areas such as effective communication and time and stress management, and embracing the iterative process of writing.

CONCLUSION

All the strategies we share are dependent upon each other; and they advocate gradual academic accomplishments through study and training in a "success-breeds-success" way. It is expected that the foregoing shared strategies in this paper, together with other previous guidance articles, can assist one with enhancing research publications, and eventually one's academic success in health research collaborations.

Enhancing primary reports of randomized controlled trials: Three most common challenges and suggested solutions

Evidence from a well-designed randomized controlled trial (RCT) is generally considered to be the gold standard that can inform clinical practice and guide decision-making. However, several deficiencies in the reporting of RCTs have frequently been identified, including incomplete, selective, and biased or inconsistent reporting. Such suboptimal reporting may lead to irreproducible results, substantial waste of resources, impaired study validity, erosion of public trust in science, and a high risk of research misconduct. In this article, we present an overview of the reporting of RCTs in the biomedical literature with a focus on the three most common reporting problems: (i) lack of adherence to reporting guidelines, (ii) inconsistencies between trial protocols or registrations and full reports, and (iii) inconsistencies between abstracts and their corresponding full reports. Unsatisfactory levels of adherence to guidelines and frequent inconsistencies between protocols or registrations and full reports, and between abstracts and full reports, were consistently found in various biomedical research fields. A variety of factors were found to be associated with these reporting challenges. Improved reporting can build public trust and credibility of science, save resources, and enhance the ethical integrity of research. Therefore, joint efforts from the various sectors of the biomedical community (researchers, journal editors and reviewers, educators, healthcare providers, and other research consumers) are needed to reduce and reverse the current suboptimal state of RCT reporting in the literature.