Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials of Orthopaedic Surgical Interventions

Abstracts of systematic reviews and meta-analyses investigating meniscal root repair exhibit a high prevalence of reporting bias

BACKGROUND

To present rates of reporting bias in systematic reviews and meta-analyses investigating meniscal root repair.

METHODS

In this systematic review, PubMed, Scopus and Web of Science databases were queried for studies that investigated meniscal root tears treated with root repair. Included studies were systematic reviews and/or meta-analyses published in peer-reviewed journals in the English language with available full-texts. Each abstract was graded in a binary fashion for 15 most severe types of spin. Fisher's exact test was used to determine if spin varied significantly by year, journal, level of evidence, funding source, or A Measurement Tool to Assess Systematic Reviews Version 2 (AMSTAR 2) confidence category.

RESULTS

Twenty studies were included. All abstracts exhibited spin with a maximum of eight types of spin. The most prevalent categories of spin were "Misleading Reporting" (n = 18), "Inappropriate Extrapolation" (n = 13), and "Misleading Interpretation" (n = 12). There were significant associations between external funding and spin types: 5 ("The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies") (p = 0.019), 9 ("Conclusion claims the beneficial effect of the experimental treatment despite reporting bias") (p < 0.001), and 15 ("Conclusion extrapolates the review's findings to a different population or setting") (p = 0.049). AMSTAR 2 confidence rating was either "low" (n = 2) or "critically low" (n = 18) in all 20 studies.

CONCLUSION

This study demonstrated a high prevalence of reporting bias in the abstracts of systematic reviews and meta-analyses investigating meniscal root repair, with significant associations with external funding.

Protocol publication rate and comparison between article, registry and protocol in RCTs

BACKGROUND

Increasing transparency in clinical research is crucial to avoid misleading conclusions. Registering clinical trials prior to participant enrolment is mandatory, and the publication of trial protocols could further enhance transparency. However, the impact of protocol publication on primary outcomes (PO) and sample sizes (SS) remains unclear. This study aimed to determine the rates of trial protocol publication and registration for a sample of randomized controlled trials (RCTs) and to compare the consistency of published and registered PO and SS.

METHODS

A search was conducted in MEDLINE via PubMed® for RCT reports indexed in May and June 2023 across various medical specialties, focusing on general and high-impact factor journals. Data were extracted regarding trial registration, protocol publication, and comparisons were made between PO and SS in articles, registries, and published protocols.

RESULTS

Out of 1119 references, 589 (52.6%) were RCTs. The corresponding protocol was published for 146 RCTs (24.8%) including 40 over 140 (28.6%) (6 without end date available) after the trial had ended. Sixty-two (42.4%) protocols were published before the trial conclusion, with no significant differences between PO and SS in published protocols and their corresponding articles. Five hundred and twenty-eight (89.6%) RCTs were registered, 225 over 510 (44%) were registered before the study start with no differences in PO and SS between article and registry. Articles published in generalist or high impact factor journals were associated with higher frequencies of published protocols and trial registration and a lower frequency of difference in PO and SS between articles, registries, and published protocols.

CONCLUSIONS

While publishing trial protocols may enhance transparency in peer-review process, the initial registered protocol alone appears sufficient for ensuring consistency in primary outcomes and sample sizes. Protocol publication does not seem to provide additional significant benefits in terms of outcome reporting.

Discrepancies between pre-specified and reported primary outcomes: A cross-sectional analysis of randomized controlled trials in gastroenterology and hepatology journals

BACKGROUND

Previous research has raised concerns regarding inconsistencies between reported and pre-specified outcomes in randomized controlled trials (RCTs) across various biomedical disciplines. However, studies examining whether similar discrepancies exist in RCTs focusing on gastrointestinal and liver diseases are limited. This study aimed to assess the extent of discrepancies between registered and published primary outcomes in RCTs featured in journals specializing in gastroenterology and hepatology.

METHODS

We retrospectively retrieved RCTs published between January 1, 2017 and December 31, 2021 in the top three journals from each quartile ranking of the 2020 Journal Citation Reports within the "Gastroenterology and Hepatology" subcategory. We extracted data on trial characteristics, registration details, and pre-specified versus published primary outcomes. Pre-specified primary outcomes were retrieved from the World Health Organization's International Clinical Trials Registry Platform. Only trials reporting specific primary outcomes were included in analyzing primary outcome discrepancies. We also assessed whether there was a potential reporting bias that deemed to favor statistically significant outcomes. Statistical analyses included chi-square tests, Fisher's exact tests, univariate analyses, and logistic regression.

RESULTS

Of 362 articles identified, 312 (86.2%) were registered, and 79.8% of the registrations (249 out of 312) were prospective. Among the 285 trials reporting primary outcomes, 76 (26.7%) exhibited at least one discrepancy between registered and published primary outcomes. The most common discrepancies included different assessment times for the primary outcome (n = 32, 42.1%), omitting the registered primary outcome in publications (n = 21, 27.6%), and reporting the registered secondary outcomes as primary outcomes (n = 13, 17.1%). Univariate analyses revealed that primary outcome discrepancies were lower in the publication year 2020 compared to year 2021 (OR = 0.267, 95% CI: 0.101, 0.706, p = 0.008). Among the 76 studies with primary outcome discrepancies, 20 (26.3%) studies were retrospectively registered, and 32 (57.1%) of the prospectively registered trials with primary outcome discrepancies showed statistically significant results. However, no significant differences were found between journal quartiles regarding primary outcome consistency and potential reporting bias (p = 0.14 and p = 0.28, respectively).

CONCLUSIONS

This study highlights the disparities between registered and published primary outcomes in RCTs within gastroenterology and hepatology journals. Attention to factors such as the timing of primary outcome assessments in published trials and the consistency between registered and published primary outcomes is crucial. Enhanced scrutiny from journal editors and peer reviewers during the review process is necessary to ensure the reliability of gastrointestinal and hepatic trials.

Compliance with requirements for registration and reporting of results in trials of mesenchymal stromal cells for musculoskeletal disorders: a systematic review

OBJECTIVE

To assess compliance with statutory requirements to register and report outcomes in interventional trials of mesenchymal stromal cells (MSCs) for musculoskeletal disorders and to describe the trials' clinical and design characteristics.

DESIGN

A systematic review of published trials and trials submitted to public registries.

DATA SOURCES

The databases Medline, Cochrane Library and McMaster; six public clinical registries. All searches were done until 31 January 2023.

ELIGIBILITY CRITERIA

Trials submitted to registries and completed before January 2021. Prospective interventional trials published in peer-reviewed journals.

DATA EXTRACTION AND SYNTHESIS

The first author searched for trials that had (1) posted trial results in a public registry, (2) presented results in a peer-reviewed publication and (3) submitted a pretrial protocol to a registry before publication. Other extracted variables included trial design, number of participants, funding source, follow-up duration and cell type.

RESULTS

In total 124 trials were found in registries and literature databases. Knee osteoarthritis was the most common indication. Of the 100 registry trials, 52 trials with in total 2 993 participants had neither posted results in the registry nor published results. Fifty-two of the registry trials submitted a protocol retrospectively. Forty-three of the 67 published trials (64%) had registered a pretrial protocol. Funding source was not associated with compliance with reporting requirements. A discrepancy between primary endpoints in the registry and publication was found in 16 of 25 trials. In 28% of trials, the treatment groups used adjuvant therapies. Only 39% of controlled trials were double-blinded.

CONCLUSIONS

A large proportion of trials failed to comply with statutory requirements for the registration and reporting of results, thereby increasing the risk of bias in outcome assessments. To improve confidence in the role of MSCs for musculoskeletal disorders, registries and medical journals should more rigorously enforce existing requirements for registration and reporting.

Publication Bias and Selective Outcome Reporting in Randomized Controlled Trials Related to Rehabilitation: A Literature Review

OBJECTIVE

To investigate the rate of registered protocols published as research papers as a measure of publication bias, and the concordance rates of the primary outcomes between research protocol and published papers as a measure of selective outcome reporting bias in randomized controlled trials (RCTs) related to rehabilitation.

DATA SOURCES

Protocols related to RCTs were extracted from electronic databases, the University Hospital Medical Information Network (UMIN), International Standard Research Clinical Trial Number (ISRCTN), ClinicalTrials.gov, and MEDLINE. Published papers were retrieved from MEDLINE.

STUDY SELECTION

The inclusion criteria were as follows: (1) initial registration (UMIN, ISRCTN, ClinicalTrials.gov) within the designated period; (2) published as a paper from a research protocol in MEDLINE (PubMed); and (3) written in English or Japanese. The search period was from January 1, 2013, to December 31, 2020.

DATA EXTRACTION

The outcome of this study was set as the rate of published papers that were consistent with the extracted research protocol and the concordance rate between the primary outcomes in published papers and in protocols. The concordance rate of the primary outcomes was evaluated by checking whether the description in the research protocol matched that in the paper's abstract and main text.

DATA SYNTHESIS

Out of the 5597 research protocols registered, only 727 were published (13.0%). The concordance rates of the primary outcomes were 48.7% and 72.6% in the abstract and main text, respectively.

CONCLUSIONS

This study revealed major discrepancies between the number of research protocols and published papers, and difference of description regarding the primary outcomes in published papers which were already defined in the research protocols.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Prospective registration of randomized clinical trials for total shoulder arthroplasty is low: a systematic review

BACKGROUND

Prospective trial registration has become an important means of improving the transparency and reproducibility of randomized controlled trials (RCTs) and is recommended by the Journal of Shoulder and Elbow Surgery (JSES) per the Consolidated Standards of Reporting Trials (CONSORT) guidelines. Herein, we performed a cross-sectional evaluation of RCTs published in JSES from 2010 to present to determine the prevalence of trial registration and consistency of outcome reporting.

METHODS

The electronic database PubMed was searched to identify all RCTs on total shoulder arthroplasty (TSA) published in JSES from 2010 to 2022 using the search terms "randomized controlled trial" AND "shoulder" AND "arthroplasty OR replacement." RCTs were considered to be registered if they provided a registration number. For articles that were registered, authors also extracted the registry name, registration date, date of first enrollment, date of last enrollment, and if the primary outcomes reported in the registry were either (1) omitted, (2) newly introduced in the publication, (3) reported as a secondary outcome or vice versa, or (4) varied in timing of assessment compared to the publication. "Early" RCTs were considered those published from 2010 to 2016, whereas "later" RCTs were from 2017 to 2022.

RESULTS

Fifty-eight RCTs met inclusion criteria. There were 16 early RCTs and 42 later RCTs. Twenty-three of the 58 (39.7%) studies were registered, with 9 of 22 with an available registry (40.9%) of those being enrolled prior to patient enrollment. Nineteen of the registered studies (82.6%) provided the name of the registry and a registration number. The proportion of later RCTs that were registered was not significantly different from the early RCTs (45.2% vs. 25.0%, P = .232). Seven RCTs (31.8%) had at least 1 inconsistency compared with the registry. The most common discrepancy was the timing of the assessment (ie, follow-up period) reported in the publication vs. the registry.

DISCUSSION

Although JSES recommends prospective trial registration, less than half of shoulder arthroplasty RCTs are registered and more than 30% registered trials have at least 1 inconsistency with their registry record. More rigorous review of trial registration and accuracy is necessary to limit bias in published shoulder arthroplasty RCTs.

Completeness and consistency of primary outcome reporting in COVID-19 publications in the early pandemic phase: a descriptive study

BACKGROUND

The COVID-19 pandemic saw a steep increase in the number of rapidly published scientific studies, especially early in the pandemic. Some have suggested COVID-19 trial reporting is of lower quality than typical reports, but there is limited evidence for this in terms of primary outcome reporting. The objective of this study was to assess the prevalence of completely defined primary outcomes reported in registry entries, preprints, and journal articles, and to assess consistent primary outcome reporting between these sources.

METHODS

This is a descriptive study of a cohort of registered interventional clinical trials for the treatment and prevention of COVID-19, drawn from the DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study dataset. The main outcomes are: 1) Prevalence of complete primary outcome reporting; 2) Prevalence of consistent primary outcome reporting between registry entry and preprint as well as registry entry and journal article pairs.

RESULTS

We analyzed 87 trials with 116 corresponding publications (87 registry entries, 53 preprints and 63 journal articles). All primary outcomes were completely defined in 47/87 (54%) registry entries, 31/53 (58%) preprints and 44/63 (70%) journal articles. All primary outcomes were consistently reported in 13/53 (25%) registry-preprint pairs and 27/63 (43%) registry-journal article pairs. No primary outcome was specified in 13/53 (25%) preprints and 8/63 (13%) journal articles. In this sample, complete primary outcome reporting occurred more frequently in trials with vs. without involvement of pharmaceutical companies (76% vs. 45%), and in RCTs vs. other study designs (68% vs. 49%). The same pattern was observed for consistent primary outcome reporting (with vs. without pharma: 56% vs. 12%, RCT vs. other: 43% vs. 22%).

CONCLUSIONS

In COVID-19 trials in the early phase of the pandemic, all primary outcomes were completely defined in 54%, 58%, and 70% of registry entries, preprints and journal articles, respectively. Only 25% of preprints and 43% of journal articles reported primary outcomes consistent with registry entries.

Arthroscopic partial meniscectomy vs non-surgical or sham treatment in patients with MRI-confirmed degenerative meniscus tears: a systematic review and meta-analysis with individual participant data from 605 randomised patients

OBJECTIVE

To identify subgroups of patients with magnetic resonance imaging (MRI)-confirmed degenerative meniscus tears who may benefit from arthroscopic partial meniscectomy (APM) in comparison with non-surgical or sham treatment.

METHODS

Individual participant data (IPD) from four RCTs were pooled (605 patients, mean age: 55 (SD: 7.5), 52.4% female) as to investigate the effectiveness of APM in patients with MRI-confirmed degenerative meniscus tears compared to non-surgical or sham treatment. Primary outcomes were knee pain, overall knee function, and health-related quality of life, at 24 months follow-up (0-100). The IPD were analysed in a one- and two-stage meta-analyses. Identification of potential subgroups was performed by testing interaction effects of predefined patient characteristics (e.g., age, gender, mechanical symptoms) and APM for each outcome. Additionally, generalized linear mixed-model trees were used for subgroup detection.

RESULTS

The APM group showed a small improvement over the non-surgical or sham group on knee pain at 24 months follow-up (2.5 points (95% CI: 0.8-4.2) and 2.2 points (95% CI: 0.9-3.6), one- and two-stage analysis, respectively). Overall knee function and health-related quality of life did not differ between the two groups. Across all outcomes, no relevant subgroup of patients who benefitted from APM was detected. The generalized linear mixed-model trees did also not identify a subgroup.

CONCLUSIONS

No relevant subgroup of patients was identified that benefitted from APM compared to non-surgical or sham treatment. Since we were not able to identify any subgroup that benefitted from APM, we recommend a restrained policy regarding meniscectomy in patients with degenerative meniscus tears.

Innovative technologies for reverse total shoulder arthroplasty in Australia: Market access challenges and implications for patients, decision-makers, and manufacturers

PURPOSE

The success of reverse total shoulder arthroplasty (RTSA) has expanded its use for a broader range of shoulder indications worldwide. Evidence regarding the relative efficacy and long-term safety of medical technologies used in RTSA is subjected to rigorous assessment. Nonetheless, substantial challenges impede market access for innovative shoulder implant technologies for RTSA in Australia, resulting in delayed patient access.

APPROACH

This paper addresses the key challenges associated with generating evidence for the health technology assessments of innovative medical technologies for RTSA that are required for access to the Australian market. The transition to value-based care requires establishing a benchmarking reference that incorporates patient-reported outcome measures (PROMs) and combines revision outcomes with additional clinical outcomes to increase patient cohort sizes. Establishing the benchmark would require agreement on the outcome measures to be collected for each indication, and investment in reporting patient-reported outcomes for RTSA to the national orthopaedic registry.

IMPLICATIONS FOR PRACTICE

The need for increased flexibility in developing evidence for health technology assessment of RTSA medical technologies is required. Optimised approaches for benchmarking RTSA require extensive stakeholder discussions, including the agreement on evidence requirements and follow-up periods, selection of clinical outcomes, as well as pre-operative and post-operative PROMs as a value assessment.

Examination of Randomized Trials and Corresponding Trial Registry Entries: Registration Timing and Primary Outcome Analysis in the Journal of Arthroplasty

BACKGROUND

Prospective trial registration enhances transparency and rigor of trial reporting. We conducted an in-depth examination of randomized clinical trials (RCTs) published in The Journal of Arthroplasty (JOA) from 2010 to 2020 and their associated trial registries.

METHODS

We examined all RCTs published in the JOA during the even years between 2010 and 2020. We determined the proportion of trials that were registered and prospectively registered as well as the extent of consistency between primary outcome characteristics in the trials vs the registries. Trial characteristics published between 2010 and 2014 were compared to trials published between 2016 and 2020.

RESULTS

A total of 57 (33.7%) of 169 primary RCTs over the study period reported being registered and of these, 20 (11.8%) were prospectively registered. For the registered primary RCTs, 75% reported primary outcome findings that were inconsistent with the corresponding registry. Trial registration proportion substantially improved from 13.6% between 2010 and 2014 to 53% between 2016 and 2020 (z-test = -5.315, P < .001).

CONCLUSION

High proportions of retrospectively registered or unregistered trials and a very high proportion of inconsistencies in reporting of primary outcomes compared to the trial registries were found. These data argue for a well-developed strategy by JOA to enhance editorial policies, reviewer and editorial board member training and oversight, and improved arthroplasty researcher awareness to improve the current state of RCT reporting in JOA.

Comparing dream to reality: an assessment of adherence of the first generation of preregistered studies

Preregistration is a method to increase research transparency by documenting research decisions on a public, third-party repository prior to any influence by data. It is becoming increasingly popular in all subfields of psychology and beyond. Adherence to the preregistration plan may not always be feasible and even is not necessarily desirable, but without disclosure of deviations, readers who do not carefully consult the preregistration plan might get the incorrect impression that the study was exactly conducted and reported as planned. In this paper, we have investigated adherence and disclosure of deviations for all articles published with the Preregistered badge in Psychological Science between February 2015 and November 2017 and shared our findings with the corresponding authors for feedback. Two out of 27 preregistered studies contained no deviations from the preregistration plan. In one study, all deviations were disclosed. Nine studies disclosed none of the deviations. We mainly observed (un)disclosed deviations from the plan regarding the reported sample size, exclusion criteria and statistical analysis. This closer look at preregistrations of the first generation reveals possible hurdles for reporting preregistered studies and provides input for future reporting guidelines. We discuss the results and possible explanations, and provide recommendations for preregistered research.

Characteristics of Randomized Clinical Trials in Surgery From 2008 to 2020: A Systematic Review

IMPORTANCE

Randomized clinical trials (RCTs) provide the highest level of evidence to evaluate 2 or more surgical interventions. Surgical RCTs, however, face unique challenges in design and implementation.

OBJECTIVE

To evaluate the design, conduct, and reporting of contemporary surgical RCTs.

EVIDENCE REVIEW

A literature search performed in the 2 journals with the highest impact factor in general medicine as well as 6 key surgical specialties was conducted to identify RCTs published between 2008 and 2020. All RCTs describing a surgical intervention in both experimental and control arms were included. The quality of included data was assessed by establishing an a priori protocol containing all the details to extract. Trial characteristics, fragility index, risk of bias (Cochrane Risk of Bias 2 Tool), pragmatism (Pragmatic Explanatory Continuum Indicator Summary 2 [PRECIS-2]), and reporting bias were assessed.

FINDINGS

A total of 388 trials were identified. Of them, 242 (62.4%) were registered; discrepancies with the published protocol were identified in 81 (33.5%). Most trials used superiority design (329 [84.8%]), and intention-to-treat as primary analysis (221 [56.9%]) and were designed to detect a large treatment effect (50.0%; interquartile range [IQR], 24.7%-63.3%). Only 123 trials (31.7%) used major clinical events as the primary outcome. Most trials (303 [78.1%]) did not control for surgeon experience; only 17 trials (4.4%) assessed the quality of the intervention. The median sample size was 122 patients (IQR, 70-245 patients). The median follow-up was 24 months (IQR, 12.0-32.0 months). Most trials (211 [54.4%]) had some concern of bias and 91 (23.5%) had high risk of bias. The mean (SD) PRECIS-2 score was 3.52 (0.65) and increased significantly over the study period. Most trials (212 [54.6%]) reported a neutral result; reporting bias was identified in 109 of 211 (51.7%). The median fragility index was 3.0 (IQR, 1.0-6.0). Multiplicity was detected in 175 trials (45.1%), and only 35 (20.0%) adjusted for multiple comparisons.

CONCLUSIONS AND RELEVANCE

In this systematic review, the size of contemporary surgical trials was small and the focus was on minor clinical events. Trial registration remained suboptimal and discrepancies with the published protocol and reporting bias were frequent. Few trials controlled for surgeon experience or assessed the quality of the intervention.

A technical review of three clinical trials register resources indicates where improvements to the search interfaces are needed

Clinical trials registers form an important part of the search for studies in systematic reviews of intervention effectiveness but the search interfaces and functionality of registers can be challenging to search systematically and resource intensive to search well. We report a technical review of the search interfaces of three leading trials register resources: ClinicalTrials.gov, the EU Clinical Trials Register and the WHO International Clinical Trials Registers Platform. The technical review used a validated checklist to identify areas where the search interfaces of these trials register resources performed well, where performance was adequate, where performance was poor, and to identify differences between search interfaces. The review found low overall scores for each of the interfaces (ClinicalTrials.gov 55/165, the EU Clinical Trials Register 25/165, the WHO International Clinical Trials Registers Platform 32/165). This finding suggests a need for joined-up dialogue between the producers of the registers and researchers who search them via these interfaces. We also set out a series of four proposed changes which might improve the search interfaces. Trials registers are an invaluable resource in systematic reviews of intervention effectiveness. With the continued growth in systematic reviews, and initiatives such as 'AllTrials', there is an anticipated need for these resources. We conclude that small changes to the search interfaces, and improved dialogue with providers, might improve the future search functionality of these valuable resources.

Levels of evidence backing the AAOS clinical practice guidelines

Purpose:

The American Academy of Orthopaedic Surgeons produces clinical practice guidelines for the treatment of orthopedic injuries. We examined the strength of the evidence underlying these recommendations in order to answer the following questions: (1) Have AAOS work groups improved guideline creation practices to locate evidence to generate strong recommendations? (2) Is there variability in the available evidence based on anatomic site or stage of care? (3) Has the level of evidence supporting improved over time?

Methods:

Twenty-two current guidelines of the Academy were examined which yielded 408 individual recommendations. These recommendations were assigned one of five strength of evidence ratings (strong, moderate, limited, inconclusive, consensus) by the guideline panel, based on the availability and quality of the supporting evidence. From these guidelines, we extracted all of the recommendations and their corresponding evidence ratings. We then classified the recommendations by stage of care, year, and anatomical site.

Results:

The distribution of the levels of evidence was as follows: 77 (18.9%) were based on consensus; 53 (13.0%) were inconclusive; 93 (22.8%) were based on limited evidence; 112 (27.5%) were based on moderate evidence; and 73 were based on (17.9%) strong evidence. Strong strength of evidence was found in 45.2% of the recommendations for preventive/screening/diagnostic care, 41.1% of nonsurgical treatment, 45.1% of surgical treatment, 51.1% of rehabilitation/postoperative treatment, and 45.5% of the recommendations that had mixed stages of care. Inconclusive strength of evidence was found to be prevalent from 2009–2013, but was eliminated starting in 2014.

Conclusions:

Only 73 (17.9%) recommendations generated by the Academy in its 22 clinical practice guidelines are based on a “strong” strength of evidence. More robust research is needed in orthopedics to bolster confidence in the recommendations in future guideline updates.

Omitting Routine Radiography of Traumatic Ankle Fractures After Initial 2-Week Follow-up Does Not Affect Outcomes: The WARRIOR Trial: A Multicenter Randomized Controlled Trial

BACKGROUND

The clinical consequences of routine follow-up radiographs for patients with ankle fracture are unclear, and their usefulness is disputed. The purpose of the present study was to determine if routine radiographs made at weeks 6 and 12 can be omitted without compromising clinical outcomes.

METHODS

This multicenter randomized controlled trial with a noninferiority design included 246 patients with an ankle fracture, 153 (62%) of whom received operative treatment. At 6 and 12 weeks of follow-up, patients in the routine-care group (n = 128) received routine radiographs whereas patients in the reduced-imaging group (n = 118) did not. The primary outcome was the Olerud-Molander Ankle Score (OMAS). Secondary outcomes were the American Academy of Orthopaedic Surgeons (AAOS) foot and ankle questionnaire, health-related quality of life (HRQoL) as measured with the EuroQol-5 Dimensions-3 Levels (EQ-5D-3L) and Short Form-36 (SF-36), complications, pain, health perception, self-perceived recovery, the number of radiographs, and the indications for radiographs to be made. The outcomes were assessed at baseline and at 6, 12, 26, and 52 weeks of follow-up. Data were analyzed with use of mixed models.

RESULTS

Reduced imaging was noninferior compared with routine care in terms of OMAS scores (difference [β], -0.9; 95% confidence interval [CI], -6.2 to 4.4). AAOS scores, HRQoL, pain, health perception, and self-perceived recovery did not differ between groups. Patients in the reduced-imaging group received a median of 4 radiographs, whereas those in the routine-care group received a median of 5 radiographs (p < 0.05). The rates of complications were similar (27.1% [32 of 118] in the reduced-imaging group, compared with 22.7% [29 of 128] in the routine-care group, p = 0.42). The types of complications were also similar.

CONCLUSIONS

Implementation of a reduced-imaging protocol following an ankle fracture has no measurable negative effects on functional outcome, pain, and complication rates during the first year of follow-up. The number of follow-up radiographs can be reduced by implementing this protocol.

LEVEL OF EVIDENCE

Therapeutic Level I. See Instructions for Authors for a complete description of levels of evidence.

Arthroscopic meniscectomy versus non-surgical or sham treatment in patients with MRI confirmed degenerative meniscus lesions: a protocol for an individual participant data meta-analysis

INTRODUCTION

Arthroscopic partial meniscectomy (APM) after degenerative meniscus tears is one of the most frequently performed surgeries in orthopaedics. Although several randomised controlled trials (RCTs) have been published that showed no clear benefit compared with sham treatment or non-surgical treatment, the incidence of APM remains high. The common perception by most orthopaedic surgeons is that there are subgroups of patients that do need APM to improve, and they argue that each study sample of the existing trials is not representative for the day-to-day patients in the clinic. Therefore, the objective of this individual participant data meta-analysis (IPDMA) is to assess whether there are subgroups of patients with degenerative meniscus lesions who benefit from APM in comparison with non-surgical or sham treatment.

METHODS AND ANALYSIS

An existing systematic review will be updated to identify all RCTs worldwide that evaluated APM compared with sham treatment or non-surgical treatment in patients with knee symptoms and degenerative meniscus tears. Time and effort will be spent in contacting principal investigators of the original trials and encourage them to collaborate in this project by sharing their trial data. All individual participant data will be validated for missing data, internal data consistency, randomisation integrity and censoring patterns. After validation, all datasets will be combined and analysed using a one-staged and two-staged approach. The RCTs' characteristics will be used for the assessment of clinical homogeneity and generalisability of the findings. The most important outcome will be the difference between APM and control groups in knee pain, function and quality of life 2 years after the intervention. Other outcomes of interest will include the difference in adverse events and mental health.

ETHICS AND DISSEMINATION

All trial data will be anonymised before it is shared with the authors. The data will be encrypted and stored on a secure server located in the Netherlands. No major ethical concerns remain. This IPDMA will provide the evidence base to update and tailor diagnostic and treatment protocols as well as (international) guidelines for patients for whom orthopaedic surgeons consider APM. The results will be submitted for publication in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42017067240.

Selective reporting bias due to discrepancies between registered and published outcomes in osteoarthritis trials

Aim: Outcome reporting bias (ORB) occurs when outcomes planned in a study protocol are subsequently not reported or are partially reported. Our aim was to analyze ORB in randomized controlled trials (RCTs) about conservative interventions for osteoarthritis (OA) by comparing registered protocols and published manuscripts, as well as association between study funding type and intervention type, and ORB in those RCTs. Materials & methods: We analyzed RCTs that were published in a peer-review journal and analyzed any type of conservative intervention for treatment of OA in humans that reported in the manuscript registration in a public clinical trial registry and provided unique registration identifier. We extracted data indicating ORB by comparing outcomes in protocol and published article, and characteristics of trials. Results: In 190 (57%) of 334 included RCTs, it was indicated in the manuscript that a trial was registered. In 48% of trials we found discrepancies in number, type or time point of primary efficacy outcome between protocol and manuscript. Significantly less discrepancies in primary efficacy outcomes between protocols and published articles were found in trials funded by a commercial sponsor (p = 0.0062) and trials of pharmacological interventions (p = 0.0016). Conclusion: Trials about conservative therapies for OA have high prevalence of discrepancies between protocol and publication, and frequent ORB. This may mislead readers of published results because it has been shown that ORB can lead to both overestimation and underestimation of effects of interventions, depending on the intervention and outcome. Efforts to prevent nonregistration of protocols and selective reporting are needed.

Discrepancies between Registered and Published Primary and Secondary Outcomes in Randomized Controlled Trials within the Plastic Surgery Literature: A Systematic Review

BACKGROUND

Recent studies have identified a high incidence of discrepancy between registered and published outcomes in registered medical and surgical randomized controlled trials. This has not yet been studied in the plastic surgery literature.

METHODS

The authors systematically assessed plastic surgery randomized controlled trials published between 2012 and 2016 in seven high-impact plastic surgery journals. Data were collected from the registration website and published articles using a standardized data extraction form.

RESULTS

A total of 145 randomized controlled trials were identified, with a 39 percent trial registration rate (n = 57). Forty-nine trials were included in the final analysis. Forty-three (88 percent) had a discrepancy between registered and published outcomes: 26 (53 percent) for primary outcome(s), and 39 (80 percent) for secondary outcome(s). The number of discrepancies in an individual trial ranged from one to seven for primary outcomes and one to 12 for secondary outcomes. Aesthetic surgery had the largest number of trials with outcome discrepancies (n = 15). The prevalence of unreported registered outcomes was 13 percent for primary outcomes and 38 percent for secondary outcomes. Registered nonsignificant primary outcomes were published as nonsignificant secondary outcomes in 30 percent of trials. Publishing new nonregistered secondary outcomes (65 percent) and changing the assessment timing of published primary outcomes (61 percent) were the most common types of discrepancies. Discrepancies favored a statistically significant positive outcome in 19 (44 percent) of the 43 trials with an outcome discrepancy. Discrepancies that resulted in published outcomes with improved patient relevance were found in eight trials (16 percent) for primary outcome discrepancies and 14 trials (29 percent) for secondary outcome discrepancies.

CONCLUSIONS

The plastic surgery literature has high rates of discrepancies between registered and published trial outcomes. Outcome reporting discrepancy is even more problematic for secondary outcomes, an area of analysis that has previously been poorly studied. The high rate of discrepancy change favoring a statistically significant outcome and more patient-relevant outcomes may indicate the pressure to demonstrate significant results to be accepted for publication in high-impact journals.

Omitting Routine Radiography of Traumatic Distal Radial Fractures After Initial 2-Week Follow-up Does Not Affect Outcomes

BACKGROUND

Routine radiography in the follow-up of distal radial fractures is common practice, although its usefulness is disputed. The aim of this study was to determine whether the number of radiographs in the follow-up period can be reduced without resulting in worse patient outcomes.

METHODS

In this multicenter, prospective, randomized controlled trial with a non-inferiority design, patients ≥18 years old with a distal radial fracture could participate. They were randomized between a regimen with routine radiographs at 6 and 12 weeks of follow-up (usual care) and a regimen without routine radiographs at those time points (reduced imaging). Randomization was performed using an online registration and randomization program. The primary outcome was the Disabilities of the Arm, Shoulder and Hand (DASH) score. Secondary outcomes included the Patient-Rated Wrist/Hand Evaluation (PRWHE) score, health-related quality of life, pain, and complications. Outcomes were assessed at baseline and after 6 weeks, 3 months, 6 months, and 1 year of follow-up. Data were analyzed using mixed models. Neither the patients nor the health-care providers were blinded.

RESULTS

Three hundred and eighty-six patients were randomized, and 326 of them were ultimately included in the analysis. The DASH scores were comparable between the usual-care group (n = 166) and the reduced-imaging group (n = 160) at all time points as well as overall. The adjusted regression coefficient for the DASH scores was 1.5 (95% confidence interval [CI] = -1.8 to 4.8). There was also no difference between the groups with respect to the overall PRWHE score (adjusted regression coefficient, 1.4 [95% CI = -2.4 to 5.2]), quality of life as measured with the EuroQol-5 Dimensions (EQ-5D) (-0.02 [95% CI = -0.05 to 0.01]), pain at rest as measured with a visual analog scale (VAS) (0.1 [95% CI = -0.2 to 0.5]), or pain when moving (0.3 [95% CI = -0.1 to 0.8]). The complication rate was similar in the reduced imaging group (11.3%) and the usual-care group (11.4%). Fewer radiographs were made for the participants in the reduced-imaging group (median, 3 versus 4; p < 0.05).

CONCLUSIONS

This study shows that omitting routine radiography after the initial 2 weeks of follow-up for patients with a distal radial fracture does not affect patient-reported outcomes or the risk of complications compared with usual care.

LEVEL OF EVIDENCE

Therapeutic Level I. See Instructions for Authors for a complete description of levels of evidence.

Comparison of Clinical Trial Changes in Primary Outcome and Reported Intervention Effect Size Between Trial Registration and Publication

IMPORTANCE

Primary outcome change could threaten the validity of a clinical trial; however, evidence about the consequences on the reported intervention effect size is unclear.

OBJECTIVES

To examine the status of randomized clinical trials whose primary outcome changed between trial registration and publication and to quantify the association of this change with the reported intervention effect size.

DESIGN, SETTING, AND PARTICIPANTS

In this cross-sectional study on the primary report of randomized clinical trials with clear prospectively registered primary outcomes, PubMed and Embase were searched for articles published between January 1, 2011, and December 31, 2015. The search was conducted in January 2016, identifying randomized clinical trials and the combination of keywords and text words related to registry.

MAIN OUTCOMES AND MEASURES

Based on the developed approach, trials were classified as having primary outcome change when there was a major discrepancy between the registered and published primary outcomes. Intervention effect was estimated or recalculated using the odds ratio (OR) for each comparison. Each component OR is structured so that an OR is less than 1 if the intervention group has a more favorable result than the control group. The ratio of ORs (ROR), which is the summary OR for trials with primary outcome change divided by those without, and its 95% CI were calculated, with a value less than 1 indicating a larger reported intervention effect size in trials with primary outcome change than those without.

RESULTS

Among 29 749 searched articles (28 810 MEDLINE and 939 Embase), 1488 articles were randomly selected for review. Of 389 trials with clear primary outcomes prospectively described in the registry (416 outcomes reported), 33.4% (130 of 389) of trials had at least 1 primary outcome change. Most (66 of 130) of the changes were either not reporting or omitting the primary outcome. In total, 338 trials (365 outcomes and 487 comparisons) were available for quantitative analysis on the reported intervention effect size bias assessment. Compared with those without primary outcome change, trials with primary outcome change showed a 16% (pooled ROR, 0.84; 95% CI, 0.73-0.96) larger reported intervention effect size. The result persisted after adjustment for potential confounders (ROR, 0.81; 95% CI, 0.71-0.93) and other sensitivity and subgroup analyses.

CONCLUSIONS AND RELEVANCE

Results of this study suggest that inconsistencies between registered and published primary outcomes of clinical trials are common, and trials with primary outcome change are likely to have a larger intervention effect than those without.

Registration Practices of Randomized Clinical Trials in Rhinosinusitis: A Cross-sectional Review

Importance

Randomized clinical trials (RCTs) play an important role in the development of clinical practice guidelines and in clinical decision making. Little is known about the registration practices of RCTs in the diagnosis and treatment of rhinosinusitis.

Objectives

The primary outcome was the frequency of reported RCT registry numbers by authors of rhinosinusitis RCTs. A secondary outcome was the rates of selective reporting bias in RCTs that were prospectively registered. A tertiary outcome end point was the frequency of publication of RCTs registered on ClinicalTrials.gov.

Evidence Review

Our sample was derived from a PubMed (MEDLINE) search performed on October 31, 2017, using the keywords "sinusitis OR rhinosinusitis OR rhinitis," and filtering by date (January 1, 2015, through October 31, 2017) and study type (RCT). Studies that were considered an RCT were included for the primary outcome analysis. All RCTs that were registered prior to or during patient enrollment were included for secondary outcome analysis. For the tertiary outcome, a search was performed on ClinicalTrials.gov on September 4, 2018, using the keywords "sinusitis OR rhinosinusitis," and filtering by date (January 1, 2013, through October 31, 2015). All analysis took place between October 29, 2018, and October 31, 2018.

Findings

A total of 179 RCTs were analyzed for our primary outcome: 94 (52.5%) included a registration number in their publication, and 70 (39.1%) were included for secondary outcome analysis of rates of selective reporting bias. Of these 70 RCTs, 22 (31%) were found to have at least 1 major discrepancy between trial registration and publication. For the tertiary outcome, 52 completed clinical trials were identified on ClinicalTrials.gov, of which 21 (40%) had listed publication in the registry.

Conclusions and Relevance

We found that published rhinosinusitis RCTs frequently do not include a trial registration number and that registered RCTs frequently are not published. Furthermore, RCTs that are registered often display selective reporting of their outcomes and frequently favor positive results. We recommend strict adherence to RCT registration policies and the enforcement of accurate reporting to help strengthen the evidence behind clinical decision making.

Outcome Selection and Methodological Quality of Major and Minor Shoulder Surgery Studies: A Scoping Review

BACKGROUND

Core outcome sets aim to select and standardize the choice of important outcomes reported in clinical trials to encourage more effective data synthesis, increase the reliability of comparing results, and minimize reporting bias. A core outcome set for elective shoulder surgery has yet to be defined, and therefore a systematic assessment of outcomes and methodology is necessary to inform the development of a core outcome set.

QUESTIONS/PURPOSES

The purpose of this study was to examine randomized controlled trials (RCTs) of patients having elective major or minor shoulder surgery to (1) identify the outcome domains reported; (2) determine specific outcome measurement tools that were utilized; and (3) assess the work for methodological quality and risk of bias.

METHODS

We conducted a scoping review (a review that identifies the nature and extent of research evidence) to explore the reported outcome domains, outcome tools, and methodological quality from RCTs conducted in shoulder surgery. We considered both major shoulder surgery (defined as arthroplasty, rotator cuff repair, stabilization procedures, biceps tenodesis, or Bankart repairs) and minor shoulder surgery (simple arthroscopy, capsular plication, lateral clavicular excisions, or subacromial decompression). We queried 10 electronic databases for studies published between January 2006 and January 2015. Studies were included if they were prospective, randomized controlled, clinical trials enrolling patients who received an elective shoulder surgical intervention. We extracted data relating to trial characteristics, primary outcomes, tools used to measure these outcomes as well as methodological quality indicators. We assessed indicators of methodological quality by exploring (1) the reproducibility of power analyses; and (2) whether the primary outcomes were powered to minimum clinically important differences. Risk of bias was also assessed with the Jadad score with scores between 0 (very high risk of bias) and 5 (very low risk). Findings were qualitatively analyzed and reported according to systematic and scoping review guidelines. We included 315 studies involving 30,232 patients; 266 studies investigated anesthetic, analgesic, or surgical interventions.

RESULTS

Of the 315 studies included, the most common outcome domains evaluated were analgesic (n = 104), functional (n = 87), anesthetic (n = 56), and radiologic (n = 29) outcomes, with temporal patterns noted. Studies of major shoulder surgery most commonly reported functional primary outcome domains, whereas minor shoulder surgery studies most frequently reported analgesic primary outcome domains. There were 85 different primary outcome tools utilized, which included 20 functional, 20 anesthetic, 13 analgesic, and 12 radiologic. A methodological quality assessment revealed that 24% of studies had reproducible power analyses, 13% were powered to minimum clinically important differences, and risk of bias assessment demonstrated a median (interquartile range [range]) Jadad score of 4 (3-5 [1-5]).

CONCLUSIONS

A wide range of outcome domains and outcome assessment tools are in common use in contemporary trials of patients undergoing elective surgery. Although some diversity is important to allow the assessment of patient populations that may have different goals, the large number of tools in common use may impair the ability of future meta-analyses to pool results effectively or even for systematic reviews to synthesize what is known. The limitations of methodological quality in RCTs may be improved by researchers following standard guidelines and considering the minimum clinically important differences in their trials to be of greater use to clinicians and their patients.

LEVEL OF EVIDENCE

Level I, therapeutic study.

Full publication of results initially presented in abstracts

BACKGROUND

Abstracts of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of results reported in these abstracts is based on the magnitude or direction of the results, publication bias may result. Publication bias creates problems for those conducting systematic reviews or relying on the published literature for evidence about health and social care.

OBJECTIVES

To systematically review reports of studies that have examined the proportion of meeting abstracts and other summaries that are subsequently published in full, the time between meeting presentation and full publication, and factors associated with full publication.

SEARCH METHODS

We searched MEDLINE, Embase, the Cochrane Library, Science Citation Index, reference lists, and author files. The most recent search was done in February 2016 for this substantial update to our earlier Cochrane Methodology Review (published in 2007).

SELECTION CRITERIA

We included reports of methodology research that examined the proportion of biomedical results initially presented as abstracts or in summary form that were subsequently published. Searches for full publications had to be at least two years after meeting presentation.

DATA COLLECTION AND ANALYSIS

Two review authors extracted data and assessed risk of bias. We calculated the proportion of abstracts published in full using a random-effects model. Dichotomous variables were analyzed using risk ratio (RR), with multivariable models taking into account various characteristics of the reports. We assessed time to publication using Kaplan-Meier survival analyses.

MAIN RESULTS

Combining data from 425 reports (307,028 abstracts) resulted in an overall full publication proportion of 37.3% (95% confidence interval (CI), 35.3% to 39.3%) with varying lengths of follow-up. This is significantly lower than that found in our 2007 review (44.5%. 95% CI, 43.9% to 45.1%). Using a survival analyses to estimate the proportion of abstracts that would be published in full by 10 years produced proportions of 46.4% for all studies; 68.7% for randomized and controlled trials and 44.9% for other studies. Three hundred and fifty-three reports were at high risk of bias on one or more items, but only 32 reports were considered at high risk of bias overall.Forty-five reports (15,783 abstracts) with 'positive' results (defined as any 'significant' result) showed an association with full publication (RR = 1.31; 95% CI 1.23 to 1.40), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; 95% CI 1.07 to 1.28) in 34 reports (8794 abstracts). Results emanating from randomized or controlled trials showed the same pattern for both definitions (RR = 1.21; 95% CI 1.10 to 1.32 (15 reports and 2616 abstracts) and RR = 1.17; 95% CI, 1.04 to 1.32 (13 reports and 2307 abstracts), respectively.Other factors associated with full publication include oral presentation (RR = 1.46; 95% CI 1.40 to 1.52; studied in 143 reports with 115,910 abstracts); acceptance for meeting presentation (RR = 1.65; 95% CI 1.48 to 1.85; 22 reports with 22,319 abstracts); randomized trial design (RR = 1.51; 95% CI 1.36 to 1.67; 47 reports with 28,928 abstracts); and basic research (RR = 0.78; 95% CI 0.74 to 0.82; 92 reports with 97,372 abstracts). Abstracts originating at an academic setting were associated with full publication (RR = 1.60; 95% CI 1.34 to 1.92; 34 reports with 16,913 abstracts), as were those considered to be of higher quality (RR = 1.46; 95% CI 1.23 to 1.73; 12 reports with 3364 abstracts), or having high impact (RR = 1.60; 95% CI 1.41 to 1.82; 11 reports with 6982 abstracts). Sensitivity analyses excluding reports that were abstracts themselves or classified as having a high risk of bias did not change these findings in any important way.In considering the reports of the methodology research that we included in this review, we found that reports published in English or from a native English-speaking country found significantly higher proportions of studies published in full, but that there was no association with year of report publication. The findings correspond to a proportion of abstracts published in full of 31.9% for all reports, 40.5% for reports in English, 42.9% for reports from native English-speaking countries, and 52.2% for both these covariates combined.

AUTHORS' CONCLUSIONS

More than half of results from abstracts, and almost a third of randomized trial results initially presented as abstracts fail to be published in full and this problem does not appear to be decreasing over time. Publication bias is present in that 'positive' results were more frequently published than 'not positive' results. Reports of methodology research written in English showed that a higher proportion of abstracts had been published in full, as did those from native English-speaking countries, suggesting that studies from non-native English-speaking countries may be underrepresented in the scientific literature. After the considerable work involved in adding in the more than 300 additional studies found by the February 2016 searches, we chose not to update the search again because additional searches are unlikely to change these overall conclusions in any important way.

Registration and Outcome-Reporting Bias in Randomized Controlled Trials of Distal Radial Fracture Treatment

BACKGROUND

The purpose of the present study was to systematically evaluate the completeness of trial registration and the extent of outcome-reporting bias in modern randomized controlled trials (RCTs) relating to the treatment of distal radial fracture.

METHODS

With use of 4 databases (PubMed, Cochrane CENTRAL, Embase, and PEDro), this systematic review identified all RCTs of distal radial fracture treatment published from January 1, 2010, to December 31, 2015. We independently determined the registration status of these trials in a public trial registry and compared the characteristics of registered and non-registered trials. We assessed the quality and consistency of primary outcome measure (POM) reporting between the registration data and the final published studies.

RESULTS

Ninety studies met the inclusion criteria. Of those, only 28 (31%) were registered, and only 3 (3%) were "appropriately registered" (i.e., prospectively registered and identifying and fully describing the POM). Registered trials had larger sample sizes and were more likely to be multicenter, to report funding sources, and to be published in higher-impact-factor journals. Sixteen (18%) of the 90 registered RCTs named a POM in the registry; 7 (44%) of those 16 registered RCTs stated a different POM, an additional POM, or no POM at all in the final publication than was stated in the registry data. Additionally, 13 (81%) of those 16 registered RCTs had discrepancies in the time point reported for the POM.

CONCLUSIONS

In an attempt to address publication and outcome-reporting bias, prospective trial registration in a public registry has been deemed a condition for publication by the International Committee of Medical Journal Editors (ICMJE) since 2005. This study shows poor registration rates as well as inconsistencies in the reporting of POMs of recent trials relating to the treatment of distal radial fracture, one of the most common and most investigated injuries in orthopaedic practice.

CLINICAL RELEVANCE

The problems of registration and outcome-reporting bias in RCTs are important to highlight and address, and to find a solution will require the cooperation of researchers, reviewers, and journal editors. Increasing the transparency and consistency of reporting will help to increase the quality of research, which can impact patient care through evidence-based guidelines.

An Evaluation of Reporting Guidelines and Clinical Trial Registry Requirements Among Orthopaedic Surgery Journals

BACKGROUND

The responsibility for ensuring that studies are adequately reported is primarily that of those conducting the study; however, journal policies may influence how thoroughly authors choose to report their research. The use of reporting guidelines and prospective trial registration are promising avenues for ensuring that published studies adhere to the highest methodological standards. The purpose of this study is to evaluate orthopaedic surgery journal policies regarding reporting guidelines and trial registration, and to evaluate the effects that these policies have on adherence to reporting.

METHODS

We conducted a cross-sectional survey of journal policies and "Instructions for Authors" to determine the journals' policies and guidance regarding use of reporting guidelines and study registration. We also examined whether trials published in journals referencing CONSORT (Consolidated Standards of Reporting Trials) had higher rates of compliance with publishing a CONSORT flow diagram and whether journals with trial registration policies were more likely to contain registered trials than journals without these requirements.

RESULTS

Of the 21 orthopaedic surgery journals, 6 (29%) did not mention a single guideline, and clinical trial registration was required by 11 (52%) orthopaedic surgery journals and recommended by 2 (10%). Of the 21 general medical journals, 3 (14%) did not mention a single guideline, and trial registration was required by 13 (62%) general medical journals and recommended by 5 (24%) others. Furthermore, journals that referenced CONSORT were more likely to publish trials with a CONSORT flow diagram. Journals with trial registration policies were more likely to publish registered trials.

CONCLUSIONS

Reporting guidelines and trial registration are suboptimally required or recommended by orthopaedic surgery journals. These 2 mechanisms may improve methodology and quality, and should be considered for adoption by journal editors in orthopaedic surgery.

CLINICAL RELEVANCE

Because orthopaedic surgeons rely on high-quality research to direct patient care, measures must be taken to ensure that published research is of the highest quality. The use of reporting guidelines and prospective clinical trial registration may improve the quality of orthopaedic research, thereby improving patient care.

A systematic review of comparisons between protocols or registrations and full reports in primary biomedical research

BACKGROUND

Prospective study protocols and registrations can play a significant role in reducing incomplete or selective reporting of primary biomedical research, because they are pre-specified blueprints which are available for the evaluation of, and comparison with, full reports. However, inconsistencies between protocols or registrations and full reports have been frequently documented. In this systematic review, which forms part of our series on the state of reporting of primary biomedical, we aimed to survey the existing evidence of inconsistencies between protocols or registrations (i.e., what was planned to be done and/or what was actually done) and full reports (i.e., what was reported in the literature); this was based on findings from systematic reviews and surveys in the literature.

METHODS

Electronic databases, including CINAHL, MEDLINE, Web of Science, and EMBASE, were searched to identify eligible surveys and systematic reviews. Our primary outcome was the level of inconsistency (expressed as a percentage, with higher percentages indicating greater inconsistency) between protocols or registration and full reports. We summarized the findings from the included systematic reviews and surveys qualitatively.

RESULTS

There were 37 studies (33 surveys and 4 systematic reviews) included in our analyses. Most studies (n = 36) compared protocols or registrations with full reports in clinical trials, while a single survey focused on primary studies of clinical trials and observational research. High inconsistency levels were found in outcome reporting (ranging from 14% to 100%), subgroup reporting (from 12% to 100%), statistical analyses (from 9% to 47%), and other measure comparisons. Some factors, such as outcomes with significant results, sponsorship, type of outcome and disease speciality were reported to be significantly related to inconsistent reporting.

CONCLUSIONS

We found that inconsistent reporting between protocols or registrations and full reports of primary biomedical research is frequent, prevalent and suboptimal. We also identified methodological issues such as the need for consensus on measuring inconsistency across sources for trial reports, and more studies evaluating transparency and reproducibility in reporting all aspects of study design and analysis. A joint effort involving authors, journals, sponsors, regulators and research ethics committees is required to solve this problem.

Published intimate partner violence studies often differ from their trial registration records

INTRODUCTION

Registering study protocols in a trial registry is important for methodologic transparency and reducing selective reporting bias. The objective of this investigation was to determine whether published studies of intimate partner violence (IPV) that had been registered matched the registration record on key study design elements.

METHODS

We systematically searched three trial registries to identify registered IPV studies and the published literature for the associated publication. Two authors independently determined for each study whether key study elements in the registry matched those in the published paper.

RESULTS

We included 66 studies published between 2006 and 2017. Nearly half (29/66, 44%) were registered after study completion. Many (26/66, 39%) had discrepancies regarding the primary outcome, and nearly two-thirds (42/66, 64%) had discrepancies in secondary outcomes. Discrepancies in study design were less frequent (13/66, 20%). However, large changes in sample size (26/66, 39%) and discrepancies in funding source (28/66, 42%) were frequently observed.

CONCLUSIONS

Trial registries are important tools for research transparency and identifying and preventing outcome switching and selective outcome reporting bias. Published IPV studies often differ from their records in trial registries. Researchers should pay close attention to the accuracy of trial registry records.

Inconsistent selection of outcomes and measurement devices found in shoulder arthroplasty research: An analysis of studies on ClinicalTrials.gov

Introduction

Recent evidence suggests a lack of standardization of shoulder arthroplasty outcomes. This issue is a limiting factor in systematic reviews. Core outcome set (COS) methodology could address this problem by delineating a minimum set of outcomes for measurement in all shoulder arthroplasty trials.

Methods

A ClinicalTrials.gov search yielded 114 results. Eligible trials were coded on the following characteristics: study status, study type, arthroplasty type, sample size, measured outcomes, outcome measurement device, specific metric of measurement, method of aggregation, outcome classification, and adverse events.

Results

Sixty-six trials underwent data abstraction and data synthesis. Following abstraction, 383 shoulder arthroplasty outcomes were organized into 11 outcome domains. The most commonly reported outcomes were shoulder outcome score (n = 58), pain (n = 33), and quality of life (n = 15). The most common measurement devices were the Constant-Murley Shoulder Outcome Score (n = 38) and American Shoulder and Elbow Surgeons Shoulder Score (n = 33). Temporal patterns of outcome use was also found.

Conclusion

Our study suggests the need for greater standardization of outcomes and instruments. The lack of consistency across trials indicates that developing a core outcome set for shoulder arthroplasty trials would be worthwhile. Such standardization would allow for more effective comparison across studies in systematic reviews, while at the same time consider important outcomes that may be underrepresented otherwise. This review of outcomes provides an evidence-based foundation for the development of a COS for shoulder arthroplasty.

Misconceptions, Misuses, and Misinterpretations of P Values and Significance Testing

The interpretation and reporting of p values and significance testing in biomedical research are fraught with misconceptions and inaccuracies. Publications of peer-reviewed research in orthopaedics are not immune to such problems. The American Statistical Association (ASA) recently published an official statement on the use, misuse, and misinterpretation of statistical testing and p values in applied research. The ASA statement discussed 6 principles: (1) "P-values can indicate how incompatible the data are with a specified statistical model." (2) "P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone." (3) "Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold." (4) "Proper inference requires full reporting and transparency." (5) "A p-value, or statistical significance, does not measure the size of an effect or the importance of a result." (6) "By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis." The purpose of this article was to discuss these principles. We make several recommendations for moving forward: (1) Authors should avoid statements such as "statistically significant" or "statistically nonsignificant." (2) Investigators should report the magnitude of effect of all outcomes together with the appropriate measure of precision or variation. (3) Orthopaedic residents and surgeons must be educated in biostatistics, the ASA principles, and clinical epidemiology. (4) Journal editors and reviewers need to be familiar with and enforce the ASA principles.

A systematic review of the processes used to link clinical trial registrations to their published results

BACKGROUND

Studies measuring the completeness and consistency of trial registration and reporting rely on linking registries with bibliographic databases. In this systematic review, we quantified the processes used to identify these links.

METHODS

PubMed and Embase databases were searched from inception to May 2016 for studies linking trial registries with bibliographic databases. The processes used to establish these links were categorised as automatic when the registration identifier was available in the bibliographic database or publication, or manual when linkage required inference or contacting of trial investigators. The number of links identified by each process was extracted where available. Linear regression was used to determine whether the proportions of links available via automatic processes had increased over time.

RESULTS

In 43 studies that examined cohorts of registry entries, 24 used automatic and manual processes to find articles; 3 only automatic; and 11 only manual (5 did not specify). Twelve studies reported results for both manual and automatic processes and showed that a median of 23% (range from 13 to 42%) included automatic links to articles, while 17% (range from 5 to 42%) of registry entries required manual processes to find articles. There was no evidence that the proportion of registry entries with automatic links had increased (R ² = 0.02, p = 0.36). In 39 studies that examined cohorts of articles, 21 used automatic and manual processes; 9 only automatic; and 2 only manual (7 did not specify). Sixteen studies reported numbers for automatic and manual processes and indicated that a median of 49% (range from 8 to 97%) of articles had automatic links to registry entries, and 10% (range from 0 to 28%) required manual processes to find registry entries. There was no evidence that the proportion of articles with automatic links to registry entries had increased (R ² = 0.01, p = 0.73).

CONCLUSIONS

The linkage of trial registries to their corresponding publications continues to require extensive manual processes. We did not find that the use of automatic linkage has increased over time. Further investigation is needed to inform approaches that will ensure publications are properly linked to trial registrations, thus enabling efficient monitoring of trial reporting.

Evidence of selective reporting bias in hematology journals: A systematic review

Introduction

Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.

Methods

Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.

Results

Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.

Conclusion

Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Shoulder Arthroplasty Trials Are Infrequently Registered: A Systematic Review of Trials

INTRODUCTION

With the intent of improving transparency in clinical research, the International Committee of Medical Journal Editors (ICMJE) established guidelines in 2005 regarding prospective clinical trial registration. This action worked to address bias related to selective outcome reporting in the medical literature. The objective of this study was to assess and characterize the quality of registration of clinical trials appearing in shoulder arthroplasty-related medical journals.

METHODS

All randomized trials involving human subjects, pertaining to shoulder arthroplasty, published between July 1, 2005 and December 31, 2015, and indexed in either PubMed or SportDISCUS were analyzed. We assessed the prevalence of registration, the timing of registration relative to patient enrollment periods, and the variable rates of orthopedic journal compliance with ICMJE and Food and Drug Administration clinical registration standards for our study.

RESULTS

Of the 382 articles identified, 345 (90.3%) were excluded due to failure to meet inclusion criteria. From the remaining 37, only 12 (32.4%) studies were found to be registered in a trial registry. Ten (10/12, 83.3%) of these provided their registration information within the body of the article. None of the included studies from ICMJE-recognized journals were registered. From 34 included studies from non-ICMJE recognized journals, 12 (35.3%) were registered.

CONCLUSION

The level of compliance with clinical trial registration guidelines in the decade since their release among shoulder arthroplasty trials in orthopedic journals is poor. Given the importance of the issue, the prevalence of the problem, and the fact that many other medical specialties have already made efforts to improve ICMJE compliance, further work on the part of orthopedic surgery journal authors and editors is needed to ensure the publication of unbiased results.

TRIAL REGISTRATION

UMIN000022487.

Editorial Commentary: Cost-Effectiveness of Orthopaedic Procedures: "A Nickel Ain't Worth a Dime Anymore" - Yogi Berra

Although cost-effectiveness research has assumed a critical role in decision making in medicine, it, by definition, must be derived from solid, germane, reproducible data. The absence of such high-quality data tends to induce compromises in analysis that may degrade a scientific article's seemingly intuitive message.