Trends in the Level of Evidence in Clinical Hand Surgery Research

Quality of Outcomes Research in Total Ankle Arthroplasty

Total ankle arthroplasty is a topic that has recently gained increasing interest, largely due to the improved outcomes, which have been demonstrated by short- and mid-term research studies on the newer, third-generation implant designs. The purpose of this review is to provide an updated assessment of the quality of outcomes research on total ankle arthroplasty.

Ten-Year Trends in Level of Evidence in Hand Surgery

BACKGROUND

The purposes of this study were to analyze the trends in Oxford level of evidence (LOE), LOE of most-cited articles, and association between LOE and journal impact factor (IF) and SCImago Journal Rank (SJR) over a 10-year period (2009-2018) in 3 prominent hand surgery journals, specifically HAND, Journal of Hand Surgery (American Volume) (JHS), and Journal of Hand Surgery (European Volume) (JHSE).

METHODS

All articles published from 2009 to 2018 in HAND, JHS, and JHSE were reviewed for assigned or available LOE. Data were pooled and analyzed for trends in LOE; relationship among IF, SJR, and LOE; and citation density.

RESULTS

A total of 3921 total publications were tabulated from 2009 to 2018, with the majority of studies being level V (1700, 43%) and fewer studies being level I (146, 4%). Over the 10-year study period, there was no significant change in frequency of level I studies for any journal. HAND trended significantly toward higher LOE, JHS trended toward higher LOE, and JHSE trended toward decreased LOE without significance. Among all journals, the annual number of articles and the average LOE were independent significant predictors of IF and SJR. Statistically significant correlations were found between citation density and LOE for JHS and HAND.

CONCLUSIONS

Higher quality evidence is becoming more prevalent in the hand surgery literature over the past 10 years. Annual articles, average LOE, and level I and II and level IV articles were significant predictors of increasing IF and SJR.

Are Observational Studies on Distal Radius Fracture Treatment Robust? An E-value Approach to Analysis

BACKGROUND

Reported complication frequencies after distal radius fracture (DRF) treatment vary widely in the literature and are based mostly on observational evidence. Whether that evidence is sufficiently robust to use in practice is controversial. The E-value is an innovative sensitivity analysis that quantitates the robustness of observational evidence against unmeasured confounders, whereby a greater E-value usually implies more robust evidence and vice versa; with DRF complications, this approach can help guide readers to a more confident interpretation of the available evidence.

QUESTIONS/PURPOSES

In this study, we sought (1) to compare the complication frequencies among different DRF treatment modalities, and (2) to evaluate the robustness of these observational studies using the E-value as an index for unmeasured confounding.

METHODS

We searched PubMed, Embase, and SCOPUS for observational studies on the management of DRFs that were published from January 2001 to July 2021 with the last database search performed on July 31, 2021. All articles that compared different DRF treatment modalities with reported complication frequencies were included to accurately capture the quality of the observational studies in research about DRF. Risk ratios (RRs) of the overall complication and major complication risks were calculated for each subgroup comparison: volar plating versus dorsal plating, casting, external fixation, and percutaneous K-wire fixation. The RRs and their corresponding lower limits of the 95% confidence intervals (CIs) were used to derive the E-values. E-values can have a minimum possible value of 1, which signifies that the treatment-outcome association is not strong and can readily be overturned by unmeasured confounders. By contrast, a large E-value means that the observed treatment-outcome association is robust against unmeasured confounders. We averaged RRs and E-values for the effect estimates and lower limits of CIs across studies in each treatment comparison group. We identified 36 comparative observational studies that met the inclusion criteria. Seven studies compared volar with dorsal plating techniques. Volar plating was also compared with casting (eight studies), external fixation (15 studies), and percutaneous K-wire fixation (six studies).

RESULTS

Total and major complication risks did not differ among different DRF treatments. The mean RRs for total and major complications were 1.2 (95% CI 0.4 to 3.9; p = 0.74) and 1.8 (95% CI 0.4 to 11.4; p = 0.52) for the volar versus dorsal plating group; 1.2 (95% CI 0.3 to 11.2; p = 0.87) and 1.5 (95% CI 0.3 to 14.9; p = 0.74) for the volar plating versus casting group; 0.6 (95% CI 0.2 to 2.2; p = 0.33) and 0.8 (95% CI 0.2 to 6.7; p = 0.86) for the volar plating versus external fixation group; and 0.6 (95% CI 0.2 to 2.6; p = 0.47) and 0.7 (95% CI 0.2 to 4.0; p = 0.67) for the volar plating versus K-wire fixation group. The mean E-values for total and major complication frequencies for the between-group comparison ranged from 3.1 to 5.8; these were relatively large in the context of a known complication risk factor, such as high-energy impact (RR 3.2), suggesting a reasonable level of robustness against unmeasured confounding. However, the E-values for lower limits of CIs remained close to 1, which indicates the observed complication frequencies in these studies were likely to have been influenced by unmeasured confounders.

CONCLUSION

Complication frequencies did not differ among different DRF treatment modalities, but the observed complication frequencies from most comparative observational studies were less robust against potential unmeasured confounders. The E-value method, or another type of sensitivity analysis, should be implemented in observational hand surgery research at the individual-study level to facilitate assessment of robustness against potential unmeasured confounders.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Bias in Hand Surgical Randomized Controlled Trials: Systematic Review and Meta-Epidemiological Study

PURPOSE

Inappropriately reported or conducted studies may decrease the quality of care due to under- or overestimation of the benefits or harms of interventions. Our aim was to evaluate how often hand surgical randomized controlled trials (RCTs) use and report adequate methods to ensure internal validity, and whether inadequate reporting or methods are associated with the magnitude of treatment effect estimates.

METHODS

Data Sources were the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase databases until November 2020. We included published RCTs investigating the effects of any surgical intervention in the hand and wrist region. We assessed internal validity using the Cochrane Risk of Bias (RoB) tool for 6 domains: selection, performance, detection, attrition, selective reporting, and "other" bias. We extracted the primary outcome and calculated the effect size for each study. We used mixed-effect meta-regression to assess whether the RoB modified the magnitude of the effects.

RESULTS

For 207 assessed trials, the RoB was unclear or high for 72% in selection, 93% in performance, 88% in detection, 25% in attrition, 22% in selective reporting, and 34% in the "other" bias domain. Trials with a high or unclear risk of selection bias yielded 0.28 standardized mean difference (95% confidence interval, 0.02-0.55) larger effect sizes compared to studies with a low risk. Risks of bias for other domains did not modify the intervention effects. The risk for selection bias declined over time: the odds ratio for a high or unclear RoB was 0.90 (95% confidence interval, 0.85-0.95) per additional year of publication CONCLUSIONS: The internal validity and credibility of hand surgical RCTs can be improved by using established methods to achieve true randomization, blinding of the participants and study personnel, publishing the trial protocol and avoiding selective reporting of the outcomes, and reporting the trial as recommended in the Consolidated Standards of Reporting Trials statement.

CLINICAL RELEVANCE

Clinicians should be aware that RCTs that do not use or report proper randomization and allocation concealment may overestimate the treatment effects.

Quality Assessment of Modern Total Ankle Arthroplasty Clinical Outcomes Research

Research demonstrating improved outcomes with third-generation ankle replacement implants has resulted in increasing utilization of total ankle arthroplasty over the past 3 decades. The purpose of this study was to examine the quality and trends of clinical outcomes research being published on third-generation total ankle arthroplasty implants. Two fellowship-trained foot and ankle surgeons reviewed all peer-reviewed, Medline-indexed English-language clinical outcomes studies evaluating total ankle arthroplasty published between 2006 and 2019. Articles were assessed for study design and indicators of study quality. A total of 694 published articles were reviewed and 231 met all inclusion criteria. The majority (78%) of studies were retrospective, most of which were case series (54%) or cohorts (32%). Ten percent (10%) of studies were funded by industry and 28% did not disclose funding sources. Thirty-eight percent (38%) of studies reported a conflict of interest and 6% did not disclose whether or not there were conflicts. The average patient follow-up time across studies was 72 months. We found that although the study of outcomes with third-generation total ankle arthroplasty prostheses is steadily increasing, most studies are Level IV, retrospective case series. Some studies have disclosed industry funding and/or a conflict of interest, and a considerable number did not disclose potential funding and/or financial conflicts. Future investigators should strive to design studies with the highest quality methodology possible.

Evaluating Strength of Evidence of Pediatric Otolaryngology Research Literature: A 20-Year Review

OBJECTIVES/HYPOTHESIS

Quantity and quality of Otolaryngology-Head and Neck Surgery (OTL-HNS) research are increasing, yet patterns within Pediatric OTL-HNS publications are unknown. This study examines trends in the level of evidence of pediatric OTL-HNS articles over a 20-year period to quantify the growth and characterize contributing factors.

STUDY DESIGN

Review article.

METHODS

A retrospective review was conducted on 12 peer-reviewed OTL-HNS journals at three time-points: 1996, 2006, and 2016. Pediatric-specific OTL-HNS journals were selected; all were among the top 10 highest impact factor journals, with one pediatric-specific and one Canadian journal. Publication details, author characteristics, and study focus were collected. Papers were classified based on the Oxford Centre for Evidence-Based Medicine Levels of Evidence by two independent reviewers.

RESULTS

Of the 1,733 articles reviewed, 727 met inclusion criteria. A greater absolute number of pediatric OTL-HNS articles were published over the years studied: from 95 in 1996 to 359 in 2016 (P < .001). As well, the absolute number of high-quality studies has increased over the study period, from 28 articles in 1996 to 100 articles in 2016. However, the relative percentage of high-quality papers remained stable between 27.9% and 32.2% with an average of 29.7% (P = .89). Higher impact factor journals did not tend to publish higher-quality pediatric OTL-HNS articles (P = .48).

CONCLUSIONS

Over the past 20 years, there is no appreciable improvement in the proportion of high-quality publications in pediatric OTL-HNS; however, there is an overall greater number of high-quality papers within OTL-HNS literature. These findings likely relate to challenges of research within pediatric surgical specialties.

LEVEL OF EVIDENCE

NA Laryngoscope, 2021.

Clinical Level of Evidence Presented at the Cervical Spine Research Society (CSRS) Annual Meeting Over 10 Years (2008-2017): A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVE

We systematically reviewed the level of clinical evidence presented at Cervical Spine Research Society annual meetings from 2008 through 2017.

SUMMARY OF BACKGROUND DATA

The Cervical Spine Research Society is dedicated to advancing knowledge of the cervical spine to promote evidence-based care. Research presented at these meetings impacts clinical practice.

METHODS

A total of 774 paper abstracts presented at Cervical Spine Research Society (CSRS) annual meetings were independently assessed by two reviewers. Reviewers designated a clinical level of evidence (LOE) to each included abstract from level I to level IV based on criteria set forth by the Oxford Centre for Evidence-Based Medicine. Reviewer agreement was assessed using Cohens Kappa coefficient (k) and disagreements were discussed until a consensus was reached. Wilcoxon rank sum test was used to assess for differences in LOE grades. Chi-squared testing was used to assess nonrandom changes in level of evidence and in excluded studies.

RESULTS

A total of 583 abstracts were included. Over the last 10 CSRS meetings, 5.15% of presentations were level I, 27.8% level II, 27.4% level III, and 39.6% level IV. The average LOE from 2008 to 2017 was 3.02 (median = 3). Additionally, 49.7% were therapeutic studies, 37.6% prognostic studies, and 12.7% diagnostic studies. When comparing the first 5 years (2008-2012) to the last 5 years (2013-2017), we observed a significant increase in Level II (P = 0.007) evidence and a corresponding decrease in level IV evidence (P < 0.001). The average LOE improved from 3.14 (2008-2012) to 2.91 (2013-2017); there was a significant improvement in LOE between the two periods (P = 0.001).

CONCLUSION

Emphasis on evidence-based medicine within cervical spine research has positively influenced the clinical level of evidence disseminated at CSRS annual meetings between 2008 and 2017. Continued focus on higher quality Level I studies is warranted.

LEVEL OF EVIDENCE

4.

Mitigating Biases in a Cohort Study: A Critical Examination of the Silicone Arthroplasty in Rheumatoid Arthritis Study

Research bias, or the systematic errors of a study, can arise before, during, or after a trial ends. These biases hinder the internal validity of the study, which is the accuracy of a study's conclusions regarding the effects of an intervention on a given group of subjects. With the growing use of evidence-based medicine, there is a demand for high-quality evidence from the research community. Systematic reviews and meta-analyses of randomized controlled trials are considered the highest level of evidence, followed by individual randomized controlled trials. However, most surgical trials cannot be conducted as randomized controlled trials because of factors such as patient preferences and lack of equipoise among surgeons. Therefore, surgical trials may lack features that are held as important standards for high-quality evidence, such as randomization and blinding. To demonstrate the biases that surgical trials may encounter, the authors examined a prospective cohort study, the Silicone Arthroplasty in Rheumatoid Arthritis study. The authors focus on the challenges that arise during a surgical trial, including the design, implementation, and methods used to report the clinical evidence. By recognizing and addressing obstacles that exist in research, investigators will provide health care providers with high-quality evidence needed to make well-informed, evidence-based clinical decisions.

Using Evidence for Hand Surgery: How to Practice Evidence-Based Hand Surgery Care

Each step of the evidence-based practice process is critical and requires clear understanding for accurate application. To practice evidence-based care, providers must acquire a specific skillset that facilitates translation of a patient problem into an answerable research question. Additional requirements are understanding of electronic databases, critical appraisal of the available evidence, and integration of the findings to generate a specific, individualized treatment plan. Although this process is demanding, evidence-based practice is essential in the delivery of optimal patient care.

Bringing Evidence into Practice in Hand Surgery

Despite the significant investment in scientific investigation to enhance clinical care, the uptake of new interventions and innovations into clinical care and policy remains slow. Understanding and examining the factors that influence the dissemination and implementation of best practices are critical to promote high-quality health care. This review provides an overview of the evidence base in hand surgery, the science that underlies dissemination and innovation, and the emergence of learning health systems in health care.

Translating Hand Surgery Evidence to Policy and Practice

Hand surgery researchers should focus on developing high-quality evidence to support the development of health policies affecting surgical care. Policy-makers and leaders of national hand societies can help reduce the variation of care for patients receiving hand surgery by incorporating evidence into guidelines and policies. Comprehensive guidelines for perioperative care help encourage the translation of evidence into practice. Moreover, the identification of institutional-level barriers and facilitators of integration ensures the successful implementation of hand surgery-specific programs. The development of robust metrics to evaluate the effect of policy on practice helps examine the feasibility of clinical guidelines.

Treatment interventions for hand fractures and joint injuries: a scoping review of randomized controlled trials

The aim of this study was to identify and assess all existing randomized studies on treatment interventions for hand fractures and joint injuries, to inform practice and plan future research. PubMed, Cochrane CENTRAL, MEDLINE and Embase were searched. We identified 78 randomized controlled trials published over 35 years, covering seven anatomical areas of the hand. We report on sources of bias, sample size, follow-up length and retention, outcome measures and reporting. In terms of interventions studied, the trials were extremely heterogeneous, so it is difficult to draw conclusions on individual treatments. The published randomized controlled clinical trial evidence for hand fractures and joint injuries is narrow in scope and of generally low methodological quality. Mapping provides a useful resource and stepping-stone for planning further research. There is a need for high-quality, collaborative research to guide management of a wider range of common hand injuries.

Establishing a National Registry for Hand Surgery

Hand surgery leadership in the United States must identify and define what quality care means for its patients. To achieve this, the surgical team needs a standardized framework to track and improve quality. This is necessary not only in our value-based health care system but also in light of considerable provider variation in the management of common hand conditions and the ongoing need for evidence-based guidelines to inform decision-making. Building a national registry for the field of hand surgery could be the solution and warrants serious consideration. A registry designed by hand surgery experts can collect data on process and outcome measures that are meaningful and specific to patients with hand conditions. These data inform the surgical team regarding where to focus their efforts for improvement. Existing methods of quality measurement are not compatible with hand surgery, a field with an ambulatory setting and rare incidence of mortality. Patient-reported outcomes, such as health-related quality of life, represent a more useful measure of quality for hand surgery and are just one example of the type of data that could be tracked using a national registry. An investment in a large-scale registry could seamlessly integrate patient preferences, values, and expectations into clinical practice so that desired outcomes can be delivered consistently across the nation.

Should We Question the External Validity of Database Studies? A Comparative Analysis of Demographics

PURPOSE

To define the external validity of national and institutional databases for common sports medicine procedures.

METHODS

Patient demographic data including age, sex, body mass index (BMI), and 4 racial categories were aggregated between 2007 and 2016 across 2 databases for 4 common sports medicine procedures: anterior cruciate ligament reconstruction, arthroscopic rotator cuff repair (RCR), partial meniscectomy (PMx), and both arthroscopic and open shoulder stabilization. The first database of interest was a prospectively collected institutional database. The second was the National Surgical Quality Improvement Program (NSQIP) database. Two-sample t tests were performed to examine mean differences (MDs) in age and BMI, and χ² testing was used to test differences in sex and race.

RESULTS

A total of 7,019 institutional and 108,881 NSQIP patients were examined. The NSQIP cohort was significantly older (MD, 1.40 years), included more female patients (42.60% female patients vs 35.67% female patients), and showed a different racial distribution compared with the institutional data (all P < .0001). The NSQIP PMx cohort (MD, 7.38 years) was significantly older and the NSQIP RCR cohort (MD, 1.97 years) was significantly younger than their institutional counterparts (all P < .0001). The NSQIP anterior cruciate ligament reconstruction cohort (MD, 2.53) showed a greater average BMI (P < .0001). The NSQIP RCR cohort (41.8% female patients vs 33.3% female patients) and PMx cohort (46.0% female patients vs 37.9% female patients) also included more female patients. Race was distributed variably between databases for each procedure code (all P < .0001).

CONCLUSIONS

Significant differences in age, BMI, sex, and race distributions were observed between an institutional database and the NSQIP database. This study underlines the importance of defining the generalizability of database research, particularly when significant demographic differences between databases may underlie differences in postoperative outcomes.

LEVEL OF EVIDENCE

Level III, cross-sectional study.

Common Topics of Publication and Levels of Evidence in the Current Hand Surgery Literature

Background Scientific publications are the primary vehicle for the distribution of scientific findings, but there has been limited research on literature topic surveillance. We sought to identify and characterize the most commonly published topic domains in the hand surgery literature. Methods We performed a 6-month hypothesis testing phase to identify the most frequently published topics in three hand surgery journals: Hand, The Journal of Hand Surgery (American), and The Journal of Hand Surgery (European). We reviewed all of the published articles in these journals from June 2010 to May 2015 to identify and characterize publications related to the three most common topic domains. Results A total of 2,146 articles were published during the 5-year study period. The three most frequent topics domains included distal radius (DR) (11% of all articles), flexor tendon (FT) (9%), and carpal tunnel (CT) (7.5%). These subjects accounted for a total of 584 articles (27% of all publications) and 3,014 published pages during the study period. FT, CT, and DR publications were cited on average 2.3 times per year (2.5, 2.4, and 2.0, respectively). Conclusion A small subset of topic domains makes up a significant proportion of scientific publications in hand surgery.

Quality of Randomized Controlled Trials for Surgical Treatment of Carpal Tunnel Syndrome: A Systematic Review

BACKGROUND

Randomized controlled trials are considered the gold standard in evidence-based medicine. The authors conducted a systematic review to evaluate the quantity, quality, and trends of randomized controlled trials that assess surgical treatment of carpal tunnel syndrome.

METHODS

The authors identified randomized controlled trials comparing two or more surgical interventions for carpal tunnel syndrome in PubMed, Cochrane, Scopus, Google Scholar, and Clinicaltrials.gov. Two independent reviewers evaluated articles for inclusion, extracted data, and assessed randomized controlled trial quality using the Jadad score.

RESULTS

Of 2253 identified studies, 58 met full inclusion criteria. They were published between 1985 and 2015, with a significant increase over time (p = 0.003). They were most frequently published in Journal of Hand Surgery (European Volume) [n = 15 (25.9 percent)]. Most randomized controlled trials were single-center studies [n = 54 (93.1 percent)] conducted in the United Kingdom [n = 13 (22.4 percent)] or the United States [n = 10 (17.2 percent)], with a mean study size of 80.1 ± 55.5 patients. Funding source was unknown in 62.1 percent (n = 36). Three-quarters [n = 44 (75.9 percent)] of randomized controlled trials did not define the primary outcome measure(s). Less than 30 percent (n = 17) of randomized controlled trials conducted a power analysis. Only four studies with patients reported lost to follow-up provided an explanation for each patient. Six randomized controlled trials (10.3 percent) conducted intention-to-treat analysis. The mean Jadad score was 2.14 ± 1.26, with no significant improvement over time (p = 0.245).

CONCLUSIONS

Despite the significant increase in the number of randomized controlled trials published studying surgical treatment of carpal tunnel syndrome over time, a mean Jadad score of 2.14 with no change over time indicates a need for improvement in quality. Proper study design is key to avoiding introduction of bias and ensuring the validity of conclusions drawn.

Quality of surgical randomized controlled trials in hand surgery: a systematic review

We assessed the quantity, quality and trends of randomized controlled trials comparing hand surgical interventions. Study characteristics were collected for 125 randomized controlled trials comparing hand surgical interventions. The Jadad scale (0-5), which assesses methodological quality of trials, was calculated. Logistic regressions were conducted to determine associations with the Jadad score. The studies were published between 1981 and 2015, with an increase over time, most often in Journal of Hand Surgery (European). Mean study size was 68 patients. Mean Jadad score was 2.1, without improvement over time. Thirty percent conducted a power analysis and 8% an intention-to-treat analysis. Studies conducted in the United Kingdom and with smaller sample sizes, power analysis and intention-to-treat analysis were associated with a higher Jadad score. The quantity of trials has increased over time while methodological quality has remained low, indicating a need to improve quality of trials in hand surgery literature.

Evidence hierarchies relating to hand surgery: current status and improvement. A bibliometric analysis study

BACKGROUND

Hierarchy of evidence is an important measurement for assessing quality of literature. Information regarding quality of evidence within the Brazilian hand surgery setting is sparse, especially regarding whether research has improved in either quality or quantity. This study aimed to identify and classify hand surgery studies published in the two most important Brazilian orthopedics journals based on hierarchy of evidence, with comparisons with previously published data.

DESIGN AND SETTING

Bibliometric analysis study performed in a federal university.

METHODS

Two independent researchers conducted an electronic database search for hand surgery studies published between 2010 and 2016 in Acta Ortopédica Brasileira and Revista Brasileira de Ortopedia. Eligible studies were subsequently classified according to methodological design, based on the Haynes pyramid model (HP) and the JBJS/AAOS levels of evidence and grades of recommendations (LOR). Qualitative and quantitative data were gathered regarding all studies. Previous data were considered to assess whether the proportion of high-quality studies had improved over time (2000-2009 versus 2010-2016).

RESULTS

The final analysis included 123 studies, mostly originating from the southeastern region (78.8%) and private institutions (65%), with self-funding (91.8%). Methodological assessment showed that 15.4% were classified as level I/II using HP and 16.4% using LOR. No significant difference in proportions of high-quality studies was found between the two periods of time assessed (5% versus 12%; P = 0.13).

CONCLUSION

Approximately 15% of hand surgery studies published in two major Brazilian journals were likely to be classified as high-quality through two different systems. Moreover, no trend towards quality-of-evidence improvement was found over the last 15 years.