Multiple testing in orthopedic literature: a common problem?

Outcome of the modified Lapidus procedure for hallux valgus deformity during the first year following surgery: A prospective clinical and gait analysis study

BACKGROUND

The modified Lapidus procedure is a surgical option to treat moderate to severe hallux valgus deformity with good radio-clinical outcome. However, comprehensive biomechanical outcome evaluation of this surgery at mid-term follow-up is missing. This study assesses and compares the radio-clinical and gait outcome at 6 and 12 months following modified Lapidus procedure.

METHOD

Ten consecutive female patients with moderate to severe hallux valgus who underwent modified Lapidus procedure participated in the study. Comprehensive gait assessment was performed preoperatively, at 6 and 12 months postoperatively. Gait parameters including spatiotemporal, kinematics and plantar pressure were analyzed using pressure insoles and 3-dimensional inertial sensors. Outcome was evaluated using two clinical questionnaires, i.e. the American Orthopaedic Foot and Ankle Score and the Foot and Ankle Ability Measure, and X-rays.

FINDINGS

Three spatiotemporal, two kinematics, and seven plantar pressure parameters significantly improved between 6 months and 12 months postoperatively. Significant improvement in radiological and clinical outcome was reported at 6 and 12 months. The Foot and Ankle Ability Measure showed non-significant improvement at 12 months.

INTERPRETATION

The outcome of this study is consistent with the previously reported good clinical and radiological results at one year following Lapidus for moderate to severe hallux valgus. Twelve gait parameters demonstrated that outcome improves from 6 months to 12 months postoperative with room for further improvement at long term. The gait outcome in this study confirms the longer rehabilitation period following modified Lapidus procedure. Studies with a larger sample size are required to confirm these findings.

P values in display items are ubiquitous and almost invariably significant: A survey of top science journals

P values represent a widely used, but pervasively misunderstood and fiercely contested method of scientific inference. Display items, such as figures and tables, often containing the main results, are an important source of P values. We conducted a survey comparing the overall use of P values and the occurrence of significant P values in display items of a sample of articles in the three top multidisciplinary journals (Nature, Science, PNAS) in 2017 and, respectively, in 1997. We also examined the reporting of multiplicity corrections and its potential influence on the proportion of statistically significant P values. Our findings demonstrated substantial and growing reliance on P values in display items, with increases of 2.5 to 14.5 times in 2017 compared to 1997. The overwhelming majority of P values (94%, 95% confidence interval [CI] 92% to 96%) were statistically significant. Methods to adjust for multiplicity were almost non-existent in 1997, but reported in many articles relying on P values in 2017 (Nature 68%, Science 48%, PNAS 38%). In their absence, almost all reported P values were statistically significant (98%, 95% CI 96% to 99%). Conversely, when any multiplicity corrections were described, 88% (95% CI 82% to 93%) of reported P values were statistically significant. Use of Bayesian methods was scant (2.5%) and rarely (0.7%) articles relied exclusively on Bayesian statistics. Overall, wider appreciation of the need for multiplicity corrections is a welcome evolution, but the rapid growth of reliance on P values and implausibly high rates of reported statistical significance are worrisome.

Statistical reporting in randomized controlled trials from the dermatology literature: a review of 44 dermatology journals

BACKGROUND

The validity of randomized controlled trials (RCTs) is determined by several statistical factors.

OBJECTIVES

To determine the level of recent statistical reporting in RCTs from the dermatology literature.

METHODS

We searched MEDLINE for all RCTs published between 1 May 2013 and 1 May 2014 in 44 dermatology journals.

RESULTS

Two hundred and ten articles were screened, of which 181 RCTs from 27 journals were reviewed. Primary study outcomes were met in 122 (67.4%) studies. Sample size calculations and beta values were reported in 52 (28.7%) and 48 (26.5%) studies, respectively, and nonsignificant findings were supported in only 31 (17.1%). Alpha values were reported in 131 (72.4%) of studies with 45 (24.9%) having two-sided P-values, although adjustment for multiple statistical tests was performed in only 16 (9.9% of studies with ≥ two statistical tests performed). Sample size calculations were performed based on a single outcome in 44 (86.3%) and multiple outcomes in six (11.8%) studies. However, among studies that were powered for a single primary outcome, 20 (45.5%) made conclusions based on multiple primary outcomes. Twenty-one (41.2%) studies relied on secondary/unspecified outcomes. There were no differences for primary outcome being met (Chi-square, P = 0.29), sample size calculations (P ≥ 0.55), beta values (P = 0.89), alpha values (P = 0.65), correction for multiple statistical testing (P = 0.59), two-sided alpha (P = 0.64), support of nonsignificant findings (Fisher's exact, P = 0.23) based on the journal's impact factor.

CONCLUSIONS

Levels of statistical reporting are low in RCTs from the dermatology literature. Future work is needed to improve these levels of reporting.

What was the survival of megaprostheses in lower limb reconstructions after tumor resections?

BACKGROUND

Prosthetic replacement is the most commonly used option for reconstruction of osteoarticular bone loss resulting from bone neoplasm resection or prosthetic failure. Starting in late 2001, we began exclusively using a single system for large-segment osteoarticular reconstruction after tumor resection; to our knowledge, there are no published series from one center evaluating the use of this implant.

QUESTIONS/PURPOSES

We investigated the following issues: (1) What is the overall survival, excluding local tumor recurrence, for these endoprostheses used for tumor reconstructions of the lower extremities (knee and hip)? (2) What types of failure were observed in these reconstructions? (3) Do the survival and complications vary according to site of implant?

METHODS

Between September 2001 and March 2012, we exclusively used this implant for tumor reconstructions. During that time, 278 patients underwent tumor reconstructions of the hip or knee, of whom 200 (72%) were available at a minimum 2 years followup. Seventy-eight patients were excluded from the study for insufficient followup as a result of early death (42) or loss at followup (36). The reconstruction types were the following: proximal femur (69 cases), distal femur (87), proximal tibia (32), and total knee (12). Failures were classified according to the Henderson classification. Nine patients among those with followup shorter than 2 years had presented one or more failures and they were included in our analysis but separately evaluated.

RESULTS

Overall survival (no further surgical procedures of any type after primary surgery), excluding Type 5 failure (tumor recurrence), was 75.9% at 5 years and 66.2% at 10 years. Seventy-one failures occurred in 58 implants (29%). Mechanical failures accounted for 59.2% and nonmechanical failures for 40.8%. The first causes of failure of the implants were the result of soft tissue failure in 6%, aseptic loosening in 3%, structural failure in 7%, infection in 8.5%, and tumor recurrence in 4.5% of the whole series. Nine implants sustained two or more failures. Overall incidence of infection was 9.5%. No statistically significant differences were observed according to anatomical site.

CONCLUSIONS

Like in the case with many such complex oncologic reconstructions, the failure rate at short- to midterm in this group was over 20%. Comparative trials are called for to ascertain whether one implant is superior to another. Infection and structural failure were the most frequent modes of failure in our experience.

LEVEL OF EVIDENCE

Level IV, therapeutic study. See Instructions for Authors for a complete description of levels of evidence.