The Robustness of Trials That Guide Evidence-Based Orthopaedic Surgery

Utilization of the Fragility Index to Assess Randomized Controlled Trials Comparing Cervical Total Disc Arthroplasty to Anterior Cervical Discectomy and Fusion

Study designSystematic Review.ObjectivesCervical total disc arthroplasty (CTDA) remains an alternative to anterior cervical discectomy and fusion (ACDF) in select patients with cervical radiculopathy or myelopathy secondary to degenerative disc disease. Studies comparing CTDA to ACDF often have conflicting conclusions and varying quality. The purpose of this study was to utilize the fragility index (FI) to assess the robustness of randomized controlled trials (RCT) comparing CTDA to ACDF.MethodsA systematic review was performed by searching PubMed, Ovid MEDLINE, Web of Science, and Embase for RCTs with 2 parallel study arms and 1:1 allocation of subjects investigating CTDA vs ACDF with at least 1 statistically significant, dichotomous outcome. The FI was calculated by individually shifting 1 patient from the event group to the non-event group with re-calculation of Fisher's Exact test until the reported P value was no longer statistically significant (P > 0.05).ResultsThe search identified 934 abstracts with 19 RCTs meeting inclusion criteria. The mean patient sample size was 276.4 (median 209, range 30-541). The number of patients lost to follow-up ranged from 0-229 (mean 69.7, median 45). The mean FI was 4.6 (range 0-30, median 2) with 3 (13.6%) of the studies having an associated FI of 0. Loss to follow up exceeded the fragility index in all but 2 studies.ConclusionRCTs comparing ACDF to CTDA are often fragile with only 1-2 patients experiencing an alternative outcome or lost to follow-up to change the studied outcome.

Beyond the Usual Significance: Fragility Indices of Randomized Controlled Trials in Top General Orthopaedic Journals

INTRODUCTION

Understanding the reliability of outcomes in randomized controlled trials (RCTs) is crucial, as standard metrics, such as P- value do not fully capture result fragility. This led to the adoption of specific indices: the fragility index (FI), which measures the strength of trial through significant results by calculating the minimum number of patient status changes from nonevent to event required to make the results statistically insignificant; reverse fragility index (RFI), used for insignificant results; and continuous fragility index (CFI), which acts similar to FI for significant continuous outcomes. The objective is to assess the robustness of orthopaedics RCTs using these indices across leading orthopaedic journals.

METHODS

A systematic review of RCTs published between 2019 and 2023 in the top five general orthopaedic journals, identified through Scientific Journal Rankings, was done. Data extraction focused on FI, RFI, and CFI and related measures for 160 RCTs. The indices were calculated using established methodologies, with sample size adjustments.

RESULTS

22 RCTs had statistically significant dichotomous primary outcomes and 17 studies had notable dichotomous secondary outcomes. Twenty-nine had negligible (insignificant) dichotomous primary outcomes, and 92 reported notable continuous outcomes. Only one RCT reported a FI in the article. The median FI was 5 with a median sample size of 142 for dichotomous outcomes. The median RFI was 3 with a median sample size of 100 for negligible outcomes, and the median CFI was 13 with a median sample size of 86.5 for continuous outcomes, showing dichotomous outcomes to be more fragile than continuous ones.

DISCUSSION

Continuous outcomes are less fragile than dichotomous outcomes, with negligible dichotomous outcomes being particularly more fragile. This fragility stems from small sample sizes and limited outcome events. Using these indices, especially when considering patient loss to follow-up, can improve the reliability of findings.

LEVEL OF EVIDENCE

I.

The Fragility of Statistical Findings Regarding Hemiarthroplasty Versus Total Hip Arthroplasty for Displaced Femoral Neck Fractures

There is debate over the treatment of displaced femoral neck fractures with either hemiarthroplasty or total hip arthroplasty. This study aimed to evaluate the fragility index (FI) of randomized controlled trials (RCTs) comparing these methods. We queried for these RCTs containing dichotomous outcomes, finding 11 RCTs with 63 dichotomous outcomes. The median FI for all outcomes was 6, signifying that 6 event reversals would change the statistical significance of an outcome. The median FI was 2 for significant outcomes and 6 for nonsignificant outcomes, indicating that outcomes favoring one surgical method are more statistically fragile than those demonstrating equivalency. [Orthopedics. 2025;48(2):69-73.].

Fragility Index Analysis of the 2018 Clinical Practice Guidelines on Tranexamic Acid Use in Total Joint Arthroplasty

BACKGROUND

The 2018 American Association of Hip and Knee Surgeons clinical practice guideline (CPG) 'tranexamic acid use in total joint arthroplasty' evaluated the efficacy and safety of tranexamic acid in primary total joint arthroplasty. The following review assessed the statistical fragility of the randomized controlled trial (RCT) outcomes on which the CPG recommendations were based using a fragility analysis.

METHODS

All dichotomous outcomes from the RCTs used to guide the CPG from its associated network, and direct meta-analyses were analyzed. Fragility and reverse fragility indices (FI and rFI) and quotients were calculated for each outcome. The mean indices and quotients were calculated for each guideline question, outcome category, and comparison of tranexamic dose, formulation, and administration timing.

RESULTS

This review evaluated 403 dichotomous outcomes on transfusion and complication rates associated with tranexamic acid (TXA) administration. The mean FI of significant outcomes of the CPG was 5.23, and the mean rFI of nonsignificant outcomes was 5.80. Outcomes assessing complication rates had a mean rFI of 6.48. Most outcomes on transfusion in categories comparing TXA to placebo administration had higher mean FIs than rFIs, and all outcomes comparing transfusion risk associated with different TXA formulations and doses had higher mean rFIs than FI or no associated significant outcomes.

CONCLUSIONS

The rFI and FIs calculated for this CPG are comparable to or higher than mean values reported across orthopaedic literature, indicating the relative statistical stability of its included outcomes. As we learn more about fragility analyses and their potential applications, this type of statistical analysis shows promise as a useful tool to incorporate into future guidelines to assess the quality of RCTs and evaluate the strength of recommendations.

Evaluating the Statistical Fragility of Comparative Studies on Autografts for Pediatric ACL Reconstruction

Background

The literature presents conflicting findings regarding outcomes after pediatric anterior cruciate ligament reconstruction (ACLR) with various autograft options, reflecting a lack of consensus on the standard of practice. Fragility analyses may assist in evaluating the statistical robustness of these studies.

Purpose

To evaluate the statistical fragility of comparative studies in pediatric ACLR through the fragility index (FI) and fragility quotient (FQ), as well as qualitative factors such as outcome type, outcome significance, and patients lost to follow-up.

Study Design

Systematic review; Level of evidence, 4.

Methods

A systematic review conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines identified 1139 studies in the PubMed and Embase databases that met the search criteria; ultimately, 6 studies were selected for inclusion. A total of 32 comparative outcomes were assessed for fragility across the 6 studies. Descriptive statistics were employed to summarize the fragility data and generate subgroup comparisons.

Results

The mean FI was 1.5, and the mean reverse FI was 3.19 (P < .01); the mean FQ was 0.0064, and the mean reverse FQ was 0.028 (P≤ .0001). No significant difference was found in the FIs between objective outcomes and patient-reported outcomes (P = .418). These findings suggested that a comparable number of patients would need to transition from a nonevent to an event to alter a statistically significant result to a nonsignificant one. The FI was lower than the estimated number of patients lost to follow-up for 30 of the 32 outcomes (93.7%).

Conclusion

Comparative studies on pediatric ACLR autograft outcomes displayed vulnerability when assessed using fragility metrics, indicating a lack of statistically robust data. The findings revealed that many reported outcomes are fragile and may require further investigation. Future research should incorporate fragility analyses-especially in studies with long-term follow-ups-to enhance the reliability of conclusions regarding optimal graft selection in pediatric ACLR.

The Statistical Fragility of Marrow Stimulation for Cartilage Defects of the Knee: A Systematic Review of Randomized Controlled Trials

OBJECTIVE

Marrow stimulation is used to address knee cartilage defects. In this study, we used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate statistical fragility of outcomes reported in randomized controlled trials (RCTs) evaluating marrow stimulation.

DESIGN

PubMed, Embase, and MEDLINE were queried for recent RCTs (January 1, 2010-September 5, 2023) assessing marrow stimulation for cartilage defects of the knee. The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the study sample size.

RESULTS

Across 155 total outcomes from 21 RCTs, the median FI was 3 (interquartile range [IQR], 2-5), with an associated median FQ of 0.067 (IQR, 0.033-0.010). Thirty-two outcomes were statistically significant, with a median FI of 2 (IQR, 1-3.25) and FQ of 0.050 (IQR, 0.025-0.069). Ten of the 32 (31.3%) outcomes reported as statistically significant had an FI of 1. In total, 123 outcomes were nonsignificant, with a median rFI of 3 (IQR, 2-5). Studies assessing stem cell augments were the most fragile, with a median FI of 2. In 55.5% of outcomes, the number of patients lost to follow-up was greater than or equal to the FI.

CONCLUSION

Statistical findings in RCTs evaluating marrow stimulation for cartilage defects of the knee are statistically fragile. We recommend combined reporting of P-values with FI and FQ metrics to aid in the interpretation of clinical findings in comparative trials assessing cartilage restoration.

Aspirin in prevention of venous thromboembolism following hip fracture surgery: A systematic review and meta-analysis

Background

Many orthopaedic surgeons routinely prescribe aspirin (ASA) as prophylaxis for venous thromboembolism (VTE) following hip fracture surgery (HFS). The purpose of this study is to assess the effectiveness of aspirin to other agents in preventing VTE and mortality following hip fracture surgery.

Methods

Following PRISMA guidelines, we performed a search for HFS studies from 1998 to 2023 reporting comparisons between aspirin and other chemoprophylaxis methods for VTE (DVT - deep vein thrombosis; PE - pulmonary embolism). SPSS Meta-analysis function was used to calculate Mean Effect Size Estimate (MESE) and 95 % Confidence Intervals for each outcome. Reverse Fragility Index (RFI) and Fragility Quotient (FQ) were calculated for each study.

Results

Of the 847 articles screened, 4 studies with 5 comparisons met the search criteria to be included for analysis. A total of 1194 participants were included in these studies. There was a decreased risk of mortality seen with use of aspirin compared to other agents (MESE = 0.86, 95 % CI: [0.07-1.66]; p=.03). There was no increased risk of DVT or PE with use of aspirin (both p>.4). The overall RFI and FQ for all 19 outcomes were 12 (IQR: 6.5-15) and 0.080 (IQR: 0.027-0.110), respectively. Ten studies (52.6 %) reported a loss-to-follow-up (LTF) greater than the overall RFI.

Conclusions

Aspirin demonstrates similar protective effects on prevention of VTE compared to other agents and may have significant protective effects on overall mortality following surgical intervention for hip fractures. However, the current evidence concerning its use in this arena is less than robust, with more than half of the studied outcomes considered statistically fragile.

The Fragility of Statistical Significance in the Use of Aspirin in Prevention of Venous Thromboembolism Events Following Total Joint Arthroplasty: Systematic Review and Meta-Analysis of Randomized Controlled Trials

Background/Objectives: Comparative studies often use the p value to convey statistical significance, but fragility indices (FI) and fragility quotients (FQ) may better signify statistical strength. The use of aspirin as venous thromboembolism (VTE) chemoprophylaxis following elective arthroplasty has been debated between the orthopedic and cardiac fields. The purpose of this study was to apply both the FI and FQ to evaluate the degree of statistical fragility in the total joint arthroplasty (TJA) literature regarding aspirin (ASA) use for VTE prevention. Methods: We performed a systematic search for TJA clinical trials from 2004 to 2023 reporting comparisons between ASA and other chemoprophylaxis methods for VTE. The FI of each outcome was calculated through reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size and interquartile range (IQR) was calculated. SPSS Meta-analysis function was used to calculate the Mean Effect Size Estimate and 95% Confidence Intervals for each outcome. Results: Of 245 articles screened, 39 met search criteria, with 10 RCTs included for analysis (n = 11,481 patients). There were 38 outcome events reported, with three significant (p < 0.05) outcomes and 35 non-significant (p > 0.05) outcomes identified. The overall FI and FQ for all 38 outcomes were 6 (IQR: 5-7) and 0.059 (IQR: 0.044-0.064), respectively. Seven studies (70%) reported a loss-to-follow-up (LTF) greater than the overall FI. There was no increased risk of DVT, PE, or mortality with use of ASA (all p > 0.2). Conclusions: Despite showing non-inferiority in preventing venous thromboembolic events in TJA overall, the highest-level peer-reviewed literature concerning aspirin use following total joint arthroplasty is considered statistically fragile due to high loss-to-follow-up. In addition to the reporting of the p value, the fragility index and quotient can further provide insight into the strength and trustworthiness of outcome measures.

The fragility of statistical findings in the reverse total shoulder arthroplasty literature: a systematic review of randomized controlled trials

BACKGROUND

Reverse total shoulder arthroplasty (RTSA) has seen increasing utilization as an effective intervention for a wide variety of shoulder pathologies. The scope and indications for growth are often driven by findings from randomized controlled trials (RCTs) guiding surgical decision-making for RTSA. In this study, we utilized the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to assess the robustness of outcomes reported in RCTs in the RTSA literature.

METHODS

PubMed, Embase, and MEDLINE were queried for RCTs (Jan. 1, 2010-Mar. 31, 2023) in the RTSA literature reporting dichotomous outcomes. The FI and rFI were defined as the number of outcome reversals required to alter statistical significance for significant and nonsignificant outcomes, respectively. The FQ was determined by dividing the FI by the sample size of each study. Subgroup analysis was performed based on outcome category.

RESULTS

One hundred seventy-six RCTs were screened with 18 studies included. The median FI across 59 total outcomes was 4 (interquartile range [IQR]: 3-5) with an associated FQ of 0.051 (IQR: 0.029-0.065). Thirteen outcomes were statistically significant with a median FI of 3 (IQR: 1-4) and FQ of 0.033 (IQR: 0.012-0.066). Forty-six outcomes were nonsignificant with a median rFI of 4 (IQR: 3-5) and FQ of 0.055 (IQR: 0.032-0.065). The most fragile outcome category was revision/reoperations with a median FI of 2.50 (IQR: 1.00-3.25), followed by clinical score/outcome (median FI: 3.00), complications (median FI: 4.00), "other" (median FI: 4.00), and radiographic findings (median FI: 5.00). Notably, the number of patients lost to follow-up was greater than or equal to the FI for 59% of outcomes.

CONCLUSION

The statistical findings in RTSA RCTs are fragile and should be interpreted with caution. Reversal of only a few outcomes, or maintaining postoperative follow-up, may be sufficient to alter significance of study findings. We recommend standardized reporting of P values with FI and FQ metrics to allow clinicians to effectively assess the robustness of study findings.

Interpreting the Current Literature on Outcomes of Robotic-Assisted Versus Conventional Total Knee Arthroplasty Using Fragility Analysis: A Systematic Review and Cross-Sectional Study of Randomized Controlled Trials

BACKGROUND

Fragility analysis is a method of further characterizing outcomes in terms of the stability of statistical findings. This study assesses the statistical fragility of recent randomized controlled trials (RCTs) evaluating robotic-assisted versus conventional total knee arthroplasty (RA-TKA versus C-TKA).

METHODS

We queried PubMed for RCTs comparing alignment, function, and outcomes between RA-TKA and C-TKA. Fragility index (FI) and reverse fragility index (RFI) (collectively, "FI") were calculated for dichotomous outcomes as the number of outcome reversals needed to change statistical significance. Fragility quotient (FQ) was calculated by dividing the FI by the sample size for that outcome event. Median FI and FQ were calculated for all outcomes collectively as well as for each individual outcome. Subanalyses were performed to assess FI and FQ based on outcome event type and statistical significance, as well as study loss to follow-up and year of publication.

RESULTS

The overall median FI was 3.0 (interquartile range, [IQR] 1.0 to 6.3) and the median reverse fragility index was 3.0 (IQR 2.0 to 4.0). The overall median FQ was 0.027 (IQR 0.012 to 0.050). Loss to follow-up was greater than FI for 23 of the 38 outcomes assessed.

CONCLUSIONS

A small number of alternative outcomes is often enough to reverse the statistical significance of findings in RCTs evaluating dichotomous outcomes in RA-TKA versus C-TKA. We recommend reporting FI and FQ alongside P values to improve the interpretability of RCT results.

The Fragility of Statistical Findings in the Femoral Neck Fracture Literature: A Systematic Review of Randomized Controlled Trials

OBJECTIVES

Randomized controlled trials (RCTs) in the femoral neck fracture literature frequently report P -values for outcomes, which have substantial implications in guiding surgical management. This study used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to assess the statistical stability of outcomes reported in RCTs evaluating the management and treatment of femoral neck fractures.

METHODS

DATA SOURCES

DESIGN

PubMed, Embase, and MEDLINE were queried for RCTs (January 1, 2010 to February 28, 2023).

SETTING

RCTs that evaluated surgical management or treatment of femoral neck fractures were included.

STUDY SELECTION CRITERIA

RCTs with 2 treatment arms reporting categorical dichotomous outcomes were included. Non-RCT studies, RCTs with greater than 2 treatment arms, and RCTs without a femoral neck fracture cohort were excluded.

DATA EXTRACTION AND SYNTHESIS

OUTCOME MEASURES AND COMPARISONS

The FI and rFI were calculated as the number of outcome event reversals required to alter statistical significance for significant ( P < 0.05) and nonsignificant ( P ≥ 0.05) outcomes, respectively. The FQ was calculated by dividing the FI by the sample size for the study.

RESULTS

Nine hundred eighty-five articles were screened, with 71 studies included for analysis. The median FI across a total of 197 outcomes was 4 [interquartile range (IQR) 2-5] with an associated FQ of 0.033 (IQR 0.017-0.060). Forty-seven outcomes were statistically significant with a median FI of 2 (IQR 1-4) and associated FQ of 0.02 (IQR 0.014-0.043). One hundred fifty outcomes were statistically nonsignificant with a median rFI of 4 (IQR 3-5) and associated FQ of 0.037 (IQR 0.019-0.065).

CONCLUSIONS

Statistical findings in femoral neck fracture RCTs are fragile, with reversal of a median 4 outcomes altering significance of study findings. The authors thus recommend standardized reporting of P -values with FI and FQ metrics to aid in interpreting the robustness of outcomes in femoral neck fracture RCTs.

LEVEL OF EVIDENCE

Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

The Statistical Fragility of Tranexamic Acid Use in the Orthopaedic Surgery Literature: A Systematic Review of Randomized Controlled Trials

INTRODUCTION

Randomized controlled trials (RCTs) represent the highest level of evidence in orthopaedic surgery literature, although the robustness of statistical findings in these trials may be unreliable. We used the fragility index (FI), reverse fragility index (rFI), and fragility quotient (FQ) to evaluate the statistical stability of outcomes reported in RCTs that assess the use of tranexamic acid (TXA) across orthopaedic subspecialties.

METHODS

PubMed, EMBASE, and MEDLINE were queried for RCTs (2010-present) reporting dichotomous outcomes with study groups stratified by TXA administration. The FI and rFI were defined as the number of outcome event reversals needed to alter the significance level of significant and nonsignificant outcomes, respectively. FQ was determined by dividing the FI or rFI by sample size. Subgroup analyses were conducted based on orthopaedic subspecialty.

RESULTS

Six hundred five RCTs were screened with 108 studies included for analysis comprising 192 total outcomes. The median FI of the 192 outcomes was 4 (IQR 2 to 5) with an associated FQ of 0.03 (IQR 0.019 to 0.050). 45 outcomes were reported as statistically significant with a median FI of 1 (IQR 1 to 5) and associated FQ of 0.02 (IQR 0.011 to 0.034). 147 outcomes were reported as nonsignificant with a median rFI of 4 (IQR 3 to 5) and associated FQ of 0.04 (IQR 0.023 to 0.051). The adult reconstruction, trauma, and spine subspecialties had a median FI of 4. Sports had a median FI of 3. Shoulder and elbow and foot and ankle had median FIs of 6.

DISCUSSION

Statistical outcomes reported in RCTs on the use of TXA in orthopaedic surgery are fragile. Reversal of a few outcomes is sufficient to alter statistical significance. We recommend reporting FI, rFI, and FQ metrics to aid in interpreting the outcomes reported in comparative trials.

The Statistical Fragility of Trials Comparing Cervical Disc Arthroplasty and Anterior Cervical Discectomy and Fusion: A Meta Analysis

STUDY DESIGN

Meta-analysis.

OBJECTIVE

To assess the robustness of randomized controlled trials (RCTs) that compared cervical disc arthroplasty (CDA) and anterior cervical discectomy and fusion (ACDF) for the treatment of symptomatic degenerative cervical pathology by using fragility indices.

SUMMARY OF BACKGROUND DATA

RCTs comparing these surgical approaches have shown that CDA may be equivalent or even superior to ACDF due to better preservation of normal spinal kinematics.

MATERIALS AND METHODS

RCTs reporting clinical outcomes after CDA versus ACDF for degenerative cervical disc disease were evaluated. Data for outcome measures were classified as continuous or dichotomous. Continuous outcomes included: Neck Disability Index, overall pain, neck pain, radicular arm pain, and modified Japanese Orthopedic Association scores. Dichotomous outcomes included: any adjacent segment disease (ASD), superior-level ASD, and inferior-level ASD. The fragility index (FI) and continuous FI (CFI) were determined for dichotomous and continuous outcomes, respectively. The corresponding fragility quotient (FQ) and continuous FQ were calculated by dividing FI/CFI by sample size.

RESULTS

Twenty-five studies (78 outcome events) were included. Thirteen dichotomous events had a median FI of 7 [interquartile range (IQR): 3-10], and the median FQ was 0.043 (IQR: 0.035-0.066). Sixty-five continuous events had a median CFI of 14 (IQR: 9-22) and a median continuous FQ of 0.145 (IQR: 0.074-0.188). This indicates that, on average, altering the outcome of 4.3 patients out of 100 for the dichotomous outcomes and 14.5 out of 100 for continuous outcomes would reverse trial significance. Of the 13 dichotomous events that included a loss to follow-up data, 8 (61.5%) represented ≥7 patients lost. Of the 65 continuous events reporting the loss to follow-up data, 22 (33.8%) represented ≥14 patients lost.

CONCLUSION

RCTs comparing ACDF and CDA have fair to moderate statistical robustness and do not suffer from statistical fragility.

Fragile Statistical Findings in Randomized Controlled Trials Evaluating Autograft Versus Allograft Use in Anterior Cruciate Ligament Reconstruction: A Systematic Review

PURPOSE

To analyze the statistical stability of randomized controlled trials (RCTs) evaluating the surgical management of autografts versus allografts in the anterior cruciate ligament reconstruction (ACLR) literature and calculate the fragility index (FI) and fragility quotient and explore a subgroup analysis by calculating the proportion of outcome events where the FI was less than the number of patients lost to follow-up.

METHODS

Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we conducted a systematic search in the PubMed and Cochrane databases to identify RCTs published between 2000 and 2022 that investigated the use of autografts versus allografts in ACLR literature and reported dichotomous data. The fragility index of each dichotomous variable was calculated through the reversal of a single outcome event until significance was reversed. The fragility quotient was calculated by dividing each fragility index by the study sample size. The interquartile range also was calculated.

RESULTS

Of the 4407 articles screened, 23 met the search criteria, with 11 RCTs evaluating ALCR using autografts and allografts included for analysis. Two hundred and 18 outcome events with 32 significant (P < .05) outcomes and 186 nonsignificant (P ≥ .05) outcomes were identified. The overall fragility index and fragility quotient for all 218 outcomes were 6 subjects (interquartile range 5-8) and 0.058 (interquartile range 0.039-0.077). Fragility analysis of statistically significant outcomes and nonsignificant outcomes had a fragility index of 3.5 (interquartile range 1-5.5) and 6 (interquartile range 5-8), respectively. All of the studies reported a loss to follow-up where 45.5% (5) reported a loss to follow-up greater or equal to 6.

CONCLUSIONS

The RCTs in the ACLR peer-reviewed literature evaluating autograft versus allograft use are vulnerable to a small number of outcome event reversals and exemplify significant statistical fragility in statistically significant findings.

LEVEL OF EVIDENCE

Level I, systematic review of Level I studies.

The statistical fragility of outcomes in calcaneus fractures: A systematic review of randomized controlled trials

INTRODUCTION

The purpose of this study was to utilize the fragility index to assess the robustness of randomized controlled trials (RCTs) evaluating the management of calcaneus fractures. We hypothesize that the dichotomous outcomes in calcaneus fracture literature will be statistically fragile and comparable to other orthopedic specialties.

METHODS

We performed a PubMed search for calcaneus fracture RCTs from 2000 to 2022 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The fragility index (FI) of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The fragility quotient (FQ) was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

RESULTS

Of the 3003 studies screened, 97 met the search criteria, with 19 RCTs evaluating calcaneus fractures included in the analysis. Seventy-nine dichotomous outcomes with 30 significant (P < 0.05) outcomes and 49 with nonsignificant (P> 0.05) outcomes were identified. The overall FI and FQ of all outcomes were 6 (IQR 3-8) and 0.067 (IQR 0.032-0.100), respectively.

CONCLUSIONS

The literature surrounding calcaneus fractures may not be as statistically stable as previously thought. The sole reliance on the P value may depict misleading results. We, therefore, recommend reporting the P value in conjunction with the FI and FQ to give a robust contextualization of clinical findings in the calcaneus fracture literature.

The Fragility of Statistical Significance in Sham Orthopaedic Surgery: A Systematic Review of Randomized Controlled Trials

OBJECTIVES

The purpose of this study was to determine the stability of statistical findings among sham surgery randomized controlled trials (RCTs) in orthopaedic surgery using fragility analysis.

METHODS

PubMed systematic review was conducted to include studies reporting dichotomous outcomes pertaining to sham surgery. The final review included eight RCTs involving only partial meniscectomies and vertebroplasties from 2009 to 2020. With a fixed sample size with dichotomous outcome measures (events versus non-events), the Total Fragility Index (TFI), which is composed of the fragility index (FI) and reverse fragility index (RFI), was calculated by altering the ratio of events to non-events in an iterative fashion until results were reversed from significant to nonsignificant findings (FI) or vice versa (RFI). The TFI, FI, and RFI were divided by their sample sizes to obtain the respective total fragility quotient, fragility quotient (FQ), and reverse fragility quotient. Median fragility indices and quotients were reported for all studies.

RESULTS

The eight RCTs included 50 dichotomous outcomes involving either partial meniscectomies or vertebroplasties, with a median TFI and total fragility quotient of 5 [interquartile range (IQR) 4 to 6] and 0.035 (IQR 0.028 to 0.048), respectively, indicating that a median of five total patients or 3.5 per 100 patients would need to experience a different outcome to reverse significant or insignificant findings for each of the eight trials. Among the 8 statistically significant ( P < 0.05) outcome events (16%), the respective FI and FQ were 2 (IQR 1 to 5) and 0.018 (IQR 0.010 to 0.044). Among the 42 statistically insignificant outcome events (84%), the respective RFI and reverse fragility quotient were 5 (IQR 4 to 6) and 0.04 (IQR 0.034 to 0.048). The median number of patients lost to follow-up was 1.5 (IQR 0.5 to 2).

CONCLUSION

The unstable findings in partial meniscectomy and vertebroplasty sham surgical RCTs undermine their study conclusions and recommendations. We recommend using fragility analysis in future sham surgical RCTs to contextualize statistical findings.

LEVEL OF EVIDENCE

Level IV; Systematic Review.

How robust are clinical trials in primary and secondary ankle sprain prevention?

OBJECTIVES

Determine the statistical stability of RCTs examining primary and secondary prevention of ankle sprains.

METHODS

Databases were searched to August 2023. We included parallel design RCTs, using conservative interventions for preventing ankle sprain, reporting dichotomous injury event outcomes. Statistical stability was quantified using Fragility Index (FI) and Fragility Quotient (FQ). Subgroup analyses were undertaken to test if FI varied based on by study objective, original approach to analysis (frequency vs time to event), follow-up duration, and pre-registration.

RESULTS

3559 studies were screened with 45 RCTs included. The median number of events required to change the statistical significance (FI) was 4 (IQR 1-6). FI was similar regardless of study objective, original analysis, follow-up duration, and pre-registration status. Median (IQR) FQ was 0.015 (0.005-0.046), therefore reversing events <2 patients/100 would alter significance. In 80% of studies the number of patients lost to follow-up was greater than the FI.

CONCLUSION

RCTs informing primary and secondary prevention of ankle sprain are fragile. Only a small percentage of outcome event reversals would reverse study significance, and this is often exceeded by the number of drop outs. Robust reporting of dichotomous outcomes requires the use P values and key metrics such as FI or FQ.

Statistical fragility of randomized clinical trials pertaining to femoral neck fractures

OBJECTIVE

P values were frequently misused and misinterpreted, the fragility index (FI) has been utilized to evaluate the robustness of randomized controlled trials (RCTs) as a complement to p-values. This study aimed to assess the statistical robustness of RCTs for femoral neck fractures through the utilization of the FI.

METHODS

We systematically reviewed PubMed, Cochrane Library, and Embase database to identify RCTs pertaining to femoral neck fractures published in the top 25 highest-impact orthopaedic journals and 4 high-impact general medical journals from January 1, 2000, to December 31, 2022. The FI was calculated for the dichotomous, categorical study outcomes in the identified RCTs using the Fisher exact test, with previously published methods. Spearman correlation analyses were used to evaluate potential associated factors associated with the FI.

RESULTS

We identified 10 eligible RCTs with a median total sample size of 101 (IQR, 79.5 to 174.75) and a number of patients lost to follow-up of 19.5 (IQR, 4.5 to 28). The median FI was 3.5 (IQR, 1 to 14.25), implying that reversal of outcome in only 4 patients was sufficient to alter trial significance. The FI was less than the number of patients lost to follow-up in seven (70%) RCTs. P values were negatively associated with the FI, while the number of patients lost to follow-up and patients enroled were not statistically significantly associated with the FI.

CONCLUSIONS

The RCTs pertaining to femoral neck fractures were not as statistically robust as previously thought and should be interpreted with caution. We recommend that the orthopaedic RCT report FI as a supplement for the P values to help readers draw reliable conclusions based on the fragility of the outcomes.

The Majority of Sports Medicine and Arthroscopy-Related Randomized Controlled Trials Reporting Nonsignificant Results Are Statistically Fragile

PURPOSE

To evaluate the robustness of sports medicine and arthroscopy related randomized controlled trials (RCTs) reporting nonsignificant results by calculating the reverse fragility index (RFI) and reverse fragility quotient (RFQ).

METHODS

All sports medicine and arthroscopic-related RCTs from January 1, 2010, through August 3, 2021, were identified. Randomized-controlled trials comparing dichotomous variables with a reported P value ≥ .05 were included. Study characteristics, such as publication year and sample size, as well as loss to follow-up and number of outcome events were recorded. The RFI at a threshold of P < .05 and respective RFQ were calculated for each study. Coefficients of determination were calculated to determine the relationships between RFI and the number of outcome events, sample size, and number of patients lost to follow-up. The number of RCTs in which the loss to follow-up was greater than the RFI was determined.

RESULTS

Fifty-four studies and 4,638 patients were included in this analysis. The mean sample size and loss to follow-up were 85.9 patients and 12.5 patients, respectively. The mean RFI was 3.7, signifying that a change of 3.7 events in one arm was needed to flip the results of the study from non-significant to significant (P < .05). Of the 54 studies investigated, 33 (61%) had a loss to follow-up greater than their calculated RFI. The mean RFQ was 0.05. A significant correlation between RFI with sample size (R2 = 0.10, P = .02) and the total number of observed events (R2 = 0.13, P < .01) was found. No significant correlation existed between RFI and loss to follow-up in the lesser arm (R2 = 0.01, P = .41).

CONCLUSIONS

The RFI and RFQ are statistical tools that allow the fragility of studies reporting nonsignificant results to be appraised. Using this methodology, we found that the majority of sports medicine and arthroscopy-related RCTs reporting nonsignificant results are fragile.

CLINICAL RELEVANCE

RFI and RFQ serve as tools that can be used to assess the validity of RCT results and provide additional context for appropriate conclusions.

The statistical fragility of the management options for reverse shoulder arthroplasty: a systematic review of randomized control trial with fragility analysis

Reverse shoulder arthroplasty (RSA) is used in the treatment of traumatic and arthritic pathologies, with expanding clinical indications and as a result there has been an increase in clinical research on the topic. The purpose of this study was to examine the statistical fragility of randomized control trials (RCTs) reporting outcomes from RSA. A systematic search was undertaken to find RCTs investigating RSA. The Fragility Index (FI) was calculated using Fisher's exact test, by sequentially altering the number of events until there was a reversal of significance. The Fragility Quotient (FQ) was calculated by dividing the FI by the trial population. Each trial was assigned an overall FI and FQ calculated as the median result of its reported findings. Overall, 19 RCTs warranted inclusion in the review, representing 1146 patients, of which 41.2% were male, with a mean age of 74.2 ± 4.3 years and mean follow-up of 22.1 ± 9.9 months. The median RCT population was 59, with a median of 9 patients lost to follow-up. The median FI was 4.5, and median FQ was 0.083, indicating more patients did not complete the trial than the number of outcomes which would have to change to reverse the finding of significance. This review found that the RCT evidence for RSA management may be vulnerable to statistical fragility, with a handful of events required to reverse a finding of significance.

Statistical Fragility of Randomized Controlled Trials Evaluating Platelet-Rich Plasma Use for Knee Osteoarthritis: A Systematic Review

Background

Numerous studies have been published on the use of platelet-rich plasma (PRP) for knee osteoarthritis (OA), with conflicting results.

Purpose

To determine the fragility index (FI) and fragility quotient (FQ) of randomized controlled trials (RCTs) that evaluated the use of PRP to treat knee OA.

Study Design

Systematic review.

Methods

RCTs evaluating the efficacy of PRP injections for knee OA from 2000 to 2020 were included for analysis according to PRISMA guidelines. The FI was determined by calculating the number of outcome event reversals required to change the statistical significance. The associated FQ was determined by dividing the FI by the sample size.

Results

Our initial search resulted in 41,149 studies, of which 8 RCTs (678 patients, 72 outcome events) were included in the analysis. One study failed to report PRP formulation details, whereas 87.5% of studies reported using either leukocyte-rich or leukocyte-poor PRP. The platelet concentration was reported in 25% of the included trials. The overall FI of the 72 outcome events was 8.5. Accounting for sample size, the associated FQ was determined to be 0.14, suggesting that the reversal of 14% of outcome events was required to change outcome significance. There were 51 statistically significant outcomes, of which the FI and FQ were 12 and 0.164, respectively.

Conclusion

Comprehensive fragility analysis suggested that the published literature evaluating the efficacy of PRP use for knee OA may lack statistical stability. We recommend the reporting of both an FI and FQ in addition to P value analysis to provide a clear and thorough understanding of the statistical integrity of studies reporting on PRP use for knee OA.

The Fragility of Tourniquet Use in Total Knee Arthroplasty: A Systematic Review of Randomized Controlled Trials

BACKGROUND

Physicians utilize P-values to interpret clinical trial data and guide patient-care decisions. Fragility analysis assesses the stability of statistical findings in relation to outcome event reversals. This study assessed the statistical fragility of recent randomized controlled trials (RCTs) investigating tourniquet use in total knee arthroplasty (TKA).

METHODS

We queried PubMed, EMBASE, and MEDLINE for RCTs comparing outcomes in TKA based on tourniquet use. Fragility index (FI) and reverse fragility index (reverse FI) were calculated - for significant and nonsignificant outcomes, respectively - as the number of outcome reversals required to change statistical significance. The fragility quotient (FQ) was calculated by dividing the FI or reverse FI by the sample size. Median overall FI and FQ were calculated for all included outcomes, and sub-analyses were performed by reported significance. The literature search yielded 23 studies reporting 91 total dichotomous outcomes.

RESULTS

Overall median FI was 4 with an interquartile range (IQR) of 3 to 6. Overall median FQ was 0.0476 (IQR 0.0291 to 0.0867). A total of 11 outcomes were statistically significant with a median FI and FQ of 2 (IQR 1.5 to 5) and 0.0200 (IQR 0.0148 to 0.0484), respectively. There were 80 outcomes that were nonsignificant with a median reverse FI of 4 (IQR 3 to 6). Loss to follow-up was greater than the median FI in 17.6% of outcomes.

CONCLUSION

Altering a small number of outcomes is often sufficient to reverse findings in RCTs evaluating tourniquet use in TKA. We recommend including fragility analyses to increase reliability in the interpretation of study conclusions.

Fragility of statistically significant outcomes in randomized trials comparing bariatric surgeries

BACKGROUND

Randomized controlled trials (RCTs) are regarded as high-level evidence, but the strength of their P values can be difficult to ascertain. The Fragility Index (FI) is a novel metric that evaluates the frailty of trial findings. It is defined as the minimum number of patients required to change from a non-event to event for the findings to lose statistical significance. This study aims to characterize the robustness of bariatric surgery RCTs by examining their FIs.

METHODS

A search was conducted in MEDLINE, EMBASE, and CENTRAL from January 2000 to February 2022 for RCTs comparing two bariatric surgeries with statistically significant dichotomous outcomes. Bivariate correlation was conducted to identify associations between FI and trial characteristics.

RESULTS

A total of 35 RCTs were included with a median sample size of 80 patients (Interquartile range [IQR] 58-109). The median FI was 2 (IQR 0-5), indicating that altering the status of two patients in one treatment arm would overturn the statistical significance of results. Subgroup analyses of RCTs evaluating diabetes-related outcomes produced a FI of 4 (IQR 2-6.5), while RCTs comparing Roux-en-Y gastric bypass and sleeve gastrectomy had an FI of 2 (IQR 0.5-5). Increasing FI was found to be correlated with decreasing P value, increasing sample size, increasing number of events, and increasing journal impact factor.

CONCLUSION

Bariatric surgery RCTs are fragile, with only a few patients required to change from non-events to events to reverse the statistical significance of most trials. Future research should examine the use of FI in trial design.

The statistical fragility of the distal fibula fracture literature: A systematic review of randomized controlled trials

BACKGROUND

The purpose of this study was to apply both the fragility index (FI) and fragility quotient (FQ) to evaluate the degree of statistical fragility in the distal fibular fracture (DFF) literature. We hypothesized that the dichotomous outcomes within the DFF literature are statistically fragile.

METHODS

We performed a PubMed search for distal fibular fractures clinical trials from 2000 to 2022 reporting dichotomous outcomes. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

RESULTS

Of the 1158 articles screened, 23 met the search criteria, with six RCTs included for analysis. Forty-five outcome events with 5 significant (p < 0.05) outcomes and 40 nonsignificant (p ≥ 0.05) outcomes were identified. The overall FI and FQ was 5 (IQR 4-6) and 0.089 (IQR 0.061-0.107), respectively.

CONCLUSIONS

The randomized controlled trials in the peer-reviewed distal fibular fracture literature may not be as robust as previously thought, as incorporating statistical analyses solely on a P value threshold is misleading. Standardized reporting of the P value, FI and FQ can help the clinician reliably draw conclusions based on the fragility of outcome measures.

The Statistical Fragility of Foot and Ankle Surgery Randomized Controlled Trials

Fragility index (FI) is a metric used to interpret the results of randomized controlled trials (RCTs), and describes the number of subjects that would need to be switched from event to non-event for a result to no longer be significant. Studies that analyze FI of RCTs in various orthopedic subspecialties have shown the RCTs to be largely underpowered and highly fragile. However, FI has not been assessed in foot and ankle RCTs. The MEDLINE and Embase online databases were searched from 1/1/2011 through 11/19/2021 for RCTs involving foot and ankle conditions. FI, fragility quotient (FQ), and difference between the FI and number of subjects lost to follow-up was calculated. Spearman correlation was performed to determine the relationship between sample size and FI. Overall, 1262 studies were identified of which 18 were included in the final analysis. The median sample size was 65 (interquartile range [IQR] 57-95.5), the median FI was 2 (IQR 1-2.5), and the median FQ was 0.026 (IQR 0.012-0.033). Ten of 15 (67%) studies with non-zero FI values had FI values less than the number of subjects lost to follow-up. There was linear association between FI and sample size (R² = 0.495, p-value: .031). This study demonstrates that RCTs in the field of foot and ankle surgery are highly fragile, similar to other orthopedic subspecialties.

The fragility of statistical significance in distal femur fractures: systematic review of randomized controlled trials

PURPOSE

The purpose of this study was to apply both the fragility index (FI) and fragility quotient (FQ) to evaluate the degree of statistical fragility in the distal femur fracture (DFF) literature. We hypothesized that the dichotomous outcomes within the DFF literature are statistically fragile.

METHODS

Using preferred reporting items for systematic reviews and meta-analyses, we performed a PubMed search for distal femur fractures clinical trials from 2000 to 2022 reporting dichotomous outcomes. The FI of each outcome was calculated through the reversal of a single outcome event until significance was reversed. The FQ was calculated by dividing each fragility index by study sample size. The interquartile range (IQR) was also calculated for the FI and FQ.

RESULTS

Of the 4258 articles screened, 92 met the search criteria, with eleven RCTs included for analysis. Ninety eight outcome events with 25 significant (P < 0.05) outcomes and 73 nonsignificant (P > 0.05) outcomes were identified. The overall FI and FQ for all 98 outcomes were 5 (IQR 4-6) and 0.130 (IQR 0.087-0.174), respectively. Three studies (33.3%) reported loss to follow (LTF) greater than 5.

CONCLUSIONS

The randomized controlled trials in the peer-reviewed distal femur fracture literature may not be as robust as previously thought, as incorporating statistical analyses solely on a P value threshold is misleading. Standardized reporting of the P value, FI and FQ can help the clinician reliably draw conclusions based on the fragility of outcome measures.

The Fragility of Statistical Significance in Patellofemoral Instability Research: A Systematic Review

BACKGROUND

Fragility analysis is increasingly utilized to evaluate the robustness of results within the orthopaedic literature and has frequently revealed instability of reported outcomes.

PURPOSE/HYPOTHESIS

The purpose of this investigation was to utilize a fragility analysis to evaluate the stability of reported results in the patellofemoral instability (PFI) literature. We hypothesized the demonstration of significant fragility in patellofemoral research to be similar to that identified throughout other areas of the orthopaedic literature.

STUDY DESIGN

Systematic review; Level of evidence, 4.

METHODS

The PubMed database was queried from January 1, 2000, to October 10, 2020 for comparative trials in 10 prominent orthopaedic journals that reported dichotomous outcomes related to the management of PFI. The fragility index (FI) and the fragility quotient (FQ) were calculated for each individual outcome event, and the overall FI and FQ were determined for all included studies.

RESULTS

A total of 22 comparative studies comprising 11 randomized controlled trials and 11 nonrandomized trials were included for the analysis. A total of 75 outcome events underwent a fragility analysis and revealed a median FI and FQ of 3 (interquartile range [IQR], 1-5) and 0.043 (IQR, 0.018-0.081), respectively. Also 27% of included studies reported loss to follow-up greater than the overall FI, therefore suggesting the maintenance of the follow-up may have resulted in the reversal of significance.

CONCLUSION

The result of the comprehensive fragility analysis demonstrated a lack of robustness in PFI research with the alteration of only a few outcome events required to reverse statistical significance. We therefore recommend the triple reporting of the P value, the FI, and the FQ to aid in the interpretation of the statistical integrity of future comparative trials in the PFI literature.

The Fragility of Statistical Findings in Distal Radius Fractures: A Systematic Review of Randomized Controlled Trials

PURPOSE

Probability analysis with reporting of P values is often used to determine the statistical significance of study findings in the orthopedic literature. The fragility index (FI) has been used to evaluate the robustness of a significant result. The purpose of this study was to determine the utility of applying a fragility analysis to randomized controlled trials (RCTs) evaluating distal radius fractures (DRFs).

METHODS

We identified all dichotomous outcome data for randomized controlled trials of distal radius fractures (DRFs) published in 13 orthopedic journals from 2000 to 2020. The fragility index (FI) was determined by the number of event reversals required to change a P value from less than 0.05 to greater than 0.05, or vice-versa. The associated fragility quotient (FQ) was determined by dividing the FI by the sample size. The interquartile ranges (IQR) for the FI and FQ were calculated as the difference between the 25th and 75th percentiles for the overall study.

RESULTS

Of the 7352 studies screened, 221 met the search criteria, with 34 randomized controlled trials evaluating distal radius fractures and 151 total outcome events included for analysis. The overall FI was 9 (IQR 6.5-11) with an associated fragility quotient of 0.097 (IQR 0.060-0.140). However, a majority of outcomes (78.8%) were not significant. Fragility analysis of statistically significant outcomes had an FI of 4 (IQR 1-10). All of the studies reported lost to follow-up (LTF) data where 20.6% reported lost to follow-up of greater or equal to 9.

CONCLUSION

The RCTs in distal radius fracture outcomes have comparable statistical robustness to literature in other orthopedic surgical specialties, are not as statistically stable as previously thought and should be interpreted with caution. We recommend that orthopedic literature report the FI and FQ along with the P values to help the reader draw reliable conclusions based on the fragility of outcome measures.

LEVEL OF EVIDENCE

Level I.

Analysis of the Evidence Underpinning the American Academy of Orthopedic Surgeons Knee Osteoarthritis Clinical Practice Guidelines

CONTEXT

Clinical practice guidelines (CPGs) are vital to establishing a standardized and evidence-based approach in medicine. These guidelines rely on the use of methodologically sound clinical trials, and the subsequent reporting of their methodology.

OBJECTIVE

To evaluate the completeness of randomized controlled trials (RCTs) underpinning CPGs published by the American Academy of Orthopedic Surgeons (AAOS) for management of osteoarthritis of the knee.

DATA SOURCES

We searched the most recent AAOS CPGs for surgical and nonsurgical management of osteoarthritis of the knee for RCTs. To estimate the necessary sample size, we performed a power analysis using OpenEpi 3.0 (openepi.com).

STUDY SELECTION

Two authors independently screened the reference sections of the included CPGs. Included studies met the definition of an RCT, were retrievable in the English language, and were cited in at least one of the included CPGs.

STUDY DESIGN

Meta-Analysis.

LEVEL OF EVIDENCE

Level 1a.

DATA EXTRACTION

We performed double-blind screening and extraction of RCTs included in the AAOS CPGs. We evaluated each RCT for adherence to the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist. A multiple regression analysis was conducted to assess CONSORT adherence against characteristics of included studies (ie, type of intervention, funding source, etc).

RESULTS

Our study included 179 RCTs. The overall adherence was 68.5% with significant differences between those published before and since the development of the 2010 CONSORT guidelines (P = 0.02). We found that RCTs receiving funding from industry/private sources as well as studies that included a conflict of interest statement showed more completeness than RCTs that reported receiving no funding (P < 0.01).

CONCLUSION

We found suboptimal CONSORT adherence for RCTs cited in AAOS CGPs for management of osteoarthritis of the knee. Therefore, the CPGs are likely supported by outdated evidence and lack of high-quality reporting. It is important that evidence used to guide clinical decision making be of the highest quality in order to optimize patient outcomes. In order for clinicians to confer the greatest benefits to their patients, CPGs should provide the totality of evidence and emphasize emerging high-quality RCTs to ensure up-to-date, evidence-based clinical decision-making.

Fragility of randomized controlled trials on treatment of proximal humeral fracture

BACKGROUND

Proximal humeral fracture represents an increasingly common pathology with evaluation and treatment often guided by evidence from randomized controlled trials (RCTs), but the strength of an RCT must be considered in this process. The purpose of this study was to evaluate the strength of outcomes in RCTs on the management of proximal humeral fractures using the fragility index (FI), a method used with statistically significant dichotomous outcomes to assess the number of patients that would change an outcome measure from significant (P ≤ .05) to nonsignificant if the patient outcome changed. We also aimed to correlate the FI with other measures of study strength.

METHODS

A systematic review was performed using Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines to evaluate RCTs on the management of proximal humeral fractures. The PubMed, Ovid MEDLINE, Web of Science, and Embase databases were searched from database inception to May 2021. RCTs with at least 1 statistically significant (P ≤ .05) dichotomous outcome were included. The FI was calculated for each included trial using the Fisher exact test. The FI was correlated with the study sample size and journal impact factor.

RESULTS

Ten RCTs reporting on 656 patients and published between 2011 and 2020 were included. The median patient sample size was 67 (mean, 65.6; range, 40-86). Complications were the most commonly reported dichotomous statistically significant outcome. The median FI was 1 (mean, 2.6; range, 0-18), with 4 studies having an FI of 0. A median FI of 1 indicates that 1 patient experiencing an alternative outcome or having not been lost to follow-up could have changed the pertinent conclusions of the trial for a given outcome. The median number of patients lost to follow-up was 3 (mean, 4.9; range, 0-16) and exceeded the FI in 50% of studies. There was no correlation between the FI and sample size (Spearman coefficient = 0.0592, P = .865) or between the FI and journal impact factor (Spearman coefficient = -0.0229, P = .522).

CONCLUSION

In most studies of proximal humeral fractures, only 1 or 2 patients experiencing an alternative outcome or lost to follow-up would change the conclusions for the dichotomous outcome studied. Although the FI cannot be used to assess continuous variables, which are often the primary outcome variables of RCTs, it does offer an additional unique measure of study strength that surgeons should consider when evaluating RCTs.

Benefits and Harms of Interventions With Surgery Compared to Interventions Without Surgery for Musculoskeletal Conditions: A Systematic Review With Meta-analysis

OBJECTIVE

To estimate the benefits and harms of interventions with and without surgery for musculoskeletal (MSK) conditions.

DESIGN

Intervention systematic review with meta-analysis of randomized controlled trials (RCTs).

LITERATURE SEARCH

MEDLINE, EMBASE, CINAHL, Web of Science, and CENTRAL, all up to January 7, 2021.

STUDY SELECTION CRITERIA

RCTs (English, German, Danish, Swedish, and Norwegian) of interventions with and without surgery conducted in any setting for any non-fracture MSK condition in adults (mean age: 18+ years) evaluating the outcomes on a continuous (benefits) or count (harms) scale. Outcomes were pain, self-reported physical function, quality of life, serious adverse events (SAEs), and death at 1 year.

DATA SYNTHESIS

Random-effects metaanalyses for MSK conditions where there were data from at least 2 trials.

RESULTS

One hundred RCTs (n = 12 645 patients) across 28 different conditions at 9 body sites were included. For 9 out of 13 conditions with data on pain (exceptions include some spine conditions), 11 out of 11 for function, and 9 out of 9 for quality of life, there were no clinically relevant differences (standardized mean difference of 0.50 or above) between interventions with and without surgery. For 13 out of 16 conditions with data on SAEs and 16 out of 16 for death, there were no differences in harms. Only 6 trials were at low risk of bias.

CONCLUSION

The low certainty of evidence does not support recommending surgery over nonsurgical alternatives for most MSK conditions with available RCTs. Further high-quality RCTs may change this conclusion. J Orthop Sports Phys Ther 2022;52(6):312-344. doi:10.2519/jospt.2022.11075.

Is Our Science Representative? A Systematic Review of Racial and Ethnic Diversity in Orthopaedic Clinical Trials from 2000 to 2020

BACKGROUND

A lack of racial and ethnic representation in clinical trials may limit the generalizability of the orthopaedic evidence base as it applies to patients in underrepresented minority populations and perpetuate existing disparities in use, complications, or functional outcomes. Although some commentators have implied the need for mandatory race or ethnicity reporting across all orthopaedic trials, the usefulness of race or ethnic reporting likely depends on the specific topic, prior evidence of disparities, and individualized study hypotheses.

QUESTIONS/PURPOSES

In a systematic review, we asked: (1) What proportion of orthopaedic clinical trials report race or ethnicity data, and of studies that do, how many report data regarding social covariates or genomic testing? (2) What trends and associations exist for racial and ethnic reporting among these trials between 2000 and 2020? (3) What is the racial or ethnic representation of United States trial participants compared with that reported in the United States Census?

METHODS

We performed a systematic review of randomized controlled trials with human participants published in three leading general-interest orthopaedic journals that focus on clinical research: The Journal of Bone and Joint Surgery, American Volume; Clinical Orthopaedics and Related Research; and Osteoarthritis and Cartilage. We searched the PubMed and Embase databases using the following inclusion criteria: English-language studies, human studies, randomized controlled trials, publication date from 2000 to 2020, and published in Clinical Orthopaedics and Related Research; The Journal of Bone and Joint Surgery, American Volume; or Osteoarthritis and Cartilage. Primary outcome measures included whether studies reported participant race or ethnicity, other social covariates (insurance status, housing or homelessness, education and literacy, transportation, income and employment, and food security and nutrition), and genomic testing. The secondary outcome measure was the racial and ethnic categorical distribution of the trial participants included in the studies reporting race or ethnicity. From our search, 1043 randomized controlled trials with 184,643 enrolled patients met the inclusion criteria. Among these studies, 21% (223 of 1043) had a small (< 50) sample size, 56% (581 of 1043) had a medium (50 to 200) sample size, and 23% (239 of 1043) had a large (> 200) sample size. Fourteen percent (141 of 1043) were based in the Northeast United States, 9.2% (96 of 1043) were in the Midwest, 4.7% (49 of 1043) were in the West, 7.2% (75 of 1043) were in the South, and 65% (682 of 1043) were outside the United States. We calculated the overall proportion of studies meeting the inclusion criteria that reported race or ethnicity. Then among the subset of studies reporting race or ethnicity, we determined the overall rate and distribution of social covariates and genomic testing reporting. We calculated the proportion of studies reporting race or ethnicity that also reported a difference in outcome by race or ethnicity. We calculated the proportion of studies reporting race or ethnicity by each year in the study period. We also calculated the proportions and 95% CIs of individual patients in each racial or ethnic category of the studies meeting the inclusion criteria.

RESULTS

During the study period (2000 to 2020), 8.5% (89 of 1043) of studies reported race or ethnicity. Of the trials reporting this factor, 4.5% (four of 89) reported insurance status, 15% (13 of 89) reported income, 4.5% (four of 89) reported housing or homelessness, 18% (16 of 89) reported education and literacy, 0% (0 of 89) reported transportation, and 2.2% (two of 89) reported food security or nutrition of trial participants. Seventy-eight percent (69 of 89) of trials reported no social covariates, while 22% (20 of 89) reported at least one. However, 0% (0 of 89) of trials reported genomic testing. Additionally, 5.6% (five of 89) of these trials reported a difference in outcomes by race or ethnicity. The proportion of studies reporting race or ethnicity increased, on average, by 0.6% annually (95% CI 0.2% to 1.0%; p = 0.02). After controlling for potentially confounding variables such as funding source, we found that studies with an increased sample size were more likely to report data by race or ethnicity; location in North America overall, Europe, Asia, and Australia or New Zealand (compared with the Northeast United States) were less likely to; and specialty-topic studies (compared with general orthopaedics research) were less likely to. Our sample of United States trials contained 18.9% more white participants than that reported in the United States Census (95% CI 18.4% to 19.4%; p < 0.001), 5.0% fewer Black participants (95% CI 4.6% to 5.3%; p < 0.001), 17.0% fewer Hispanic participants (95% CI 16.8% to 17.1%; p < 0.001), 5.3% fewer Asian participants (95% CI 5.2% to 5.4%; p < 0.001), and 7.5% more participants from other groups (95% CI 7.2% to 7.9%; p < 0.001).

CONCLUSION

Reporting of race or ethnicity data in orthopaedic clinical trials is low compared with other medical fields, although the proportion of diseases warranting this reporting might be lower in orthopaedics.

CLINICAL RELEVANCE

Investigators should initiate discussions about race and ethnicity reporting in the early stages of clinical trial development by surveying available published evidence for relevant health disparities, social determinants, and, when warranted, genomic risk factors. The decision to include or exclude race and ethnicity data in study protocols should be based on specific hypotheses, necessary statistical power, and an appreciation for unmeasured confounding. Future studies should evaluate cost-efficient mechanisms for obtaining baseline social covariate data and investigate researcher perspectives on current administrative workflows and decision-making algorithms for race and ethnicity reporting.

The Fragility Index of Statistically Significant Findings From Randomized Controlled Trials Comparing the Management Strategies of Anterior Shoulder Instability

BACKGROUND

Debate centering on the management of anterior shoulder instability (ASI) in recent years has led to many randomized controlled trials (RCTs) being published on the topic. The fragility index (FI) has subsequently emerged as a novel method of assessing significant findings reported in RCTs, particularly those with small sample sizes.

PURPOSE

To evaluate the FI of statistically significant findings in RCTs that reported the outcomes of management strategies of patients with ASI.

STUDY DESIGN

Systematic review; Level of evidence, 1.

METHODS

Using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, 2 independent reviewers performed a systematic review of RCTs focusing on the outcomes of management strategies of patients with ASI. There were 3 main categories of RCTs included: (1) nonoperative management in internal rotation (IR) versus external rotation (ER), (2) nonoperative management versus a surgical intervention, and (3) surgical management with arthroscopic Bankart repair versus open Bankart repair. The Fisher exact test was utilized to calculate the FI for the reversal of statistical significance in all statistically significant dichotomous outcomes.

RESULTS

A total of 21 RCTs were included, including 1589 shoulders (mean age, 29.4 years) with a mean follow-up of 26.8 months. There were 10 RCTs (831 shoulders) that reported outcomes after the nonoperative management of ASI in IR versus ER, with a mean FI of 6.8. There were 5 RCTs (324 shoulders) that reported outcomes comparing the nonoperative and operative management of ASI, with a mean FI of 3.5. There were 6 RCTs (434 shoulders) that reported outcomes after the operative management of ASI with either arthroscopic Bankart repair or open Bankart repair, with a mean FI of 9.6.

CONCLUSION

The overall FI of RCTs reporting the outcomes of management strategies for patients with ASI was high, suggesting a moderate fragility of statistically significant outcomes including recurrence, revision stabilization, and return to play.

An Analysis of the Evidence Underpinning the American Academy Orthopaedic Surgery Pediatrics Clinical Practice Guidelines

BACKGROUND

Randomized control trials (RCTs) serve as evidentiary support for recommendations underpinning clinical practice guidelines (CPGs) with the goal of optimizing patient care. A knowledge gap exists within scientific literature when evaluating the quality of RCTs used as evidence in the American Academy of Orthopaedic Surgery (AAOS) pediatric CPGs. We aim to evaluate the reporting quality and risk of bias in RCTs underlying AAOS Pediatric CPG recommendations.

METHODS

We located all AAOS Pediatric CPGs. We then extracted all RCTs from the CPG reference sections. All included RCTs were evaluated using the Consolidated Standards of Reporting Trials (CONSORT) checklist and Cochrane Collaboration risk of bias assessment tool (RoB 2.0). Descriptive statistics were recorded, and bivariate analysis was used to account for variance in CONSORT scores. A Mann-Whitney U test was completed to compare CONSORT studies published before and after 2010.

RESULTS

Three CPGs and 23 RCTs met inclusion criteria. Mean CONSORT adherence was 69.8% (21.6/31). The lowest adhered to CONSORT items were 10, 23, and 24, while items 2a, 13a, and 18 displayed the highest adherence. Ten RCTs (43.5%, 10/23) had "low" risk of bias, 5 RCTs (21.7%, 5/23) were of "some concerns," and 8 RCTs (34.8%, 8/23) received a "high" designation for risk of bias. There were no statistically significant associations in the bivariate regression analysis or Mann-Whitney U test.

CONCLUSIONS

Our results suggest that CONSORT adherence within RCTs used as evidence in AAOS Pediatric CPGs is substandard-relying on evidence that, in some cases, is >20 years old. Many of the RCTs cited as supporting evidence have a "high" risk of bias. Altogether, these CPGs may need to be updated or expanded to include more recent evidence relevant to pediatric orthopaedic surgery.

The Statistical Fragility of Single-Bundle vs Double-Bundle Autografts for ACL Reconstruction: A Systematic Review of Comparative Studies

Background:

The statistical significance of a given study outcome can be liable to small changes in findings. P values are common, but imperfect statistical methods to convey significance, and inclusion of the fragility index (FI) and fragility quotient (FQ) may provide a clearer perception of statistical strength.

Purpose/Hypothesis:

The purpose was to examine the statistical stability of studies comparing primary single-bundle to double-bundle anterior cruciate ligament reconstruction (ACLR) utilizing autograft and independent tunnel drilling. It was hypothesized that the study findings would be vulnerable to a small number of outcome event reversals, often less than the number of patients lost to follow-up.

Study Design:

Systematic review; Level of evidence, 2.

Methods:

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, the authors searched PubMed for comparative studies and randomized controlled trials (RCTs) published in select journals, based on impact factor, between 2005 and 2020. Risk-of-bias assessment and methodology scoring were conducted for the included studies. A total of 48 dichotomous outcome measures were examined for possible event reversals. The FI for each outcome was determined by the number of event reversals necessary to alter significance. The FQ was calculated by dividing the FI by the respective sample size.

Results:

Of the 1794 studies screened, 15 comparative studies were included for analysis; 13 studies were RCTs. Overall, the mean FI and FQ were 3.14 (IQR, 2-4) and 0.050 (IQR, 0.032-0.062), respectively. For 72.9% of outcomes, the FI was less than the number of patients lost to follow-up.

Conclusion:

Studies comparing single-bundle versus double-bundle ACLR may not be as statistically stable as previously thought. Comparative studies and RCTs are at substantial risk for statistical fragility, with few event reversals required to alter significance. The reversal of fewer than 4 outcome events in a treatment group can alter the statistical significance of a given result; this is commonly less than the number of patients lost to follow-up. Future comparative study analyses might consider including FI and FQ with P values in their statistical analysis.

The fragility of statistically significant results in gynecologic surgery: A systematic review

OBJECTIVE

To use the fragility index (FI) to evaluate the robustness of gynaecologic surgery trials that report statistically significant results. The FI defines the minimum number of patients who must have an alternative outcome to alter statistical significance.

DATA SOURCES

We searched MEDLINE, Web of Science, Embase, and ClinicalTrials.gov from 2011 to 2021 to identify gynaecologic surgery randomized controlled trials (RCTs).

STUDY SELECTION

A total of 4775 trials were screened for eligibility. All included studies evaluated benign gynaecologic surgery interventions or peri-operative medical interventions. Only two-arm RCTs with statistically significant dichotomous primary outcomes were included. Ninety-three trials were ultimately included for analysis.

DATA EXTRACTION AND SYNTHESIS

Data from the included studies, including sample size, loss to follow-up, and number of events, were recorded. The FI of each study was calculated using a predefined technique. The overall FI and FIs by subgroup (clinical subspecialty, country of origin, and statistical test used) are reported as medians alongside their interquartile ranges (IQRs). The Kruskal-Wallis test was applied to find possible statistically significant relationships between FI and the nominal subgroups. Among this cohort, the median FI was 3 (IQR 1-7). The FI was 0 in 13 trials (14%), and in 39 trials (42%), the number of patients lost to follow-up was greater than the FI. The median FI within clinical subspecialty groups (general gynaecology, anaesthesia, urogynaecology, and fertility) did not differ (P = 0.122).

CONCLUSION

Statistically significant results of RCTs in gynaecologic surgery are fragile, suggesting that clinicians should interpret results with caution. This is particularly true when the number of patients lost to follow-up is greater than the FI. The FI serves as a quality metric that can be used to evaluate robustness of results when applying the outcomes of RCTs to clinical practice or guideline development.

The Fragility of Statistical Significance in Cartilage Restoration of the Knee: A Systematic Review of Randomized Controlled Trials

OBJECTIVE

The purpose of this study was to utilize fragility analysis to assess the robustness of randomized controlled trials (RCTs) evaluating the management of articular cartilage defects of the knee. We hypothesize that the cartilage restorative literature will be fragile with the reversal of only a few outcome events required to change statistical significance.

DESIGN

RCTs from 11 orthopedic journals indexed on PubMed from 2000 to 2020 reporting dichotomous outcome measures relating to the management of articular cartilage defects of the knee were included. The Fragility Index (FI) for each outcome was calculated through the iterative reversal of a single outcome event until significance was reversed. The Fragility Quotient (FQ) was calculated by dividing each FI by study sample size. Additional statistical analysis was performed to provide median FI and FQ across subgroups.

RESULTS

Nineteen RCTs containing 60 dichotomous outcomes were included for analysis. The FI and FQ of all outcomes was 4 (IQR 2-7) and 0.067 (IQR 0.034-0.096), respectively. The average number of patients lost to follow-up (LTF) was 3.9 patients with 15.8% of the included studies reporting LTF greater than or equal to 4, the FI of all included outcomes.

CONCLUSIONS

The orthopedic literature evaluating articular cartilage defects of the knee is fragile as the reversal of relatively few outcome events may alter the significance of statistical findings. We therefore recommend comprehensive fragility analysis and triple reporting of the P value, FI, and FQ to aid in the interpretation and contextualization of clinical findings reported in the cartilage restoration literature.

Statistical Fragility of Surgical Clinical Trials in Orthopaedic Trauma

The Fragility Index (FI) and the Fragility Quotient (FQ) are powerful statistical tools that can aid clinicians in assessing clinical trial results. The purpose of this study was to use the FI and FQ to evaluate the statistical robustness of widely cited surgical clinical trials in orthopaedic trauma.

A Call for a Standardized Approach to Reporting Patient-Reported Outcome Measures: Clinical Relevance Ratio

A shift toward a value-driven health-care model has made prospective collection of patient-reported outcome measures (PROMs) inextricably tied to measuring the success of orthopaedic surgery and patient satisfaction. While progress has been made in optimizing the utilization of PROM data, including establishing appropriate PROMs for a procedure and determining the clinical importance of unique tools, if these PROMs are not accurately analyzed and reported, a proportion of patients who do not reach the clinical thresholds may go unnoticed. Furthermore, parameters are unclear for setting a statistically and clinically important PROM threshold along with a minimum period for follow-up data collection.In this forum, we walk through simulated data sets modeling PROMs with the example of total joint arthroplasty. We discuss how the commonly used method of reporting PROMs by mean change can overestimate the treatment effects for the cohort as a whole and fail to capture distinct populations that are below a clinically relevant threshold. We demonstrate that when a study's outcome is PROMs, clinical importance should be reported using clinical thresholds such as the minimum clinically important difference (MCID), the smallest change in the treatment outcome that a patient perceives as beneficial, and the patient acceptable symptom state (PASS), the highest level of symptoms beyond which a patient considers himself or herself well. Finally, we propose a standardized reporting of PROMs that incorporates both the MCID and the PASS, and introduce a "clinical relevance ratio," which relies on a clinically relevant threshold to dichotomize outcomes and reports the number of patients achieving clinical importance at a given time point divided by the total number of patients included in the study. Unlike other common PROM-reporting approaches, the clinical relevance ratio is not skewed by patients who are lost to follow-up with increased time.

Static versus dynamic fixation of distal tibiofibular syndesmosis: a systematic review of overlapping meta-analyses

PURPOSE

Multiple Level I meta-analyses were conducted comparing traditional static vs. more recently introduced dynamic strategies of fixation for injuries of the distal tibiofibular syndesmosis (TFS). The aim of this review was to assess their robustness and methodological quality, providing support in the choice of a treatment strategy in case of TFS injury using the highest level of evidence.

METHODS

In this systematic review, conducted in accordance with the PRISMA guidelines, meta-analyses/systematic reviews comparing static and dynamic fixation methods after acute TFS injury were identified. The robustness of studies was evaluated using the fragility index (FI) for meta-analysis and the fragility quotient (FQ). The risk of bias was evaluated using the Assessment of Multiple Systematic Reviews (AMSTAR) instrument. Finally, the Jadad was applied to select the study which provided the highest quality of evidence to develop recommendations for the fixation strategy of these lesions.

RESULTS

Out of 1.302 records, four Level I meta-analyses were included in this study. Analyzing the statistically significant dichotomous outcomes, the median FI was 3.5 (IQR, 2 to 5.5; range, 1 to 9), while the median FQ was 1.9% (IQR, 1 to 3.5; range 0.35 to 4.4). In total, 37% had an FI of 2 or less and 75% of outcomes had a FI of 4 or less. According to the AMSTAR score and Jadad algorithm, the largest meta-analysis was selected as the highest evidence provided so far.

CONCLUSION

The meta-analyses with statistically significant dichotomous outcomes comparing dynamic and static fixation for treating injuries of the distal tibiofibular syndesmosis are fragile, with a change in less than four patients or less than 2% of the study population sufficient to reverse a significant outcome to nonsignificant.

LEVEL OF EVIDENCE

Level I.

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

Background

The Fragility Index (FI) and Reverse Fragility Index are powerful tools to supplement the P value in evaluation of randomized clinical trial (RCT) outcomes. These metrics are defined as the number of patients needed to change the significance level of an outcome. The purpose of this study was to calculate these metrics for published RCTs in total joint arthroplasty (TJA).

Methods

We performed a systematic review of RCTs in TJA over the last decade. For each study, we calculated the FI (for statistically significant outcomes) or Reverse Fragility Index (for nonstatistically significant outcomes) for all dichotomous, categorical outcomes. We also used the Pearson correlation coefficient to evaluate publication-level variables.

Results

We included 104 studies with 473 outcomes; 92 were significant, and 381 were nonstatistically significant. The median FI was 6 overall and 4 and 7 for significant and nonsignificant outcomes, respectively. There was a positive correlation between FI and sample size (R = 0.14, P = .002) and between FI and P values (R = 0.197, P = .000012).

Conclusions

This study is the largest evaluation of FI in orthopedics literature to date. We found a median FI that was comparable to or higher than FIs calculated in other orthopedic subspecialties. Although the mean and median FIs were greater than the 2 recommended by the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines to demonstrate strong evidence, a large percentage of studies have an FI < 2. This suggests that the TJA literature is on par or slightly better than other subspecialties, but improvements must be made.

Level of Evidence

Level I; Systematic Review.

A review of trauma and orthopaedic randomised clinical trials published in high-impact general medical journals

Introduction

The recent past has seen a significant increase in the number of trauma and orthopaedic randomised clinical trials published in “the big five” general medical journals. The quality of this research has, however, not yet been established.

Methods

We therefore set out to critically appraise the quality of available literature over a 10-year period (April 2010–April 2020) through a systematic search of these 5 high-impact general medical journals (JAMA, NEJM, BMJ, Lancet and Annals). A standardised data extraction proforma was utilised to gather information regarding: trial design, sample size calculation, results, study quality and pragmatism. Quality assessment was performed using the Cochrane Risk of Bias 2 tool and the modified Delphi list. Study pragmatism was assessed using the PRECIS-2 tool.

Results

A total of 25 studies were eligible for inclusion. Over half of the included trials did not meet their sample size calculation for the primary outcome, with a similar proportion of these studies at risk of type II error for their non-significant results. There was a high degree of pragmatism according to PRECIS-2. Non-significant studies had greater pragmatism that those with statistically significant results (p < 0.001). Only 56% studies provided adequate justification for the minimum clinically important difference (MCID) in the population assessed. Overall, very few studies were deemed high quality/low risk of bias.

Conclusions

These findings highlight that there are some important methodological concerns present within the current evidence base of RCTs published in high-impact medical journals. Potential strategies that may improve future trial design are highlighted.

Level of evidence

Level 1.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00590-021-03137-3.

The Fragility of Statistical Findings in Achilles Tendon Injury Research: A Systematic Review

Probability analysis with the reporting of P values is often used to determine the statistical significance of study findings in the Achilles literature. The purpose of this study was to determine the utility of applying a fragility analysis to comparative trials evaluating Achilles tendon injuries.

Statistical fragility of randomized clinical trials in shoulder arthroplasty

BACKGROUND

The P value is a statistical tool used to assess the statistical significance of clinical trial outcomes in orthopedic surgery. However, the P value does not evaluate research quality or clinical significance. The Fragility Index (FI) is an alternative statistical method that can be used to assess the quality and significance of clinical research and is defined as the number of patients in a study intervention group necessary to convert an outcome from statistically significant to statistically insignificant or vice versa. The primary purpose of this study was to evaluate the statistical robustness of clinical trials regarding shoulder arthroplasty using the FI. The secondary goal was to identify trial characteristics associated greater statistical fragility.

METHODS

A systematic review of randomized clinical trials in shoulder arthroplasty was performed. The FI was calculated for all dichotomous, categorical study outcomes discussed in the identified studies. Descriptive statistics and the Pearson correlation coefficient were used to evaluate all studies and characterize associations between study variables.

RESULTS

A total of 13 randomized controlled trials were identified and evaluated; these trials had a median sample size of 47 patients (mean, 54 patients; range, 26-102 patients) and a median of 7 patients (mean, 5.8 patients; range, 0-14 patients) lost to follow-up. The median FI was 6 (mean, 5; range, 1-11), a higher FI than what has been observed in other orthopedic subspecialties. However, the majority of outcomes (74.4%) had an FI that was less than the number of patients lost to follow-up, and most outcomes (89.7%) were statistically insignificant.

CONCLUSION

Randomized controlled trials in shoulder arthroplasty have comparable statistical robustness to the literature in other orthopedic surgical subspecialties. We believe that the inclusion of the FI in future comparative studies in the shoulder arthroplasty literature will allow surgeons to better assess the statistical robustness of future research.

The fragility and reverse fragility indices of proximal humerus fracture randomized controlled trials: a systematic review

BACKGROUND

The quality of evidence of the orthopedic literature has been often called into question. The fragility index (FI) has emerged as a means to evaluate the robustness of a significant result. Similarly, reverse fragility index (RFI) can be used for nonsignificant results to evaluate whether one can confidently conclude that there is no difference between groups. The analysis of FI and RFI in proximal humerus fracture (PHF) management is of particular interest, given ongoing controversy regarding optimal management and patient selection. The aim of this study was to report the FI, RFI and quality of the evidence in the proximal humerus fracture literature.

METHODS

A systematic review was conducted based on the PRISMA (preferred reporting items for systematic reviews and meta-analyses) guidelines, which utilized EMBASE, MEDLINE and Cochrane Library databases. Inclusion criteria included randomized controlled clinical trials related to the management of proximal humerus fractures, published from 2000 to 2020 with dichotomous outcome measures and 1:1 allocation. The FI and RFI were calculated by successively changing one nonevent to an event for each outcome measure until the result was made nonsignificant or significant, respectively. The fragility quotient, (FQ), calculated by dividing the FI by the total sample size, was calculated as well.

RESULTS

There were 25 studies that met our criteria with 48 outcome measures recorded. A total of 21 studies had at least one fragile result, with ten studies including a fragile result in the conclusion of the abstract. A total of 31 outcome measures had nonsignificant results and the median RFI was found to be 4, with 71% greater than number of patients lost to follow up. Seventeen outcomes had significant results, with a median FI of 1, with 65% greater than or equal to the number patients lost to follow up. A total of 18 of 25 studies (72%) included a power analysis. In particular, ten studies reported a statistical analysis of complication rates, 90% of which were fragile. The median FQ was found to be 0.037.

CONCLUSIONS

The literature on PHF management is frequently fragile. Outcome measures are often fragile, particularly with regards to comparing complication rates and reoperation rates in treatment arms. Comparing to the studies in other subspecialties PHF RCTs are relatively more fragile and underpowered. Standardized reporting of FI, FQ and RFI can help the reader to reliably draw conclusions based on the fragility of outcome measures.

Evaluation of Risks of Bias in Addiction Medicine Randomized Controlled Trials

AIMS

Perhaps the most important step when designing and conducting randomized controlled trials (RCTs) in addiction is to put methodological safeguards in place to minimize the likelihood for bias to affect trial outcomes. In this study, we applied the revised Cochrane risk of bias tool (ROB 2) to RCTs of drug, alcohol or tobacco interventions.

METHODS

We searched for trials published in 15 addiction medicine journals over a 7-year period. Our primary endpoint is the risk of bias of included studies. We conducted a sensitivity analysis of publicly funded trials.

RESULTS

Overall, included RCTs were most often at high risk of bias per our judgments (244/487, 50.1%). However, significant proportions of included RCTs were at low risk of bias (123/487, 25.3%) or some concerns for bias (120/497, 24.6%). RCTs with behavioral modification interventions (19/44, 43.2%) and alcohol interventions (80/150, 53.3%) had the highest proportion of high-risk judgments. In a sensitivity analysis of publicly funded RCTs), 195/386 (50.5%) were at high risk of bias.

CONCLUSIONS

Approximately half of included drug, alcohol or tobacco RCTs in our sample were judged to be at high risk of bias with the most common reason being a lack of proper blinding or proper description of blinding. Key action items to reduce bias in future addiction RCTs include adequate randomization, blinding and inclusion of a trial registry number and protocol.