Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package

Fragility of Statistical Differences Between Surgical Approaches in Total Hip Arthroplasty

BACKGROUND

Total hip arthroplasty (THA) is a successful treatment for hip osteoarthritis, but the optimal surgical approach for this procedure is a topic of debate. This review uses fragility analysis to determine the statistical fragility of randomized controlled trials (RCTs) comparing the 3 most common surgical approaches for THA: direct anterior, direct lateral, and posterior.

MATERIALS AND METHODS

A systematic review was conducted to identify RCTs comparing 2 of the 3 surgical approaches for THA. Dichotomous outcomes and study characteristics were extracted from each study that met the inclusion criteria. Fragility index (FI) and fragility quotient (FQ) were calculated for each significant outcome (P≤.05), and reverse fragility index and quotient (rFI and FQ) were calculated for each nonsignificant outcome (P>.05).

RESULTS

Our search yielded 20 studies in total, with 146 identified dichotomous outcomes. In total, the outcomes had a median FI of 5 and a median rFI of 5. Significant outcomes that favored the anterior approach had a median FI of 6, and significant outcomes that favored the posterior or lateral approach both had a median FI of 1.

CONCLUSION

The RCTs comparing approaches to THA had a median FI of 5, signifying that the reversal of 5 events would be sufficient to change the significance of the entire outcome. This value is comparable to other FI values within the orthopedic literature, but subgroup analyses elucidated areas of greater statistical fragility, particularly in outcomes favoring either the lateral or posterior approach in THA. [Orthopedics. 2025;48(3):166-173.].

The Fragility of Statistical Findings Regarding Hemiarthroplasty Versus Total Hip Arthroplasty for Displaced Femoral Neck Fractures

There is debate over the treatment of displaced femoral neck fractures with either hemiarthroplasty or total hip arthroplasty. This study aimed to evaluate the fragility index (FI) of randomized controlled trials (RCTs) comparing these methods. We queried for these RCTs containing dichotomous outcomes, finding 11 RCTs with 63 dichotomous outcomes. The median FI for all outcomes was 6, signifying that 6 event reversals would change the statistical significance of an outcome. The median FI was 2 for significant outcomes and 6 for nonsignificant outcomes, indicating that outcomes favoring one surgical method are more statistically fragile than those demonstrating equivalency. [Orthopedics. 2025;48(2):69-73.].

The Role of Double-Zero-Event Studies in Evidence Synthesis: Evaluating Robustness Using the Fragility Index

RATIONALE

Zero-event counts are common in clinical studies, particularly when assessing rare adverse events. These occurrences can result from low event rates, short follow-up periods, and small sample sizes. When both intervention and control groups report zero events in a clinical trial, the study is referred to as a double-zero-event study, which presents methodological challenges for evidence synthesis. There has been ongoing debate about whether these studies should be excluded from evidence synthesis, as traditional two-stage meta-analysis methods may not estimate an effect size for them. Recent research suggests that these studies may still contain valuable clinical and statistical information.

AIMS AND OBJECTIVES

This study examines the role of double-zero-event studies from the perspective of the fragility index (FI), a popular metric for assessing the robustness of clinical results. We aim to determine how including or excluding double-zero-event studies affects FI derivations in meta-analyses.

METHODS

We conducted an illustrative case study to demonstrate how double-zero-event studies can impact FI derivations. Additionally, we performed a large-scale analysis of 12,184 Cochrane meta-analyses involving zero-event studies to assess the prevalence and effect of double-zero-event studies on FI calculations.

RESULTS

Our analysis revealed that FI derivations in 6608 (54.2%) of these meta-analyses involved double-zero-event studies. Excluding double-zero-event studies could lead to artificially inflated FI values, potentially misrepresenting the results as more robust than they are.

CONCLUSIONS

We advocate for retaining double-zero-event studies in meta-analyses and emphasise the importance of carefully considering their role in FI assessments. Including these studies ensures a more accurate evaluation of the robustness of clinical results in evidence synthesis.

What is the robustness of randomized controlled trials supporting rhinosinusitis guidelines?

PURPOSE

To determine the robustness of randomized controlled trials (RCTs) supporting the current rhinosinusitis guideline; International Consensus Statement on Allergy and Rhinology: rhinosinusitis (ICAR-RS).

MATERIALS & METHODS

RCTs referenced by ICAR-RS with primary dichotomous outcomes were analyzed. The Fragility Index (FI) was calculated for trials with statistically significant findings. Trial characteristics, the FI, and FI minus number lost to follow-up (LTF) were assessed for associations.

RESULTS

A total of 317 RCTs were identified, with 38 trials possessing a primary dichotomous outcome. Thirty-one percent evaluated surgical interventions and 24 % were industry-sponsored. The mean sample size was 116 with 9 patients, on average, LTF. Sixty-three percent were eligible for FI calculation and had a median FI of 2.5 (IQR 1, 4.25). Sixty-seven percent of trials had an FI ≤ 3, indicating low robustness. No difference in FI was observed between trials with and without industry support (p = 0.577). The FI was less than or equal to the number of patients LTF in 33 % of trials (n = 8). Higher FI was strongly correlated with higher sample size, total number of events, p-value, and grade of recommendation (p < 0.001). After adjusting for covariates, higher sample size and total number of events were associated with higher FI.

CONCLUSION

The RCTs used to support the ICAR-RS have an overall low robustness and future rhinosinusitis trials should report FI measures to provide improved context of their results.

Lupus nephritis randomised controlled trials: evidence gaps and under-represented groups

OBJECTIVE

We performed a scoping review of randomised clinical trials (RCTs) assessing pharmacological therapies for the initial management of lupus nephritis (LN), focusing on study design, included populations and outcome definitions, to assess the generalisability of their results and identify gaps in the evidence.

METHODS

RCTs evaluating pharmacological interventions for the initial therapy of LN published between 2000 and 2024 were evaluated. Extracted variables included study design, selection criteria, outcome definitions, populations recruited and clinical characteristics of participants. Each study arm was included as intervention and segregated into guideline-recommended regimens (cyclophosphamide (CYC), mycophenolic acid analogues (MPAAs), calcineurin inhibitors and belimumab) or other regimens. Data were analysed by descriptive statistics, and Fragility Index (FI) was estimated to assess robustness of studies.

RESULTS

We included 124 intervention arms within 61 RCT, involving 7058 participants. Seventy-nine arms (63.7%) corresponded to guideline-recommended therapies: 33 (26.6%) MPAA, 28 (22.6%) NIH-CYC and 7 (5.6%) triple-drug therapies. While 100% of triple-drug therapies RCT were multinational, only 7.1% of NIH-CYC and 0% of tacrolimus RCTs were conducted in more than one country. Only 9 (14.8%) had follow-up ≥24 months. Ten (16.4%) RCTs exclusively included participants with severe or refractory LN. Only 29 (47.5%) reported serious adverse events, and few described patient-reported outcomes. Black and other race participants were under-represented, as well as participants from Middle East, North Africa, and the sub-Saharan African region. Response was variably defined and assessed at different intervals. Robustness of RCTs evaluating double-drug guideline-recommended therapies were mostly low, with FI ranging from 1 to 3.

CONCLUSIONS

Considering new recommendations for the management of LN, we call for broader inclusion of under-represented populations and homogenisation of study design. This study provides the rationale for evaluating unexplored treatment comparisons and conducting research on newer interventions in clinical settings where evidence is currently lacking.

Robustness Assessment of Oncology Dose-Finding Trials Using the Modified Fragility Index

OBJECTIVES

The sample sizes of phase I trials are typically small; some designs may lead to inaccurate estimation of the maximum tolerated dose (MTD). The objective of this study was to propose a metric assessing whether the MTD decision is sensitive to enrolling a few additional subjects in a phase I dose-finding trial.

METHODS

Numerous model-based and model-assisted designs have been proposed to improve the efficiency and accuracy of finding the MTD. The Fragility Index (FI) is a widely used metric quantifying the statistical robustness of randomized controlled trials by estimating the number of events needed to change a statistically significant result to non-significant (or vice versa). We propose a modified Fragility Index (mFI), defined as the minimum number of additional participants required to potentially change the estimated MTD, to supplement existing designs identifying fragile phase I trial results.

FINDINGS

Three oncology trials were used to illustrate how to evaluate the fragility of phase I trials using mFI. The results showed that two of the trials were not sensitive to additional subjects' participation while the third trial was quite fragile to one or two additional subjects.

CONCLUSIONS

The mFI can be a useful metric assessing the fragility of phase I trials and facilitating robust identification of MTD.

Platelet-Rich Plasma Versus Alternative Injections for Osteoarthritis of the Knee: A Systematic Review and Statistical Fragility Index-Based Meta-analysis of Randomized Controlled Trials

BACKGROUND

Based in part on the results of randomized controlled trials (RCTs) that suggest a beneficial effect over alternative treatment options, the use of platelet-rich plasma (PRP) for the management of knee osteoarthritis (OA) is widespread and increasing. However, the extent to which these studies are vulnerable to slight variations in the outcomes of patients remains unknown.

PURPOSE

To evaluate the statistical fragility of conclusions from RCTs that reported outcomes of patients with knee OA who were treated with PRP versus alternative nonoperative management strategies.

STUDY DESIGN

Systematic review and meta-analysis; Level of evidence, 2.

METHODS

All RCTs comparing PRP with alternative nonoperative treatment options for knee OA were identified. The fragility index (FI) and reverse FI were applied to assess the robustness of conclusions regarding the efficacy of PRP for knee OA. Meta-analyses were performed to determine the minimum number of patients from ≥1 trials included in the meta-analysis for which a modification on the event status would change the statistical significance of the pooled treatment effect.

RESULTS

In total, this analysis included outcomes from 1993 patients with a mean ± SD age of 58.0 ± 3.8 years. The mean number of events required to reverse significance of individual RCTs (FI) was 4.57 ± 5.85. Based on random-effects meta-analyses, PRP demonstrated a significantly higher rate of successful outcomes when compared with hyaluronic acid (P = .002; odds ratio [OR], 2.19; 95% CI, 1.33-3.62), as well as higher rates of patient-reported symptom relief (P = .019; OR, 1.55; 95% CI, 1.07-2.24), not requiring a reintervention after the initial injection treatment (P = .002; OR, 2.17; 95% CI, 1.33-3.53), and achieving the minimal clinically important difference (MCID) for pain improvement (P = .007; OR, 6.19; 95% CI, 1.63-23.42) when compared with all alternative nonoperative treatments. Overall, the mean number of events per meta-analysis required to change the statistical significance of the pooled treatment effect was 8.67 ± 4.50.

CONCLUSION

Conclusions drawn from individual RCTs evaluating PRP for knee OA demonstrated slight robustness. On meta-analysis, PRP demonstrated a significant advantage over hyaluronic acid as well as improved symptom relief, lower rates of reintervention, and more frequent achievement of the MCID for pain improvement when compared with alternative nonoperative treatment options. Statistically significant pooled treatment effects evaluating PRP for knee OA are more robust than approximately half of all comparable meta-analyses in medicine and health care. Future RCTs and meta-analyses should consider reporting FIs and fragility quotients to facilitate interpretation of results in their proper context.

Statistical Fragility Analysis of Open Reduction Internal Fixation vs Primary Arthrodesis to Treat Lisfranc Injuries: A Systematic Review

BACKGROUND

There is a lack of consensus in the use of open reduction internal fixation (ORIF) vs primary arthrodesis (PA) in the management of Lisfranc injuries. Statistical fragility represents the number of events needed to flip statistical significance and provides context to interpret P values of outcomes from conflicting studies. The current study evaluates the statistical fragility of existing research with an outcome-specific approach to provide statistical clarity to the ORIF vs PA discussion. We hypothesized that statistical fragility analysis would offer clinically relevant insight when interpreting conflicting outcomes regarding ORIF vs PA management of Lisfranc injuries.

METHODS

All comparative studies, RCTs, and case-series investigating ORIF vs PA management of Lisfranc injuries published through October 5, 2023, were identified. Descriptive characteristics, dichotomous outcomes, and continuous outcomes were extracted. Fragility index and continuous fragility index were calculated by the number of event reversals needed to alter significance. Outcomes were categorized by clinical relevance, and median FI and CFI were reported.

RESULTS

A total of 244 studies were screened. Ten studies and 67 outcomes (44 dichotomous, 23 continuous) were included in the fragility analysis. Of the 10 studies, 4 studies claimed PA to correlate with superior outcomes compared to ORIF with regard to functional scores and return to function outcomes. Of these 4 studies, 3 were statistically robust. Six studies claimed PA and ORIF to have no differences in outcomes, in which only 2 studies were statistically robust.

CONCLUSION

The overall research regarding ORIF vs PA is relatively robust compared with other orthopaedic areas of controversy. Although the full statistical context of each article must be considered, studies supporting PA superiority with regard to functional scores and return to function metrics were found to be statistically robust. Outcome-specific analysis revealed moderate fragility in several clinically relevant outcomes such as functional score, return to function, and wound complications.

Comprehensive analyses of mitophagy-related genes and mitophagy-related lncRNAs for patients with ovarian cancer

BACKGROUND

Both mitophagy and long non-coding RNAs (lncRNAs) play crucial roles in ovarian cancer (OC). We sought to explore the characteristics of mitophagy-related gene (MRG) and mitophagy-related lncRNAs (MRL) to facilitate treatment and prognosis of OC.

METHODS

The processed data were extracted from public databases (TCGA, GTEx, GEO and GeneCards). The highly synergistic lncRNA modules and MRLs were identified using weighted gene co-expression network analysis. Using LASSO Cox regression analysis, the MRL-model was first established based on TCGA and then validated with four external GEO datasets. The independent prognostic value of the MRL-model was evaluated by Multivariate Cox regression analysis. Characteristics of functional pathways, somatic mutations, immunity features, and anti-tumor therapy related to the MRL-model were evaluated using abundant algorithms, such as GSEA, ssGSEA, GSVA, maftools, CIBERSORT, xCELL, MCPcounter, ESTIMATE, TIDE, pRRophetic and so on.

RESULTS

We found 52 differentially expressed MRGs and 22 prognostic MRGs in OC. Enrichment analysis revealed that MRGs were involved in mitophagy. Nine prognostic MRLs were identified and eight optimal MRLs combinations were screened to establish the MRL-model. The MRL-model stratified patients into high- and low-risk groups and remained a prognostic factor (P < 0.05) with independent value (P < 0.05) in TCGA and GEO. We observed that OC patients in the high-risk group also had the unfavorable survival in consideration of clinicopathological parameters. The Nomogram was plotted to make the prediction results more intuitive and readable. The two risk groups were enriched in discrepant functional pathways (such as Wnt signaling pathway) and immunity features. Besides, patients in the low-risk group may be more sensitive to immunotherapy (P = 0.01). Several chemotherapeutic drugs (Paclitaxel, Veliparib, Rucaparib, Axitinib, Linsitinib, Saracatinib, Motesanib, Ponatinib, Imatinib and so on) were found with variant sensitivity between the two risk groups. The established ceRNA network indicated the underlying mechanisms of MRLs.

CONCLUSIONS

Our study revealed the roles of MRLs and MRL-model in expression, prognosis, chemotherapy, immunotherapy, and molecular mechanism of OC. Our findings were able to stratify OC patients with high risk, unfavorable prognosis and variant treatment sensitivity, thus improving clinical outcomes for OC patients.

Impact of functional status on 30-day resource utilization and organ system complications following index bariatric surgery: a cohort study

BACKGROUND

Bariatric surgical procedures carry an appreciable risk profile despite their elective nature. Identified risk factors for procedural complications are often limited to medical comorbidities. This study assesses the impact of functional status on resource utilization and organ system complications following bariatric surgery.

MATERIALS AND METHODS

This retrospective cohort study analyzed patients undergoing elective, index bariatric surgery from American College of Surgeons National Surgical Quality Improvement Program participating hospitals from 2015 to 2019 ( n =65 627). The primary independent variable was functional status. The primary outcome was unplanned resource utilization. Secondary outcomes included composite organ system complications and mortality. The impact of functional status was first investigated with univariate analyses. Survival and multivariate analyses were then performed on select complications with clinically and statistically significant incidence in the dependent cohort.

RESULTS

On univariate analysis, dependent functional status was associated with unplanned resource utilization [12.1% (27/223) vs. 4.1% (2661/65 404)]; relative risk, 2.98 (95% CI, 2.09-4.25); P < 0.001] and haematologic/infectious complications [6.7% (15/223) vs. 2.4% (1540/65 404); relative risk, 2.86 (95% CI, 1.75-4.67); P < 0.001]. Survival analysis demonstrated a significantly shorter time to both events in patients with dependent functional status ( P < 0.001). On multivariate analysis, dependent functional status was an independent predictor of unplanned resource utilization[adjusted odds ratio 2.17 (95% CI, 1.27-3.50); P = 0.003; model c-statistic, 0.572]) and haematologic/infectious complications [adjusted odds ratio, 2.20 ([95% CI, 1.14-3.86); P = 0.011; model c-statistic, 0.579].

CONCLUSION

Patients with dependent functional status are at an elevated risk of unplanned resource utilization and haematologic/infectious complications following index bariatric surgery. The increased risk cannot be explained by medical comorbidities alone.

Statistical Significance Versus Clinical Relevance: A Head-to-Head Comparison of the Fragility Index and Relative Risk Index

Background In biostatistics, assessing the fragility of research findings is crucial for understanding their clinical significance. This study focuses on the fragility index, unit fragility index, and relative risk index as measures to evaluate statistical fragility. The fragility indices assess the susceptibility of p-values to change significance with minor alterations in outcomes within a 2x2 contingency table. In contrast, the relative risk index quantifies the deviation of observed findings from therapeutic equivalence, the point at which the relative risk equals 1. While the fragility indices have intuitive appeal and have been widely applied, their behavior across a wide range of contingency tables has not been rigorously evaluated. Methods Using a Python software program, a simulation approach was employed to generate random 2x2 contingency tables. All tables under consideration exhibited p-values < 0.05 according to Fisher's exact test. Subsequently, the fragility indices and the relative risk index were calculated. To account for sample size variations, the indices were divided by the sample size to give fragility and risk quotients. A correlation matrix assessed the collinearity between each metric and the p-value. Results The analysis included 2,000 contingency tables with cell counts ranging from 20 to 480. Notably, the formulas for calculating the fragility indices encountered limitations when cell counts approached zero or duplicate cell counts hindered standardized application. The correlation coefficients with p-values were as follows: unit fragility index (-0.806), fragility index (-0.802), fragility quotient (-0.715), unit fragility quotient (-0.695), relative risk index (-0.403), and risk quotient (-0.261). Conclusion The fragility indices and fragility quotients demonstrated a strong correlation with p-values below 0.05, while the relative risk index and relative risk quotient exhibited a weak association with p-values below this threshold. This implies that the fragility indices offer limited additional information beyond the p-value alone. In contrast, the relative risk index and risk quotient exhibit independence from the p-value, indicating that they may provide important additional information about statistical fragility by evaluating the divergence of observed results from therapeutic equivalence, irrespective of the p-value-based statistical significance.

Empirical assessment of fragility index based on a large database of clinical studies in the Cochrane Library

RATIONALE AIMS AND OBJECTIVES

The fragility index (FI) and fragility quotient (FQ) are increasingly used measures for assessing the robustness of clinical studies with binary outcomes in terms of statistical significance. The FI is the minimum number of event status modifications that can alter a study result's statistical significance (or nonsignificance), and the FQ is calculated as the FI divided by the study's total sample size. The literature has no widely recognized criteria for interpreting the fragility measures' magnitudes. This article aims to provide an empirical assessment for the FI and FQ based on a large database of clinical studies in the Cochrane Library.

METHODS

We explored the overall empirical distributions of the FI and FQ based on five common methods (Fisher's exact test, χ2 test, risk difference, odds ratio, and relative risk) for determining statistical significance of binary outcomes in clinical research. We also considered three different scenarios for the FI calculation and evaluated the relationship between p values and FIs or FQs using Spearman'sρ $\rho $ . Finally, we summarized empirical thresholds based on the overall distributions of the FI and FQ to facilitate their interpretations in future research.

RESULTS

For about 20% of studies with significant results, the statistical significance was changed after modifying the event status of only one participant. Studies with significant results were considered slightly fragile if the significance hinged on the statuses of about five events. Studies were extremely fragile if FI≤ $\le $ 1 or FQ≤ $\le $ 0.01. The FIs were strongly correlated with p values for significant studies, while Spearman'sρ $\rho $ varied according to the total sample sizes of studies.

CONCLUSIONS

The statistical significance of clinical studies could be changed after modifying a few events' statuses. Many studies' findings are fairly fragile. The distributions of the FI and FQ provide insights for appraising the robustness of evidence in clinical decision-making.

Assessing the robustness of results from clinical trials and meta-analyses with the fragility index

The fragility index has been increasingly used to assess the robustness of the results of clinical trials since 2014. It aims at finding the smallest number of event changes that could alter originally statistically significant results. Despite its popularity, some researchers have expressed several concerns about the validity and usefulness of the fragility index. It offers a comprehensive review of the fragility index's rationale, calculation, software, and interpretation, with emphasis on application to studies in obstetrics and gynecology. This article presents the fragility index in the settings of individual clinical trials, standard pairwise meta-analyses, and network meta-analyses. Moreover, this article provides worked examples to demonstrate how the fragility index can be appropriately calculated and interpreted. In addition, the limitations of the traditional fragility index and some solutions proposed in the literature to address these limitations were reviewed. In summary, the fragility index is recommended to be used as a supplemental measure in the reporting of clinical trials and a tool to communicate the robustness of trial results to clinicians. Other considerations that can aid in the fragility index's interpretation include the loss to follow-up and the likelihood of data modifications that achieve the loss of statistical significance.