About half of the noninferiority trials tested superior treatments: a trial-register based study

Management of hard-to-heal leg ulcers with an acid-oxidising solution versus standard of care: the MACAN study

OBJECTIVE

The efficacy of available wound dressings in the treatment of hard-to-heal wounds is limited. A new therapeutic approach using an acid-oxidising solution (AOS) was developed. Its effect on healing progress, tolerability and safety properties were investigated in a clinical study, and compared with standard of care (SOC) wound dressings. The study aimed to demonstrate the non-inferiority of AOS to SOC in terms of wound healing progress.

METHOD

This open-label, randomised controlled trial was conducted at two study centres in Austria with patients with either infected or non-infected hard-to-heal leg ulcers of different aetiology. Patients were treated for six weeks either with AOS or SOC wound dressings. Outcome assessments included the percentage of granulation and re-epithelialisation tissue, wound size reduction, changes in wound pH, infection control and wound pain, local tolerability and adverse events (AEs). Healing time and rate were also assessed.

RESULTS

A total of 50 patients took part. In the AOS group, wounds exhibited higher amounts of granulation and re-epithelialisation tissue, and a faster and more pronounced wound size reduction compared with wounds in the SOC group. In the AOS-treated versus SOC-treated patients, a greater percentage of complete healing of hard-to-heal ulcers was achieved by the end of the study period (32% versus 8%, respectively). Furthermore, the wound pH decreased significantly faster in these wounds (p<0.0001). In all patients with infected leg ulcers, local infection was overcome more rapidly under AOS treatment. In the AOS group, one AE and no serious adverse events (SAEs) were detected versus 24 AEs and two SAEs in the SOC group.

CONCLUSION

In this study, AOS proved to be a highly effective treatment to support wound healing in infected or non-infected hard-to-heal leg ulcers of different aetiology. Efficacy was found to be not only non-inferior but superior to SOC wound dressings. Furthermore, tolerability and safety profiles were favourable for AOS.

Exploring how non-inferiority and equivalence are assessed in paediatrics: a systematic review

OBJECTIVE

To review characteristics, methodology and reporting of non-inferiority and equivalence trials in the specific context of paediatrics.

DESIGN

PubMed and Cochrane databases were searched (up to September 2016) for non-inferiority/equivalence randomised controlled trials conducted in children published in high-impact-factor journals (>5.0 for general/specialist medical journals; >2.2 for paediatric journals).

RESULTS

We found that the statistical hypothesis was inconsistent with the objective in 12 (10%) of the 125 reports included. Non-inferiority (n=98) and equivalence trials (n=27) were mostly used to evaluate interventions with easier administration (45%, n=54/120) and/or better safety profile (34%, n=41/120). All the data needed for targeted sample size recalculation were available for 39 reports (31%). The margin-representing the largest difference between arms that would be clinically acceptable-was reported in 119 (95%), and 44/119 (37%) reported the method used for margin determination. The median sample size was 268 (IQR 125-531). Margins were wider in smaller trials (<125 randomised patients) than in larger trials (p=0.04/p<0.01 for binary/continuous outcomes, respectively). We did not agree with the authors' conclusions in 11% (11/103) of the reports that provided sufficient information.

CONCLUSIONS

There is still a need to improve the quality of methodology, reporting and interpretation of non-inferiority/equivalence trials in paediatrics. In particular, the margins were often not justified and the conclusion was often not supported by the design and/or the results. As researchers have to cope with small sample size and with lack of evidence, methods for non-inferiority/equivalence trials need to be used and/or developed in this vulnerable population.

Do non-inferiority trials of reduced intensity therapies show reduced effects? A descriptive analysis

OBJECTIVES

To identify non-inferiority trials within a cohort where the experimental therapy is the same as the active control comparator but at a reduced intensity and determine if these non-inferiority trials of reduced intensity therapies have less favourable results than other non-inferiority trials in the cohort. Such a finding would provide suggestive evidence of biocreep in these trials.

DESIGN

This metaresearch study used a cohort of non-inferiority trials published in the five highest impact general medical journals during a 5-year period. Data relating to the characteristics and results of the trials were abstracted.

PRIMARY OUTCOME MEASURES

Proportions of trials with a declaration of superiority, non-inferiority and point estimates favouring the experimental therapy and mean absolute risk differences for trials with outcomes expressed as a proportion.

RESULTS

Our search yielded 163 trials reporting 182 non-inferiority comparisons; 36 comparisons from 31 trials were between the same therapy at reduced and full intensity. Compared with trials not evaluating reduced intensity therapies, fewer comparisons of reduced intensity therapies demonstrated a favourable result (non-inferiority or superiority) (58.3%vs82.2%; P=0.002) and fewer demonstrated superiority (2.8%vs18.5%; P=0.019). Likewise, point estimates for reduced intensity therapies more often favoured active control than those for other trials (77.8%vs39.7%; P<0.001) as did mean absolute risk differences (+2.5% vs -0.7%; P=0.018).

CONCLUSIONS

Non-inferiority trials comparing a therapy at reduced intensity to the same therapy at full intensity showed reduced effects compared with other non-inferiority trials. This suggests these trials may have a high rate of type 1 errors and biocreep, with significant implications for the design and interpretation of future non-inferiority trials.

Empirical Consequences of Current Recommendations for the Design and Interpretation of Noninferiority Trials

BACKGROUND

Noninferiority trials are increasingly common, though they have less standardized designs and their interpretation is less familiar to clinicians than superiority trials.

OBJECTIVE

To empirically evaluate a cohort of noninferiority trials to determine 1) their interpretation as recommended by CONSORT, 2) choice of alpha threshold and its sidedness, and 3) differences between methods of analysis such as intention-to-treat and per-protocol.

DESIGN

We searched MEDLINE for parallel-group randomized controlled noninferiority trials published in the five highest-impact general medical journals between 2011 and 2016.

MAIN MEASURES

Data abstracted included trial design parameters, results, and interpretation of results based on CONSORT recommendations.

KEY RESULTS

One hundred sixty-three trials and 182 noninferiority comparisons were included in our analysis. Based on CONSORT-recommended interpretation, 79% of experimental therapies met criteria for noninferiority, 13% met criteria for superiority, 20% were declared inconclusive, and 2% met criteria for inferiority. However, for 12% of trials, the experimental therapy was statistically significantly worse than the active control, but CONSORT recommended an interpretation of inconclusive or noninferior. A two-sided alpha equivalent of greater than 0.05 was used in 34% of the trials, and in five of these trials, the use of a standard two-sided alpha of 0.05 led to changes in the interpretation of results that disfavored the experimental therapy. In four of the five comparisons where different methods of analysis (e.g., intention-to-treat and per-protocol) yielded different results, the intention-to-treat analysis was the more conservative. In 11% of trials, a secondary advantage of the new therapy was neither reported nor could it be inferred by reviewers.

CONCLUSIONS

In this cohort, the design and interpretation of noninferiority trials led to significant and systematic bias in favor of the experimental therapy. Clinicians should exercise caution when interpreting these trials. Future trials may be more reliable if design parameters are standardized.

A framework to assess the value of application of formal criteria to check clinical relevance in RCTs as part of a benefit assessment strategy

Recently, the topic of assessing clinical relevance on top of statistical significance in the analysis of randomized control trials (RCTs) has got increasing attention, in particular as part of benefit assessments. Several formal criteria to serve this purpose have been published. In this paper, we present a framework to assess the value of the application of such criteria. We propose to quantify the need for the assessment of clinical relevance by the actual risk of having accepted a benefit for a treatment with an irrelevant effect in a successful RCT. We then study how this risk can be controlled by two popular criteria based on comparing the effect estimate or the lower bound of the confidence interval with a given threshold. We further propose to quantify the impact of using formal criteria by considering the expected costs when specifying error-specific costs for each of the three possible types of errors: A benefit may be accepted for a treatment, which is actually inferior, or which is not inferior, but only implies an irrelevant improvement, or a benefit may be rejected for a treatment implying a relevant improvement. This way we can demonstrate that the impact depends on parameters which are typically not explicitly defined in the frame of benefit assessments. Depending on the values of these parameters, formal checks of clinical relevance may imply better decisions on average, but they may also imply more harm than good on average.

Choice of non-inferiority (NI) margins does not protect against degradation of treatment effects on an average--an observational study of registered and published NI trials

OBJECTIVE

NI margins have to be chosen appropriately to control the risk of degradation of treatment effects in non-inferiority (NI) trials. We aimed to study whether the current choice of NI margins protects sufficiently against a degradation of treatment effect on an average.

STUDY DESIGN AND SETTING

NI trials reflecting current practice were assembled and for each trial, the NI margin was translated into a likelihood of degradation. The likelihood of degradation was calculated as the conditional probability of a treatment being harmful given that it is declared non-inferior in the trial, using simulation. Its distribution among the NI trials was then studied to assess the potential risk of degradation.

RESULTS

The median (lower/upper quartile) NI margin among 112 binary outcome NI trials corresponded to an odds ratio of 0.57(0.45, 0.66), while among 38 NI trials with continuous outcome, to a Cohen's d of -0.42(-0.54, -0.31) and a hazard ratio of 0.82(0.73, 0.86) among 24 survival outcome NI trials. Overall, the median likelihood of degradation was 56% (45%, 62%).

CONCLUSION

Only two fifths of the current NI trials had a likelihood of degradation lower than 50%, suggesting that, in majority of the NI trials, there is no sufficient protection against degradation on an average. We suggest a third hurdle for the choice of NI margins, thus contributing a sufficient degree of protection.

Assessing additional benefit in noninferiority trials

A method for simultaneously assessing noninferiority with respect to efficacy and superiority with respect to another endpoint in two-arm noninferiority trials is presented. The procedure controls both the average type I error rate for the intersection-union test problem and the frequentist type I error rate for the noninferiority test by α while allowing an increased level for the superiority test. For normally distributed outcomes, two methods are presented to deal with the uncertainty about the correlation between the endpoints which defines the adjusted levels. The operating characteristics of these procedures are investigated. Furthermore, the sample size required when applying the proposed method is compared with that of alternative procedures. Application of the method in the situation of binary endpoints and mixed normal and binary endpoints, respectively, is sketched. An illustrative example is provided demonstrating implementation of the proposed approach in a clinical trial.