Choice of non-inferiority (NI) margins does not protect against degradation of treatment effects on an average--an observational study of registered and published NI trials

StudiCare procrastination - Randomized controlled non-inferiority trial of a persuasive design-optimized internet- and mobile-based intervention with digital coach targeting procrastination in college students

BACKGROUND

Academic procrastination is widespread among college students. Procrastination is strongly negatively correlated with psychological well-being, thus early interventions are needed. Internet- and mobile-based cognitive behavioral therapy (iCBT) could provide a low-threshold treatment option. Human guidance seems to be a decisive mechanism of change in iCBT. Persuasive design optimization of iCBT and guidance by a digital coach might represent a resource-saving alternative. The study evaluated the non-inferiority of a digital coach in comparison to human guidance with regard to the primary outcome procrastination.

METHODS

The iCBT StudiCare procrastination was optimized by principles of the Persuasive System Design (PSD). A total of 233 college students were randomly assigned to either StudiCare procrastination guided by a digital coach (intervention group, IG) or by a human eCoach (control group, CG). All participants were assessed at baseline, 4-, 8- and 12-weeks post-randomization. Symptom change and between-group differences were assessed with latent growth curve models and supported by effect size levels. The non-inferiority margin was set at Cohen's d = - 0.3.

RESULTS

The primary outcome procrastination measured by the Irrational Procrastination scale (IPS) significantly decreased across groups (γ = - 0.79, p < .001, Cohen's d = -0.43 to -0.89) from baseline to 12-weeks post-randomization. There were no significant differences between groups (γ = -0.03, p = .84, Cohen's d = -0.03 to 0.08). Regarding symptoms of depression, no significant time x group effect was found (γ = 0.26, p = .09; Cohen's d = -0.15 to 0.21). There was also no significant time x group effect on the improvement of symptoms of anxiety (γ = 0.25, p = .09). However, Cohen's ds were above the non-inferiority margin 8-weeks (Cohen's d = 0.51) and 12-weeks post-randomization (Cohen's d = 0.37), preferring the CG. Of the IG, 34% and of the CG, 36% completed 80% of the modules.

CONCLUSIONS

The PSD optimized version of StudiCare procrastination is effective in reducing procrastination. The digital coach was not inferior to human guidance. Guidance by a digital coach in iCBT against procrastination for college students could be a resource-saving alternative to human guidance.

TRIAL REGISTRATION

The trial was registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trial Register (ID: DRKS00025209, 30/04/2021).

Parent Training via Internet or in Group for Disruptive Behaviors: A Randomized Clinical Noninferiority Trial

OBJECTIVE

To evaluate if an internet-delivered parent training program is noninferior to its group-delivered counterpart in reducing child disruptive behavior problems (DBP).

METHOD

This noninferiority randomized clinical trial enrolled families seeking treatment in primary care in Stockholm, Sweden, for DBP in a child 3-11 years of age. Participants were randomized to internet-delivered (iComet) or group-delivered (gComet) parent training. The primary outcome was parent-rated DBP. Assessments were made at baseline and 3, 6, and 12 months. Secondary outcomes included child and parent behaviors and well-being and treatment satisfaction. The noninferiority analysis was determined by a one-sided 95% CI of the mean difference between gComet and iComet using multilevel modeling.

RESULTS

This trial included 161 children (mean age 8.0); 102 (63%) were boys. In both intention-to-treat and per-protocol analyses, iComet was noninferior to gComet. There were small differences in between-group effect sizes (d = -0.02 to 0.13) on the primary outcome with the upper limit of the one-sided 95% CI below the noninferiority margin at 3-, 6-, and 12-month follow-up. Parents were more satisfied with gComet (d = 0.49, 95% CI [0.26, 0.71]). At 3-month follow-up, there were also significant differences in treatment effect on attention-deficit/hyperactivity disorder symptoms (d = 0.34, 95% CI [0.07, 0.61]) and parenting behavior (d = 0.41, 95% CI [0.17, 0.65]) favoring gComet. At 12-month follow-up, there were no differences in any outcomes.

CONCLUSION

Internet-delivered parent training was noninferior to group-delivered parent training in reducing child DBP. The results were maintained at 12-month follow-up. This study supports internet-delivered parent training being used as an alternative to group-delivered parent training in clinical settings.

CLINICAL TRIAL REGISTRATION INFORMATION

Randomized Controlled Trial of Comet via the Internet or in Group Format; https://www.

CLINICALTRIALS

gov/; NCT03465384.

Health-related quality of life after restrictive versus liberal RBC transfusion for cardiac surgery: Sub-study from a randomized clinical trial

BACKGROUND

Transfusion Requirements in Cardiac Surgery III (TRICS III), a multi-center randomized controlled trial, demonstrated clinical non-inferiority for restrictive versus liberal RBC transfusion for patients undergoing cardiac surgery. However, it is uncertain if transfusion strategy affects long-term health-related quality of life (HRQOL).

STUDY DESIGN AND METHODS

In this planned sub-study of Australian patients in TRICS III, we sought to determine the non-inferiority of restrictive versus liberal transfusion strategy on long-term HRQOL and to describe clinical outcomes 24 months postoperatively. The restrictive strategy involved transfusing RBCs when hemoglobin was <7.5 g/dl; the transfusion triggers in the liberal group were: <9.5 g/L intraoperatively, <9.5 g/L in intensive care, or <8.5 g/dl on the ward. HRQOL assessments were performed using the 36-item short form survey version 2 (SF-36v2). Primary outcome was non-inferiority of summary measures of SF-36v2 at 12 months, (non-inferiority margin: -0.25 effect size; restrictive minus liberal scores). Secondary outcomes included non-inferiority of HRQOL at 18 and 24 months.

RESULTS

Six hundred seventeen Australian patients received allocated randomization; HRQOL data were available for 208/311 in restrictive and 217/306 in liberal group. After multiple imputation, non-inferiority of restrictive transfusion at 12 months was not demonstrated for HRQOL, and the estimates were directionally in favor of liberal transfusion. Non-inferiority also could not be concluded at 18 and 24 months. Sensitivity analyses supported these results. There were no differences in quality-adjusted life years or composite clinical outcomes up to 24 months after surgery.

DISCUSSION

The non-inferiority of a restrictive compared to a liberal transfusion strategy was not established for long-term HRQOL in this dataset.

Testing for non-inferior mortality: a systematic review of non-inferiority margin sizes and trial characteristics

OBJECTIVE

To describe the size and variability of non-inferiority margins used in non-inferiority trials of medications with primary outcomes involving mortality, and to examine the association between trial characteristics and non-inferiority margin size.

DESIGN

Systematic review.

DATA SOURCES

Medline, Medline In Process, Medline Epub Ahead of Print and Embase Classic+Embase databases from January 1989 to December 2019.

ELIGIBILITY CRITERIA

Prospective non-inferiority randomised controlled trials comparing pharmacological therapies, with primary analyses for non-inferiority and primary outcomes involving mortality alone or as part of a composite outcome. Trials had to prespecify non-inferiority margins as absolute risk differences or relative to risks of outcome and provide a baseline risk of primary outcome in the control intervention.

RESULTS

3992 records were screened, 195 articles were selected for full text review and 111 articles were included for analyses. 82% of trials were conducted in thrombosis, infectious diseases or oncology. Mortality was the sole primary outcome in 23 (21%) trials, and part of a composite primary outcome in 88 (79%) trials. The overall median non-inferiority margin was an absolute risk difference of 9% (IQR 4.2%-10%). When non-inferiority margins were expressed relative to the baseline risk of primary outcome in control groups, the median relative non-inferiority margin was 1.5 (IQR 1.3-1.7). In multivariable regression analyses examining the association between trial characteristics (medical specialty, inclusion of paediatric patients, mortality as a sole or part of a composite primary outcome, presence of industry funding) and non-inferiority margin size, only medical specialty was significantly associated with non-inferiority margin size.

CONCLUSION

Absolute and relative non-inferiority margins used in published trials comparing medications are large, allowing conclusions of non-inferiority in the context of large differences in mortality. Accepting the potential for large increases in outcomes involving mortality while declaring non-inferiority is a challenging methodological issue in the conduct of non-inferiority trials.

Optimal contrast analysis with heterogeneous variances and budget concerns

The omnibus test is commonly applied to evaluate the overall disparity between group means in ANOVA. Alternatively, linear contrasts are more informative in detecting specific pattern of mean differences that cannot be obtained via the omnibus test. This article concerns power and sample size calculations for contrast analysis with heterogeneous variances and budget concerns. Optimal allocation procedures for the Welch-Satterthwaite tests of standardized and unstandardized contrasts are presented to minimize the total sample size with the designated ratios, to meet a desirable power level for the least cost, and to attain the maximum power performance under a fixed cost. Currently available methods rely exclusively on simple allocation formula and direct rounding rule. The proposed allocation strategies combine the computing techniques of nonlinear optimization search and iterative screening process. Numerical assessments of a randomized control trial for the overcoming depression on the Internet are conducted to demonstrate and confirm that the approximate procedures do not guarantee optimal solution. The suggested approaches extend and outperform the existing findings in methodological soundness and overall performance. The corresponding computer algorithms are developed to implement the recommended power and sample size calculations for optimal contrast analysis.

Exploring how non-inferiority and equivalence are assessed in paediatrics: a systematic review

OBJECTIVE

To review characteristics, methodology and reporting of non-inferiority and equivalence trials in the specific context of paediatrics.

DESIGN

PubMed and Cochrane databases were searched (up to September 2016) for non-inferiority/equivalence randomised controlled trials conducted in children published in high-impact-factor journals (>5.0 for general/specialist medical journals; >2.2 for paediatric journals).

RESULTS

We found that the statistical hypothesis was inconsistent with the objective in 12 (10%) of the 125 reports included. Non-inferiority (n=98) and equivalence trials (n=27) were mostly used to evaluate interventions with easier administration (45%, n=54/120) and/or better safety profile (34%, n=41/120). All the data needed for targeted sample size recalculation were available for 39 reports (31%). The margin-representing the largest difference between arms that would be clinically acceptable-was reported in 119 (95%), and 44/119 (37%) reported the method used for margin determination. The median sample size was 268 (IQR 125-531). Margins were wider in smaller trials (<125 randomised patients) than in larger trials (p=0.04/p<0.01 for binary/continuous outcomes, respectively). We did not agree with the authors' conclusions in 11% (11/103) of the reports that provided sufficient information.

CONCLUSIONS

There is still a need to improve the quality of methodology, reporting and interpretation of non-inferiority/equivalence trials in paediatrics. In particular, the margins were often not justified and the conclusion was often not supported by the design and/or the results. As researchers have to cope with small sample size and with lack of evidence, methods for non-inferiority/equivalence trials need to be used and/or developed in this vulnerable population.

Some problems with non-inferiority tests in psychotherapy research: psychodynamic therapies as an example

In virtually every field of medicine, non-inferiority trials and meta-analyses with non-inferiority conclusions are increasingly common. This non-inferiority approach has been frequently used by a group of authors favoring psychodynamic therapies (PDTs), concluding that PDTs are just as effective as cognitive-behavioral therapies (CBT). We focus on these examples to exemplify some problems associated with non-inferiority tests of psychological treatments, although the problems also apply to psychopharmacotherapy research, CBT research, and others. We conclude that non-inferiority trials have specific risks of different types of validity problems, usually favoring an (erroneous) non-inferiority conclusion. Non-inferiority trials require the definition of non-inferiority margins, and currently used thresholds have a tendency to be inflationary, not protecting sufficiently against degradation. The use of non-inferiority approaches can lead to the astonishing result that one single analysis can suggest both, superiority of the comparator (here: CBT) and non-inferiority of the other treatment (here PDT) at the same time. We provide recommendations how to improve the quality of non-inferiority trials, and we recommend to consider them among other criteria when evaluating manuscripts examining non-inferiority trials. If psychotherapeutic families (such as PDT and CBT) differ on the number of investigating trials, and in the fields of clinical applications, and in other validity aspects mentioned above, conclusions about their general non-inferiority are no more than a best guess, typically expressing the favored approach of the lead author.

Do non-inferiority trials of reduced intensity therapies show reduced effects? A descriptive analysis

OBJECTIVES

To identify non-inferiority trials within a cohort where the experimental therapy is the same as the active control comparator but at a reduced intensity and determine if these non-inferiority trials of reduced intensity therapies have less favourable results than other non-inferiority trials in the cohort. Such a finding would provide suggestive evidence of biocreep in these trials.

DESIGN

This metaresearch study used a cohort of non-inferiority trials published in the five highest impact general medical journals during a 5-year period. Data relating to the characteristics and results of the trials were abstracted.

PRIMARY OUTCOME MEASURES

Proportions of trials with a declaration of superiority, non-inferiority and point estimates favouring the experimental therapy and mean absolute risk differences for trials with outcomes expressed as a proportion.

RESULTS

Our search yielded 163 trials reporting 182 non-inferiority comparisons; 36 comparisons from 31 trials were between the same therapy at reduced and full intensity. Compared with trials not evaluating reduced intensity therapies, fewer comparisons of reduced intensity therapies demonstrated a favourable result (non-inferiority or superiority) (58.3%vs82.2%; P=0.002) and fewer demonstrated superiority (2.8%vs18.5%; P=0.019). Likewise, point estimates for reduced intensity therapies more often favoured active control than those for other trials (77.8%vs39.7%; P<0.001) as did mean absolute risk differences (+2.5% vs -0.7%; P=0.018).

CONCLUSIONS

Non-inferiority trials comparing a therapy at reduced intensity to the same therapy at full intensity showed reduced effects compared with other non-inferiority trials. This suggests these trials may have a high rate of type 1 errors and biocreep, with significant implications for the design and interpretation of future non-inferiority trials.

Non-inferiority versus superiority drug claims: the (not so) subtle distinction

BACKGROUND

Current regulatory guidance and practice of non-inferiority trials are asymmetric in favor of the test treatment (Test) over the reference treatment (Control). These trials are designed to compare the relative efficacy of Test to Control by reference to a clinically important margin, M.

MAIN TEXT

Non-inferiority trials allow for the conclusion of: (a) non-inferiority of Test to Control if Test is slightly worse than Control but by no more than M; and (b) superiority if Test is slightly better than Control even if it is by less than M. From Control's perspective, (b) should lead to a conclusion of non-inferiority of Control to Test. The logical interpretation ought to be that, while Test is statistically better, it is not clinically superior to Control (since Control should be able to claim non-inferiority to Test). This article makes a distinction between statistical and clinical significance, providing for symmetry in the interpretation of results. Statistical superiority and clinical superiority are achieved, respectively, when the null and the non-inferiority margins are exceeded. We discuss a similar modification to placebo-controlled trials.

CONCLUSION

Rules for interpretation should not favor one treatment over another. Claims of statistical or clinical superiority should depend on whether or not the null margin or the clinically relevant margin is exceeded.

The choice of the noninferiority margin in clinical trials was driven by baseline risk, type of primary outcome, and benefits of new treatment

OBJECTIVES

To explore characteristics of clinical trials that influence the choice of the noninferiority margin (NIM) when planning the trial.

STUDY DESIGN AND SETTING

We conducted an experimental survey among corresponding authors of randomized controlled trials indexed in MEDLINE. We described two hypothetical studies and asked the respondents' opinion on the largest loss of effectiveness that is clinically negligible (or the smallest lost of effectiveness that is clinically important in the superiority scenario). We randomly manipulated four study attributes in each vignette, using a factorial design.

RESULTS

A total of 364 researchers participated. The values for NIMs were significantly lower than the differences to be detected in a superiority trial. The NIM was smaller when the primary outcome was mortality compared with treatment failure, when baseline risk in the control arm was lower, and when the advantage of the new treatment was a lower cost compared with having fewer side effects. In contrast, the population age group under study and the difficulty to recruit patients showed no effect on the choice of the NIM.

CONCLUSION

In our experimental study, the factors associated with lower NIMs were mortality as a primary outcome, low baseline risk, and a less costly new treatment.