Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance

Editorial Commentary: Knee Lateral Extra-articular Tenodesis Procedures: Appropriate Study Design Is Essential for Reducing Conflicting Findings and Unnecessary Controversy in the Orthopaedic Literature

Knee lateral extra-articular tenodesis procedures (LEAPs) reduce graft rupture rates when performed at the time of anterior cruciate ligament (ACL) reconstruction. However, in the setting of revision ACL reconstruction, LEAPs are less studied and remain controversial. Many studies support combined procedures (ACL + LEAP), yet others do not. When the literature comprises small patient cohorts and short follow-up periods, conflicting results often arise. The controversy surrounding them may be unnecessarily generated by the publication of low-quality studies. Future studies should focus on adequate power; appropriate design and methodology, including matching or randomization to account for potential confounding factors; proper statistical analyses; and avoidance of spin bias.

(Re)animation of a stuffed plush bunny: A case report

An expressed and constant wish of the first author's oldest daughter to enhance interaction with her favourite toy animal led to a (re)animation/resuscitation attempt of a 1½-year-old stuffed plush bunny. Initial physical examination found no vital signs. Based on the lack of identifiable airways, we hypothesised that tissue oxygenation might be caused by passive diffusion throughout the body. Hence, animation was attempted by mechanical chest compressions without including airway management or positive-pressure ventilation. Multimodal monitoring of arterial blood pressure (by proxy), intra-'cranial' pressure and oxygen tension, near-infrared spectroscopy of the head and laser-Doppler blood flow was successfully initiated, whereas an attempt at intracranial microdialysis was unsuccessful. Despite achieving measurable arterial blood pressure (by proxy) (12/3 mmHg) and an increase of cerebral perfusion by 30 points, spontaneous circulation or diffusion was not achieved apparently, and ultimately, animation attempts were ceased. Clinical experience, as well as common sense, forces us to conclude that our measurements were contaminated by the intervention, and that we must rethink the method for the animation of stuffed plush bunnies.

Influence of daily usage times on patients' compliance during at-home bleaching: a randomized clinical trial

The effectiveness of at-home dental bleaching treatments depends on the time that bleaching products are in contact with the teeth surface and, consequently, on the adequate use of associated custom acetate trays.

OBJECTIVE

This randomized single-blinded trial aimed to analyze if the daily usage time of these products influences the patient's compliance behavior when submitted to monitored at-home dental bleaching. Secondary outcomes were color change and tooth sensitivity.

METHODOLOGY

Sixty-six volunteers were randomly distributed into three groups (n=22): patients were instructed to use the trays for 2 (G2), 4 (G4), and 8 (G8) hours daily. The daily dental bleaching compliance behavior was measured using a microsensor inserted into the trays. Subjective and objective color evaluation assessments were adopted at baseline (T0), one (T1), two (T2), and three weeks (T3) after the beginning of the bleaching treatment, as well as two weeks after the treatment (T4). Tooth sensitivity was analyzed using the VAS scale, ranging from T1 to T4.

RESULTS

G2 showed a greater degree of cooperation than G8 and cooperation was inversely proportional to the recommended usage time. Significantly higher color change was observed in the upper arch for G8 when compared to G2 in subjective analysis, from T1 to T4. There were no statistical differences between the groups in objective analysis.

CONCLUSION

Shorter recommended usage time of the bleaching product may improve the patient's compliance with at-home dental bleaching treatments. However, increased daily usage time may promote better subjective color change. Bleaching sensitivity was more significant in the first week for a longer time of use.

A randomized, sham-controlled, quintuple-blinded trial to evaluate the NET device as an alternative to medication for promoting opioid abstinence

Background

There is an unmet need for non-medication approaches to illicit opioid discontinuation and relapse prevention. The NET (NeuroElectric Therapy) Device is a non-invasive, battery-operated, portable, re-useable device designed to deliver bilateral transcranial transcutaneous alternating current electrical stimulation, and is intended to treat opioid use disorder (OUD) without medication. The device is a CE-marked Class IIa, non-significant risk, investigational medical device.

Objective

This prospective trial (NRC021) tests the hypothesis that the NET Device provides safe and effective neurostimulation treatment for persons with OUD who express a desire to be opioid abstinent without medications for opioid use disorder (MOUD).

Methods

NRC021 is a randomized, parallel-group, sham-controlled, quintuple-blinded, single-site study. Persons with OUD entering a residential treatment facility for opioid detoxification are assigned to active or sham treatment (n = 50/group). Group assignment is stratified on presence of any current non-opioid substance use disorder and by sex. The biostatistician maintains the blinding so that the study sponsor, principal investigator, research assistants, treatment staff, and participants remain blinded. Following discharge from the inpatient facility, participants are assessed once weekly over 12 weeks for substance use (using timeline followback interview and video assessment of observed oral fluid sample provision and testing). The primary efficacy endpoint is each participant's overall percentage of weekly abstinence from illicit opioid use without use of MOUD. The secondary efficacy endpoint is each participant's percentage of non-opioid drug-free weeks. Safety outcomes are also measured.

Conclusion

NRC021 is designed to assess the efficacy of a novel non-medication treatment for OUD.

Clinical trial registration

ClinicalTrials.gov with the identifier NCT04916600.

Primary and Secondary Outcome Reporting in Randomized Trials: JACC State-of-the-Art Review

Consensus as to best practices for the selection, reporting, and interpretation of primary and secondary outcomes of randomized controlled trials is lacking. We reviewed the strategies adopted in publications of randomized controlled trials (RCTs) for the analysis, presentation, and interpretation of efficacy outcomes from a survey of all cardiovascular RCTs published in the New England Journal of Medicine, Lancet, and the Journal of the American Medical Association during 2019. We focus on the choice of primary outcomes, the variety of approaches to selecting secondary outcomes, the options sometimes used to control type I error, and the common practice to not correct for multiple testing in reporting secondary outcomes. We comment on current practice across journals in the reporting of P values and also how conclusions in trial reports frequently adhere to an undue reliance on P < 0.05 as a basis for positive claims of treatment efficacy. We conclude with recommendations for how future RCT reports could best select, report, and interpret their findings on primary and secondary outcomes.

Contemporary narrative review of treatment options for COVID-19

The coronavirus disease 2019 (COVID-19) pandemic is ongoing and many drugs have been studied in clinical trials. From a pathophysiological perspective, anti-viral drugs may be more effective in the early stage while immunomodulators may be more effective in severe patients in later stages of infection. While drugs such as lopinavir-ritonavir, hydroxychloroquine and azithromycin have proved to be ineffective in randomized controlled trials, corticosteroids, neutralizing monoclonal antibodies, remdesivir, tocilizumab and baricitinib have been reported to benefit certain groups of patients with COVID-19. In this review, we will present the key clinical evidence and progress in promising COVID-19 therapeutics, as well as summarize the experience and lessons learned from the development of the current therapeutics.

Comparative safety and effectiveness of vedolizumab to tumour necrosis factor antagonist therapy for Crohn's disease

BACKGROUND

Direct comparisons are lacking between vedolizumab and tumour necrosis factor (TNF)-antagonist therapy in Crohn's disease (CD).

AIM

To compare safety and effectiveness of vedolizumab and TNF-antagonist therapy in adult CD patients.

METHODS

Retrospective observational cohort (May 2014-December 2017) propensity score-weighted comparison of vedolizumab vs TNF-antagonist therapy (infliximab, adalimumab, certolizumab) in CD. Propensity scores were weighted for age, prior treatments, disease complications, extent and severity, steroid dependence, and concomitant immunosuppressive drug use. The primary outcome was comparative risk for infections or non-infectious serious adverse events (requiring antibiotics, antivirals, antifungals, hospitalisation, or treatment discontinuation, or resulting in death). Secondary comparative effectiveness outcomes were clinical remission (resolution of CD-related symptoms), steroid-free clinical remission and endoscopic remission (absence of ulcers/erosions).

RESULTS

We included 1266 patients (n = 659 vedolizumab). Rates of non-infectious serious adverse events (odds ratio [OR] 0.072, 95% confidence interval [CI] 0.012-0.242), but not serious infections (OR 1.183, 95% CI 0.786-1.795), were significantly lower with vedolizumab vs TNF-antagonist therapy. Safety comparisons for non-infectious serious adverse events remained significant after adjusting for differences in duration of exposure. No significant difference was observed between vedolizumab and TNF-antagonist therapy for clinical remission (hazard ratio [HR] 0.932, 95% CI 0.707-1.228), steroid-free clinical remission (HR 1.250, 95% CI 0.677-2.310) or endoscopic remission (HR 0.827, 95% CI 0.595-1.151). TNF-antagonist therapy was associated with higher treatment persistence compared with vedolizumab.

CONCLUSIONS

There was a lower risk of non-infectious serious adverse events, but not serious infections, with vedolizumab vs TNF-antagonist therapy, with no significant difference for achieving disease remission.

Comparisons of global tests on intersection hypotheses and their application in matched parallel gatekeeping procedures

A clinical trial often has primary and secondary endpoints and comparisons of high and low doses of a study drug to a control. Multiplicity is not only caused by the multiple comparisons of study drugs versus the control, but also from the hierarchical structure of the hypotheses. Closed test procedures were proposed as general methods to address multiplicity. Two commonly used tests for intersection hypotheses in closed test procedures are the Simes test and the average method. When the treatment effect of a less efficacious dose is not much smaller than the treatment effect of a more efficacious dose for a specific endpoint, the average method has better power than the Simes test for the comparison of two doses versus control. Accordingly, for inferences for primary and secondary endpoints, the matched parallel gatekeeping procedure based on the Simes test for testing intersection hypotheses is extended here to allow the average method for such testing. This procedure is further extended to clinical trials with more than two endpoints as well as to clinical trials with more than two active doses and a control.

Improving the Standards of Reporting of Clinical Trial Data

Although checklists and guidelines for reporting and interpretation of clinical trial results are of immense value there is still room for a biased presentation in journal publications. Two important sources of bias that remain are as follows: (1) The absence of a principle guiding the display of point estimates in abstracts. For example, bias arises, even for a primary endpoint, when the reported point estimate is preferentially selected and does not correspond to the prespecified method of analysis. The benefit of treatment on an endpoint is often communicated through point estimates, and as abstracts contain the main takeaways, establishing ground rules for what to include and what not to include is crucial. (2) A commingling in the body of the publication of results from α-controlled endpoints, non-α-controlled endpoints, and post hoc analyses. The total number of non-α-controlled and post hoc analyses are unknown. Blending a favored selection of these with α-controlled results provides opportunities to overstate or understate findings as desired. Publicly available results provide the grist for the changes proposed to improve reporting standards. Additional changes are recommended as well, including a threshold of significance more stringent than 0.05 for non-α-controlled analyses. For safety, the proposal is to display the data via the mean cumulative function graph for prespecified adverse events of interest. The bottom line is that more objective reporting can be achieved if journals establish standards for reporting of point estimates in abstracts and require a hierarchical display of results in the main body.

Adaptive Antiretroviral Therapy Adherence Interventions for Youth Living With HIV Through Text Message and Cell Phone Support With and Without Incentives: Protocol for a Sequential Multiple Assignment Randomized Trial (SMART)

BACKGROUND

Youth living with HIV (YLH) aged 13 to 24 years made up over a fifth (21%) of new HIV diagnoses in 2016, yet only 27% of YLH are virally suppressed. YLH have been shown to be poorly adherent to antiretroviral therapy (ART); however, there has been limited research investigating how to increase adherence in YLH. Mobile health (mHealth) interventions may be one promising way to do this.

OBJECTIVE

This study (ATN [Adolescent Trials Network] 144 SMART) aimed to compare adaptive interventions that could increase ART adherence in YLH aged 15 to 24 years. This includes mHealth initiatives, the tapering of interventions, and the use of incentives. Cost-effectiveness of sequencing the interventions without incentives before providing incentives and the savings on societal costs due to suppressed viral loads will be determined. This protocol is part of the ATN Scale It Up program described in this issue by Naar et al.

METHODS

This study uses a Sequential Multiple Assignment Randomized Trial design. Approximately 190 participants are being recruited, enrolled, and randomized to either cell phone support or text message support. Both intervention groups receive 3 months of intervention, followed by a second randomization based on response to the intervention. Responders test tapering their intervention, and nonresponders test receiving incentives.

RESULTS

Data collection for this study is projected to begin in August 2018 and last until June 2020.

CONCLUSIONS

This is an innovative study, particularly in terms of population, intervention types, focus on cost-effectiveness, and recruitment. This study could be particularly effective in improving adherence in YLH while reducing long-term individual and societal costs.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03535337; https://clinicaltrials.gov/ct2/show/NCT03535337 (Archived by WebCite at http://www.webcitation.org/74alXb92z).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/11183.

Interpretation of clinical endpoints in trials of acute myeloid leukemia

Treatment regimens for acute myeloid leukemia (AML) have remained largely unchanged until recently. Molecular advances have opened the door to targeted therapies, many of which are in late-phase clinical trials. As new therapeutic opportunities arise, it is appropriate to review key aspects of clinical trial design, statistical interpretation of outcomes, and methods of data reporting. Complete remission and overall survival (OS) are common primary endpoints in early-phase AML clinical trials. OS and event-free survival are frequent primary endpoints in phase 3 trials. Clinical trials are designed to address the primary endpoint using prespecified α and power levels. Interpretation of additional endpoints (eg, secondary endpoints and subgroup analyses) must be viewed in light of a trial's statistical design. Furthermore, variations in reporting of endpoints must be considered in order to understand trial outcomes. Time-to-event endpoints are typically reported using Kaplan-Meier curves, which are visually informative. Statistical data derived from these curves can be complex, and a variety of factors may impact interpretation. The purpose of this review is to discuss the nuances of common AML trial endpoints and their data presentation to better inform evaluation and understanding of clinical trial data.

Individualized support for informal caregivers of people with dementia - effectiveness of the German adaptation of REACH II

Background

Individualized, outreach and structured multicomponent interventions are a promising intervention approach to relieve the burden of informal caregivers of people with dementia. In this study, we adapted and evaluated a multicomponent intervention (Resources for Enhancing Alzheimer’s Caregiver Health II, REACH II), which was developed in the USA, to the German health-care system. Therefore the project is called the German adaptation of REACH II (in German: Deutsche Adaptation der REACH II, DE-REACH).

Methods

The effectiveness of DE-REACH was examined in a randomized, controlled trial on 92 informal caregivers of people with dementia. The intervention comprised 12 individual two-weekly sessions (9 at home with the informal caregiver and 3 via telephone) and combined five modules. The reduction of the burden of the informal caregivers was chosen as the primary outcome.

Results

The results showed a great stabilizing effect of the intervention on caregiver burden (effect size d = 0.91), that is, comparing pre- and post-measurements the burden decreased very slightly in the intervention group whereas it increased very strongly in the control group. After a three-month follow-up period this effect decreased from a great to a moderate effect. There were also improvements as a result of the intervention in somatization, health-related psychological quality of life and the reaction of the informal caregivers in response to challenging behaviors of the relative with dementia. Moreover, the frequency of challenging behaviors of the affected person itself was reduced in favor of the intervention.

Conclusion

The findings of this study provide further evidence for the impact of multicomponent support interventions for informal caregivers of people with dementia.

Clinical trial registration

NCT01690117. Registered September 17, 2012.

Advances in p-Value Based Multiple Test Procedures

In this article we review recent advances in [Formula: see text]-value-based multiple test procedures (MTPs). We begin with a brief review of the basic tests of Bonferroni and Simes. Standard stepwise MTPs derived from them using the closure method of Marcus et al. (1976) are discussed next. They include the well-known MTPs of Holm (1979), Hochberg (1988) and Hommel (1988), and their extensions and improvements. This is followed by stepwise MTPs for a priori ordered hypotheses. Next we present gatekeeping MTPs (Dmitrienko and Tamhane, 2007) for hierarchically ordered families of hypotheses with logical relations among them. Finally, we give a brief review of the graphical approach (Bretz et al., 2009) to constructing and visualizing gatekeeping and other MTPs. Simple numerical examples are given to illustrate the various procedures.

Systematic literature review of health-related quality of life in locally-advanced non-small cell lung cancer: Has it yet become state-of-the-art?

Lung cancer and its treatment have an important impact on the patients' health-related quality-of-life (HRQoL). A systematic literature review of prospective clinical studies published since 2005 and measuring HRQoL in patients with locally-advanced non-small cell lung cancer (LA- NSCLC) was performed. Besides reviewing the HRQoL impact of LA-NSCLC treatment, it critically examined the frequency, methodology and quality of HRQoL data collection and analysis in LA-NSCLC clinical studies. Out of 814 potentially eligible publications, only 27 (representing 19 individual studies) met the inclusion criteria. Eight studies documented an impact on HRQoL. Large variability in use of HRQoL instruments, statistical analysis and methodological quality was observed. Reporting of HRQoL data lacks standardization, but recent initiatives establishing recommendations to standardize the analysis and reporting of HRQoL in cancer trials are expected to address these issues. Overall, more research is needed to evaluate the treatment impact on HRQoL in both clinical trials and daily care.

Impact of Induction Therapy on Delayed Graft Function Following Kidney Transplantation in Mated Kidneys

BACKGROUND

Delayed graft function (DGF) is defined as the need for dialysis within 1 week of transplantation and occurs in 20%-50% of deceased-donor kidney transplant recipients. Although recovery from DGF often occurs within a few days, many cases may take weeks to months before the transplant function begins. The delay in function increases the complexity of recipient care, makes the diagnosis of acute rejection more difficult, prolongs length of stay, and increases hospital costs. Although several authors have proposed nomograms to predict DGF, there is no identifiable strategy to ameliorate it, except for the possible use of a specific type of induction therapy called Thymoglobulin.

METHODS

In this retrospective analysis we included 407 subjects, of which 76 were mated (left and right kidney transplanted at Montefiore from the same donor). We used conditional logistic regression analysis while adjusting for the mated kidneys. We adjusted for age, gender, and race a priori, as well as cold ischemia time.

RESULTS

There was a 36% decrease in odds of DGF when Thymoglobulin was used as induction when compared with basiliximab in mated kidneys 0.64 (0.10-4.05) (odds ratio [OR] with 95% confidence interval [CI]).

CONCLUSIONS

Thymoglobulin did have a protective effect in these data when analyzed in mated kidneys, however, we need a larger amount of data to concretely conclude this effect.

Robust inference for group sequential trials

For ethical reasons, group sequential trials were introduced to allow trials to stop early in the event of extreme results. Endpoints in such trials are usually mortality or irreversible morbidity. For a given endpoint, the norm is to use a single test statistic and to use that same statistic for each analysis. This approach is risky because the test statistic has to be specified before the study is unblinded, and there is loss in power if the assumptions that ensure optimality for each analysis are not met. To minimize the risk of moderate to substantial loss in power due to a suboptimal choice of a statistic, a robust method was developed for nonsequential trials. The concept is analogous to diversification of financial investments to minimize risk. The method is based on combining P values from multiple test statistics for formal inference while controlling the type I error rate at its designated value.This article evaluates the performance of 2 P value combining methods for group sequential trials. The emphasis is on time to event trials although results from less complex trials are also included. The gain or loss in power with the combination method relative to a single statistic is asymmetric in its favor. Depending on the power of each individual test, the combination method can give more power than any single test or give power that is closer to the test with the most power. The versatility of the method is that it can combine P values from different test statistics for analysis at different times. The robustness of results suggests that inference from group sequential trials can be strengthened with the use of combined tests.

A Benefit-Risk Analysis Approach to Capture Regulatory Decision-Making: Non-Small Cell Lung Cancer

Drug regulators around the world make decisions about drug approvability based on qualitative benefit-risk analyses. There is much interest in quantifying regulatory approaches to benefit and risk. In this work the use of a quantitative benefit-risk analysis was applied to regulatory decision-making about new drugs to treat advanced non-small cell lung cancer (NSCLC). Benefits and risks associated with 20 US Food and Drug Administration (FDA) decisions associated with a set of candidate treatments submitted between 2003 and 2015 were analyzed. For benefit analysis, the median overall survival (OS) was used where available. When not available, OS was estimated based on overall response rate (ORR) or progression-free survival (PFS). Risks were analyzed based on magnitude (or severity) of harm and likelihood of occurrence. Additionally, a sensitivity analysis was explored to demonstrate analysis of systematic uncertainty. FDA approval decision outcomes considered were found to be consistent with the benefit-risk logic.

Twenty-five years of confirmatory adaptive designs: opportunities and pitfalls

'Multistage testing with adaptive designs' was the title of an article by Peter Bauer that appeared 1989 in the German journal Biometrie und Informatik in Medizin und Biologie. The journal does not exist anymore but the methodology found widespread interest in the scientific community over the past 25 years. The use of such multistage adaptive designs raised many controversial discussions from the beginning on, especially after the publication by Bauer and Köhne 1994 in Biometrics: Broad enthusiasm about potential applications of such designs faced critical positions regarding their statistical efficiency. Despite, or possibly because of, this controversy, the methodology and its areas of applications grew steadily over the years, with significant contributions from statisticians working in academia, industry and agencies around the world. In the meantime, such type of adaptive designs have become the subject of two major regulatory guidance documents in the US and Europe and the field is still evolving. Developments are particularly noteworthy in the most important applications of adaptive designs, including sample size reassessment, treatment selection procedures, and population enrichment designs. In this article, we summarize the developments over the past 25 years from different perspectives. We provide a historical overview of the early days, review the key methodological concepts and summarize regulatory and industry perspectives on such designs. Then, we illustrate the application of adaptive designs with three case studies, including unblinded sample size reassessment, adaptive treatment selection, and adaptive endpoint selection. We also discuss the availability of software for evaluating and performing such designs. We conclude with a critical review of how expectations from the beginning were fulfilled, and - if not - discuss potential reasons why this did not happen.

Deficiencies in reporting of statistical methodology in recent randomized trials of nonpharmacologic pain treatments: ACTTION systematic review

OBJECTIVE

The goal of this study was to assess the quality of reporting of statistical methods in randomized clinical trials (RCTs), including identification of primary analyses, missing data accommodation, and multiplicity adjustment, in studies of nonpharmacologic, noninterventional pain treatments (e.g., physical therapy, cognitive behavioral therapy, acupuncture, and massage).

STUDY DESIGN

Systematic review of 101 articles reporting RCTs of pain treatments that were published between January 2006 and June 2013 in the European Journal of Pain, the Journal of Pain, and Pain.

SETTING

Systematic review.

RESULTS

Sixty-two percent of studies identified a primary outcome variable, 46% identified a primary analysis, and of those with multiple primary analyses, only 21% adjusted for multiplicity. Slightly over half (55%) of studies reported using at least one method to accommodate missing data. Only four studies reported prespecifying at least one of these four study methods.

CONCLUSION

This review identified deficiencies in the reporting of primary analyses and methods to adjust for multiplicity and accommodate missing data in articles disseminating results of nonpharmacologic, noninterventional trials. Investigators should be encouraged to indicate whether their analyses were prespecified and to clearly and completely report statistical methods in clinical trial publications to maximize the interpretability of trial results.

What can we do about exploratory analyses in clinical trials?

The research community has alternatively embraced then repudiated exploratory analyses since the inception of clinical trials in the middle of the twentieth century. After a series of important but ultimately unreproducible findings, these non-prospectively declared evaluations were relegated to hypothesis generating. Since the majority of evaluations conducted in clinical trials with their rich data sets are exploratory, the absence of their persuasive power adds to the inefficiency of clinical trial analyses in an atmosphere of fiscal frugality. However, the principle argument against exploratory analyses is not based in statistical theory, but pragmatism and observation. The absence of any theoretical treatment of exploratory analyses postpones the day when their statistical weaknesses might be repaired. Here, we introduce examination of the characteristics of exploratory analyses from a probabilistic and statistical framework. Setting the obvious logistical concerns aside (i.e., the absence of planning produces poor precision), exploratory analyses do not appear to suffer from estimation theory weaknesses. The problem appears to be a difficulty in what is actually reported as the p-value. The use of Bayes Theorem provides p-values that are more in line with confirmatory analyses. This development may inaugurate a body of work that would lead to the readmission of exploratory analyses to a position of persuasive power in clinical trials.

Clinically meaningful end points in the evolving treatment paradigm for radioactive iodine-refractory differentiated thyroid cancer

Radioactive iodine (RAI) is often used post-operatively for treatment of differentiated thyroid cancer (DTC), but many patients develop RAI-refractory disease. Patients with RAI-refractory DTC may be asymptomatic and stable for long periods of time, so identifying tumors that are no longer likely to respond to RAI treatment and determining when to transition to systemic therapy are critical issues for optimal patient care. The purpose of this paper is to review and assess the end points used in studies of RAI-refractory DTC in relation to the issues facing clinicians in transitioning patients to systemic therapy. Our goals are to provide a framework to help evaluate whether study results are clinically meaningful in guiding treatment decisions and to make recommendations to better define these end points for RAI-refractory DTC.

Adaptive graph-based multiple testing procedures

Multiple testing procedures defined by directed, weighted graphs have recently been proposed as an intuitive visual tool for constructing multiple testing strategies that reflect the often complex contextual relations between hypotheses in clinical trials. Many well-known sequentially rejective tests, such as (parallel) gatekeeping tests or hierarchical testing procedures are special cases of the graph based tests. We generalize these graph-based multiple testing procedures to adaptive trial designs with an interim analysis. These designs permit mid-trial design modifications based on unblinded interim data as well as external information, while providing strong family wise error rate control. To maintain the familywise error rate, it is not required to prespecify the adaption rule in detail. Because the adaptive test does not require knowledge of the multivariate distribution of test statistics, it is applicable in a wide range of scenarios including trials with multiple treatment comparisons, endpoints or subgroups, or combinations thereof. Examples of adaptations are dropping of treatment arms, selection of subpopulations, and sample size reassessment. If, in the interim analysis, it is decided to continue the trial as planned, the adaptive test reduces to the originally planned multiple testing procedure. Only if adaptations are actually implemented, an adjusted test needs to be applied. The procedure is illustrated with a case study and its operating characteristics are investigated by simulations.

Data interpretation in analgesic clinical trials with statistically nonsignificant primary analyses: an ACTTION systematic review

Peer-reviewed publications of randomized clinical trials (RCTs) are the primary means of disseminating research findings. "Spin" in RCT publications is misrepresentation of statistically nonsignificant research findings to suggest treatment benefit. Spin can influence the way readers interpret clinical trials and use the information to make decisions about treatments and medical policies. The objective of this study was to determine the frequency with which 4 types of spin were used in publications of analgesic RCTs with nonsignificant primary analyses in 6 major pain journals. In the 76 articles included in our sample, 28% of the abstracts and 29% of the main texts emphasized secondary analyses with P values <.05; 22% of abstracts and 29% of texts emphasized treatment benefit based on nonsignificant primary results; 14% of abstracts and 18% of texts emphasized within-group improvements over time, rather than primary between-group comparisons; and 13% of abstracts and 10% of texts interpreted a nonsignificant difference between groups in a superiority study as comparable effectiveness. When considering the article conclusion sections, 21% did not mention the nonsignificant primary result, 22% were presented with no uncertainty or qualification, 30% did not acknowledge that future research was required, and 8% recommended the intervention for clinical use.

PERSPECTIVE

This article identifies relatively frequent "spin" in analgesic RCTs. These findings highlight a need for authors, reviewers, and editors to be more cognizant of how analgesic RCT results are presented and attempt to minimize spin in future clinical trial publications.

An Audit of the Statistical Validity of Conclusions of Clinical Superiority in Anaesthesia Journals

Making a statistically valid conclusion of the superiority of a clinical intervention in a clinical trial requires not only a statistically significant P value, but also adequate a priori power and an observed effect size larger than the clinically important value specified in the sample size calculation. We scrutinised the five most highly cited clinical trials reporting one or more conclusions of clinical superiority published in Anesthesiology, the British Journal of Anaesthesia, Anaesthesia, Anesthesia and Analgesia and Anaesthesia and Intensive Care in 2011 or 2012 to determine how many met all three requisite criteria. In the 25 articles, there were a total of 36 unconditional conclusions of the superiority of a clinical intervention. All were supported by a statistically significant P value. However, only 15 (41.6%) met all three requisite statistical criteria to support clinical superiority. The remainder included secondary outcomes without specific reference to their observational nature, and primary outcomes whose observed effect size was smaller than the clinically important value specified in the sample size calculation. These findings indicate that clinicians should closely scrutinise conclusions of clinical superiority in anaesthesia journals. Many will be ‘hypothesis-generating observations’ without adequate statistical support for a conclusion of clinical superiority in their own right.

Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations

Performing multiple analyses in clinical trials can inflate the probability of a type I error, or the chance of falsely concluding a significant effect of the treatment. Strategies to minimize type I error probability include prespecification of primary analyses and statistical adjustment for multiple comparisons, when applicable. The objective of this study was to assess the quality of primary analysis reporting and frequency of multiplicity adjustment in 3 major pain journals (ie, European Journal of Pain, Journal of Pain, and PAIN®). A total of 161 randomized controlled trials investigating noninvasive pharmacological treatments or interventional treatments for pain, published between 2006 and 2012, were included. Only 52% of trials identified a primary analysis, and only 10% of trials reported prespecification of that analysis. Among the 33 articles that identified a primary analysis with multiple testing, 15 (45%) adjusted for multiplicity; of those 15, only 2 (13%) reported prespecification of the adjustment methodology. Trials in clinical pain conditions and industry-sponsored trials identified a primary analysis more often than trials in experimental pain models and non-industry-sponsored trials, respectively. The results of this systematic review demonstrate deficiencies in the reporting and possibly the execution of primary analyses in published analgesic trials. These deficiencies can be rectified by changes in, or better enforcement of, journal policies pertaining to requirements for the reporting of analyses of clinical trial data.

Challenges in design and interpretation of chronic pain trials

The process of systematic review has shone a light on the methodology of randomized controlled trials. Notably, a range of potential biases hinders the interpretation of chronic pain trials. These include a consistent bias favouring active over placebo in trials that are small and of short duration. The use of the 'last observation carried forward' imputation method is known to inflate results, often generating statistically significance when adverse event withdrawals are high; in clinical practice terms, this is the wrong answer. Patients want outcomes of low pain scores, large reductions in pain and relief from associated symptoms, with improvements in ability to function and in quality of life. Some patients achieve this, but many do not. The distribution of benefit is skewed and the use of average pain scores, or change in pain, can be misleading compared with responder analysis in which withdrawal is regarded as non-response. Historically, chronic pain trials have had a simple classic or a crossover design. They have been small and short, and used inappropriate imputation and outcomes unconnected to the experiences of most patients. While these designs are useful for answering some questions, they may be insensitive for many interventions. Newer designs, like enriched enrolment randomized withdrawal (EERW) trials or clinical effectiveness trials, are potentially more interesting and informative.

Joint hypothesis testing and gatekeeping procedures for studies with multiple endpoints

A claim of superiority of one intervention over another often depends naturally on results from several outcomes of interest. For such studies the common practice of making conclusions about individual outcomes in isolation can be problematic. For example, an intervention might be shown to improve one outcome (e.g., pain score) but worsen another (e.g., opioid consumption), making interpretation difficult. We thus advocate joint hypothesis testing, in which the decision rule used to claim success of an intervention over its comparator with regard to the multiple outcomes are specified a priori, and the overall type I error is protected. Success might be claimed only if there is a significant improvement detected in all primary outcomes, or alternatively, in at least one of them. We focus more specifically on demonstrating superiority on at least one outcome and noninferiority (i.e., not worse) on the rest. We also advocate the more general "gatekeeping" procedures (both serial and parallel), in which primary and secondary hypotheses of interest are a priori organized into ordered sets, and testing does not proceed to the next set, i.e., through the "gate," unless the significance criteria for the previous sets are satisfied, thus protecting the overall type I error. We demonstrate methods using data from a randomized controlled trial assessing the effects of transdermal nicotine on pain and opioids after pelvic gynecological surgery. Joint hypothesis testing and gatekeeping procedures are shown to substantially improve the efficiency and interpretation of randomized and nonrandomized studies having multiple outcomes of interest.

Three methods for constructing parallel gatekeeping procedures in clinical trials

This paper gives a review of three classes of parallel gatekeeping procedures that can be used in clinical trials with multiple objectives grouped into two or more families. We begin with a high-level summary of three methods for building parallel gatekeeping procedures proposed in the literature and provide a detailed comparison of the three methods. The comparison is based on analytical arguments as well as simulation studies and helps us develop general recommendations on the use of these methods in clinical trial applications. The methods discussed in this paper are illustrated using clinical trial examples with two families of objectives.

Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests

The confirmatory analysis of pre-specified multiple hypotheses has become common in pivotal clinical trials. In the recent past multiple test procedures have been developed that reflect the relative importance of different study objectives, such as fixed sequence, fallback, and gatekeeping procedures. In addition, graphical approaches have been proposed that facilitate the visualization and communication of Bonferroni-based closed test procedures for common multiple test problems, such as comparing several treatments with a control, assessing the benefit of a new drug for more than one endpoint, combined non-inferiority and superiority testing, or testing a treatment at different dose levels in an overall and a subpopulation. In this paper, we focus on extended graphical approaches by dissociating the underlying weighting strategy from the employed test procedure. This allows one to first derive suitable weighting strategies that reflect the given study objectives and subsequently apply appropriate test procedures, such as weighted Bonferroni tests, weighted parametric tests accounting for the correlation between the test statistics, or weighted Simes tests. We illustrate the extended graphical approaches with several examples. In addition, we describe briefly the gMCP package in R, which implements some of the methods described in this paper.

Testing a primary and a secondary endpoint in a group sequential design

We consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The familywise error rate (FWER) of falsely concluding significance on either endpoint is to be controlled at a nominal level α. The type I error rate for the primary endpoint is controlled by choosing any α-level stopping boundary, e.g., the standard O'Brien-Fleming or the Pocock boundary. Given any particular α-level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient ρ between the two endpoints equals 1. For the four combinations consisting of O'Brien-Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of ρ. An ad hoc boundary is proposed for the secondary endpoint to address a practical concern that may be at issue in some applications. Numerical studies indicate that the O'Brien-Fleming boundary for the primary endpoint and the Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance. The Pocock boundary may be replaced by the ad hoc boundary for the secondary endpoint with a very little loss of secondary power if the practical concern is at issue. A clinical trial example is given to illustrate the methods.

Understanding the FDA guidance on adaptive designs: historical, legal, and statistical perspectives

The recent Food and Drug Administration (FDA) guidance for industry on adaptive designs is perhaps one of the important undertakings by CDER/CBER Office of Biostatistics. Undoubtedly, adaptive designs may affect almost all phases of clinical development and impact nearly all aspects of clinical trial planning, execution and statistical inference. Thus, it is a significant accomplishment for the Office of Biostatistics to develop this well-thought-out and all-encompassing guidance document. In this paper, we discuss some critical topical issues of adaptive designs with supporting methodological work from either existing literature, additional technical notes, or accompanying papers. In particular, we provide numerous sources of design, conduct, analysis, and interpretation bias that arise from statistical procedures. We illustrate, as a result, and caution that substantial research is necessary for many adaptive designs to meet required scientific standards prior to their applications in clinical trials.

Randomized controlled trial of 2.5-cm versus 3.5-cm mesial temporal resection--Part 2: volumetric resection extent and subgroup analyses

BACKGROUND

This paper is addressing outcome differences in interesting subgroups from a previous randomized controlled trial of the extent of mesial temporal lobe resection (TLR) for drug-resistant epilepsy, by looking at effects of randomization, intended resection group, center, and true resection extent on seizure outcome.

METHODS

One hundred and seventy-nine cases with volumetrically assessed resection extent were used. Analyses of the extent of resection and subgroups and within subgroups for the two treatment arms will be performed, looking for confounding factors and using statistical methods (chi-square test, logistic regression analysis, and two-factorial ANOVA).

RESULTS

True resection extent varied considerably. Outcome comparison for right versus left resections, subgroups with mesial temporal sclerosis (MTS), or largest and smallest resections revealed no remarkable difference, compared to overall class I outcome. The intent-to-treat analyses within these subgroups revealed differences for class I outcome, albeit lacking in significance, except for better TLR outcome. Small true resection volume differences or randomization into the two resection groups could not explain the outcome differences between the selective amygdalohippocampectomy (SAH) and TLR subgroups. Logistic regression analysis showed an interaction between intended resection length and surgery type, confirming the impression of different impacts of the intended resection length under the two surgery types. The outcome difference between SAH and TLR was more likely explained by a center effect. In a two-factorial ANOVA for resected hippocampal volume, Engel outcome class I, and resection type, the outcome was not found to be correlated with true resection volume. A multifactorial logistic regression showed a mild interaction between the resection type with center on the Engel outcome class, extent of resection, and surgery type interacted, as did the extent of resection and center.

CONCLUSION

Patients with quite similar extent of resection can be seizure free or non-seizure free. In this cohort, seizure freedom rates fell again when the extent of mesial resection was maximized. Differences in class I outcome for SAH and TLR were not due to erroneous randomization, true resection extent, or presence of MTS, but were influenced by a center effect. Subgroup analyses did not help to provide arguments to favor one surgery type over the other.

A consistency-adjusted alpha-adaptive strategy for sequential testing

In a clinical trial with two clinically important endpoints, each of which can fully characterize a treatment benefit to support an efficacy claim by itself, a minimum degree of consistency in the findings is expected; otherwise interpretation of study findings can be problematic. Clinical trial literature contains examples where lack of consistency in the findings of clinically relevant endpoints led to difficulties in interpreting study results. The aim of this paper is to introduce this consistency concept at the study design stage and investigate the consequences of its implementation in the statistical analysis plan. The proposed methodology allows testing of hierarchically ordered endpoints to proceed as long as a pre-specified consistency criterion is met. In addition, while an initial allocation of the alpha level is specified for the ordered endpoints at the design stage, the methodology allows the alpha level allocated to the second endpoint to be adaptive to the findings of the first endpoint. In addition, the methodology takes into account the correlation between the endpoints in calculating the significance level and the power of the test for the next endpoint. The proposed Consistency-Adjusted Alpha-Adaptive Strategy (CAAAS) is very general. Several of the well-known multiplicity adjustment approaches arise as special cases of this strategy by appropriate selection of the consistency level and the form of alpha-adaptation function. We discuss control of the Type I error rate as well as power of the proposed methodology and consider its application to clinical trial data.

A nutrient approach to prostate cancer prevention: The Selenium and Vitamin E Cancer Prevention Trial (SELECT)

The Selenium and Vitamin E Cancer Prevention Trial (SELECT) randomized 35,533 healthy men, >55 yr old (>50 yr if African American), with normal digital rectal exams and prostate specific antigens <4 ng/ml to 1) 200 μg/day l-selenomethionine, 2) 400 IU/day all-rac-alpha-tocopheryl acetate (vitamin E), 3) both supplements, or 4) placebo for 7 to 12 yr. The hypotheses underlying SELECT, that selenium and vitamin E individually and together decrease prostate cancer incidence, derived from epidemiologic and laboratory evidence and significant secondary endpoints in the Nutritional Prevention of Cancer (selenium) and Alpha-Tocopherol Beta-Carotene (vitamin E) trials. In SELECT, prostate cancer incidence did not differ among the 4 arms: hazard ratios [99% confidence intervals (CIs)] for prostate cancer were 1.13 (99% CI = 0.95-1.35, P = 0.06; n = 473) for vitamin E, 1.04 (99% CI = 0.87-1.24, P = 0.62; n = 432) for selenium, and 1.05 (99% CI = 0.88-1.25, P = 0.52; n = 437) for selenium + vitamin E vs. 1.00 (n = 416) for placebo. Statistically nonsignificant increased risks of prostate cancer with vitamin E alone [relative risk (RR) = 1.13, P = 0.06) and newly diagnosed Type 2 diabetes mellitus with selenium alone (RR = 1.07, P = 0.16) were observed. SELECT data show that neither selenium nor vitamin E, alone or together, in the doses and formulations used, prevented prostate cancer in this heterogeneous population of healthy men.

Endpoints in paediatric oncology

PURPOSE

The purpose of this review is to address the issue of endpoints in paediatric oncology. Oncologists use this term to refer to an outcome they are trying to measure with a clinical trial, which may become accordingly the object of scientific articles. The outcome measured may concern both efficacy and safety, although from different perspectives.

METHODS

Based on both literature and experience developed in clinical trials, the different types of endpoints have been critically analysed in their power to provide the highest information of therapeutic interest (efficacy and safety) with the least risk and discomfort for the individual. Primary, secondary and surrogate endpoints have been distinguished. The most relevant differences have been discussed in comparison with adult oncology settings of endpoints.

RESULTS

The rarity of cancer in childhood and adolescence and the objective difficulty of enrolling statistically conceivable numbers of individuals have determined the utmost positive development of large scale, multinational clinical trials. The most interesting consequence is that the impact of multiplicity interferences, which is usually present in virtually all clinical trials developed for adults with cancer, is not a common event in paediatric oncology. Nevertheless, many of the questions concerning the different impact on outcome and survival of clinical trials developed in adult oncology remain unanswered due to the objective limitations still existing in terms of cure compared with paediatric oncology. The powerful consistency of cure rate, as the most relevant endpoint of clinical trials developed in paediatric oncology, addresses additional considerations to support the relevant differences existing between adult and paediatric oncology: both the development of clinical trials with different aims (confirmatory versus primary response) and the limited impact of multiplicity limitations may determine different implications regarding the meaning of endpoints in paediatric and adult oncology.

CONCLUSION

The aim of cancer treatment is to improve survival (SUR) and quality of life (QoL), but some restraints on the conduct of clinical trials may make these goals unattainable. Clinical trial endpoints represent a measure method aimed to grant answers to questions addressed by the clinical trial itself. The effect of the new regulation is expected to stimulate high-quality research and provide robust information on paediatric drugs to increase the availability of such drugs to children.