When to Stop a Clinical Trial Early for Benefit: Lessons Learned and Future Approaches

Lost in Translation: Why Are Rates of Hypertension Control Getting Worse Over Time?

Treatment of hypertension to decrease rates of cardiovascular disease is the most well studied and most broadly applicable treatment in cardiovascular prevention. Blood pressure can be measured anywhere, not just in a physician's office; medications are readily available, inexpensive, and have highly favorable benefit/harm ratios with relatively minimal side effects; and stepped medication regimens can be prescribed in algorithmic fashion by a variety of practitioners. Yet overall hypertension control rates in the United States have never exceeded 60%, and the last 5-10 years have seen decreased, rather than increased, rates of control. Here, I describe the scale of this massive failure to deliver on the promise of preventive hypertension care; outline the populations most affected and the contemporaneous events that have impacted hypertension control; discuss the disparate paths of hypertension science and health care delivery; and highlight novel interventions, approaches, and future opportunities to bend the curve back toward improvements in hypertension control.

A systematic review of randomised controlled trials with adaptive and traditional group sequential designs - applications in cardiovascular clinical trials

BACKGROUND

Trial design plays a key role in clinical trials. Traditional group sequential design has been used in cardiovascular clinical trials over decades as the trials can potentially be stopped early, therefore, it can reduce pre-planned sample size and trial resources. In contrast, trials with adoptive designs provide greater flexibility and are more efficient due to the ability to modify trial design according to the interim analysis results. In this systematic review, we aim to explore characteristics of adaptive and traditional group sequential trials in practice and to gain an understanding how these trial designs are currently being reported in cardiology.

METHODS

PubMed, Embase and Cochrane Central Register of Controlled Trials database were searched from January 1980 to June 2022. Randomised controlled phase 2/3 trials with either adaptive or traditional group sequential design in patients with cardiovascular disease were included. Descriptive statistics were used to present the collected data.

RESULTS

Of 456 articles found in the initial search, 56 were identified including 43 (76.8%) trials with traditional group sequential design and 13 (23.2%) with adaptive. Most trials were large, multicentre, led by the USA (50%) and Europe (28.6%), and were funded by companies (78.6%). For trials with group sequential design, frequency of interim analyses was determined mainly by the number of events (47%). 67% of the trials stopped early, in which 14 (32.6%) were due to efficacy, and 5 (11.6%) for futility. The commonly used stopping rule to terminate trials was O'Brien- Fleming-type alpha spending function (10 (23.3%)). For trials with adaptive designs, 54% of the trials stopped early, in which 4 (30.8%) were due to futility, and 2 (15.4%) for efficacy. Sample size re-estimation was commonly used (8 (61.5%)). In 69% of the trials, simulation including Bayesian approach was used to define the statistical stopping rules. The adaptive designs have been increasingly used (from 0 to 1999 to 38.6% after 2015 amongst adaptive trials). 25% of the trials reported "adaptive" in abstract or title of the studies.

CONCLUSIONS

The application of adaptive trials is increasingly popular in cardiovascular clinical trials. The reporting of adaptive design needs improving.

Randomised trials at the level of the individual

In global health research, short-term, small-scale clinical trials with fixed, two-arm trial designs that generally do not allow for major changes throughout the trial are the most common study design. Building on the introductory paper of this Series, this paper discusses data-driven approaches to clinical trial research across several adaptive trial designs, as well as the master protocol framework that can help to harmonise clinical trial research efforts in global health research. We provide a general framework for more efficient trial research, and we discuss the importance of considering different study designs in the planning stage with statistical simulations. We conclude this second Series paper by discussing the methodological and operational complexity of adaptive trial designs and master protocols and the current funding challenges that could limit uptake of these approaches in global health research.

Efficacy and safety of cilostazol-nimodipine combined therapy on delayed cerebral ischaemia after aneurysmal subarachnoid haemorrhage: a prospective, randomised, double-blinded, placebo-controlled trial protocol

INTRODUCTION

Delayed cerebral ischaemia (DCI) due to cerebral vasospasm (cVS) remains the foremost contributor to morbidity and mortality following aneurysmal subarachnoid haemorrhage (aSAH). Past efforts in preventing and treating DCI have failed to make any significant progress. To date, our most effective treatment involves the use of nimodipine, a calcium channel blocker. Recent studies have suggested that cilostazol, a platelet aggregation inhibitor, may prevent cVS. Thus far, no study has evaluated the effect of cilostazol plus nimodipine on the rate of DCI following aSAH.

METHODS AND ANALYSIS

This is a multicentre, double-blinded, randomised, placebo-controlled superiority trial investigating the effect of cilostazol on DCI. Data concerning rates of DCI, symptomatic and radiographic vasospasm, length of intensive care unit stay, and long-term functional and quality-of-life (QoL) outcomes will be recorded. All data will be collected with the aim of demonstrating that the use of cilostazol plus nimodipine will safely decrease the incidence of DCI, and decrease the rates of both radiographic and symptomatic vasospasm with subsequent improvement in long-term functional and QoL outcomes when compared with nimodipine alone.

ETHICS AND DISSEMINATION

Ethical approval was obtained from all participating hospitals by the Ascension Providence Hospital Institutional Review Board. The results of this study will be submitted for publication in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

NCT04148105.

Early termination of cardiovascular trials as a consequence of poor accrual: analysis of ClinicalTrials.gov 2006-2015

OBJECTIVES

To present a snapshot of experimental cardiovascular research with a focus on geographical and temporal patterns of early termination due to poor accrual.

SETTING

The Aggregate Analysis of ClinicalTrials.gov (AACT) database, reflecting ClinicalTrials.gov as of 27 March 2016.

DESIGN

The AACT database was searched for all cardiovascular clinical trials that started from January 2006 up to December 2015.

RESULTS

Thirteen thousand and seven hundred twenty-nine cardiovascular trials were identified. Of these, 8900 (65%) were classified as closed studies. Globally, 11% of closed trials were terminated. This proportion varied from 9.6% to 14% for trials recruiting from Europe and Americas, respectively, with a slightly decreasing trend (p=0.02) over the study period. The most common reason for trials failing to complete was poor accrual (41%). Intercontinental trials exhibited lower figures of poor accrual as the reason for their early stopping, as compared with trials recruiting in a single continent (28% vs 44%, p=0.002).

CONCLUSIONS

Poor accrual significantly challenges the successful completion of cardiovascular clinical trials. Findings are suggestive of a positive effect of globalisation of cardiovascular clinical research on the achievement of enrolment goals within a reasonable time frame.

Independent academic Data Monitoring Committees for clinical trials in cardiovascular and cardiometabolic diseases

Data Monitoring Committees (DMCs) play a crucial role in the conducting of clinical trials to ensure the safety of study participants and to maintain a trial's scientific integrity. Generally accepted standards exist for DMC composition and operational conduct. However, some relevant issues are not specifically addressed in current guidance documents, resulting in uncertainties regarding optimal approaches for communication between the DMC, steering committee, and sponsors, release of information, and liability protection for DMC members. The Heart Failure Association (HFA) of the European Society of Cardiology (ESC), in collaboration with the Clinical Trials Unit of the European Heart Agency (EHA) of the ESC convened a meeting of international experts in DMCs for cardiovascular and cardiometabolic clinical trials to identify specific issues and develop steps to resolve challenges faced by DMCs.The main recommendations from the meeting relate to methodological consistency, independence, managing conflicts of interest, liability protection, and training of future DMC members. This paper summarizes the key outcomes from this expert meeting, and describes the core set of activities that might be further developed and ultimately implemented by the ESC, HFA, and other interested ESC constituent bodies. The HFA will continue to work with stakeholders in cardiovascular and cardiometabolic clinical research to promote these goals.

Randomized trials addressing a similar question are commonly published after a trial stopped early for benefit

OBJECTIVE

We explored how investigators of ongoing or planned trials respond to the publication of a trial stopped early for benefit addressing a similar question.

STUDY DESIGN AND SETTING

We searched multiple databases from the date of publication of the truncated trial through August, 2015. Independent reviewers selected trials and extracted data.

RESULTS

We identified 207 trials truncated for early benefit; of which 102 (49%) were followed by subsequent trials (262 subsequent trials, median 2 per truncated trial, range 1-13). Only 99 (38%) provided a rationale justifying conducting a trial despite prior stopping. The top reasons were to address different population or setting (33%), skepticism of truncated trials findings because of small sample size (12%), inconsistency with other evidence (11%), or increased risk of bias (7%). We did not identify significant associations between subsequent trials and characteristics of truncated ones (risk of bias, precision, funding, or rigor of stopping decision).

CONCLUSION

About half of the trials stopped early for benefit were followed by subsequent trials addressing a similar question. This suggests that future trialists may have been skeptic about the decision to stop prior trials. A more rigorous threshold for stopping early for benefit is needed.

The Impact of Clinical Trials Conducted by Research Networks in Pediatric Critical Care

OBJECTIVES

Research networks in adult and neonatal critical care have demonstrated collaborative and successful execution of clinical trials. Such networks appear to have been relatively recently established in the field of pediatric critical care. The objective of this study was to evaluate the productivity and impact of randomized controlled trials conducted by pediatric critical care research networks, compared with nonnetwork trials.

DATA SOURCES, STUDY SELECTION, AND DATA ABSTRACTION

We searched multiple online databases including MEDLINE, reference lists of randomized controlled trials, and relevant systematic reviews. Independent pairs of reviewers identified published randomized controlled trials administering any intervention to children in a PICU and abstracted data. A research network was defined as a formal consortium or collaborative research group established for the purpose of conducting clinical research. Data were independently abstracted in duplicate.

MAIN RESULTS

There were 288 pediatric critical care randomized controlled trials published in English between 1986 and July 2015, of which 15 randomized controlled trials (5.2%) were conducted by a total of five research networks. Network randomized controlled trials were more often multicentered, multinational, and larger in size (p < 0.001), compared with nonnetwork randomized controlled trials. Accordingly, their trials took longer to complete (median, 36 vs 21 mo; p < 0.001). Early stopping occurred in 46.7% of network randomized controlled trials (46.7%) and 27% of nonnetwork randomized controlled trials (p = 0.14), most commonly for futility. None of the network, but 45% of the nonnetwork trials found a significant difference in their primary outcome (p < 0.001). Network trials were more frequently cited (median, 6 vs 2 citations per year) and published in higher impact journals (median impact factor, 21.8 vs 3; p < 0.001).

CONCLUSIONS

Research networks have conducted a minority of randomized controlled trials in pediatric critical care. They infrequently demonstrate significant differences in their primary outcomes. Despite this, network trials are cited more frequently and appear to have greater impact. There are important lessons to learn from both individual researchers as well as research networks that may guide the successful conduct of collaborative, high-quality randomized controlled trials in critically ill children.

Advantages of Bayesian monitoring methods in deciding whether and when to stop a clinical trial: an example of a neonatal cooling trial

BACKGROUND

Decisions to stop randomized trials are often based on traditional P value thresholds and are often unconvincing to clinicians. To familiarize clinical investigators with the application and advantages of Bayesian monitoring methods, we illustrate the steps of Bayesian interim analysis using a recent major trial that was stopped based on frequentist analysis of safety and futility.

METHODS

We conducted Bayesian reanalysis of a factorial trial in newborn infants with hypoxic-ischemic encephalopathy that was designed to investigate whether outcomes would be improved by deeper (32 °C) or longer cooling (120 h), as compared with those achieved by standard whole body cooling (33.5 °C for 72 h). Using prior trial data, we developed neutral and enthusiastic prior probabilities for the effect on predischarge mortality, defined stopping guidelines for a clinically meaningful effect, and derived posterior probabilities for predischarge mortality.

RESULTS

Bayesian relative risk estimates for predischarge mortality were closer to 1.0 than were frequentist estimates. Posterior probabilities suggested increased predischarge mortality (relative risk > 1.0) for the three intervention groups; two crossed the Bayesian futility threshold.

CONCLUSIONS

Bayesian analysis incorporating previous trial results and different pre-existing opinions can help interpret accruing data and facilitate informed stopping decisions that are likely to be meaningful and convincing to clinicians, meta-analysts, and guideline developers.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01192776 . Registered on 31 August 2010.

Stopping guidelines for an effectiveness trial: what should the protocol specify?

Background

Despite long-standing problems in decisions to stop clinical trials, stopping guidelines are often vague or unspecified in the trial protocol. Clear, well-conceived guidelines are especially important to assist the data monitoring committees for effectiveness trials.

Main text

To specify better stopping guidelines in the protocol for such trials, the clinical investigators and trial statistician should carefully consider the following kinds of questions:

How should the relative importance of the treatment benefits and hazards be assessed?

For decisions to stop a trial for benefit:

What would be the minimum clinically important difference for the study population?

How should the probability that the benefit exceeds that difference be assessed?

When should the interim analyses include data from other trials?

Would the evidence meet state-of-the-art standards for treatment recommendations and practice guidelines?

Should less evidence be required to stop the trial for harm than for benefit?

When should conventional stopping guidelines for futility be used for comparative effectiveness trials?

Conclusion

Both clinical and statistical expertise are required to address such challenging questions for effectiveness trials. Their joint consideration by clinical investigators and statisticians is needed to define better stopping guidelines before starting the trial.

An Investigation of the Shortcomings of the CONSORT 2010 Statement for the Reporting of Group Sequential Randomised Controlled Trials: A Methodological Systematic Review

BACKGROUND

It can be argued that adaptive designs are underused in clinical research. We have explored concerns related to inadequate reporting of such trials, which may influence their uptake. Through a careful examination of the literature, we evaluated the standards of reporting of group sequential (GS) randomised controlled trials, one form of a confirmatory adaptive design.

METHODS

We undertook a systematic review, by searching Ovid MEDLINE from the 1st January 2001 to 23rd September 2014, supplemented with trials from an audit study. We included parallel group, confirmatory, GS trials that were prospectively designed using a Frequentist approach. Eligible trials were examined for compliance in their reporting against the CONSORT 2010 checklist. In addition, as part of our evaluation, we developed a supplementary checklist to explicitly capture group sequential specific reporting aspects, and investigated how these are currently being reported.

RESULTS

Of the 284 screened trials, 68(24%) were eligible. Most trials were published in "high impact" peer-reviewed journals. Examination of trials established that 46(68%) were stopped early, predominantly either for futility or efficacy. Suboptimal reporting compliance was found in general items relating to: access to full trials protocols; methods to generate randomisation list(s); details of randomisation concealment, and its implementation. Benchmarking against the supplementary checklist, GS aspects were largely inadequately reported. Only 3(7%) trials which stopped early reported use of statistical bias correction. Moreover, 52(76%) trials failed to disclose methods used to minimise the risk of operational bias, due to the knowledge or leakage of interim results. Occurrence of changes to trial methods and outcomes could not be determined in most trials, due to inaccessible protocols and amendments.

DISCUSSION AND CONCLUSIONS

There are issues with the reporting of GS trials, particularly those specific to the conduct of interim analyses. Suboptimal reporting of bias correction methods could potentially imply most GS trials stopping early are giving biased results of treatment effects. As a result, research consumers may question credibility of findings to change practice when trials are stopped early. These issues could be alleviated through a CONSORT extension. Assurance of scientific rigour through transparent adequate reporting is paramount to the credibility of findings from adaptive trials. Our systematic literature search was restricted to one database due to resource constraints.

Initiation and continuation of randomized trials after the publication of a trial stopped early for benefit asking the same study question: STOPIT-3 study design

BACKGROUND

Randomized control trials (RCTs) stopped early for benefit (truncated RCTs) are increasingly common and, on average, overestimate the relative magnitude of benefit by approximately 30%. Investigators stop trials early when they consider it is no longer ethical to enroll patients in a control group. The goal of this systematic review is to determine how investigators of ongoing or planned RCTs respond to the publication of a truncated RCT addressing a similar question.

METHODS/DESIGN

We will conduct systematic reviews to update the searches of 210 truncated RCTs to identify similar trials ongoing at the time of publication, or started subsequently, to the truncated trials ('subsequent RCTs'). Reviewers will determine in duplicate the similarity between the truncated and subsequent trials. We will analyze the epidemiology, distribution, and predictors of subsequent RCTs. We will also contact authors of subsequent trials to determine reasons for beginning, continuing, or prematurely discontinuing their own trials, and the extent to which they rely on the estimates from truncated trials.

DISCUSSION

To the extent that investigators begin or continue subsequent trials they implicitly disagree with the decision to stop the truncated RCT because of an ethical mandate to administer the experimental treatment. The results of this study will help guide future decisions about when to stop RCTs early for benefit.