Implementing and assessing Bayesian response-adaptive randomisation for backfilling in dose-finding trials

Traditional approaches in dose-finding trials, such as the continual reassessment method, focus on identifying the maximum tolerated dose. In contemporary early-phase dose-finding trials, especially in oncology with targeted agents or immunotherapy, a more relevant aim is to identify the lowest dose level that maximises efficacy whilst remaining tolerable. Backfilling, defined as the practice of assigning patients to dose levels lower than the current highest tolerated dose, has been proposed to gather additional pharmacokinetic, pharmacodynamic and biomarker data to recommend the most appropriate dose to carry forward for subsequent studies. The first formal framework [5] for backfilling proposed randomising backfill patients with equal probability among those doses below the dose level where the study is currently at. Here, we propose to use Bayesian response-adaptive randomisation to backfill patients. This patient-oriented approach to backfilling aims to allocate more patients to dose levels in the backfill set with higher expected efficacy based on emerging data. The backfill set constitutes of the doses below the dose the dose-finding algorithm is at. At study completion, collective patient data inform the dose-response curve, suggesting an optimal dose level balancing toxicity and efficacy. Our simulation study across diverse clinical trial settings demonstrates that a backfilling strategy using Bayesian response-adaptive randomisation can result in a patient-oriented patient assignment procedure which simultaneously enhances the likelihood of correctly identifying the most appropriate dose level. This contribution offers a methodological framework and practical implementation for patient-oriented backfilling, encompassing design and analysis considerations in early-phase trials.

Evaluating pooled testing for asymptomatic screening of healthcare workers in hospitals

BACKGROUND

There is evidence that during the COVID pandemic, a number of patient and HCW infections were nosocomial. Various measures were put in place to try to reduce these infections including developing asymptomatic PCR (polymerase chain reaction) testing schemes for healthcare workers. Regularly testing all healthcare workers requires many tests while reducing this number by only testing some healthcare workers can result in undetected cases. An efficient way to test as many individuals as possible with a limited testing capacity is to consider pooling multiple samples to be analysed with a single test (known as pooled testing).

METHODS

Two different pooled testing schemes for the asymptomatic testing are evaluated using an individual-based model representing the transmission of SARS-CoV-2 in a 'typical' English hospital. We adapt the modelling to reflect two scenarios: a) a retrospective look at earlier SARS-CoV-2 variants under lockdown or social restrictions, and b) transitioning back to 'normal life' without lockdown and with the omicron variant. The two pooled testing schemes analysed differ in the population that is eligible for testing. In the 'ward' testing scheme only healthcare workers who work on a single ward are eligible and in the 'full' testing scheme all healthcare workers are eligible including those that move across wards. Both pooled schemes are compared against the baseline scheme which tests only symptomatic healthcare workers.

RESULTS

Including a pooled asymptomatic testing scheme is found to have a modest (albeit statistically significant) effect, reducing the total number of nosocomial healthcare worker infections by about 2[Formula: see text] in both the lockdown and non-lockdown setting. However, this reduction must be balanced with the increase in cost and healthcare worker isolations. Both ward and full testing reduce HCW infections similarly but the cost for ward testing is much less. We also consider the use of lateral flow devices (LFDs) for follow-up testing. Considering LFDs reduces cost and time but LFDs have a different error profile to PCR tests.

CONCLUSIONS

Whether a PCR-only or PCR and LFD ward testing scheme is chosen depends on the metrics of most interest to policy makers, the virus prevalence and whether there is a lockdown.

Medicines and Healthcare products Regulatory Agency's "Consultation on proposals for legislative changes for clinical trials": a response from the Trials Methodology Research Partnership Adaptive Designs Working Group, with a focus on data sharing

In the UK, the Medicines and Healthcare products Regulatory Agency consulted on proposals "to improve and strengthen the UK clinical trials legislation to help us make the UK the best place to research and develop safe and innovative medicines". The purpose of the consultation was to help finalise the proposals and contribute to the drafting of secondary legislation. We discussed these proposals as members of the Trials Methodology Research Partnership Adaptive Designs Working Group, which is jointly funded by the Medical Research Council and the National Institute for Health and Care Research. Two topics arose frequently in the discussion: the emphasis on legislation, and the absence of questions on data sharing. It is our opinion that the proposals rely heavily on legislation to change practice. However, clinical trials are heterogeneous, and as a result some trials will struggle to comply with all of the proposed legislation. Furthermore, adaptive design clinical trials are even more heterogeneous than their non-adaptive counterparts, and face more challenges. Consequently, it is possible that increased legislation could have a greater negative impact on adaptive designs than non-adaptive designs. Overall, we are sceptical that the introduction of legislation will achieve the desired outcomes, with some exceptions. Meanwhile the topic of data sharing - making anonymised individual-level clinical trial data available to other investigators for further use - is entirely absent from the proposals and the consultation in general. However, as an aspect of the wider concept of open science and reproducible research, data sharing is an increasingly important aspect of clinical trials. The benefits of data sharing include faster innovation, improved surveillance of drug safety and effectiveness and decreasing participant exposure to unnecessary risk. There are already a number of UK-focused documents that discuss and encourage data sharing, for example, the Concordat on Open Research Data and the Medical Research Council's Data Sharing Policy. We strongly suggest that data sharing should be the norm rather than the exception, and hope that the forthcoming proposals on clinical trials invite discussion on this important topic.

Response-adaptive randomization in clinical trials: from myths to practical considerations

Response-Adaptive Randomization (RAR) is part of a wider class of data-dependent sampling algorithms, for which clinical trials are typically used as a motivating application. In that context, patient allocation to treatments is determined by randomization probabilities that change based on the accrued response data in order to achieve experimental goals. RAR has received abundant theoretical attention from the biostatistical literature since the 1930's and has been the subject of numerous debates. In the last decade, it has received renewed consideration from the applied and methodological communities, driven by well-known practical examples and its widespread use in machine learning. Papers on the subject present different views on its usefulness, and these are not easy to reconcile. This work aims to address this gap by providing a unified, broad and fresh review of methodological and practical issues to consider when debating the use of RAR in clinical trials.

A novel statistical test for treatment differences in clinical trials using a response-adaptive forward-looking Gittins Index Rule

The most common objective for response-adaptive clinical trials is to seek to ensure that patients within a trial have a high chance of receiving the best treatment available by altering the chance of allocation on the basis of accumulating data. Approaches that yield good patient benefit properties suffer from low power from a frequentist perspective when testing for a treatment difference at the end of the study due to the high imbalance in treatment allocations. In this work we develop an alternative pairwise test for treatment difference on the basis of allocation probabilities of the covariate-adjusted response-adaptive randomization with forward-looking Gittins Index (CARA-FLGI) Rule for binary responses. The performance of the novel test is evaluated in simulations for two-armed studies and then its applications to multiarmed studies are illustrated. The proposed test has markedly improved power over the traditional Fisher exact test when this class of nonmyopic response adaptation is used. We also find that the test's power is close to the power of a Fisher exact test under equal randomization.

Effect of Hypoxemia on Outcome in Respiratory Failure Supported With Extracorporeal Membrane Oxygenation: A Cardinality Matched Cohort Study

Venovenous extracorporeal membrane oxygenation (ECMO) is recommended in adult patients with refractory acute respiratory failure (ARF), but there is limited evidence for its use in patients with less severe hypoxemia. Prior research has suggested a lower PaO 2 /FiO 2 at cannulation is associated with higher short-term mortality, but it is unclear whether this is due to less severe illness or a potential benefit of earlier ECMO support. In this exploratory cardinality-matched observational cohort study, we matched 668 patients who received venovenous ECMO as part of a national severe respiratory failure service into cohorts of "less severe" and "very severe" hypoxemia based on the median PaO 2 /FiO 2 at ECMO institution of 68 mmHg. Before matching, ICU mortality was 19% in the 'less severe' hypoxemia group and 28% in the "very severe" hypoxemia group (RR for mortality = 0.69, 95% CI 0.54-0.88). After matching on key prognostic variables including underlying diagnosis, this difference remained statistically present but smaller: (23% vs. 30%, RR = 0.76, 95% CI 0.59-0.99). This may suggest the observed survival benefit of venovenous ECMO is not solely due to reduced disease severity. Further research is warranted to examine the potential role of ECMO in ARF patients with less severe hypoxemia.

Practical guidance for planning resources required to support publicly-funded adaptive clinical trials

Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.

Response adaptive intervention allocation in stepped-wedge cluster randomized trials

BACKGROUND

Stepped-wedge cluster randomized trial (SW-CRT) designs are often used when there is a desire to provide an intervention to all enrolled clusters, because of a belief that it will be effective. However, given there should be equipoise at trial commencement, there has been discussion around whether a pre-trial decision to provide the intervention to all clusters is appropriate. In pharmaceutical drug development, a solution to a similar desire to provide more patients with an effective treatment is to use a response adaptive (RA) design.

METHODS

We introduce a way in which RA design could be incorporated in an SW-CRT, permitting modification of the intervention allocation during the trial. The proposed framework explicitly permits a balance to be sought between power and patient benefit considerations. A simulation study evaluates the methodology.

RESULTS

In one scenario, for one particular RA design, the proportion of cluster-periods spent in the intervention condition was observed to increase from 32.2% to 67.9% as the intervention effect was increased. A cost of this was a 6.2% power drop compared to a design that maximized power by fixing the proportion of time in the intervention condition at 45.0%, regardless of the intervention effect.

CONCLUSIONS

An RA approach may be most applicable to settings for which the intervention has substantial individual or societal benefit considerations, potentially in combination with notable safety concerns. In such a setting, the proposed methodology may routinely provide the desired adaptability of the roll-out speed, with only a small cost to the study's power.

Costs and staffing resource requirements for adaptive clinical trials: quantitative and qualitative results from the Costing Adaptive Trials project

BACKGROUND

Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials.

METHODS

We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences.

RESULTS

Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had.

CONCLUSIONS

This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Positioning imatinib for pulmonary arterial hypertension: A phase I/II design comprising dose finding and single-arm efficacy

Pulmonary arterial hypertension is an unmet clinical need. Imatinib, a tyrosine kinase inhibitor, 200 to 400 mg daily reduces pulmonary artery pressure and increases functional capacity in this patient group, but is generally poorly tolerated at the higher dose. We have designed an open-label, single-arm clinical study to investigate whether there is a tolerated dose of imatinib that can be better targeted to patients who will benefit. The study consists of two parts. Part 1 seeks to identify the best tolerated dose of Imatinib in the range from 100 and up to 400 mg using a Bayesian Continuous Reassessment Method. Part 2 will measure efficacy after 24 weeks treatment with the best tolerated dose using a Simon's two-stage design. The primary efficacy endpoint is a binary variable. For patients with a baseline pulmonary vascular resistance (PVR) >1000 dynes · s · cm-5, success is defined by an absolute reduction in PVR of ≥300 dynes · s · cm-5 at 24 weeks. For patients with a baseline PVR ≤1000 dynes · s · cm-5, success is a 30% reduction in PVR at 24 weeks. PVR will also be evaluated as a continuous variable by genotype as an exploratory analysis. Evaluating the response to that dose by genotype may inform a prospective biomarker-driven study.

Overcoming hurdles to intervention studies with autistic children with profound communication difficulties and their families

Autistic children and adults who are non-verbal/minimally verbal or have an intellectual disability have often been excluded from Autism Spectrum Disorder research. Historical, practical and theoretical reasons for this exclusion continue to deter some researchers from work with this underserved population. We discuss why these reasons are neither convincing nor ethical, and provide strategies for dealing with practical issues. As part of a randomised controlled trial of an intervention for children with profound autism, we reflected as a multi-disciplinary team on what we had learnt from these children, their families and each other. We provide 10 strategies to overcome what appeared initially to be barriers to collecting data with this population. These hurdles and our solutions are organised by theme: interacting physically with children, how to play and test, navigating difficult behaviours, selecting suitable outcome measures, relating with parents, managing siblings, involving stakeholders, timing interactions, the clinician’s role in managing expectations, and recruitment. The aim of this article is to provide researchers with the tools to feel motivated to conduct research with children with profound autism and their families, a difficult but worthwhile endeavour. Many of these lessons also apply to conducting research with non-autistic children with intellectual disabilities.

Outcomes of the NHS England National Extracorporeal Membrane Oxygenation Service for adults with respiratory failure: a multicentre observational cohort study

BACKGROUND

Extracorporeal membrane oxygenation (ECMO) is increasingly used to support adults with severe respiratory failure refractory to conventional measures. In 2011, NHS England commissioned a national service to provide ECMO to adults with refractory acute respiratory failure. Our aims were to characterise the patients admitted to the service, report their outcomes, and highlight characteristics potentially associated with survival.

METHODS

An observational cohort study was conducted of all patients treated by the NHS England commissioned ECMO service between December 1, 2011 and December 31, 2017. Analysis was conducted according to a prespecified protocol (NCT: 03979222). Data are presented as median [inter-quartile range, IQR].

RESULTS

A total of 1205 patients were supported with ECMO during the study period; the majority (n=1150; 95%) had veno-venous ECMO alone. The survival rate at ECMO ICU discharge was 74% (n=887). Survivors had a lower median age (43 yr [32-52]), compared with non-survivors (49 y [39-60]). Increased severity of hypoxaemia at time of decision-to-cannulate was associated with a lower probability of survival: survivors had a median Sao2 of 90% (84-93%; median Pao2/Fio2, 9.4 kPa [7.7-12.6]), compared with non-survivors (Sao2 88% [80-92%]; Pao2/Fio2 ratio: 8.5 kPa [7.1-11.5]). Patients requiring ECMO because of asthma were more likely to survive (95% survival rate (95% CI, 91-99%), compared with a survival of 71% (95% CI, 69-74%) in patients with respiratory failure attributable to other diagnoses.

CONCLUSION

A national ECMO service can achieve good short-term outcomes for patients with undifferentiated respiratory failure refractory to conventional management.

CLINICAL TRIAL REGISTRATION

NCT03979222.

Point OutWords: protocol for a feasibility randomised controlled trial of a motor skills intervention to promote communicative development in non-verbal children with autism

BACKGROUND

Point OutWords is a caregiver-delivered, iPad-assisted intervention for non-verbal or minimally verbal children with autism. It aims to develop prerequisite skills for communication such as manual and oral motor skills, sequencing, and symbolic representation. This feasibility trial aims to determine the viability of evaluating the clinical efficacy of Point OutWords.

METHODOLOGY

We aim to recruit 46 non-verbal or minimally verbal children with autism and their families, approximately 23 per arm. Children in the intervention group will use Point OutWords for half an hour, five times a week, for 8 weeks. Children in the control group will have equal caregiver-led contact time with the iPad using a selection of control apps (e.g. sensory apps, drawing apps). Communication, motor, and daily living skills are assessed at baseline and post-intervention. Parents will keep diaries during the intervention period and will take part in focus groups when the intervention is completed.

DISCUSSION

Point OutWords was developed in collaboration with children with autism and their caregivers, to provide an intervention for a subgroup of autism that has been historically underserved. As autism is a heterogeneous condition, it is unlikely that one style of intervention will address all aspects of its symptomatology; the motor skills approach of Point OutWords can complement other therapies that address core autistic symptoms of social cognition and communication more directly. The current feasibility trial can inform the selection of outcome measures and design for future full-scale randomised controlled trials of Point OutWords and of other early interventions in autism.

TRIAL REGISTRATION

ISRCTN, ISRCTN12808402. Prospectively registered on 12 March 2019.

Acute kidney injury after thoracic surgery: a proposal for a multicentre evaluation (MERITS)

OBJECTIVES

Because the mortality rate is very low in thoracic surgery, its use as a quality discriminator is limited. Acute kidney injury (AKI) is a candidate measure because it is associated with increased rates of morbidity and mortality and is partly preventable. The incidence of AKI after thoracic surgery is not well documented. We conducted an audit to determine the incidence and outcomes of AKI. This audit became a pilot project, and the results indicate the feasibility of a larger study.

METHODS

Retrospective data on renal function post-thoracic surgery were collected at a tertiary cardiothoracic unit over 12 months. Renal impairment was classified according to the Kidney Disease Improving Global Outcomes criteria.

RESULTS

Of 568 patients (mean = 59  ±  SD 18; 38% women), AKI was diagnosed in 86 (15.1%) within 72 h post-thoracic surgery based on the Kidney Disease Improving Global Outcomes staging system (stage 1, n = 55; stage 2, n = 25; stage 3, n = 6). Significant differences were found in postoperative length of stay (3 vs 5 days; P < 0.001) of patients with and without AKI. There was a significant difference between the age groups of patients with and without AKI (P < 0.05) in the open surgical group but not in the group having video-assisted thoracoscopic surgery (VATS). There was no significant difference in the mortality rates between patients with and without AKI.

CONCLUSIONS

The incidence of AKI after thoracic surgery was 15.1%. AKI was associated with longer hospital stays and was more likely in ≥60-year-old patients after open surgery than after VATS. Reducing AKI could improve patient outcomes. We propose that AKI may be a useful quality measure in thoracic surgery. We are developing a multicentre audit based on this approach.

A response-adaptive randomization procedure for multi-armed clinical trials with normally distributed outcomes

We propose a novel response‐adaptive randomization procedure for multi‐armed trials with continuous outcomes that are assumed to be normally distributed. Our proposed rule is non‐myopic, and oriented toward a patient benefit objective, yet maintains computational feasibility. We derive our response‐adaptive algorithm based on the Gittins index for the multi‐armed bandit problem, as a modification of the method first introduced in Villar et al. (Biometrics, 71, pp. 969‐978). The resulting procedure can be implemented under the assumption of both known or unknown variance. We illustrate the proposed procedure by simulations in the context of phase II cancer trials. Our results show that, in a multi‐armed setting, there are efficiency and patient benefit gains of using a response‐adaptive allocation procedure with a continuous endpoint instead of a binary one. These gains persist even if an anticipated low rate of missing data due to deaths, dropouts, or complete responses is imputed online through a procedure first introduced in this paper. Additionally, we discuss how there are response‐adaptive designs that outperform the traditional equal randomized design both in terms of efficiency and patient benefit measures in the multi‐armed trial context.

Measurement of extravascular lung water to diagnose severe reperfusion lung injury following pulmonary endarterectomy: a prospective cohort clinical validation study

The measurement of extravascular lung water is a relatively new technology which has not yet been well validated as a clinically useful tool. We studied its utility in patients undergoing pulmonary endarterectomy as they frequently suffer reperfusion lung injury and associated oedematous lungs. Such patients are therefore ideal for evaluating this new monitor. We performed a prospective observational cohort study during which extravascular lung water index measurements were taken before and immediately after surgery and postoperatively in intensive care. Data were analysed for 57 patients; 21 patients (37%) experienced severe reperfusion lung injury. The first extravascular lung water index measurement after cardiopulmonary bypass failed to predict severe reperfusion lung injury, area under the receiver operating characteristic curve 0.59 (95%CI 0.44–0.74). On intensive care, extravascular lung water index correlated most strongly at 36 h, area under the receiver operating characteristic curve 0.90 (95%CI 0.80–1.00). Peri‐operative extravascular lung water index is not a useful measure to predict severe reperfusion lung injury after pulmonary endarterectomy, however, it does allow monitoring and measurement during the postoperative period. This study implies that extravascular lung water index can be used to directly assess pulmonary fluid overload and that monitoring patients by measuring extravascular lung water index during their intensive care stay is useful and correlates with their clinical course. This may allow directed, pre‐empted therapy to attenuate the effects and improve patient outcomes and should prompt further studies.

Logistic early warning scores to predict death, cardiac arrest or unplanned intensive care unit re-admission after cardiac surgery

NHS England recently mandated that the National Early Warning Score of vital signs be used in all acute hospital trusts in the UK despite limited validation in the postoperative setting. We undertook a multicentre UK study of 13,631 patients discharged from intensive care after risk‐stratified cardiac surgery in four centres, all of which used VitalPAC^TM to electronically collect postoperative National Early Warning Score vital signs. We analysed 540,127 sets of vital signs to generate a logistic score, the discrimination of which we compared with the national additive score for the composite outcome of: in‐hospital death; cardiac arrest; or unplanned intensive care admission. There were 578 patients (4.2%) with an outcome that followed 4300 sets of observations (0.8%) in the preceding 24 h: 499 out of 578 (86%) patients had unplanned re‐admissions to intensive care. Discrimination by the logistic score was significantly better than the additive score. Respective areas (95%CI) under the receiver‐operating characteristic curve with 24‐h and 6‐h vital signs were: 0.779 (0.771–0.786) vs. 0.754 (0.746–0.761), p < 0.001; and 0.841 (0.829–0.853) vs. 0.813 (0.800–0.825), p < 0.001, respectively. Our proposed logistic Early Warning Score was better than the current National Early Warning Score at discriminating patients who had an event after cardiac surgery from those who did not.

Bandit strategies evaluated in the context of clinical trials in rare life-threatening diseases

In a rare life-threatening disease setting the number of patients in the trial is a high proportion of all patients with the condition (if not all of them). Further, this number is usually not enough to guarantee the required statistical power to detect a treatment effect of a meaningful size. In such a context, the idea of prioritizing patient benefit over hypothesis testing as the goal of the trial can lead to a trial design that produces useful information to guide treatment, even if it does not do so with the standard levels of statistical confidence. The idealised model to consider such an optimal design of a clinical trial is known as a classic multi-armed bandit problem with a finite patient horizon and a patient benefit objective function. Such a design maximises patient benefit by balancing the learning and earning goals as data accumulates and given the patient horizon. On the other hand, optimally solving such a model has a very high computational cost (many times prohibitive) and more importantly, a cumbersome implementation, even for populations as small as a hundred patients. Several computationally feasible heuristic rules to address this problem have been proposed over the last 40 years in the literature. In this article we study a novel heuristic approach to solve it based on the reformulation of the problem as a Restless bandit problem and the derivation of its corresponding Whittle index rule. Such rule was recently proposed in the context of a clinical trial in Villar et al (2015). We perform extensive computational studies to compare through both exact value calculations and simulated values the performance of this rule, other index rules and simpler heuristics previously proposed in the literature. Our results suggest that for the two and three-armed case and a patient horizon less or equal than a hundred patients, all index rules are a priori practically identical in terms of the expected proportion of success attained when all arms start with a uniform prior. However, we find that a posteriori, for specific values of the parameters of interest, the index policies outperform the simpler rules in every instance and specially so in the case of many arms and a larger, though still relatively small, total number of patients with the diseases. The very good performance of bandit rules in terms of patient benefit (i.e. expected number of successes and mean number of patients allocated to the best arm, if it exists) makes them very appealing in context of the challenge posed by drug development for rare life threatening diseases.

Adaptive designs in clinical trials: why use them, and how to run and report them

Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial's course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented.We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

Response-adaptive designs for binary responses: How to offer patient benefit while being robust to time trends?

Response‐adaptive randomisation (RAR) can considerably improve the chances of a successful treatment outcome for patients in a clinical trial by skewing the allocation probability towards better performing treatments as data accumulates. There is considerable interest in using RAR designs in drug development for rare diseases, where traditional designs are not either feasible or ethically questionable. In this paper, we discuss and address a major criticism levelled at RAR: namely, type I error inflation due to an unknown time trend over the course of the trial. The most common cause of this phenomenon is changes in the characteristics of recruited patients—referred to as patient drift. This is a realistic concern for clinical trials in rare diseases due to their lengthly accrual rate. We compute the type I error inflation as a function of the time trend magnitude to determine in which contexts the problem is most exacerbated. We then assess the ability of different correction methods to preserve type I error in these contexts and their performance in terms of other operating characteristics, including patient benefit and power. We make recommendations as to which correction methods are most suitable in the rare disease context for several RAR rules, differentiating between the 2‐armed and the multi‐armed case. We further propose a RAR design for multi‐armed clinical trials, which is computationally efficient and robust to several time trends considered.

Covariate-adjusted response-adaptive randomization for multi-arm clinical trials using a modified forward looking Gittins index rule

We introduce a non-myopic, covariate-adjusted response adaptive (CARA) allocation design for multi-armed clinical trials. The allocation scheme is a computationally tractable procedure based on the Gittins index solution to the classic multi-armed bandit problem and extends the procedure recently proposed in Villar et al. (2015). Our proposed CARA randomization procedure is defined by reformulating the bandit problem with covariates into a classic bandit problem in which there are multiple combination arms, considering every arm per each covariate category as a distinct treatment arm. We then apply a heuristically modified Gittins index rule to solve the problem and define allocation probabilities from the resulting solution. We report the efficiency, balance, and ethical performance of our approach compared to existing CARA methods using a recently published clinical trial as motivation. The net savings in terms of expected number of treatment failures is considerably larger and probably enough to make this design attractive for certain studies where known covariates are expected to be important, stratification is not desired, treatment failures have a high ethical cost, and the disease under study is rare. In a two-armed context, this patient benefit advantage comes at the expense of increased variability in the allocation proportions and a reduction in statistical power. However, in a multi-armed context, simple modifications of the proposed CARA rule can be incorporated so that an ethical advantage can be offered without sacrificing power in comparison with balanced designs.

Bayesian adaptive bandit-based designs using the Gittins index for multi-armed trials with normally distributed endpoints

Adaptive designs for multi-armed clinical trials have become increasingly popular recently because of their potential to shorten development times and to increase patient response. However, developing response-adaptive designs that offer patient-benefit while ensuring the resulting trial provides a statistically rigorous and unbiased comparison of the different treatments included is highly challenging. In this paper, the theory of Multi-Armed Bandit Problems is used to define near optimal adaptive designs in the context of a clinical trial with a normally distributed endpoint with known variance. We report the operating characteristics (type I error, power, bias) and patient-benefit of these approaches and alternative designs using simulation studies based on an ongoing trial. These results are then compared to those recently published in the context of Bernoulli endpoints. Many limitations and advantages are similar in both cases but there are also important differences, specially with respect to type I error control. This paper proposes a simulation-based testing procedure to correct for the observed type I error inflation that bandit-based and adaptive rules can induce.

A Bayesian adaptive design for clinical trials in rare diseases

Development of treatments for rare diseases is challenging due to the limited number of patients available for participation. Learning about treatment effectiveness with a view to treat patients in the larger outside population, as in the traditional fixed randomised design, may not be a plausible goal. An alternative goal is to treat the patients within the trial as effectively as possible. Using the framework of finite-horizon Markov decision processes and dynamic programming (DP), a novel randomised response-adaptive design is proposed which maximises the total number of patient successes in the trial and penalises if a minimum number of patients are not recruited to each treatment arm. Several performance measures of the proposed design are evaluated and compared to alternative designs through extensive simulation studies using a recently published trial as motivation. For simplicity, a two-armed trial with binary endpoints and immediate responses is considered. Simulation results for the proposed design show that: (i) the percentage of patients allocated to the superior arm is much higher than in the traditional fixed randomised design; (ii) relative to the optimal DP design, the power is largely improved upon and (iii) it exhibits only a very small bias and mean squared error of the treatment effect estimator. Furthermore, this design is fully randomised which is an advantage from a practical point of view because it protects the trial against various sources of bias. As such, the proposed design addresses some of the key issues that have been suggested as preventing so-called bandit models from being implemented in clinical practice.

INDEXABILITY AND OPTIMAL INDEX POLICIES FOR A CLASS OF REINITIALISING RESTLESS BANDITS

Motivated by a class of Partially Observable Markov Decision Processes with application in surveillance systems in which a set of imperfectly observed state processes is to be inferred from a subset of available observations through a Bayesian approach, we formulate and analyze a special family of multi-armed restless bandit problems. We consider the problem of finding an optimal policy for observing the processes that maximizes the total expected net rewards over an infinite time horizon subject to the resource availability. From the Lagrangian relaxation of the original problem, an index policy can be derived, as long as the existence of the Whittle index is ensured. We demonstrate that such a class of reinitializing bandits in which the projects' state deteriorates while active and resets to its initial state when passive until its completion possesses the structural property of indexability and we further show how to compute the index in closed form. In general, the Whittle index rule for restless bandit problems does not achieve optimality. However, we show that the proposed Whittle index rule is optimal for the problem under study in the case of stochastically heterogenous arms under the expected total criterion, and it is further recovered by a simple tractable rule referred to as the 1-limited Round Robin rule. Moreover, we illustrate the significant suboptimality of other widely used heuristic: the Myopic index rule, by computing in closed form its suboptimality gap. We present numerical studies which illustrate for the more general instances the performance advantages of the Whittle index rule over other simple heuristics.

Response-adaptive randomization for multi-arm clinical trials using the forward looking Gittins index rule

The Gittins index provides a well established, computationally attractive, optimal solution to a class of resource allocation problems known collectively as the multi-arm bandit problem. Its development was originally motivated by the problem of optimal patient allocation in multi-arm clinical trials. However, it has never been used in practice, possibly for the following reasons: (1) it is fully sequential, i.e., the endpoint must be observable soon after treating a patient, reducing the medical settings to which it is applicable; (2) it is completely deterministic and thus removes randomization from the trial, which would naturally protect against various sources of bias. We propose a novel implementation of the Gittins index rule that overcomes these difficulties, trading off a small deviation from optimality for a fully randomized, adaptive group allocation procedure which offers substantial improvements in terms of patient benefit, especially relevant for small populations. We report the operating characteristics of our approach compared to existing methods of adaptive randomization using a recently published trial as motivation.

Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges

Multi-armed bandit problems (MABPs) are a special type of optimal control problem well suited to model resource allocation under uncertainty in a wide variety of contexts. Since the first publication of the optimal solution of the classic MABP by a dynamic index rule, the bandit literature quickly diversified and emerged as an active research topic. Across this literature, the use of bandit models to optimally design clinical trials became a typical motivating application, yet little of the resulting theory has ever been used in the actual design and analysis of clinical trials. To this end, we review two MABP decision-theoretic approaches to the optimal allocation of treatments in a clinical trial: the infinite-horizon Bayesian Bernoulli MABP and the finite-horizon variant. These models possess distinct theoretical properties and lead to separate allocation rules in a clinical trial design context. We evaluate their performance compared to other allocation rules, including fixed randomization. Our results indicate that bandit approaches offer significant advantages, in terms of assigning more patients to better treatments, and severe limitations, in terms of their resulting statistical power. We propose a novel bandit-based patient allocation rule that overcomes the issue of low power, thus removing a potential barrier for their use in practice.