Evidence, eminence and extrapolation

Sharing information across patient subgroups to draw conclusions from sparse treatment networks

Network meta-analysis (NMA) usually provides estimates of the relative effects with the highest possible precision. However, sparse networks with few available studies and limited direct evidence can arise, threatening the robustness and reliability of NMA estimates. In these cases, the limited amount of available information can hamper the formal evaluation of the underlying NMA assumptions of transitivity and consistency. In addition, NMA estimates from sparse networks are expected to be imprecise and possibly biased as they rely on large-sample approximations that are invalid in the absence of sufficient data. We propose a Bayesian framework that allows sharing of information between two networks that pertain to different population subgroups. Specifically, we use the results from a subgroup with a lot of direct evidence (a dense network) to construct informative priors for the relative effects in the target subgroup (a sparse network). This is a two-stage approach where at the first stage, we extrapolate the results of the dense network to those expected from the sparse network. This takes place by using a modified hierarchical NMA model where we add a location parameter that shifts the distribution of the relative effects to make them applicable to the target population. At the second stage, these extrapolated results are used as prior information for the sparse network. We illustrate our approach through a motivating example of psychiatric patients. Our approach results in more precise and robust estimates of the relative effects and can adequately inform clinical practice in presence of sparse networks.

Composite Likelihoods with Bounded Weights in Extrapolation of Data

Among many efforts to facilitate timely access to safe and effective medicines to children, increased attention has been given to extrapolation. Loosely, it is the leveraging of conclusions or available data from adults or older age groups to draw conclusions for the target pediatric population when it can be assumed that the course of the disease and the expected response to a medicinal product would be sufficiently similar in the pediatric and the reference population. Extrapolation then can be characterized as a statistical mapping of information from the reference (adults or older age groups) to the target pediatric population. The translation, or loosely mapping of information, can be through a composite likelihood approach where the likelihood of the reference population is weighted by exponentiation and that this exponent is related to the value of the mapped information in the target population. The weight is bounded above and below recognizing the fact that similarity (of the disease and the expected response) is still valid despite variability of response between the cohorts. Maximum likelihood approaches are then used for estimation of parameters, and asymptotic theory is used to derive distributions of estimates for use in inference. Hence, the estimation of effects in the target population borrows information from the reference population. In addition, this manuscript also talks about how this method is related to the Bayesian statistical paradigm.

Bayesian Strategies in Rare Diseases

Bayesian strategies for planning and analyzing clinical trials have become a viable choice, especially in rare diseases where drug development faces many challenges and stakeholders are interested in innovations that may help overcome them. Disease natural history and clinical outcomes occurrence and variability are often poorly understood. Standard trial designs are not optimized to obtain adequate safety and efficacy data from small numbers of patients. Bayesian methods are well-suited for adaptive trials, with an accelerated learning curve. Using Bayesian statistics can be advantageous in that design choices and their consequences are considered carefully, continuously monitored, and updated where necessary, which ultimately provides a natural and principled way of seamlessly combining prior clinical information with data, within a solid decision theoretical framework. In this article, we introduce the Bayesian option in the rare disease context to support clinical decision-makers in selecting the best choice for their drug development project. Many researchers in drug development show reluctance to using Bayesian statistics, and the top-two reported barriers are insufficient knowledge of Bayesian approaches and a lack of clarity or guidance from regulators. Here we introduce concepts of borrowing, extrapolation, adaptation, and modeling and illustrate them with examples that have been discussed or developed with regulatory bodies to show how Bayesian strategies can be applied to drug development in rare diseases.

Evidence-Based Practices and Use among Employees and Students at an Austrian Medical University

Developed in the pre-internet era in the early 1980s, empirical medical practice, i.e., evidence-based practice (EBP) has become crucial in critical thinking and statistical reasoning at the point-of-care. As little evidence is available so far on how EBP is perceived in the Austrian academic context, we conducted a cross-sectional online survey among a nonrandom purposive sample of employees and students at the Medical University Vienna, Austria (total n = 1247, 59.8% females). The German questionnaire assessed both EBP capability beliefs and EBP use, with the respective indices both yielding good internal consistency. We conducted subgroup comparisons between employees (n = 638) and students (n = 609). In line with Bandura’s self-efficacy theory, we found a correlation between EBP capability beliefs and EBP use, with higher scores reported in the employee group. The results indicated that the participants did not strictly follow the sequential EBP steps as grounded in the item-response theory. Since its emergence, EBP has struggled to overcome the dominating traditional way of conducting medicine, which is also known as eminence-based medicine, where ad hoc decisions are based upon expert opinions, and nowadays frequently supplemented by quick online searches. Medical staff and supervisors of medical students should be aware of the existing overlaps and synergies of these potentially equivalent factors in clinical care. There is a need for intensifying the public and scientific debate on how to deal with the divergence between EBP theory and EBP practice.

Optimising the trade-off between type I and II error rates in the Bayesian context

For any decision-making study, there are two sorts of errors that can be made, declaring a positive result when the truth is negative, and declaring a negative result when the truth is positive. Traditionally, the primary analysis of a study is a two-sided hypothesis test, the type I error rate will be set to 5% and the study is designed to give suitably low type II error - typically 10 or 20% - to detect a given effect size. These values are standard, arbitrary and, other than the choice between 10 and 20%, do not reflect the context of the study, such as the relative costs of making type I and II errors and the prior belief the drug will be placebo-like. Several authors have challenged this paradigm, typically for the scenario where the planned analysis is frequentist. When resource is limited, there will always be a trade-off between the type I and II error rates, and this article explores optimising this trade-off for a study with a planned Bayesian statistical analysis. This work provides a scientific basis for a discussion between stakeholders as to what type I and II error rates may be appropriate and some algebraic results for normally distributed data.

Outcomes of the Pediatric Development Plan of Tapentadol

The opioid analgesic tapentadol was the first pain medication to be developed for the treatment of pain in children under a formal process established by the regulatory authorities. This article summarizes the outcomes of the pediatric development program for tapentadol across the entire age range from birth (including neonates) to adolescents <18 years of age. In addition, the challenges experienced when designing and conducting the pediatric tapentadol clinical trials as well as the interactions with the regulatory authorities are discussed. As a first outcome, the oral solution of tapentadol was authorized in the EU in 2018 as a new treatment option in the hospital setting for moderate to severe acute pain in children from 2 to <18 years of age.

Tapentadol Prolonged Release for Long-Term Treatment of Pain in Children

Purpose

Investigation of the efficacy and safety of tapentadol prolonged release (PR) compared with morphine PR for long-term treatment of pain in children.

Patients and Methods

Children aged 6 to <18 years requiring long-term treatment with opioids were studied in a 12-month, 2-part, multi-center trial: Part 1, 14-day open-label, randomized, active-controlled, parallel group non-inferiority trial comparing twice daily tapentadol PR with morphine PR; Part 2, open-label treatment with tapentadol PR for up to 12 months or no treatment “safety observation period”. Pain intensity was rated with visual analogue scale or Faces Pain Scale-Revised, and non-inferiority was assessed by comparison of “treatment responders” (those completing the 14-day treatment period and showing pre-defined changes in pain rating) in each group.

Results

Twenty-three of 48 centers enrolled 73 patients. In Part 1, 45 and 24 patients received tapentadol or morphine, respectively, of which 40 and 22 completed 14-day treatment. In Part 2, thirty-six and 58 patients entered the tapentadol PR or observation periods, respectively, with 20/36 completing at least 12 weeks of treatment; 10 of the 36 had received morphine in Part 1. Forty-four of the 58 patients in the safety observation period had received tapentadol. Tapentadol PR was non-inferior to morphine PR (lower limit of confidence interval above negative non-inferiority margin of −0.2) in Part 1. Rates of adverse events were as expected with nausea (22.2%) and constipation (15.6%) in the tapentadol PR group, and with vomiting (33.3%), nausea and constipation (each 16.7%) in the morphine PR group. No new safety issues were identified; the safety profile of tapentadol over the 12 months treatment and observation periods was comparable to that established in subjects >18 years old.

Conclusion

Tapentadol PR was well tolerated and equivalent to morphine PR for both efficacy and safety in children (6 to <18 years old) requiring long-term treatment with opioids.

A clinimetric analysis of the Pressure Ulcer Risk Primary or Secondary Evaluation Tool: PURPOSE-T

The assessment of patients' risk for developing pressure ulcers is a routine and fundamental nursing process undertaken to prevent avoidable harm to patients in all care settings. Many risk assessment tools are currently used in clinical practice, however no individual tool is recommended by advisory bodies such as the National Institute for Health and Care Excellence or the European Pressure Ulcer Advisory Panel. The evidence base on the value of structured risk assessment tools in reducing the incidence or severity of pressure ulcers is poor. This purpose of this article is to provide a clinimetric analysis of the recently developed Pressure Ulcer Risk Primary or Secondary Evaluation Tool (PURPOSE-T) and identify areas for future research to improve the utility of structured risk assessment in identifying patients at risk of developing pressure ulcers.

Group sequential adaptive designs in series of time-to-event randomised trials in rare diseases: A simulation study

In rare diseases, fully powered large trials may not be doable in a reasonable time frame even with international collaborations. In a previous work, we proposed an approach based on a series of smaller parallel group two-arm randomised controlled trials (RCT) performed over a long research horizon. Within the series of trials, the treatment selected after each trial becomes the control treatment of the next one. We concluded that running more trials with smaller sample sizes and relaxed α-levels leads in the long term and under reasonable assumptions to larger survival benefits with a moderate increase of risk as compared to traditional designs based on larger but fewer trials designed to meet stringent evidence criteria. We now extend this quantitative framework with more 'flexible' designs including interim analyses for futility and/or efficacy, and three-arm adaptive designs with treatment selection at interim. In the simulation study, we considered different disease severities, accrual rates, and hypotheses of how treatments improve over time. For each design, we estimated the long-term survival benefit as the relative difference in hazard rates between the end and the start of the research horizon, and the risk defined as the probability of selecting at the end of the research horizon a treatment inferior to the initial control. We assessed the impact of the α-level and the choice of the stopping rule on the operating characteristics. We also compared the performance of series based on two- vs. three-arm trials. We show that relaxing α-levels within the limit of 0.1 is associated with larger survival gains and moderate increase of risk which remains within acceptable ranges. Including an interim analysis with a futility rule is associated with an additional survival gain and a better risk control as compared to series with no interim analysis, when the α-level is below or equal to 0.1, whereas the benefit of including an interim analysis is rather small for higher α-levels. Including an interim analysis for efficacy yields almost no additional gain. Series based on three-arm trials are associated with a systematic improvement in terms of survival gain and risk control as compared to series of two-arm trials.

Commentary on the EMA Reflection Paper on the use of extrapolation in the development of medicines for paediatrics

Adopted guidelines reflect a harmonised European approach to a specific scientific issue and should reflect the most recent scientific knowledge. However, whilst EU regulations are mandatory for all member states and EU directives must be followed by national laws in line with the directive, EMA guidelines do not have legal force and alternative approaches may be taken, but these obviously require more justification. This new series of the BJCP, developed in collaboration with the EMA, aims to address this issue by providing an annotated version of some relevant EMA guidelines and regulatory documents by experts. Hopefully, this will help in promoting their diffusion and in opening a forum for discussion with our readers.

Applicability and added value of novel methods to improve drug development in rare diseases

BACKGROUND

The ASTERIX project developed a number of novel methods suited to study small populations. The objective of this exercise was to evaluate the applicability and added value of novel methods to improve drug development in small populations, using real world drug development programmes as reported in European Public Assessment Reports.

METHODS

The applicability and added value of thirteen novel methods developed within ASTERIX were evaluated using data from 26 European Public Assessment Reports (EPARs) for orphan medicinal products, representative of rare medical conditions as predefined through six clusters. The novel methods included were 'innovative trial designs' (six methods), 'level of evidence' (one method), 'study endpoints and statistical analysis' (four methods), and 'meta-analysis' (two methods) and they were selected from the methods developed within ASTERIX based on their novelty; methods that discussed already available and applied strategies were not included for the purpose of this validation exercise. Pre-requisites for application in a study were systematized for each method, and for each main study in the selected EPARs it was assessed if all pre-requisites were met. This direct applicability using the actual study design was firstly assessed. Secondary, applicability and added value were explored allowing changes to study objectives and design, but without deviating from the context of the drug development plan. We evaluated whether differences in applicability and added value could be observed between the six predefined condition clusters.

RESULTS AND DISCUSSION

Direct applicability of novel methods appeared to be limited to specific selected cases. The applicability and added value of novel methods increased substantially when changes to the study setting within the context of drug development were allowed. In this setting, novel methods for extrapolation, sample size re-assessment, multi-armed trials, optimal sequential design for small sample sizes, Bayesian sample size re-estimation, dynamic borrowing through power priors and fall-back tests for co-primary endpoints showed most promise - applicable in more than 40% of evaluated EPARs in all clusters. Most of the novel methods were applicable to conditions in the cluster of chronic and progressive conditions, involving multiple systems/organs. Relatively fewer methods were applicable to acute conditions with single episodes. For the chronic clusters, Goal Attainment Scaling was found to be particularly applicable as opposed to other (non-chronic) clusters.

CONCLUSION

Novel methods as developed in ASTERIX can improve drug development programs. Achieving optimal added value of these novel methods often requires consideration of the entire drug development program, rather than reconsideration of methods for a specific trial. The novel methods tested were mostly applicable in chronic conditions, and acute conditions with recurrent episodes.

Model averaging for robust extrapolation in evidence synthesis

Extrapolation from a source to a target, eg, from adults to children, is a promising approach to utilize external information when data are sparse. In the context of meta-analyses, one is commonly faced with a small number of studies, whereas potentially relevant additional information may also be available. Here, we describe a simple extrapolation strategy using heavy-tailed mixture priors for effect estimation in meta-analysis, which effectively results in a model-averaging technique. The described method is robust in the sense that a potential prior-data conflict, ie, a discrepancy between source and target data, is explicitly anticipated. The aim of this paper is to develop a solution for this particular application to showcase the ease of implementation by providing R code, and to demonstrate the robustness of the general approach in simulations.

Lessons learned from IDeAl - 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials

BACKGROUND

IDeAl (Integrated designs and analysis of small population clinical trials) is an EU funded project developing new statistical design and analysis methodologies for clinical trials in small population groups. Here we provide an overview of IDeAl findings and give recommendations to applied researchers.

METHOD

The description of the findings is broken down by the nine scientific IDeAl work packages and summarizes results from the project's more than 60 publications to date in peer reviewed journals. In addition, we applied text mining to evaluate the publications and the IDeAl work packages' output in relation to the design and analysis terms derived from in the IRDiRC task force report on small population clinical trials.

RESULTS

The results are summarized, describing the developments from an applied viewpoint. The main result presented here are 33 practical recommendations drawn from the work, giving researchers a comprehensive guidance to the improved methodology. In particular, the findings will help design and analyse efficient clinical trials in rare diseases with limited number of patients available. We developed a network representation relating the hot topics developed by the IRDiRC task force on small population clinical trials to IDeAl's work as well as relating important methodologies by IDeAl's definition necessary to consider in design and analysis of small-population clinical trials. These network representation establish a new perspective on design and analysis of small-population clinical trials.

CONCLUSION

IDeAl has provided a huge number of options to refine the statistical methodology for small-population clinical trials from various perspectives. A total of 33 recommendations developed and related to the work packages help the researcher to design small population clinical trial. The route to improvements is displayed in IDeAl-network representing important statistical methodological skills necessary to design and analysis of small-population clinical trials. The methods are ready for use.

How to use prior knowledge and still give new data a chance?

A common challenge for the development of drugs in rare diseases and special populations, eg, paediatrics, is the small numbers of patients that can be recruited into clinical trials. Extrapolation can be used to support development and licensing in paediatrics through the structured integration of available data in adults and prospectively generated data in paediatrics to derive conclusions that support licensing decisions in the target paediatric population. In this context, Bayesian analyses have been proposed to obtain formal proof of efficacy of a new drug or therapeutic principle by using additional information (data, opinion, or expectation), expressed through a prior distribution. However, little is said about the impact of the prior assumptions on the evaluation of outcome and prespecified strategies for decision‐making as required in the regulatory context.

On the basis of examples, we explore the use of data‐based Bayesian meta‐analytic–predictive methods and compare these approaches with common frequentist and Bayesian meta‐analysis models. Noninformative efficacy prior distributions usually do not change the conclusions irrespective of the chosen analysis method. However, if heterogeneity is considered, conclusions are highly dependent on the heterogeneity prior. When using informative efficacy priors based on previous study data in combination with heterogeneity priors, these may completely determine conclusions irrespective of the data generated in the target population. Thus, it is important to understand the impact of the prior assumptions and ensure that prospective trial data in the target population have an appropriate chance, to change prior belief to avoid trivial and potentially erroneous conclusions.

A systematic literature review of evidence-based clinical practice for rare diseases: what are the perceived and real barriers for improving the evidence and how can they be overcome?

BACKGROUND

Evidence-based clinical practice is challenging in all fields, but poses special barriers in the field of rare diseases. The present paper summarises the main barriers faced by clinical research in rare diseases, and highlights opportunities for improvement.

METHODS

Systematic literature searches without meta-analyses and internal European Clinical Research Infrastructure Network (ECRIN) communications during face-to-face meetings and telephone conferences from 2013 to 2017 within the context of the ECRIN Integrating Activity (ECRIN-IA) project.

RESULTS

Barriers specific to rare diseases comprise the difficulty to recruit participants because of rarity, scattering of patients, limited knowledge on natural history of diseases, difficulties to achieve accurate diagnosis and identify patients in health information systems, and difficulties choosing clinically relevant outcomes.

CONCLUSIONS

Evidence-based clinical practice for rare diseases should start by collecting clinical data in databases and registries; defining measurable patient-centred outcomes; and selecting appropriate study designs adapted to small study populations. Rare diseases constitute one of the most paradigmatic fields in which multi-stakeholder engagement, especially from patients, is needed for success. Clinical research infrastructures and expertise networks offer opportunities for establishing evidence-based clinical practice within rare diseases.

Enhancing the Pediatric Drug Development Framework to Deliver Better Pediatric Therapies Tomorrow

Health care professionals involved in the clinical management of children have long appreciated the limited number of therapies suitably evaluated for their optimal use in the pediatric population. In the past century, advances in regulatory policy significantly evolved adult drug evaluation. The scarcity of available patient populations, practical complexities of drug development research, and minimal financial returns have hampered pharmaceutical investment in the study of therapies for children. More recently, pediatric policy and legislation in the United States and Europe have instituted a system of obligations and incentives to stimulate investment in pediatric drug development. These initiatives, in conjunction with a more sophisticated process of drug discovery and development, have led to significant advancements in the labeling of drugs for pediatric use. Facilitated by the emergence of new targets, precision medicine, and innovations in regulatory science, there is now a subtle shift in focus toward drug development research for children rather than simply in children. Although there has been an increase in pediatric studies of investigational agents and labeling of pediatric information for use, there have been unintended consequences of existing policies. As a result, limited progress has been made in certain therapeutic areas and for off-patent therapies. Future policy reform to enhance the availability and accessibility of pediatric medicines should not only reflect an understanding not only of the successes of existing policy and legislative initiatives but also constructively address failures and unintended consequences. Taken together, policy reform, global cooperation, and innovation in regulatory science will more ably deliver better pediatric therapies tomorrow.

Model-Based Assessment of Alternative Study Designs in Pediatric Trials. Part II: Bayesian Approaches

This study presents a pharmacokinetic‐pharmacodynamic based clinical trial simulation framework for evaluating the performance of a fixed‐sample Bayesian design (BD) and two alternative Bayesian sequential designs (BSDs) (i.e., a non‐hierarchical (NON‐H) and a semi‐hierarchical (SEMI‐H) one). Prior information was elicited from adult trials and weighted based on the expected similarity of response to treatment between the pediatric and adult populations. Study designs were evaluated in terms of: type I and II errors, sample size per arm (SS), trial duration (TD), and estimate precision. No substantial differences were observed between NON‐H and SEMI‐H. BSDs require, on average, smaller SS and TD compared to the BD, which, on the other hand, guarantees higher estimate precision. When large differences between children and adults are expected, BSDs can return very large SS. Bayesian approaches appear to outperform their frequentist counterparts in the design of pediatric trials even when little weight is given to prior information from adults.

Adaptive paediatric investigation plans, a small step to improve regulatory decision making in drug development for children?

Different arguments have been put forward why drug developers should commit themselves early for what they are planning to do for children. By EU regulation, paediatric investigation plans should be agreed on in early phases of drug development in adults. Here, extrapolation from adults to children is widely applied to reduce the burden and avoids unnecessary clinical trials in children, but early regulatory decisions on how far extrapolation can be used may be highly uncertain. Under special circumstances, the regulatory process should allow for adaptive paediatric investigation plans explicitly foreseeing a re-evaluation of the early decision based on the information accumulated later from adults or elsewhere. A small step towards adaptivity and learning from experience may improve the quality of regulatory decisions in particular with regard to how much information can be borrowed from adults. © 2016 The Authors. Pharmaceutical Statistics Published by John Wiley & Sons Ltd.

Model-Based Assessment of Alternative Study Designs in Pediatric Trials. Part I: Frequentist Approaches

Alternative designs can increase the feasibility of pediatric trials when compared to classical parallel designs (PaD). In this work we present a model-based approach based on clinical trial simulations for the comparison of PaD with the alternative sequential, crossover, and randomized withdrawal (RWD) designs. Study designs were evaluated in terms of: type I and II errors, sample size per arm (SS), trial duration (TD), treatment exposures, and parameter estimate precision (EP). The crossover requires the lowest SS and TD, although it implies higher placebo and no treatment exposures. RWD maximizes exposure to active treatment while minimizing that to placebo, but requires the largest SS. SS of sequential designs can sometimes be smaller than the crossover one, although with poorer EP. This pharmacometric framework allows a multiscale comparison of alternative study designs that can be used for design selection in future pediatric trials.

Evidence, eminence and extrapolation

A full independent drug development programme to demonstrate efficacy may not be ethical and/or feasible in small populations such as paediatric populations or orphan indications. Different levels of extrapolation from a larger population to smaller target populations are widely used for supporting decisions in this situation. There are guidance documents in drug regulation, where a weakening of the statistical rigour for trials in the target population is mentioned to be an option for dealing with this problem. To this end, we propose clinical trials designs, which make use of prior knowledge on efficacy for inference. We formulate a framework based on prior beliefs in order to investigate when the significance level for the test of the primary endpoint in confirmatory trials can be relaxed (and thus the sample size can be reduced) in the target population while controlling a certain posterior belief in effectiveness after rejection of the null hypothesis in the corresponding confirmatory statistical test. We show that point‐priors may be used in the argumentation because under certain constraints, they have favourable limiting properties among other types of priors. The crucial quantity to be elicited is the prior belief in the possibility of extrapolation from a larger population to the target population. We try to illustrate an existing decision tree for extrapolation to paediatric populations within our framework. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.