A comparison of the performance of contemporary, historical, and cross-lab controls in QTc assessment in conscious nonhuman primates

Cardiovascular safety pharmacology and toxicology studies include vehicle control animals in most studies. Electrocardiogram data on common vehicles is accumulated relatively quickly. In the interests of the 3Rs principles it may be useful to use this historical information to reduce the use of animals or to refine the sensitivity of studies. We used implanted telemetry data from a large nonhuman primate (NHP) cardiovascular study (n = 48) evaluating the effect of moxifloxacin. We extracted 24 animals to conduct a n = 3/sex/group analysis. The remaining 24 animals were used to generate 1000 unique combinations of 3 male and 3 female NHP to act as control groups for the three treated groups in the n = 3/sex/group analysis. The distribution of treatment effects, median minimum detectable difference (MDD) values were gathered from the 1000 studies. These represent contemporary controls. Data were available from 42 NHP from 3 other studies in the same laboratory using the same technology. These were used to generate 1000 unique combinations of 6, 12, 18, 24 and 36 NHP to act as historical control animals for the 18 animals in the treated groups of the moxifloxacin study. Data from an additional laboratory were also available for 20 NHP. The QT, RR and QT-RR data from the three sources were comparable. However, differences in the time course of QTc effect in the vehicle data from the two laboratories meant that it was not possible to use cross-lab controls. In the case of historical controls from the same laboratory, these could be used in place of the contemporary controls in determining a treatment's effect. There appeared to be an advantage in using larger (≥18) group sizes for historical controls. These data support the opportunity of using historical controls to reduce the number of animals used in new cardiovascular studies.

The 4th NextGen therapies for SJIA and MAS: part 3 clinical trials in refractory SJIA: historic controls as an alternative to a withdrawal design study

The substantial morbidity and mortality associated with refractory systemic JIA underlies the need for new treatment approaches. However, progress in this area has been limited by the difficulty of enrolling these patients in clinical trials with traditional designs, particularly in patients presenting with the life-threatening macrophage activation syndrome. At the NextGen 2022 conference, there was group consensus that using historical cohorts as a control group to avoid the need for a placebo-arm or drug withdrawal was highly desirable and might be acceptable for clinical trials in MAS to support medication efficacy and safety. However, if historic controls were used in a trial, it would be important to ensure that the historic cohort matches the study group in terms of clinical characteristics (such as disease severity and exposure to other medications), and that disease outcome in both groups is assessed using the same outcome measures. The discussions at the NextGen 2022 conference focused on the potential strategies to achieve these goals.

Increasing FDA-accelerated approval of single-arm trials in oncology (1992 to 2020)

OBJECTIVES

We aimed to map the characteristics of single-arm trials (SAT), report the Food and Drug Administration (FDA) transparency in presenting historical control, and to assess the confirmatory randomized controlled trials (RCTs).

STUDY DESIGN AND SETTING

This metaresearch included a review of all oncology indication approved using SAT by FDA-AA (FDA-Accelerated Approval) from 1992 to 2020. Two independent reviewers identified SAT, extracted data from FDA full medical reviews for historical controls reported and MEDLINE for searching for confirmatory RCT published.

RESULTS

Of 254 FDA-AA approvals, 119 (47%) were approved for oncologic indications using SAT. Fifty-four drugs for 72 oncology indications were for leukemia, lymphoma, lung cancer, urothelial cancer, multiple myeloma, and thyroid cancer. Overall, 37 (52%) treatments were converted into regular approval. Of these, 17 (46%) were based on confirmatory RCTs using overall survival (OS) as an outcome. Five indications were withdrawn from the market. Most trials outcomes were blindly assessed by independent research committees. Median trial sample size was 105 patients (min:8 to max:532). The FDA did not fully specify historical control selection in 75% of cases.

CONCLUSION

The granting of FDA-AAs based on SAT in oncology is increasing with more target drugs approved over time. Transparency in historical control reporting is necessary.

The Use of External Controls in FDA Regulatory Decision Making

The regulatory standards of the United States Food and Drug Administration (FDA) require substantial evidence of effectiveness from adequate and well-controlled trials that typically use a valid comparison to an internal concurrent control. However, when it is not feasible or ethical to use an internal control, particularly in rare disease populations, relying on external controls may be acceptable. To better understand the use of external controls to support product development and approval, we reviewed FDA regulatory approval decisions between 2000 and 2019 for drug and biologic products to identify pivotal studies that leveraged external controls, with a focus on select therapeutic areas. Forty-five approvals were identified where FDA accepted external control data in their benefit/risk assessment; they did so for many reasons including the rare nature of the disease, ethical concerns regarding use of a placebo or no-treatment arm, the seriousness of the condition, and the high unmet medical need. Retrospective natural history data, including retrospective reviews of patient records, was the most common source of external control (44%). Other types of external control were baseline control (33%); published data (11%); and data from a previous clinical study (11%). To gain further insights, a comprehensive evaluation of selected approvals utilizing different types of external control is provided to highlight the variety of approaches used by sponsors and the challenges encountered in supporting product development and FDA decision making; particularly, the value and use of retrospective natural history in the development of products for rare diseases. Education on the use of external controls based on FDA regulatory precedent will allow for continued use and broader application of innovative approaches to clinical trial design, while avoiding delays in product development for rare diseases. Learnings from this review also highlight the need to update regulatory guidance to acknowledge the utility of external controls, particularly retrospective natural history data.

Using historical data to facilitate clinical prevention trials in Alzheimer disease? An analysis of longitudinal MCI (mild cognitive impairment) data sets

BACKGROUND

The Placebo Group Simulation Approach (PGSA) aims at partially replacing randomized placebo-controlled trials (RPCTs), making use of data from historical control groups in order to decrease the needed number of study participants exposed to lengthy placebo treatment. PGSA algorithms to create virtual control groups were originally derived from mild cognitive impairment (MCI) data of the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. To produce more generalizable algorithms, we aimed to compile five different MCI databases in a heuristic manner to create a "standard control algorithm" for use in future clinical trials.

METHODS

We compared data from two North American cohort studies (n=395 and 4328, respectively), one company-sponsored international clinical drug trial (n=831) and two convenience patient samples, one from Germany (n=726), and one from Switzerland (n=1558).

RESULTS

Despite differences between the five MCI samples regarding inclusion and exclusion criteria, their baseline demographic and cognitive performance data varied less than expected. However, the five samples differed markedly with regard to their subsequent cognitive performance and clinical development: (1) MCI patients from the drug trial did not deteriorate on verbal fluency over 3 years, whereas patients in the other samples did; (2) relatively few patients from the drug trial progressed from MCI to dementia (about 10% after 4 years), in contrast to the other four samples with progression rates over 30%.

CONCLUSION

Conventional MCI criteria were insufficient to allow for the creation of well-defined and internationally comparable samples of MCI patients. More recently published criteria for MCI or "MCI due to AD" are unlikely to remedy this situation. The Alzheimer scientific community needs to agree on a standard set of neuropsychological tests including appropriate selection criteria to make MCI a scientifically more useful concept. Patient data from different sources would then be comparable, and the scientific merits of algorithm-based study designs such as the PGSA could be properly assessed.

Implementing Historical Controls in Oncology Trials

Drug development in oncology has broadened from mainly considering randomized clinical trials to also including single-arm trials tailored for very specific subtypes of cancer. They often use historical controls, and this article discusses benefits and risks of this paradigm and provide various regulatory and statistical considerations. While leveraging the information brought by historical controls could potentially shorten development time and reduce the number of patients enrolled, a careful selection of the past studies, a prespecified statistical analysis accounting for the heterogeneity between studies, and early engagement with regulators will be key to success. Although both the European Medicines Agency and the U.S. Food and Drug Administration have already approved medicines based on nonrandomized experiments, the evidentiary package can be perceived as less comprehensive than randomized experiments. Use of historical controls, therefore, is better suited for cases of high unmet clinical need, where the disease course is well characterized and the primary endpoint is objective. IMPLICATIONS FOR PRACTICE: Incorporating historical data in single-arm oncology trials has the potential to accelerate drug development and to reduce the number of patients enrolled, compared with standard randomized controlled clinical trials. Given the lack of blinding and randomization, such an approach is better suited for cases of high unmet clinical need and/or difficult experimental situations, in which the trajectory of the disease is well characterized and the endpoint can be measured objectively. Careful pre-specification and selection of the historical data, matching of the patient characteristics with the concurrent trial data, and innovative statistical methodologies accounting for between-study variation will be needed. Early engagement with regulators (e.g., via Scientific Advice) is highly recommended.

A comprehensive analysis of the animal carcinogenicity data for glyphosate from chronic exposure rodent carcinogenicity studies

Since the introduction of glyphosate-tolerant genetically-modified plants, the global use of glyphosate has increased dramatically making it the most widely used pesticide on the planet. There is considerable controversy concerning the carcinogenicity of glyphosate with scientists and regulatory authorities involved in the review of glyphosate having markedly different opinions. One key aspect of these opinions is the degree to which glyphosate causes cancer in laboratory animals after lifetime exposure. In this review, twenty-one chronic exposure animal carcinogenicity studies of glyphosate are identified from regulatory documents and reviews; 13 studies are of sufficient quality and detail to be reanalyzed in this review using trend tests, historical control tests and pooled analyses. The analyses identify 37 significant tumor findings in these studies and demonstrate consistency across studies in the same sex/species/strain for many of these tumors. Considering analyses of the individual studies, the consistency of the data across studies, the pooled analyses, the historical control data, non-neoplastic lesions, mechanistic evidence and the associated scientific literature, the tumor increases seen in this review are categorized as to the strength of the evidence that glyphosate causes these cancers. The strongest evidence shows that glyphosate causes hemangiosarcomas, kidney tumors and malignant lymphomas in male CD-1 mice, hemangiomas and malignant lymphomas in female CD-1 mice, hemangiomas in female Swiss albino mice, kidney adenomas, liver adenomas, skin keratoacanthomas and skin basal cell tumors in male Sprague-Dawley rats, adrenal cortical carcinomas in female Sprague-Dawley rats and hepatocellular adenomas and skin keratocanthomas in male Wistar rats.

Reducing Patient Burden in Clinical Trials Through the Use of Historical Controls: Appropriate Selection of Historical Data to Minimize Risk of Bias

Historical data have been used to augment or replace control arms in some rare disease and pediatric clinical trials. With greater availability of historical data and new methodology such as dynamic borrowing, the inclusion of historical data in clinical trials is an increasingly appealing approach for larger disease areas as well, as this can result in increased power and precision and can minimize the burden on patients in clinical trials. However, sponsors must assess whether the potential biases incurred with this approach outweigh the benefits and discuss this trade-off with the regulatory agencies. This paper discusses important points for the appropriate selection of historical controls for inclusion in the analysis of primary and/or key secondary endpoint(s) in clinical trials. The general steps are as follows: (1) Assess whether a trial is a suitable candidate for this approach. (2) If it is, then carefully identify appropriate historical trials to minimize selection bias. (3) Refine the historical control set if appropriate, for example, by selecting subsets of studies or patients. Identification of trial settings that are amenable to historical borrowing and selection of appropriate historical data using the principles discussed in this paper has the potential to lead to more efficient estimation and decision making. Ultimately, this efficiency gain results in lower patient burden and gets effective drugs to patients more quickly.

Clustered allocation as a way of understanding historical controls: Components of variation and regulatory considerations

There has been increasing interest in recent years in the possibility of increasing the efficiency of clinical trials by using historical controls. There has been a general recognition that in replacing concurrent by historical controls, the potential for bias is serious and requires some down-weighting to the apparent amount of historical information available. However, such approaches have generally assumed that what is required is some modification to the standard inferential model offered by the parallel group trial. In our opinion, the correct starting point that requires modification is a trial in which treatments are allocated to clusters. This immediately shows that the amount of information available is governed not just by the number of historical patients but also by the number of centres and of historical studies. Furthermore, once one accepts that external patients may be used as controls, this raises the issue as to which patients should be used. Thus, abandoning concurrent control has implications for many aspects of design and analysis of trials, including (a) identification, pre-specification and agreement on a suitable historical dataset; (b) an agreed, enforceable and checkable plan for recruiting the experimental arm; (c) a finalised analysis plan prior to beginning the trial and (d) use of a hierarchical model with sufficient complexity. We discuss these issues and suggest approaches to design and analysis making extensive reference to the partially randomised Therapeutic Arthritis Research and Gastrointestinal Event Trial study. We also compare some Bayesian and frequentist approaches and provide some important regulatory considerations. We conclude that effective use of historical data will require considerable circumspection and discipline.

Out of Control? Managing Baseline Variability in Experimental Studies with Control Groups

Control groups are expected to show what happens in the absence of the intervention of interest (negative control) or the effect of an intervention expected to have an effect (positive control). Although they usually give results we can anticipate, they are an essential component of all experiments, both in vitro and in vivo, and fulfil a number of important roles in any experimental design. Perhaps most importantly they help you understand the influence of variables that you cannot fully eliminate from your experiment and thus include them in your analysis of treatment effects. Because of this it is essential that they are treated as any other experimental group in terms of subjects, randomisation, blinding, etc. It also means that in almost all cases, contemporaneous control groups are required. Historical and baseline control groups serve a slightly different role and cannot fully replace control groups run as an integral part of the experiment. When used correctly, a good control group not only validates your experiment; it provides the basis for evaluating the effect of your treatments.

Design and analysis of a clinical trial using previous trials as historical control

BACKGROUND/AIMS

For single arm trials, a treatment is evaluated by comparing an outcome estimate to historically reported outcome estimates. Such a historically controlled trial is often analyzed as if the estimates from previous trials were known without variation and there is no trial-to-trial variation in their estimands. We develop a test of treatment efficacy and sample size calculation for historically controlled trials that considers these sources of variation.

METHODS

We fit a Bayesian hierarchical model, providing a sample from the posterior predictive distribution of the outcome estimand of a new trial, which, along with the standard error of the estimate, can be used to calculate the probability that the estimate exceeds a threshold. We then calculate criteria for statistical significance as a function of the standard error of the new trial and calculate sample size as a function of difference to be detected. We apply these methods to clinical trials for amyotrophic lateral sclerosis using data from the placebo groups of 16 trials.

RESULTS

We find that when attempting to detect the small to moderate effect sizes usually assumed in amyotrophic lateral sclerosis clinical trials, historically controlled trials would require a greater total number of patients than concurrently controlled trials, and only when an effect size is extraordinarily large is a historically controlled trial a reasonable alternative. We also show that utilizing patient level data for the prognostic covariates can reduce the sample size required for a historically controlled trial.

CONCLUSION

This article quantifies when historically controlled trials would not provide any sample size advantage, despite dispensing with a control group.

Disease Risk Score Approach

Many community-based translations of evidence-based interventions are designed as one-arm studies due to ethical and other considerations. Evaluating the impacts of such programs is challenging. Here, we examine the effectiveness of the lifestyle intervention implemented by the Special Diabetes Program for Indians Diabetes Prevention (SDPI-DP) demonstration project, a translational lifestyle intervention among American Indian and Alaska Native communities. Data from the landmark Diabetes Prevention Program placebo group was used as a historical control. We compared the use of propensity score (PS) and disease risk score (DRS) matching to adjust for potential confounder imbalance between groups. The unadjusted hazard ratio (HR) for diabetes risk was 0.35 for SDPI-DP lifestyle intervention vs. control. However, when relevant diabetes risk factors were considered, the adjusted HR estimates were attenuated toward 1, ranging from 0.56 (95% CI 0.44-0.71) to 0.69 (95% CI 0.56-0.96). The differences in estimated HRs using the PS and DRS approaches were relatively small but DRS matching resulted in more participants being matched and smaller standard errors of effect estimates. Carefully employed, publicly available randomized clinical trial data can be used as a historical control to evaluate the intervention effectiveness of one-arm community translational initiatives. It is critical to use a proper statistical method to balance the distributions of potential confounders between comparison groups in this kind of evaluations.

Evaluating Anti-Infective Drugs in the Resistant Pathogen Setting: Can we Use External Controls?

Decreased efficacy of antibiotics due to resistant pathogens has created a need for the development of more effective medical interventions. Despite the increasing prevalence of pathogens resistant to one or more drugs, identifying and enrolling participants into clinical trials that evaluate new interventions for the treatment of some diseases can be challenging given the low prevalence of disease in which there are no effective treatments. Thus researchers might be tempted to consider externally-controlled trials that may allow for a reduction of the necessary number of prospectively-identified trial participants, thus easing recruitment burden and resulting in more timely trial completion relative to randomized controlled trials. We discuss advantages and disadvantages in externally controlled trials and review requirements for a valid externally-controlled trial. As ECTs are subject to the bias of observational studies, the criteria for a valid ECT should be carefully evaluated before these designs are implemented. Given considerable variation in study results in the resistant pathogen setting, the lack of information on important patient characteristics that may confound estimates of treatment effects, as well as the improvements in medical practice and evolving antibiotic resistance, the use of ECTs in the resistant pathogen setting, is not recommended. ECTs should be should be limited to specific situations where superiority of the effect of the new intervention is dramatic, the usual course of the disease highly predictable, the endpoints are objective (e.g., all-cause mortality) and the impact of baseline and treatment variables on outcomes is well characterized. Given that the resistant pathogen setting does not satisfy these criteria, we conclude that that randomized clinical trials are needed to evaluate new treatments for resistant pathogens. Innovative approaches to trial design that may ease recruitment burden while evaluating the benefits and harms of new treatments are being developed and utilized.

Commensurate Priors on a Finite Mixture Model for Incorporating Repository Data in Clinical Trials

Docosahexaenoic acid (DHA) is a good source of fat that can be taken up through food, such as fish, or taken as a supplement. Evidence is building that DHA provides a high yield, low risk strategy to reduce preterm birth and/or low birth weight. These births are great costs to society. A recently completed phase III trial revealed that higher birth weight and gestational age were associated with DHA dosed at 600 mg/day. In this paper we take a posterior predictive approach to assess impacts of these findings on public health. Simple statistical models are not adequate for accurate posterior predictive distribution estimation. Of particular interest is a paper by Schwartz et al. (2010) who discovered that the joint distribution of birth weight and gestational age is well modeled by a finite mixture of three normal distributions. Data from our own clinical trial exhibit similar features. Using the mean and variance-covariance matrices from Schwartz et al. (2010) and flexible commensurate priors (Hobbs et al., 2012) for the mixing parameters, we estimate the effect of DHA supplementation on the over 20,000 infants born in hospitals demographically similar to the hospital where the clinical trial was conducted.