Missing Data: Turning Guidance Into Action

An illness-death multistate model to implement delta adjustment and reference-based imputation with time-to-event endpoints

With a treatment policy strategy, therapies are evaluated regardless of the disturbance caused by intercurrent events (ICEs). Implementing this estimand is challenging if subjects are not followed up after the ICE. This circumstance can be dealt with using delta adjustment (DA) or reference-based (RB) imputation. In the survival field, DA and RB imputation have been researched so far using multiple imputation (MI). Here, we present a fully analytical solution. We use the illness-death multistate model with the following transitions: (a) from the initial state to the event of interest, (b) from the initial state to the ICE, and (c) from the ICE to the event. We estimate the intensity function of transitions (a) and (b) using flexible parametric survival models. Transition (c) is assumed unobserved but identifiable using DA or RB imputation assumptions. Various rules have been considered: no ICE effect, DA under proportional hazards (PH) or additive hazards (AH), jump to reference (J2R), and (either PH or AH) copy increment from reference. We obtain the marginal survival curve of interest by calculating, via numerical integration, the probability of transitioning from the initial state to the event of interest regardless of having passed or not by the ICE state. We use the delta method to obtain standard errors (SEs). Finally, we quantify the performance of the proposed estimator through simulations and compare it against MI. Our analytical solution is more efficient than MI and avoids SE misestimation-a known phenomenon associated with Rubin's variance equation.

Integrating Intensive Longitudinal Data (ILD) to Inform the Development of Dynamic Theories of Behavior Change and Intervention Design: a Case Study of Scientific and Practical Considerations

The increasing sophistication of mobile and sensing technology has enabled the collection of intensive longitudinal data (ILD) concerning dynamic changes in an individual's state and context. ILD can be used to develop dynamic theories of behavior change which, in turn, can be used to provide a conceptual framework for the development of just-in-time adaptive interventions (JITAIs) that leverage advances in mobile and sensing technology to determine when and how to intervene. As such, JITAIs hold tremendous potential in addressing major public health concerns such as cigarette smoking, which can recur and arise unexpectedly. In tandem, a growing number of studies have utilized multiple methods to collect data on a particular dynamic construct of interest from the same individual. This approach holds promise in providing investigators with a significantly more detailed view of how a behavior change processes unfold within the same individual than ever before. However, nuanced challenges relating to coarse data, noisy data, and incoherence among data sources are introduced. In this manuscript, we use a mobile health (mHealth) study on smokers motivated to quit (Break Free; R01MD010362) to illustrate these challenges. Practical approaches to integrate multiple data sources are discussed within the greater scientific context of developing dynamic theories of behavior change and JITAIs.

Standard and reference-based conditional mean imputation

Clinical trials with longitudinal outcomes typically include missing data due to missed assessments or structural missingness of outcomes after intercurrent events handled with a hypothetical strategy. Approaches based on Bayesian random multiple imputation and Rubin's rules for pooling results across multiple imputed data sets are increasingly used in order to align the analysis of these trials with the targeted estimand. We propose and justify deterministic conditional mean imputation combined with the jackknife for inference as an alternative approach. The method is applicable to imputations under a missing-at-random assumption as well as for reference-based imputation approaches. In an application and a simulation study, we demonstrate that it provides consistent treatment effect estimates with the Bayesian approach and reliable frequentist inference with accurate standard error estimation and type I error control. A further advantage of the method is that it does not rely on random sampling and is therefore replicable and unaffected by Monte Carlo error.

Modified reference based imputation and tipping point analysis in the presence of missing data due to COVID-19

In longitudinal clinical trials, missing data are inevitable due to intercurrent events (ICEs) such as treatment interruption or premature discontinuation for different reasons. The COVID-19 pandemic has had substantial impact on clinical trials since early 2020 as it may result in missing data due to missed visits and premature discontinuations. The missing data due to COVID-19 can reasonably be assumed as missing at random (MAR).

We propose a combined hypothetical strategy for sensitivity analyses to handle missing data due to both COVID-19 and non-COVID reasons. We modify the commonly used missing not at random (MNAR) methods, reference based imputation (RBI) and tipping point analysis, under this strategy. We propose the standard multiple imputation approach and derive an analytic likelihood based approach to implement the proposed methods to improve efficiency in applications. The proposed strategy and methods are applicable to a more general scenario when there are missing data due to both MAR and MNAR reasons.

Estimand framework: Delineating what to be estimated with clinical questions of interest in clinical trials

ICH (International Council for Harmonization) E9 R1 (2019) proposes a framework to define estimands in clinical trials. Although the concept of estimand was proposed previously when US Food and Drug Administration (FDA) issued the panel report on handling missing data in clinical trials, many details including attributes and different strategies have not been developed until the recent ICH E9 (R1) addendum. A clearly defined estimand should include considerations of five attributes including patient population, treatment regimen of interest, endpoint/variables, handling of intercurrent events (IEs), and summary measures for assessing treatment effect. To evaluate the underlying treatment effects of a new investigational drug or biologic product, it is desirable to consider estimands that are aligned with the objectives of the study and that are meaningful to the stakeholders such as physicians or patients, health authority administration, and payers, etc.. In this paper, the concepts, attributes and strategies of the estimand framework will be reviewed and illustrated with clinical trial examples. Some common estimands and their associated scientific questions are discussed within a causal inference framework for longitudinal clinical trials.

Reference-based pattern-mixture models for analysis of longitudinal binary data

Pattern-mixture model (PMM)-based controlled imputations have become a popular tool to assess the sensitivity of primary analysis inference to different post-dropout assumptions or to estimate treatment effectiveness. The methodology is well established for continuous responses but less well established for binary responses. In this study, we formulate the copy-reference and jump-to-reference PMMs for longitudinal binary data using a multivariate probit model with latent variables. We discuss the maximum likelihood, Bayesian, and multiple imputation methods for estimating the treatment effect under the specified PMM. Simulation studies are conducted to evaluate the performance of these methods. These methods are also illustrated using data from a bipolar mania study.

Two-level approaches to missing data in longitudinal trials with daily patient-reported outcomes

In longitudinal clinical trials with daily patient-reported outcomes, the analysis endpoints are often defined as the averaged daily diary outcomes in a treatment cycle (such as a month or a week). Conventional methods often deal with missing data at the cycle level by imputing the average, and the cycle average is treated as missing if the number of days with available outcomes in the treatment cycle is less than a certain number. This was the method used for a case study of a phase 3 clinical trial evaluating a treatment for insomnia with daily patient-reported outcomes. Such methods may introduce bias. Motivated by this, we propose methods to impute missing daily outcomes in this paper. Specifically, we define a two-level missing pattern for clinical trials with daily patient-reported outcomes, and propose two-level methods to impute missing data at daily base. Other than the standard methods by multiple imputations, we derive analytic formulas for the proposed two-level methods to reduce computational intensity and improve the estimates of variances. The proposed two-level methods provide more powerful approaches to estimate the treatment difference compared to the conventional cycle-level methods, which are evaluated by theoretical development and simulation studies. In addition, the methods are applied to the motivating phase 3 trial evaluating a treatment for insomnia with daily patient-reported outcomes.

A monotone data augmentation algorithm for longitudinal data analysis via multivariate skew-t, skew-normal or t distributions

The mixed effects model for repeated measures has been widely used for the analysis of longitudinal clinical data collected at a number of fixed time points. We propose a robust extension of the mixed effects model for repeated measures for skewed and heavy-tailed data on basis of the multivariate skew-t distribution, and it includes the multivariate normal, t, and skew-normal distributions as special cases. An efficient Markov chain Monte Carlo algorithm is developed using the monotone data augmentation and parameter expansion techniques. We employ the algorithm to perform controlled pattern imputations for sensitivity analyses of longitudinal clinical trials with nonignorable dropouts. The proposed methods are illustrated by real data analyses. Sample SAS programs for the analyses are provided in the online supplementary material.

A Parent Treatment Program for Preschoolers With Obesity: A Randomized Controlled Trial

BACKGROUND AND OBJECTIVES

Early obesity treatment seems to be the most effective, but few treatments exist. In this study, we examine the effectiveness of a parent-only treatment program with and without booster sessions (Booster or No Booster) focusing on parenting practices and standard treatment (ST).

METHODS

Families of children 4 to 6 years of age with obesity were recruited from 68 child care centers in Stockholm County and randomly assigned to a parent-only program (10 weeks) with or without boosters (9 months) or to ST. Treatment effects on primary outcomes (BMI z score) and secondary outcomes (BMI and waist circumference) during a 12-month period were examined with linear mixed models. The influence of sociodemographic factors was examined by 3-way interactions. The clinically significant change in BMI z score (-0.5) was assessed with risk ratios.

RESULTS

A total of 174 children (mean age: 5.3 years [SD = 0.8]; BMI z score: 3.0 [SD = 0.6], 56% girls) and their parents (60% foreign background; 39% university degree) were included in the analysis (Booster, n = 44; No Booster, n = 43; ST, n = 87). After 12 months, children in the parent-only treatment had a greater reduction in their BMI z score (0.30; 95% confidence interval [CI]: -0.45 to -0.15) compared with ST (0.07; 95% CI: -0.19 to 0.05). Comparing all 3 groups, improvements in weight status were only seen for the Booster group (-0.54; 95% CI: -0.77 to -0.30). The Booster group was 4.8 times (95% CI: 2.4 to 9.6) more likely to reach a clinically significant reduction of ≥0.5 of the BMI z score compared with ST.

CONCLUSION

A parent-only treatment with boosters outperformed standard care for obesity in preschoolers.

A monotone data augmentation algorithm for multivariate nonnormal data: With applications to controlled imputations for longitudinal trials

An efficient monotone data augmentation (MDA) algorithm is proposed for missing data imputation for incomplete multivariate nonnormal data that may contain variables of different types and are modeled by a sequence of regression models including the linear, binary logistic, multinomial logistic, proportional odds, Poisson, negative binomial, skew-normal, skew-t regressions, or a mixture of these models. The MDA algorithm is applied to the sensitivity analyses of longitudinal trials with nonignorable dropout using the controlled pattern imputations that assume the treatment effect reduces or disappears after subjects in the experimental arm discontinue the treatment. We also describe a heuristic approach to implement the controlled imputation, in which the fully conditional specification method is used to impute the intermediate missing data to create a monotone missing pattern, and the missing data after dropout are then imputed according to the assumed nonignorable mechanisms. The proposed methods are illustrated by simulation and real data analyses. Sample SAS code for the analyses is provided in the supporting information.

On the multiple imputation variance estimator for control-based and delta-adjusted pattern mixture models

Control-based pattern mixture models (PMM) and delta-adjusted PMMs are commonly used as sensitivity analyses in clinical trials with non-ignorable dropout. These PMMs assume that the statistical behavior of outcomes varies by pattern in the experimental arm in the imputation procedure, but the imputed data are typically analyzed by a standard method such as the primary analysis model. In the multiple imputation (MI) inference, Rubin's variance estimator is generally biased when the imputation and analysis models are uncongenial. One objective of the article is to quantify the bias of Rubin's variance estimator in the control-based and delta-adjusted PMMs for longitudinal continuous outcomes. These PMMs assume the same observed data distribution as the mixed effects model for repeated measures (MMRM). We derive analytic expressions for the MI treatment effect estimator and the associated Rubin's variance in these PMMs and MMRM as functions of the maximum likelihood estimator from the MMRM analysis and the observed proportion of subjects in each dropout pattern when the number of imputations is infinite. The asymptotic bias is generally small or negligible in the delta-adjusted PMM, but can be sizable in the control-based PMM. This indicates that the inference based on Rubin's rule is approximately valid in the delta-adjusted PMM. A simple variance estimator is proposed to ensure asymptotically valid MI inferences in these PMMs, and compared with the bootstrap variance. The proposed method is illustrated by the analysis of an antidepressant trial, and its performance is further evaluated via a simulation study.

On inference of control-based imputation for analysis of repeated binary outcomes with missing data

Missing data are common in longitudinal clinical trials. How to handle missing data is critical for both sponsors and regulatory agencies to assess treatment effect from the trials. Recently, a control-based imputation has been proposed, where the missing data are imputed based on the assumption that patients who discontinued the test drug will have a similar response profile to the patients in the control group. Under control-based imputation, the variance estimation may be biased using Rubin's formula which could produce biased statistical inferences. We evaluate several statistical methods for obtaining appropriate variances under control-based imputation for analysis of repeated binary outcomes with monotone missing data and show that both the analytical method developed by Robins & Wang and the nonparametric bootstrap method provide more appropriate variance estimates under various simulation settings. We use the methods in an application of an antidepressant Phase III clinical trial and give discussion and recommendations on method performance and preference.

Current Practices in Choosing Estimands and Sensitivity Analyses in Clinical Trials: Results of the ICH E9 Survey

BACKGROUND

An addendum to the International Conference on Harmonisation E9 (ICH E9) guidance document (Statistical Principles for Clinical Trials) is currently under development. The aim of the addendum is to promote harmonized standards on the choice of estimand (a well-defined measure of the treatment effect that is being estimated) in clinical trials and to describe a consensual framework for planning, conducting, and interpreting sensitivity analyses of clinical trial data.

METHODS

In order to help understand current practices relating to the choice of estimands and sensitivity analyses for clinical trials, the ICH E9 working group developing the addendum conducted a survey with a primary focus on clinical trials involving drugs, vaccines, and biologics. The survey was distributed electronically between May 19, 2015, and June 11, 2015, to various stakeholder groups within ICH, including industry, regulatory, and academic communities. A total of 1305 respondents participated.

RESULTS

Of the 1305 respondents 547 (42%), 344 (26%) and 283 (22%) were from Europe, USA and Japan respectively. Over half of the respondents work in pharmaceutical companies, and approximately a quarter of respondents noted oncology as the primary therapeutic area they work in. Over half of the respondents (595, 55%) noted the treatment effect being estimated was 'in the entire target population of patients regardless of whether they will take treatment as instructed'. The most common methods for handling missing data in primary analyses were mixed-models repeated measures (555, 56% respondents) and last observation carried forward (549, 55% respondents). The majority of respondents (816, 83%) noted they conducted sensitivity analyses to estimate treatment effects in different ways compared to the primary analysis by using alternative assumptions (627, 78%) and/or using alternative statistical methods (616, 76%).

CONCLUSIONS

The survey results have provided useful information to the ICH E9 working group on current practices on the choice of primary estimands for measuring treatment effects in confirmatory clinical trials, and approaches used to select sensitivity analyses.

Good intentions and ICH-GCP: Trial conduct training needs to go beyond the ICH-GCP document and include the intention-to-treat principle

BACKGROUND

It is assumed investigators and statisticians fully understand the importance of avoiding missing outcomes and the intention-to-treat principle during design and analysis phases of a randomised controlled trial in order to obtain the most valuable and reliable results. However, many personnel undertaking day-to-day trial conduct and data collection commonly rely exclusively for guidance on the widely implemented, indeed regulated, International Conference on Harmonisation-Good Clinical Practice document as the guideline and standard for trial conduct.

PURPOSE

This article describes adverse consequences of omission of intention-to-treat principles from training for trial personnel and explores the need for training in addition to the International Conference on Harmonisation-Good Clinical Practice guideline document.

METHODS

Data from the Breast Boost Study were used to illustrate a comparison of actual results, where vigilant senior investigators re-enforced intention-to-treat requirements throughout all aspects of trial conduct with results that could easily have occurred if study personnel did not understand the importance of intention-to-treat principles. Experience as a co-ordinating centre for an international trial (Trans-Tasman Radiation Oncology Group 08.06 Breast STARS) acted as an audit of data-management culture regarding intention-to-treat in Australia and New Zealand.

RESULTS

Despite the Breast Boost Study exceeding planned accrual, it was demonstrated that the study, which found a statistically significant result, could have reported a negative or inconclusive result under the scenario of trial conduct personnel having lack of understanding of the importance of avoiding losses to follow-up. Trans-Tasman Radiation Oncology 08.06 co-ordination experience verified that data-management culture in Australia and New Zealand does not adequately recognise intention-to-treat principles, and this is reflected in trial conduct.

LIMITATIONS

Trial data described are limited to two trials and in the Australian and New Zealand setting.

CONCLUSION

To be both scientifically and ethically valid, guidelines for trial conduct should include and stress the importance of the intention-to-treat principle and in particular avoiding missing outcomes. Our discussion highlights the vitally important role played by personnel involved in day-to-day trial conduct. Inclusion of scientific principles in guideline documents and/or training which goes beyond International Conference on Harmonisation-Good Clinical Practice to include intention-to-treat is essential to achieve robust research results. Related aspects of randomised trial consent and ethics are discussed.

Missing data sensitivity analysis for recurrent event data using controlled imputation

Statistical analyses of recurrent event data have typically been based on the missing at random assumption. One implication of this is that, if data are collected only when patients are on their randomized treatment, the resulting de jure estimator of treatment effect corresponds to the situation in which the patients adhere to this regime throughout the study. For confirmatory analysis of clinical trials, sensitivity analyses are required to investigate alternative de facto estimands that depart from this assumption. Recent publications have described the use of multiple imputation methods based on pattern mixture models for continuous outcomes, where imputation for the missing data for one treatment arm (e.g. the active arm) is based on the statistical behaviour of outcomes in another arm (e.g. the placebo arm). This has been referred to as controlled imputation or reference-based imputation. In this paper, we use the negative multinomial distribution to apply this approach to analyses of recurrent events and other similar outcomes. The methods are illustrated by a trial in severe asthma where the primary endpoint was rate of exacerbations and the primary analysis was based on the negative binomial model.