Analysis of ordered composite endpoints

Use of win time for ordered composite endpoints in clinical trials

Consider the choice of outcome for overall treatment benefit in a clinical trial which measures the first time to each of several clinical events. We describe several new variants of the win ratio that incorporate the time spent in each clinical state over the common follow-up, where clinical state means the worst clinical event that has occurred by that time. One version allows restriction so that death during follow-up is most important, while time spent in other clinical states is still accounted for. Three other variants are described; one is based on the average pairwise win time, one creates a continuous outcome for each participant based on expected win times against a reference distribution and another that uses the estimated distributions of clinical state to compare the treatment arms. Finally, a combination testing approach is described to give robust power for detecting treatment benefit across a broad range of alternatives. These new methods are designed to be closer to the overall treatment benefit/harm from a patient's perspective, compared to the ordinary win ratio. The new methods are compared to the composite event approach and the ordinary win ratio. Simulations show that when overall treatment benefit on death is substantial, the variants based on either the participants' expected win times (EWTs) against a reference distribution or estimated clinical state distributions have substantially higher power than either the pairwise comparison or composite event methods. The methods are illustrated by re-analysis of the trial heart failure: a controlled trial investigating outcomes of exercise training.

Desirability of Outcome Ranking for Status Epilepticus: A Benefit-Risk Approach to Design and Analyses of Clinical SE Trials

Most clinical trials of treatment efficacy evaluate benefits and harms separately. Investigators generally rate the primary outcome of a trial with a binary outcome measure and consider harms separately as adverse events. This approach fails to recognize finer gradations of patient response, correlations between benefits and harms, and the overall effects on individual patients. For example, in status epilepticus trials, efficacy is often defined as the absence of clinically apparent seizures with recovery of consciousness. Such an efficacy outcome fails to recognize that some causes of status epilepticus, such as subarachnoid hemorrhage or stroke, may not be accompanied by return of consciousness, and the need to intubate a patient may be classified as treatment failure even if status was successfully terminated. The Desirability of Outcome Ranking (DOOR) method uses a different approach. The DOOR method involves comparing the experiences of trial participants in different treatment arms by the desirability of the overall patient outcome. Using status epilepticus treatment as an example, a patient who experiences successful termination of status epilepticus but with major side effects would have a less desirable outcome than a patient with treatment success and minor side effects, who in turn would have a less desirable outcome than a patient with treatment success but no side effects. This is a patient-centered approach because it considers treatment efficacy in the context of the costs borne by the patient, for example, toxicity in achieving efficacy. Thus, DOOR considers both the benefits and harms to individual patients in assessing the outcome of a clinical trial. In this article, we present the rationale for the use of DOOR, the issues involved in the development of and statistical analyses of an ordinal outcome, and an example of the potential application of the DOOR method to a clinical trial of convulsive status epilepticus.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Multiparametric Quantitative Imaging Biomarker as a Multivariate Descriptor of Health: A Roadmap

Multiparametric quantitative imaging biomarkers (QIBs) offer distinct advantages over single, univariate descriptors because they provide a more complete measure of complex, multidimensional biological systems. In disease, where structural and functional disturbances occur across a multitude of subsystems, multivariate QIBs are needed to measure the extent of system malfunction. This paper, the first Use Case in a series of articles on multiparameter imaging biomarkers, considers multiple QIBs as a multidimensional vector to represent all relevant disease constructs more completely. The approach proposed offers several advantages over QIBs as multiple endpoints and avoids combining them into a single composite that obscures the medical meaning of the individual measurements. We focus on establishing statistically rigorous methods to create a single, simultaneous measure from multiple QIBs that preserves the sensitivity of each univariate QIB while incorporating the correlation among QIBs. Details are provided for metrological methods to quantify the technical performance. Methods to reduce the set of QIBs, test the superiority of the mp-QIB model to any univariate QIB model, and design study strategies for generating precision and validity claims are also provided. QIBs of Alzheimer's Disease from the ADNI merge data set are used as a case study to illustrate the methods described.

A desirability of outcome ranking (DOOR) for periprosthetic joint infection - a Delphi analysis

Background: Treatment outcomes in studies on prosthetic joint infection are generally assessed using a dichotomous outcome relating to treatment success or failure. These outcome measures neither include patient-centred outcome measures including joint function and quality of life, nor do they account for adverse effects of treatment. A desirability of outcome ranking (DOOR) measure can include these factors and has previously been proposed and validated for other serious infections. We aimed to develop a novel DOOR for prosthetic joint infections (PJIs). Methods: The Delphi method was used to develop a DOOR for PJI research. An international working group of 18 clinicians (orthopaedic surgeons and infectious disease specialists) completed the Delphi process. The final DOOR comprised the dimensions established to be most important by consensus with > 75  % of participant agreement. Results: The consensus DOOR comprised four main dimensions. The primary dimension was patient-reported joint function. The secondary dimensions were infection cure and mortality. The final dimension of quality of life was selected as a tie-breaker. Discussion: A desirability of outcome ranking for periprosthetic joint infection has been proposed. It focuses on patient-centric outcome measures of joint function, cure and quality of life. This DOOR provides a multidimensional assessment to comprehensively rank outcomes when comparing treatments for prosthetic joint infection.

A Composite Endpoint for Treatment Benefit According to Patient Preference

Complex disorders usually affect multiple symptom domains measured by several outcomes. The importance of these outcomes is often different among patients. Current approaches integrate multiple outcomes without considering patient preferences at the individual level. In this paper, we propose a new composite Desirability of Outcome Ranking (DOOR) that integrates individual level ranking of outcome importance and define a winning probability measuring the overall treatment effect. Stratified randomization can be performed based on the participants' baseline outcome rankings. A Wilcoxon-Mann-Whitney U-statistic is used to average the pairwise DOOR between one treated and one control patient, considering the difference in these patients' ranking of outcome importance. We use both theoretical and empirical methods to examine the statistical properties of our method and to compare with conventional approaches. We conclude that the proposed composite DOOR properly reflects patient-level preferences and can be used in pivotal trials or comparative effectiveness trials for a patient-centered evaluation of overall treatment benefits.

Statistical models for composite endpoints of death and non-fatal events: a review

The proper analysis of composite endpoints consisting of both death and non-fatal events is an intriguing and sometimes contentious topic. The current practice of analyzing time to the first event often draws criticisms for ignoring the unequal importance between component events and for leaving recurrent-event data unused. Novel methods that address these limitations have recently been proposed. To compare the novel versus traditional approaches, we review three typical models for composite endpoints based on time to the first event, composite event process, and pairwise hierarchical comparisons. The pros and cons of these models are discussed with reference to the relevant regulatory guidelines, such as the recently released ICH-E9(R1) Addendum "Estimands and Sensitivity Analysis in Clinical Trials". We also discuss the impact of censoring when the model assumptions are violated and explore sensitivity analysis strategies. Simulation studies are conducted to assess the performance of the reviewed methods under different settings. As demonstration, we use publicly available R-packages to analyze real data from a major cardiovascular trial.

Pharmacological Prevention of Neonatal Opioid Withdrawal in a Pregnant Guinea Pig Model

Newborns exposed to prenatal opioids often experience intense postnatal withdrawal after cessation of the opioid, called neonatal opioid withdrawal syndrome (NOWS), with limited pre- and postnatal therapeutic options available. In a prior study in pregnant mice we demonstrated that the peripherally selective opioid antagonist, 6β-naltrexol (6BN), is a promising drug candidate for preventive prenatal treatment of NOWS, and a therapeutic mechanism was proposed based on preferential delivery of 6BN to fetal brain with relative exclusion from maternal brain. Here, we have developed methadone (MTD) treated pregnant guinea pigs as a physiologically more suitable model, enabling detection of robust spontaneous neonatal withdrawal. Prenatal MTD significantly aggravates two classic maternal separation stress behaviors in newborn guinea pigs: calling (vocalizing) and searching (locomotion) - natural attachment behaviors thought to be controlled by the endogenous opioid system. In addition, prenatal MTD significantly increases the levels of plasma cortisol in newborns, showing that cessation of MTD at birth engages the hypothalamic-pituitary-adrenal (HPA) axis. We find that co-administration of 6BN with MTD prevents these withdrawal symptoms in newborn pups with extreme potency (ID50 ∼0.02 mg/kg), at doses unlikely to induce maternal or fetal withdrawal or to interfere with opioid antinociception based on many prior studies in rodents and non-human primates. Furthermore, we demonstrate a similarly high potency of 6BN in preventing opioid withdrawal in adult guinea pigs (ID50 = 0.01 mg/kg). This high potency appears to run counter to our pharmacokinetic studies showing slow 6BN transit of both the placenta and maternal blood brain barrier in guinea pigs, and calls into question the preferential delivery mechanism. Rather, it suggests a novel receptor mechanism to account for the selectively high potency of 6BN to suppress opioid dependence at all developmental stages, even in adults, as compared to its well-established low potency as a classical opioid antagonist. In conclusion, 6BN is an attractive compound for development of a preventive therapy for NOWS.

A class of proportional win-fractions regression models for composite outcomes

The win ratio is gaining traction as a simple and intuitive approach to analysis of prioritized composite endpoints in clinical trials. To extend it from two-sample comparison to regression, we propose a novel class of semiparametric models that includes as special cases both the two-sample win ratio and the traditional Cox proportional hazards model on time to the first event. Under the assumption that the covariate-specific win and loss fractions are proportional over time, the regression coefficient is unrelated to the censoring distribution and can be interpreted as the log win ratio resulting from one-unit increase in the covariate. U-statistic estimating functions, in the form of an arbitrary covariate-specific weight process integrated by a pairwise residual process, are constructed to obtain consistent estimators for the regression parameter. The asymptotic properties of the estimators are derived using uniform weak convergence theory for U-processes. Visual inspection of a "score" process provides useful clues as to the plausibility of the proportionality assumption. Extensive numerical studies using both simulated and real data from a major cardiovascular trial show that the regression methods provide valid inference on covariate effects and outperform the two-sample win ratio in both efficiency and robustness. The proposed methodology is implemented in the R-package WR, publicly available from the Comprehensive R Archive Network (CRAN).

Methodologies for pragmatic and efficient assessment of benefits and harms: Application to the SOCRATES trial

BACKGROUND/AIMS

Standard approaches to trial design and analyses can be inefficient and non-pragmatic. Failure to consider a range of outcomes impedes evidence-based interpretation and reduces power. Traditional approaches synthesizing information obtained from separate analysis of each outcome fail to incorporate associations between outcomes and recognize the cumulative nature of outcomes in individual patients, suffer from competing risk complexities during interpretation, and since efficacy and safety analyses are often conducted on different populations, generalizability is unclear. Pragmatic and efficient approaches to trial design and analyses are needed.

METHODS

Approaches providing a pragmatic assessment of benefits and harms of interventions, summarizing outcomes experienced by patients, and providing sample size efficiencies are described. Ordinal outcomes recognize finer gradations of patient responses. Desirability of outcome ranking is an ordinal outcome combining benefits and harms within patients. Analysis of desirability of outcome ranking can be based on rank-based methodologies including the desirability of outcome ranking probability, the win ratio, and the proportion in favor of treatment. Partial credit analyses, involving grading the levels of the desirability of outcome ranking outcome similar to an academic test, provides an alternative approach. The methodologies are demonstrated using the acute stroke or transient ischemic attack treated with aspirin or ticagrelor and patient outcomes study (SOCRATES; NCT01994720), a randomized clinical trial.

RESULTS

Two 5-level ordinal outcomes were developed for SOCRATES. The first was based on a modified Rankin scale. The odds ratio is 0.86 (95% confidence interval = 0.75, 0.99; p = 0.04) indicating that the odds of worse stroke categorization for a trial participant assigned to ticagrelor is 0.86 times that of a trial participant assigned to aspirin. The 5-level desirability of outcome ranking outcome incorporated and prioritized survival; the number of strokes, myocardial infarction, and major bleeding events; and whether a stroke event was disabling. The desirability of outcome ranking probability and win ratio are 0.504 (95% confidence interval = 0.499, 0.508; p = 0.10) and 1.11 (95% confidence interval = 0.98, 1.26; p = 0.10), respectively, implying that the probability of a more desirable result with ticagrelor is 50.4% and that a more desirable result occurs 1.11 times more frequently on ticagrelor versus aspirin.

CONCLUSION

Ordinal outcomes can improve efficiency through required pre-specification, careful construction, and analyses. Greater pragmatism can be obtained by composing outcomes within patients. Desirability of outcome ranking provides a global assessment of the benefits and harms that more closely reflect the experience of patients. The desirability of outcome ranking probability, the proportion in favor of treatment, the win ratio, and partial credit can more optimally inform patient treatment, enhance the understanding of the totality of intervention effects on patients, and potentially provide efficiencies over standard analyses. The methods provide the infrastructure for incorporating patient values and estimating personalized effects.