Generated effect modifiers (GEM's) in randomized clinical trials

A Bayesian multivariate hierarchical model for developing a treatment benefit index using mixed types of outcomes

BACKGROUND

Precision medicine has led to the development of targeted treatment strategies tailored to individual patients based on their characteristics and disease manifestations. Although precision medicine often focuses on a single health outcome for individualized treatment decision rules (ITRs), relying only on a single outcome rather than all available outcomes information leads to suboptimal data usage when developing optimal ITRs.

METHODS

To address this limitation, we propose a Bayesian multivariate hierarchical model that leverages the wealth of correlated health outcomes collected in clinical trials. The approach jointly models mixed types of correlated outcomes, facilitating the "borrowing of information" across the multivariate outcomes, and results in a more accurate estimation of heterogeneous treatment effects compared to using single regression models for each outcome. We develop a treatment benefit index, which quantifies the relative benefit of the experimental treatment over the control treatment, based on the proposed multivariate outcome model.

RESULTS

We demonstrate the strengths of the proposed approach through extensive simulations and an application to an international Coronavirus Disease 2019 (COVID-19) treatment trial. Simulation results indicate that the proposed method reduces the occurrence of erroneous treatment decisions compared to a single regression model for a single health outcome. Additionally, the sensitivity analyses demonstrate the robustness of the model across various study scenarios. Application of the method to the COVID-19 trial exhibits improvements in estimating the individual-level treatment efficacy (indicated by narrower credible intervals for odds ratios) and optimal ITRs.

CONCLUSION

The study jointly models mixed types of outcomes in the context of developing ITRs. By considering multiple health outcomes, the proposed approach can advance the development of more effective and reliable personalized treatment.

Improving Individualized Treatment Decisions: A Bayesian Multivariate Hierarchical Model for Developing a Treatment Benefit Index using Mixed Types of Outcomes

Background

Precision medicine has led to the development of targeted treatment strategies tailored to individual patients based on their characteristics and disease manifestations. Although precision medicine often focuses on a single health outcome for individualized treatment decision rules (ITRs), relying only on a single outcome rather than all available outcomes information leads to suboptimal data usage when developing optimal ITRs.

Methods

To address this limitation, we propose a Bayesian multivariate hierarchical model that leverages the wealth of correlated health outcomes collected in clinical trials. The approach jointly models mixed types of correlated outcomes, facilitating the "borrowing of information" across the multivariate outcomes, and results in a more accurate estimation of heterogeneous treatment effects compared to using single regression models for each outcome. We develop a treatment benefit index, which quantifies the relative treatment benefit of the experimental treatment over the control treatment, based on the proposed multivariate outcome model.

Results

We demonstrate the strengths of the proposed approach through extensive simulations and an application to an international Coronavirus Disease 2019 (COVID-19) treatment trial. Simulation results indicate that the proposed method reduces the occurrence of erroneous treatment decisions compared to a single regression model for a single health outcome. Additionally, the sensitivity analysis demonstrates the robustness of the model across various study scenarios. Application of the method to the COVID-19 trial exhibits improvements in estimating the individual-level treatment efficacy (indicated by narrower credible intervals for odds ratios) and optimal ITRs.

Conclusion

The study jointly models mixed types of outcomes in the context of developing ITRs. By considering multiple health outcomes, the proposed approach can advance the development of more effective and reliable personalized treatment.

A Single Index Model for Longitudinal Outcomes to Optimize Individual Treatment Decision Rules

A pressing challenge in medical research is to identify optimal treatments for individual patients. This is particularly challenging in mental health settings where mean responses are often similar across multiple treatments. For example, the mean longitudinal trajectories for patients treated with an active drug and placebo may be very similar but different treatments may exhibit distinctly different individual trajectory shapes. Most precision medicine approaches using longitudinal data often ignore information from the longitudinal data structure. This paper investigates a powerful precision medicine approach by examining the impact of baseline covariates on longitudinal outcome trajectories to guide treatment decisions instead of traditional scalar outcome measures derived from longitudinal data, such as a change score. We introduce a method of estimating "biosignatures" defined as linear combinations of baseline characteristics (i.e., a single index) that optimally separate longitudinal trajectories among different treatment groups. The criterion used is to maximize the Kullback-Leibler Divergence between different treatment outcome distributions. The approach is illustrated via simulation studies and a depression clinical trial. The approach is also contrasted with more traditional methods and compares performance in the presence of missing data.

Development and Validation of a Treatment Benefit Index to Identify Hospitalized Patients With COVID-19 Who May Benefit From Convalescent Plasma

Importance

Identifying which patients with COVID-19 are likely to benefit from COVID-19 convalescent plasma (CCP) treatment may have a large public health impact.

Objective

To develop an index for predicting the expected relative treatment benefit from CCP compared with treatment without CCP for patients hospitalized for COVID-19 using patients' baseline characteristics.

Design, Setting, and Participants

This prognostic study used data from the COMPILE study, ie, a meta-analysis of pooled individual patient data from 8 randomized clinical trials (RCTs) evaluating CCP vs control in adults hospitalized for COVID-19 who were not receiving mechanical ventilation at randomization. A combination of baseline characteristics, termed the treatment benefit index (TBI), was developed based on 2287 patients in COMPILE using a proportional odds model, with baseline characteristics selected via cross-validation. The TBI was externally validated on 4 external data sets: the Expanded Access Program (1896 participants), a study conducted under Emergency Use Authorization (210 participants), and 2 RCTs (with 80 and 309 participants).

Exposure

Receipt of CCP.

Main Outcomes and Measures

World Health Organization (WHO) 11-point ordinal COVID-19 clinical status scale and 2 derivatives of it (ie, WHO score of 7-10, indicating mechanical ventilation to death, and WHO score of 10, indicating death) at day 14 and day 28 after randomization. Day 14 WHO 11-point ordinal scale was used as the primary outcome to develop the TBI.

Results

A total of 2287 patients were included in the derivation cohort, with a mean (SD) age of 60.3 (15.2) years and 815 (35.6%) women. The TBI provided a continuous gradation of benefit, and, for clinical utility, it was operationalized into groups of expected large clinical benefit (B1; 629 participants in the derivation cohort [27.5%]), moderate benefit (B2; 953 [41.7%]), and potential harm or no benefit (B3; 705 [30.8%]). Patients with preexisting conditions (diabetes, cardiovascular and pulmonary diseases), with blood type A or AB, and at an early COVID-19 stage (low baseline WHO scores) were expected to benefit most, while those without preexisting conditions and at more advanced stages of COVID-19 could potentially be harmed. In the derivation cohort, odds ratios for worse outcome, where smaller odds ratios indicate larger benefit from CCP, were 0.69 (95% credible interval [CrI], 0.48-1.06) for B1, 0.82 (95% CrI, 0.61-1.11) for B2, and 1.58 (95% CrI, 1.14-2.17) for B3. Testing on 4 external datasets supported the validation of the derived TBIs.

Conclusions and Relevance

The findings of this study suggest that the CCP TBI is a simple tool that can quantify the relative benefit from CCP treatment for an individual patient hospitalized with COVID-19 that can be used to guide treatment recommendations. The TBI precision medicine approach could be especially helpful in a pandemic.

The concept of justifiable healthcare and how big data can help us to achieve it

Over the last decades, the face of health care has changed dramatically, with big improvements in what is technically feasible. However, there are indicators that the current approach to evaluating evidence in health care is not holistic and hence in the long run, health care will not be sustainable. New conceptual and normative frameworks for the evaluation of health care need to be developed and investigated. The current paper presents a novel framework of justifiable health care and explores how the use of artificial intelligence and big data can contribute to achieving the goals of this framework.

Predictive approaches to heterogeneous treatment effects: a scoping review

BACKGROUND

Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial.

METHODS

We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel.

RESULTS

The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers).

CONCLUSIONS

Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Predicting Depression Symptoms in Families at Risk for Depression: Interrelations of Posterior EEG Alpha and Religion/Spirituality

BACKGROUND

Posterior EEG alpha has been identified as a putative biomarker of clinical outcomes in major depression (MDD). Separately, personal importance of religion and spirituality (R/S) has been shown to provide protective benefits for individuals at high familial risk for MDD. This study directly explored the joint value of posterior alpha and R/S on predicting clinical health outcomes of depression.

METHODS

Using a mixed-effects model approach, we obtained virtual estimates of R/S at age 21 using longitudinal data collected at 5 timepoints spanning 25 years. Current source density and frequency principal component analysis was used to quantify posterior alpha in 72-channel resting EEG (eyes open/closed). Depression severity was measured between 5 and 10 years after EEG collection using PHQ-9 and IDAS-GD scales.

RESULTS

Greater R/S (p = .008, η²_p = 0.076) and higher alpha (p = .02, η²_p = 0.056) were separately associated with fewer symptoms across scales. However, an interaction between alpha and R/S (p = .02, η²_p = 0.062) was observed, where greater R/S predicted fewer symptoms with low alpha but high alpha predicted fewer symptoms with lower R/S.

LIMITATIONS

Small-to-medium effect sizes and homogeneity of sample demographics caution overall interpretation and generalizability.

CONCLUSIONS

Findings revealed a complementary role of R/S and alpha in that either variable exerted protective effects only if the other was present at low levels. These findings confirm the relevance of R/S importance and alpha oscillations as predictors of depression symptom severity. More research is needed on the neurobiological mechanism underlying the protective effects of R/S importance for MDD.

A single-index model with multiple-links

In a regression model for treatment outcome in a randomized clinical trial, a treatment effect modifier is a covariate that has an interaction with the treatment variable, implying that the treatment efficacies vary across values of such a covariate. In this paper, we present a method for determining a composite variable from a set of baseline covariates, that can have a nonlinear association with the treatment outcome, and acts as a composite treatment effect modifier. We introduce a parsimonious generalization of the single-index models that targets the effect of the interaction between the treatment conditions and the vector of covariates on the outcome, a single-index model with multiple-links (SIMML) that estimates a single linear combination of the covariates (i.e., a single-index), with treatment-specific nonparametric link functions. The approach emphasizes a focus on the treatment-by-covariates interaction effects on the treatment outcome that are relevant for making optimal treatment decisions. Asymptotic results for estimator are obtained under possible model misspecification. A treatment decision rule based on the derived single-index is defined, and it is compared to other methods for estimating optimal treatment decision rules. An application to a clinical trial for the treatment of depression is presented.

The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration

The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneity of treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risk with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made on the basis of only an overall average treatment effect. The authors distinguished 2 categories of predictive HTE approaches (a "risk-modeling" and an "effect-modeling" approach) and developed 4 sets of guidance statements: criteria to determine when risk-modeling approaches are likely to identify clinically meaningful HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. They discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation and discusses related analytic considerations, caveats, and reservations.

Optimising treatment decision rules through generated effect modifiers: a precision medicine tutorial

This tutorial introduces recent developments in precision medicine for estimating treatment decision rules. The objective of these developments is to advance personalised healthcare by identifying an optimal treatment option for each individual patient based on each patient's characteristics. The methods detailed in this tutorial define composite variables from the patient measures that can be viewed as 'biosignatures' for differential treatment response, which we have termed 'generated effect modifiers'. In contrast to most machine learning approaches to precision medicine, these biosignatures are derived from linear and non-linear regression models and thus have the advantage of easy visualisation and ready interpretation. The methods are illustrated using examples from randomised clinical trials.

A Machine Learning Approach to Identifying Placebo Responders in Late-Life Depression Trials

OBJECTIVE

Despite efforts to identify characteristics associated with medication-placebo differences in antidepressant trials, few consistent findings have emerged to guide participant selection in drug development settings and differential therapeutics in clinical practice. Limitations in the methodologies used, particularly searching for a single moderator while treating all other variables as noise, may partially explain the failure to generate consistent results. The present study tested whether interactions between pretreatment patient characteristics, rather than a single-variable solution, may better predict who is most likely to benefit from placebo versus medication.

METHODS

Data were analyzed from 174 patients aged 75 years and older with unipolar depression who were randomly assigned to citalopram or placebo. Model-based recursive partitioning analysis was conducted to identify the most robust significant moderators of placebo versus citalopram response.

RESULTS

The greatest signal detection between medication and placebo in favor of medication was among patients with fewer years of education (≤12) who suffered from a longer duration of depression since their first episode (>3.47 years) (B = 2.53, t(32) = 3.01, p = 0.004). Compared with medication, placebo had the greatest response for those who were more educated (>12 years), to the point where placebo almost outperformed medication (B = -0.57, t(96) = -1.90, p = 0.06).

CONCLUSION

Machine learning approaches capable of evaluating the contributions of multiple predictor variables may be a promising methodology for identifying placebo versus medication responders. Duration of depression and education should be considered in the efforts to modulate placebo magnitude in drug development settings and in clinical practice.

Statistical Analysis Plan for Stage 1 EMBARC (Establishing Moderators and Biosignatures of Antidepressant Response for Clinical Care) Study

Antidepressant medications are commonly used to treat depression, but only about 30% of patients reach remission with any single first-step antidepressant. If the first-step treatment fails, response and remission rates at subsequent steps are even more limited. The literature on biomarkers for treatment response is largely based on secondary analyses of studies designed to answer primary questions of efficacy, rather than on a planned systematic evaluation of biomarkers for treatment decision. The lack of evidence-based knowledge to guide treatment decisions for patients with depression has lead to the recognition that specially designed studies with the primary objective being to discover biosignatures for optimizing treatment decisions are necessary. Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) is one such discovery study. Stage 1 of EMBARC is a randomized placebo controlled clinical trial of 8 week duration. A wide array of patient characteristics is collected at baseline, including assessments of brain structure, function and connectivity along with electrophysiological, biological, behavioral and clinical features. This paper reports on the data analytic strategy for discovering biosignatures for treatment response based on Stage 1 of EMBARC.

An OMERACT Initiative Toward Consensus to Identify and Characterize Candidate Contextual Factors: Report from the Contextual Factors Working Group

Objective.

The importance of contextual factors (CF) for appropriate patient-specific care is widely acknowledged. However, evidence in clinical trials on how CF influence outcomes remains sparse. The 2014 Outcome Measures in Rheumatology (OMERACT) Handbook introduced the role of CF in outcome assessment and defined them as “potential confounders and/or effect modifiers of outcomes in randomized controlled trials.” Subsequently, the CF Methods Group (CFMG) was formed to develop guidance on how to address CF in clinical trials.

Methods.

First, the CFMG conducted an e-mail survey of OMERACT working groups (WG) to analyze how they had addressed CF in outcome measurement so far. The results facilitated an informed discussion at the OMERACT 2016 CFMG Special Interest Group (SIG) session, with the aim of gaining preliminary consensus regarding an operational definition of CF and to make a first selection of potentially relevant CF.

Results.

The survey revealed that the WG had mostly used the OMERACT Handbook and/or the International Classification of Functioning, Disability and Health (ICF) definition. However, significant heterogeneity was found in the methods used to identify, refine, and categorize CF candidates. The SIG participants agreed on using the ICF as a framework along with the OMERACT Handbook definition. A list with 28 variables was collected including person-related factors and physical and social environments. Recommendations from the SIG guided the CFMG to formulate 3 preliminary projects on how to identify and analyze CF.

Conclusion.

New methods are urgently needed to assist researchers to identify and characterize CF that significantly influence the interpretation of results in clinical trials. The CFMG defined first steps to develop further guidance.