Missing the trees for the forest: most subgroup analyses using forest plots at the ASCO annual meeting are inconclusive

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials

Most oncology trials define superiority of an experimental therapy compared to a control therapy according to frequentist significance thresholds, which are widely misinterpreted. Posterior probability distributions computed by Bayesian inference may be more intuitive measures of uncertainty, particularly for measures of clinical benefit such as the minimum clinically important difference (MCID). Here, we manually reconstructed 194,129 individual patient-level outcomes across 230 phase III, superiority-design, oncology trials. Posteriors were calculated by Markov Chain Monte Carlo sampling using standard priors. All trials interpreted as positive had probabilities > 90% for marginal benefits (HR < 1). However, 38% of positive trials had ≤ 90% probabilities of achieving the MCID (HR < 0.8), even under an enthusiastic prior. A subgroup analysis of 82 trials that led to regulatory approval showed 30% had ≤ 90% probability for meeting the MCID under an enthusiastic prior. Conversely, 24% of negative trials had > 90% probability of achieving marginal benefits, even under a skeptical prior, including 12 trials with a primary endpoint of overall survival. Lastly, a phase III oncology-specific prior from a previous work, which uses published summary statistics rather than reconstructed data to compute posteriors, validated the individual patient-level data findings. Taken together, these results suggest that Bayesian models add considerable unique interpretative value to phase III oncology trials and provide a robust solution for overcoming the discrepancies between refuting the null hypothesis and obtaining a MCID.

Is there a preferred platinum and fluoropyrimidine regimen for advanced HER2-negative esophagogastric adenocarcinoma? Insights from 1293 patients in AGAMENON-SEOM registry

BACKGROUND

The optimal chemotherapy backbone for HER2-negative advanced esophagogastric cancer, either in combination with targeted therapies or as a comparator in clinical trials, is uncertain. The subtle yet crucial differences in platinum-based regimens' safety and synergy with combination treatments need consideration.

METHODS

We analyzed cases from the AGAMENON-SEOM Spanish registry of HER2-negative advanced esophagogastric adenocarcinoma treated with platinum and fluoropyrimidine from 2008 to 2021. This study focused exclusively on patients receiving one of the four regimens: FOLFOX (5-FU and oxaliplatin), CAPOX (capecitabine and oxaliplatin), CP (capecitabine and cisplatin) and FP (5-FU and cisplatin). The aim was to determine the most effective and tolerable platinum and fluoropyrimidine-based chemotherapy regimen and to identify any prognostic factors.

RESULTS

Among 1293 patients, 36% received either FOLFOX (n = 468) or CAPOX (n = 466), 20% CP (n = 252), and 8% FP (n = 107). FOLFOX significantly increased PFS (progression free survival) compared to CP, with a hazard ratio of 0.73 (95% CI 0.58-0.92, p = 0.009). The duration of treatment was similar across all groups. Survival outcomes among regimens were similar, but analysis revealed worse ECOG-PS (Eastern Cooperative Oncology Group-Performance Status), > 2 metastatic sites, bone metastases, hypoalbuminemia, higher NLR (neutrophil-to-lymphocyte ratio), and CP regimen as predictors of poor PFS. Fatigue was common in all treatments, with the highest incidence in FOLFOX (77%), followed by FP (72%), CAPOX (68%), and CP (60%). Other notable toxicities included neuropathy (FOLFOX 69%, CAPOX 62%), neutropenia (FOLFOX 52%, FP 55%), hand-foot syndrome in CP (46%), and thromboembolic events (FP 12%, CP 11%).

CONCLUSIONS

FOLFOX shown better PFS than CP. Adverse effects varied: neuropathy was more common with oxaliplatin, while thromboembolism was more frequent with cisplatin.

Risk-benefit trade-offs and precision utilities in phase I-II clinical trials

BACKGROUND

Identifying optimal doses in early-phase clinical trials is critically important. Therapies administered at doses that are either unsafe or biologically ineffective are unlikely to be successful in subsequent clinical trials or to obtain regulatory approval. Identifying appropriate doses for new agents is a complex process that involves balancing the risks and benefits of outcomes such as biological efficacy, toxicity, and patient quality of life.

PURPOSE

While conventional phase I trials rely solely on toxicity to determine doses, phase I-II trials explicitly account for both efficacy and toxicity, which enables them to identify doses that provide the most favorable risk-benefit trade-offs. It is also important to account for patient covariates, since one-size-fits-all treatment decisions are likely to be suboptimal within subgroups determined by prognostic variables or biomarkers. Notably, the selection of estimands can influence our conclusions based on the prognostic subgroup studied. For example, assuming monotonicity of the probability of response, higher treatment doses may yield more pronounced efficacy in favorable prognosis compared to poor prognosis subgroups when the estimand is mean or median survival. Conversely, when the estimand is the 3-month survival probability, higher treatment doses produce more pronounced efficacy in poor prognosis compared to favorable prognosis subgroups.

METHODS AND CONCLUSIONS

Herein, we first describe why it is essential to consider clinical practice when designing a clinical trial and outline a stepwise process for doing this. We then review a precision phase I-II design based on utilities tailored to prognostic subgroups that characterize efficacy-toxicity risk-benefit trade-offs. The design chooses each patient's dose to optimize their expected utility and allows patients in different prognostic subgroups to have different optimal doses. We illustrate the design with a dose-finding trial of a new therapeutic agent for metastatic clear cell renal cell carcinoma.

Impact of covariate model building methods on their clinical relevance evaluation in population pharmacokinetic analyses: comparison of the full model, stepwise covariate model (SCM) and SCM+ approaches

Covariate analysis in population pharmacokinetics is key for adjusting doses for patients. The main objective of this work was to compare the adequacy of various modeling approaches on covariate clinical relevance decision-making. The full model, stepwise covariate model (SCM) and SCM+ PsN algorithms were compared in a clinical trial simulation of a 383-patient population pharmacokinetic study mixing rich and sparse designs. A one-compartment model with first-order absorption was used. A base model including a body weight effect on CL/F and V/F and a covariate model including 4 additional covariates-parameters relationships were simulated. As for forest plots, ratios between covariates at a specific value and that of a typical individual were calculated with their 90% confidence interval (CI90) using standard errors. Covariates on CL, V and KA were considered relevant if their CI90 fell completely outside the reference area [0.8-1.2]. All approaches provided unbiased covariate ratio estimates. For covariates with a simulated effect, the 3 approaches correctly identify their clinical relevance. However, significant covariates were missed in up to 15% of cases with SCM/SCM+. For covariate with no simulated effects, the full model mainly identified them as non-relevant or with insufficient information while SCM/SCM+ mainly did not select them. SCM/SCM+ assume that non-selected covariates are non-relevant when it could be due to insufficient information, whereas the full model does not make this assumption and is faster. This study must be extended to other methods and completed by a more complex high-dimensional simulation framework.

Differential Treatment Effects of Subgroup Analyses in Phase 3 Oncology Trials From 2004 to 2020

Importance

Subgroup analyses are often performed in oncology to investigate differential treatment effects and may even constitute the basis for regulatory approvals. Current understanding of the features, results, and quality of subgroup analyses is limited.

Objective

To evaluate forest plot interpretability and credibility of differential treatment effect claims among oncology trials.

Design, Setting, and Participants

This cross-sectional study included randomized phase 3 clinical oncology trials published prior to 2021. Trials were screened from ClinicalTrials.gov.

Main Outcomes and Measures

Missing visual elements in forest plots were defined as a missing point estimate or use of a linear x-axis scale for hazard and odds ratios. Multiplicity of testing control was recorded. Differential treatment effect claims were rated using the Instrument for Assessing the Credibility of Effect Modification Analyses. Linear and logistic regressions evaluated associations with outcomes.

Results

Among 785 trials, 379 studies (48%) enrolling 331 653 patients reported a subgroup analysis. The forest plots of 43% of trials (156 of 363) were missing visual elements impeding interpretability. While 4148 subgroup effects were evaluated, only 1 trial (0.3%) controlled for multiple testing. On average, trials that did not meet the primary end point conducted 2 more subgroup effect tests compared with trials meeting the primary end point (95% CI, 0.59-3.43 tests; P = .006). A total of 101 differential treatment effects were claimed across 15% of trials (55 of 379). Interaction testing was missing in 53% of trials (29 of 55) claiming differential treatment effects. Trials not meeting the primary end point were associated with greater odds of no interaction testing (odds ratio, 4.47; 95% CI, 1.42-15.55, P = .01). The credibility of differential treatment effect claims was rated as low or very low in 93% of cases (94 of 101).

Conclusions and Relevance

In this cross-sectional study of phase 3 oncology trials, nearly half of trials presented a subgroup analysis in their primary publication. However, forest plots of these subgroup analyses largely lacked essential features for interpretation, and most differential treatment effect claims were not supported. Oncology subgroup analyses should be interpreted with caution, and improvements to the quality of subgroup analyses are needed.

Interpreting Randomized Controlled Trials

This article describes rationales and limitations for making inferences based on data from randomized controlled trials (RCTs). We argue that obtaining a representative random sample from a patient population is impossible for a clinical trial because patients are accrued sequentially over time and thus comprise a convenience sample, subject only to protocol entry criteria. Consequently, the trial's sample is unlikely to represent a definable patient population. We use causal diagrams to illustrate the difference between random allocation of interventions within a clinical trial sample and true simple or stratified random sampling, as executed in surveys. We argue that group-specific statistics, such as a median survival time estimate for a treatment arm in an RCT, have limited meaning as estimates of larger patient population parameters. In contrast, random allocation between interventions facilitates comparative causal inferences about between-treatment effects, such as hazard ratios or differences between probabilities of response. Comparative inferences also require the assumption of transportability from a clinical trial's convenience sample to a targeted patient population. We focus on the consequences and limitations of randomization procedures in order to clarify the distinctions between pairs of complementary concepts of fundamental importance to data science and RCT interpretation. These include internal and external validity, generalizability and transportability, uncertainty and variability, representativeness and inclusiveness, blocking and stratification, relevance and robustness, forward and reverse causal inference, intention to treat and per protocol analyses, and potential outcomes and counterfactuals.

Less is More? First Impressions From COSMIC-313

The COSMIC-313 phase 3 randomized controlled trial tested the triplet combination of cabozantinib with nivolumab and ipilimumab in comparison with nivolumab plus ipilimumab control as fist-line systemic therapy in metastatic clear cell renal cell carcinoma. The first results presented at the 2022 European Society of Medical Oncology Congress are a milestone for the renal cell carcinoma field because they signal the advent of triplet combinations as potential treatment options for our patients. The present commentary highlights some considerations and potential next steps based on these first impressions.

Analysis of new treatments proposed for malignant pleural mesothelioma raises concerns about the conduction of clinical trials in oncology

In this commentary, using existing clinical trial data and FDA approvals we propose that there is currently a critical need for an appropriate balancing between the financial impact of new cancer drugs and their actual benefit for patients. By adopting "pleural mesothelioma" as our clinical model we summarize the most relevant pertinent and available literature on this topic, and use an analysis of the reliability of the trials submitted for registration and/or recently published as a case in point to raise concerns with respect to appropriate trial design, biomarker based stratification and to highlight the ongoing need for balancing the benefit/cost ratio for both patients and healthcare providers.

The significance of sarcomatoid and rhabdoid dedifferentiation in renal cell carcinoma

Dedifferentiation in renal cell carcinoma (RCC), either sarcomatoid or rhabdoid, is an infrequent event that may occur heterogeneously in the setting of any RCC histology and is associated with poor outcomes. Sarcomatoid dedifferentiation is associated with inferior survival with angiogenesis targeted therapy and infrequent responses to cytotoxic chemotherapy. However, immune checkpoint therapy has significantly improved outcomes for patients with sarcomatoid dedifferentiation. Biologically, sarcomatoid dedifferentiation has increased programmed death-ligand 1 (PD-L1) expression and an inflamed tumor microenvironment, in addition to other distinct molecular alterations. Less is known about rhabdoid dedifferentiation from either a clinical, biological, or therapeutic perspective. In this focused review, we will discuss the prognostic implications, outcomes with systemic therapy, and underlying biology in RCC with either sarcomatoid or rhabdoid dedifferentiation present.

A Causal Framework for Making Individualized Treatment Decisions in Oncology

Simple Summary

Physicians routinely make individualized treatment decisions by accounting for the joint effects of patient prognostic covariates and treatments on clinical outcomes. Ideally, this is performed using historical randomized clinical trial (RCT) data. Randomization ensures that unbiased estimates of causal treatment effect parameters can be obtained from the historical RCT data and used to predict each new patient’s outcome based on the joint effect of their baseline covariates and each treatment being considered. However, this process becomes problematic if a patient seen in the clinic is very different from the patients who were enrolled in the RCT. That is, if a new patient does not satisfy the entry criteria of the RCT, then the patient does not belong to the population represented by the patients who were studied in the RCT. In such settings, it still may be possible to utilize the RCT data to help choose a new patient’s treatment. This may be achieved by combining the RCT data with data from other clinical trials, or possibly preclinical experiments, and using the combined dataset to predict the patient’s expected outcome for each treatment being considered. In such settings, combining data from multiple sources in a way that is statistically reliable is not entirely straightforward, and correctly identifying and estimating the effects of treatments and patient covariates on clinical outcomes can be complex. Causal diagrams provide a rational basis to guide this process. The first step is to construct a causal diagram that reflects the plausible relationships between treatment variables, patient covariates, and clinical outcomes. If the diagram is correct, it can be used to determine what additional data may be needed, how to combine data from multiple sources, how to formulate a statistical model for clinical outcomes as a function of treatment and covariates, and how to compute an unbiased treatment effect estimate for each new patient. We use adjuvant therapy of renal cell carcinoma to illustrate how causal diagrams may be used to guide these steps.

Abstract

We discuss how causal diagrams can be used by clinicians to make better individualized treatment decisions. Causal diagrams can distinguish between settings where clinical decisions can rely on a conventional additive regression model fit to data from a historical randomized clinical trial (RCT) to estimate treatment effects and settings where a different approach is needed. This may be because a new patient does not meet the RCT’s entry criteria, or a treatment’s effect is modified by biomarkers or other variables that act as mediators between treatment and outcome. In some settings, the problem can be addressed simply by including treatment–covariate interaction terms in the statistical regression model used to analyze the RCT dataset. However, if the RCT entry criteria exclude a new patient seen in the clinic, it may be necessary to combine the RCT data with external data from other RCTs, single-arm trials, or preclinical experiments evaluating biological treatment effects. For example, external data may show that treatment effects differ between histological subgroups not recorded in an RCT. A causal diagram may be used to decide whether external observational or experimental data should be obtained and combined with RCT data to compute statistical estimates for making individualized treatment decisions. We use adjuvant treatment of renal cell carcinoma as our motivating example to illustrate how to construct causal diagrams and apply them to guide clinical decisions.