Combined test versus logrank/Cox test in 50 randomised trials

On the role of benchmarking data sets and simulations in method comparison studies

Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favor a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data for such method comparison studies. Most manuscripts on statistical methodology rely on simulation studies and provide a single real-world data set as an example to motivate and illustrate the methodology investigated. In the context of supervised learning, in contrast, methods are often evaluated using so-called benchmarking data sets, that is, real-world data that serve as gold standard in the community. Simulation studies, on the other hand, are much less common in this context. The aim of this paper is to investigate differences and similarities between these approaches, to discuss their advantages and disadvantages, and ultimately to develop new approaches to the evaluation of methods picking the best of both worlds. To this aim, we borrow ideas from different contexts such as mixed methods research and Clinical Scenario Evaluation.

Dealing With Nonproportional Hazards in Coronary Revascularisation Studies

The Cox proportional hazards model is one of the most popular statistical tools to model time to event outcomes without the need for specifying the hazards or survival time distributions. The Cox model requires that the ratio of the hazards of the occurrence of the outcome for any 2 individuals remains constant during the entire follow-up. Studies comparing coronary revascularisation strategies, however, might be prone to violations of proportionality by the crossing of the hazard functions over time. Early increases in the risk of cardiovascular outcomes are commonly observed when comparing coronary artery bypass grafting vs percutaneous coronary intervention, whereas decreased risk might be observed later during the follow-up. The same is valid for comparisons between invasive vs conservative coronary revascularisation strategies. In these situations, the statistical power of the Cox model is reduced, and hazard ratios might not be an informative summary measure of treatment effect. In this article, we discuss methods to identify and account for nonproportionality. We illustrate the use of these methods in a case study based on reconstructed data from a coronary revascularisation clinical trial. And finally, we review the cardiovascular literature to estimate how the proportionality assumption has been reported in coronary revascularisation studies recently.

Insights from UKCTOCS for design, conduct and analyses of large randomised controlled trials

Abstract

Randomised controlled trials are challenging to deliver. There is a constant need to review and refine recruitment and implementation strategies if they are to be completed on time and within budget. We present the strategies adopted in the United Kingdom Collaborative Trial of Ovarian Cancer Screening, one of the largest individually randomised controlled trials in the world. The trial recruited over 202,000 women (2001-5) and delivered over 670,000 annual screens (2001-11) and over 3 million women-years of follow-up (2001-20). Key to the successful completion were the involvement of senior investigators in the day-to-day running of the trial, proactive trial management and willingness to innovate and use technology. Our underlying ethos was that trial participants should always be at the centre of all our processes. We ensured that they were able to contact either the site or the coordinating centre teams for clarifications about their results, for follow-up and for rescheduling of appointments. To facilitate this, we shared personal identifiers (with consent) with both teams and had dedicated reception staff at both site and coordinating centre. Key aspects were a comprehensive online trial management system which included an electronic data capture system (resulting in an almost paperless trial), biobanking, monitoring and project management modules. The automation of algorithms (to ascertain eligibility and classify results and ensuing actions) and processes (scheduling of appointments, printing of letters, etc.) ensured the protocol was closely followed and timelines were met. Significant engagement with participants ensured retention and low rates of complaints. Our solutions to the design, conduct and analyses issues we faced are highly relevant, given the renewed focus on trials for early detection of cancer.

Future work

There is a pressing need to increase the evidence base to support decision making about all aspects of trial methodology.

Trial registration

ISRCTN-22488978; ClinicalTrials.gov-NCT00058032.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme as award number 16/46/01. The long-term follow-up UKCTOCS (2015 20) was supported by National Institute for Health and Care Research (NIHR HTA grant 16/46/01), Cancer Research UK, and The Eve Appeal. UKCTOCS (2001-14) was funded by the MRC (G9901012 and G0801228), Cancer Research UK (C1479/A2884), and the UK Department of Health, with additional support from The Eve Appeal. Researchers at UCL were supported by the NIHR UCL Hospitals Biomedical Research Centre and by the MRC Clinical Trials Unit at UCL core funding (MC_UU_00004/09, MC_UU_00004/08, MC_UU_00004/07). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the UK Department of Health and Social Care.

Testing for Differences in Survival When Treatment Effects Are Persistent, Decaying, or Delayed

A statistical test for the presence of treatment effects on survival will be based on a null hypothesis (absence of effects) and an alternative (presence of effects). The null is very simply expressed. The most common alternative, also simply expressed, is that of proportional hazards. For this situation, not only do we have a very powerful test in the log-rank test but also the outcome is readily interpreted. However, many modern treatments fall outside this relatively straightforward paradigm and, as such, have attracted attention from statisticians eager to do their best to avoid losing power as well as to maintain interpretability when the alternative hypothesis is less simple. Examples include trials where the treatment effect decays with time, immunotherapy trials where treatment effects may be slow to manifest themselves as well as the so-called crossing hazards problem. We review some of the solutions that have been proposed to deal with these issues. We pay particular attention to the integrated log-rank test and how it can be combined with the log-rank test itself to obtain powerful tests for these more complex situations.

Differential Gene Expression and Weighted Correlation Network Dynamics in High-Throughput Datasets of Prostate Cancer

Precision oncology is an absolute need today due to the emergence of treatment resistance and heterogeneity among cancerous profiles. Target-propelled cancer therapy is one of the treasures of precision oncology which has come together with substantial medical accomplishment. Prostate cancer is one of the most common cancers in males, with tremendous biological heterogeneity in molecular and clinical behavior. The spectrum of molecular abnormalities and varying clinical patterns in prostate cancer suggest substantial heterogeneity among different profiles. To identify novel therapeutic targets and precise biomarkers implicated with prostate cancer, we performed a state-of-the-art bioinformatics study, beginning with analyzing high-throughput genomic datasets from The Cancer Genome Atlas (TCGA). Weighted gene co-expression network analysis (WGCNA) suggests a set of five dysregulated hub genes (MAF, STAT6, SOX2, FOXO1, and WNT3A) that played crucial roles in biological pathways associated with prostate cancer progression. We found overexpressed STAT6 and SOX2 and proposed them as candidate biomarkers and potential targets in prostate cancer. Furthermore, the alteration frequencies in STAT6 and SOX2 and their impact on the patients' survival were explored through the cBioPortal platform. The Kaplan-Meier survival analysis suggested that the alterations in the candidate genes were linked to the decreased overall survival of the patients. Altogether, the results signify that STAT6 and SOX2 and their genomic alterations can be explored in therapeutic interventions of prostate cancer for precision oncology, utilizing early diagnosis and target-propelled therapy.

Which test for crossing survival curves? A user’s guideline

Background

The exchange of knowledge between statisticians developing new methodology and clinicians, reviewers or authors applying them is fundamental. This is specifically true for clinical trials with time-to-event endpoints. Thereby, one of the most commonly arising questions is that of equal survival distributions in two-armed trial. The log-rank test is still the gold-standard to infer this question. However, in case of non-proportional hazards, its power can become poor and multiple extensions have been developed to overcome this issue. We aim to facilitate the choice of a test for the detection of survival differences in the case of crossing hazards.

Methods

We restricted the review to the most recent two-armed clinical oncology trials with crossing survival curves. Each data set was reconstructed using a state-of-the-art reconstruction algorithm. To ensure reproduction quality, only publications with published number at risk at multiple time points, sufficient printing quality and a non-informative censoring pattern were included. This article depicts the p-values of the log-rank and Peto-Peto test as references and compares them with nine different tests developed for detection of survival differences in the presence of non-proportional or crossing hazards.

Results

We reviewed 1400 recent phase III clinical oncology trials and selected fifteen studies that met our eligibility criteria for data reconstruction. After including further three individual patient data sets, for nine out of eighteen studies significant differences in survival were found using the investigated tests. An important point that reviewers should pay attention to is that 28% of the studies with published survival curves did not report the number at risk. This makes reconstruction and plausibility checks almost impossible.

Conclusions

The evaluation shows that inference methods constructed to detect differences in survival in presence of non-proportional hazards are beneficial and help to provide guidance in choosing a sensible alternative to the standard log-rank test.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01520-0.

Critical review of oncology clinical trial design under non-proportional hazards

In trials of novel immuno-oncology drugs, the proportional hazards (PH) assumption often does not hold for the primary time-to-event (TTE) efficacy endpoint, likely due to the unique mechanism of action of these drugs. In practice, when it is anticipated that PH may not hold for the TTE endpoint with respect to treatment, the sample size is often still calculated under the PH assumption, and the hazard ratio (HR) from the Cox model is still reported as the primary measure of the treatment effect. Sensitivity analyses of the TTE data using methods that are suitable under non-proportional hazards (non-PH) are commonly pre-planned. In cases where a substantial deviation from the PH assumption is likely, we suggest designing the trial, calculating the sample size and analyzing the data, using a suitable method that accounts for non-PH, after gaining alignment with regulatory authorities. In this comprehensive review article, we describe methods to design a randomized oncology trial, calculate the sample size, analyze the trial data and obtain summary measures of the treatment effect in the presence of non-PH. For each method, we provide examples of its use from the recent oncology trials literature. We also summarize in the Appendix some methods to conduct sensitivity analyses for overall survival (OS) when patients in a randomized trial switch or cross-over to the other treatment arm after disease progression on the initial treatment arm, and obtain an adjusted or weighted HR for OS in the presence of cross-over. This is an example of the treatment itself changing at a specific point in time - this cross-over may lead to a non-PH pattern of diminishing treatment effect.

Using the geometric average hazard ratio in sample size calculation for time-to-event data with composite endpoints

Background

Sample size calculation is a key point in the design of a randomized controlled trial. With time-to-event outcomes, it’s often based on the logrank test. We provide a sample size calculation method for a composite endpoint (CE) based on the geometric average hazard ratio (gAHR) in case the proportional hazards assumption can be assumed to hold for the components, but not for the CE.

Methods

The required number of events, sample size and power formulae are based on the non-centrality parameter of the logrank test under the alternative hypothesis which is a function of the gAHR. We use the web platform, CompARE, for the sample size computations. A simulation study evaluates the empirical power of the logrank test for the CE based on the sample size in terms of the gAHR. We consider different values of the component hazard ratios, the probabilities of observing the events in the control group and the degrees of association between the components. We illustrate the sample size computations using two published randomized controlled trials. Their primary CEs are, respectively, progression-free survival (time to progression of disease or death) and the composite of bacteriologically confirmed treatment failure or Staphylococcus aureus related death by 12 weeks.

Results

For a target power of 0.80, the simulation study provided mean (± SE) empirical powers equal to 0.799 (±0.004) and 0.798 (±0.004) in the exponential and non-exponential settings, respectively. The power was attained in more than 95% of the simulated scenarios and was always above 0.78, regardless of compliance with the proportional-hazard assumption.

Conclusions

The geometric average hazard ratio as an effect measure for a composite endpoint has a meaningful interpretation in the case of non-proportional hazards. Furthermore it is the natural effect measure when using the logrank test to compare the hazard rates of two groups and should be used instead of the standard hazard ratio.

Cediranib in addition to chemotherapy for women with relapsed platinum-sensitive ovarian cancer (ICON6): overall survival results of a phase III randomised trial

BACKGROUND

Cediranib, an oral anti-angiogenic VEGFR 1-3 inhibitor, was studied at a daily dose of 20 mg in combination with platinum-based chemotherapy and as maintenance in a randomised trial in patients with first relapse of 'platinum-sensitive' ovarian cancer and has been shown to improve progression-free survival (PFS).

PATIENTS AND METHODS

ICON6 (NCT00532194) was an international three-arm, double-blind, placebo-controlled randomised trial. Between December 2007 and December 2011, 456 women were randomised, using stratification, to receive either chemotherapy with placebo throughout (arm A, reference); chemotherapy with concurrent cediranib, followed by maintenance placebo (arm B, concurrent); or chemotherapy with concurrent cediranib, followed by maintenance cediranib (arm C, maintenance). Due to an enforced redesign of the trial in September 2011, the primary endpoint became PFS between arms A and C which we have previously published, and the overall survival (OS) was defined as a secondary endpoint, which is reported here.

RESULTS

After a median follow-up of 25.6 months, strong evidence of an effect of concurrent plus maintenance cediranib on PFS was observed [hazard ratio (HR) 0.56, 95% confidence interval (CI) 0.44-0.72, P < 0.0001]. In this final update of the survival analysis, 90% of patients have died. There was a 7.4-month difference in median survival and an HR of 0.86 (95% CI: 0.67-1.11, P = 0.24) in favour of arm C. There was strong evidence of a departure from the assumption of non-proportionality using the Grambsch-Therneau test (P = 0.0031), making the HR difficult to interpret. Consequently, the restricted mean survival time (RMST) was used and the estimated difference over 6 years by the RMST was 4.8 months (95% CI: -0.09 to 9.74 months).

CONCLUSIONS

Although a statistically significant difference in time to progression was seen, the enforced curtailment in recruitment meant that the secondary analysis of OS was underpowered. The relative reduction in the risk of death of 14% risk of death was not conventionally statistically significant, but this improvement and the increase in the mean survival time in this analysis suggest that cediranib may have worthwhile activity in the treatment of recurrent ovarian cancer and that further research should be undertaken.

UKCTOCS update: applying insights of delayed effects in cancer screening trials to the long-term follow-up mortality analysis

BACKGROUND

During trials that span decades, new evidence including progress in statistical methodology, may require revision of original assumptions. An example is the continued use of a constant-effect approach to analyse the mortality reduction which is often delayed in cancer-screening trials. The latter led us to re-examine our approach for the upcoming primary mortality analysis (2020) of long-term follow-up of the United Kingdom Collaborative Trial of Ovarian Cancer Screening (LTFU UKCTOCS), having initially (2014) used the proportional hazards (PH) Cox model.

METHODS

We wrote to 12 experts in statistics/epidemiology/screening trials, setting out current evidence, the importance of pre-specification, our previous mortality analysis (2014) and three possible choices for the follow-up analysis (2020) of the mortality outcome: (A) all data (2001-2020) using the Cox model (2014), (B) new data (2015-2020) only and (C) all data (2001-2020) using a test that allows for delayed effects.

RESULTS

Of 11 respondents, eight supported changing the 2014 approach to allow for a potential delayed effect (option C), suggesting various tests while three favoured retaining the Cox model (option A). Consequently, we opted for the Versatile test introduced in 2016 which maintains good power for early, constant or delayed effects. We retained the Royston-Parmar model to estimate absolute differences in disease-specific mortality at 5, 10, 15 and 18 years.

CONCLUSIONS

The decision to alter the follow-up analysis for the primary outcome on the basis of new evidence and using new statistical methodology for long-term follow-up is novel and has implications beyond UKCTOCS. There is an urgent need for consensus building on how best to design, test, estimate and report mortality outcomes from long-term randomised cancer screening trials.

TRIAL REGISTRATION

ISRCTN22488978 . Registered on 6 April 2000.

A non-linear optimisation method to extract summary statistics from Kaplan-Meier survival plots using the published P value

Background

Meta-analyses of studies evaluating survival (time-to-event) outcomes are a powerful technique to assess the strength of evidence for a given disease or treatment. However, these studies rely on the adequate reporting of summary statistics in the source articles to facilitate further analysis. Unfortunately, many studies, especially within the field of prognostic research do not report such statistics, making secondary analyses challenging. Consequently, methods have been developed to infer missing statistics from the commonly published Kaplan-Meier (KM) plots but are liable to error especially when the published number at risk is not included.

Methods

We therefore developed a method using non-linear optimisation (nlopt) that only requires the KM plot and the commonly published P value to better estimate the underlying censoring pattern. We use this information to then calculate the natural logarithm of the hazard ratio (ln (HR)) and its variance (var) ln (HR), statistics important for meta-analyses.

Results

We compared this method to the Parmar method which also does not require the number at risk to be published. In a validation set consisting of 13 KM studies, a statistically significant improvement in calculating ln (HR) when using an exact P value was obtained (mean absolute error 0.014 vs 0.077, P = 0.003). Thus, when the true HR has a value of 1.5, inference of the HR using the proposed method would set limits between 1.49/1.52, an improvement of the 1.39/1.62 limits obtained using the Parmar method. We also used Monte Carlo simulations to establish recommendations for the number and positioning of points required for the method.

Conclusion

The proposed non-linear optimisation method is an improvement on the existing method when only a KM plot and P value are included and as such will enhance the accuracy of meta-analyses performed for studies analysing time-to-event outcomes. The nlopt source code is available, as is a simple-to-use web implementation of the method.

Construction and Comprehensive Analysis of a ceRNA Network to Reveal Potential Novel Biomarkers for Triple-Negative Breast Cancer

Background

Triple-negative breast cancer (TNBC) is the most common and aggressive type of breast cancer with an unfavourable outcome worldwide. Novel therapeutic targets are urgently required to explore this malignancy. This study explored the ceRNA network and the important genes for predicting the therapeutic targets.

Methods

It identified the differentially expressed genes of mRNAs, lncRNAs and miRNAs between TNBC and non-TNBC samples in four cohorts (TCGA, GSE38959, GSE45827 and GSE65194) to explore the novel therapeutic targets for TNBC. Downstream analyses, including functional enrichment analysis, ceRNA network, protein–protein interaction and survival analysis, were then conducted by bioinformatics analysis. Finally, the potential core protein of the ceRNA network in TNBC was validated by immunohistochemistry.

Results

A total of 1,045 lncRNAs and 28 miRNAs were differentially expressed in the TCGA TNBC samples, and the intersections of 282 mRNAs (176 upregulations and 106 downregulations) between the GEO and TCGA databases were identified. A ceRNA network composed of 7 lncRNAs, 62 mRNAs, 12 miRNAs and 244 edges specific to TNBC was established. The functional assay showed dysregulated genes, and GO, DO and KEGG enrichment analysis were performed. Survival analysis showed that mRNA LIFR and lncRNA AC124312.3 were significantly correlated with the overall survival of patients with TNBC in the TCGA databases (P < 0.05). Finally, the LIFR protein was validated, and immunohistochemical results showed the upregulated expression of LIFR in TNBC tissues.

Conclusion

Thus, our study presents an enhanced understanding of the ceRNA network in TNBC, where the key gene LIFR may be a new promising potential therapeutic target for patients with TNBC.

A simulation study comparing the power of nine tests of the treatment effect in randomized controlled trials with a time-to-event outcome

BACKGROUND

The logrank test is routinely applied to design and analyse randomized controlled trials (RCTs) with time-to-event outcomes. Sample size and power calculations assume the treatment effect follows proportional hazards (PH). If the PH assumption is false, power is reduced and interpretation of the hazard ratio (HR) as the estimated treatment effect is compromised. Using statistical simulation, we investigated the type 1 error and power of the logrank (LR)test and eight alternatives. We aimed to identify test(s) that improve power with three types of non-proportional hazards (non-PH): early, late or near-PH treatment effects.

METHODS

We investigated weighted logrank tests (early, LRE; late, LRL), the supremum logrank test (SupLR) and composite tests (joint, J; combined, C; weighted combined, WC; versatile and modified versatile weighted logrank, VWLR, VWLR2) with two or more components. Weighted logrank tests are intended to be sensitive to particular non-PH patterns. Composite tests attempt to improve power across a wider range of non-PH patterns. Using extensive simulations based on real trials, we studied test size and power under PH and under simple departures from PH comprising pointwise constant HRs with a single change point at various follow-up times. We systematically investigated the influence of high or low control-arm event rates on power.

RESULTS

With no preconceived type of treatment effect, the preferred test is VWLR2. Expecting an early effect, tests with acceptable power are SupLR, C, VWLR2, J, LRE and WC. Expecting a late effect, acceptable tests are LRL, VWLR, VWLR2, WC and J. Under near-PH, acceptable tests are LR, LRE, VWLR, C, VWLR2 and SupLR. Type 1 error was well controlled for all tests, showing only minor deviations from the nominal 5%. The location of the HR change point relative to the cumulative proportion of control-arm events considerably affected power.

CONCLUSIONS

Assuming ignorance of the likely treatment effect, the best choice is VWLR2. Several non-standard tests performed well when the correct type of treatment effect was assumed. A low control-arm event rate reduced the power of weighted logrank tests targeting early effects. Test size was generally well controlled. Further investigation of test characteristics with different types of non-proportional hazards of the treatment effect is warranted.

Novel Risk Modeling Approach of Atrial Fibrillation With Restricted Mean Survival Times: Application in the Framingham Heart Study Community-Based Cohort

BACKGROUND

Risk prediction models for atrial fibrillation (AF) do not give information about when AF might develop. Restricted mean survival time (RMST) quantifies risk into the time domain. Our objective was to use RMST to re-express individualized AF risk predictions.

METHODS AND RESULTS

We included AF-free participants from the Framingham Heart Study community-based cohorts. We predicted new-onset AF over 10-year follow-up according to baseline covariates: age, height, weight, systolic blood pressure, diastolic blood pressure, current smoking, antihypertensive treatment, diabetes mellitus, prevalent heart failure, and prevalent myocardial infarction. First, we fitted a Cox regression model and estimated the 10-year predicted risk of AF. Second, we fitted an RMST model and estimated the predicted mean time free of AF and alive over a time horizon of 10 years. We included 7586 AF-free participants contributing to 11 088 examinations (mean age 61±11 years, 44% were men). During 10-year follow-up, 822 participants developed AF. The Cox and RMST models were in agreement regarding the direction, strength, and statistical significance of associations for all covariates. Low (<5%), intermediate (5%-15%), and high (>15%) 10-year predicted risk of AF corresponded to predicted mean time alive and free of AF of 9.9, 9.6, and 8.8 years, respectively. A 60-year-old woman with a body mass index of 25 kg/m2, no use of hypertension treatment and no history of heart failure had a predicted mean time alive and free of AF of 9.9 years, whereas a 70-year-old man with a body mass index of 30 kg/m2, use of hypertension treatment, and with prevalent heart failure had a predicted mean time alive and free of AF of 7.9 years.

CONCLUSIONS

The RMST can be used to develop risk prediction models to express results in a time scale. RMST may offer a complementary risk communication tool for AF in clinical practice.