Statistical controversies in clinical research: the importance of importance

Visualizing a marker's degrees of necessity and of sufficiency in the predictiveness curve

BACKGROUND

The degrees to which a factor is necessary or sufficient for an event have been proposed as generalizations of attributable risk based on simple functions of unconditional and conditional event probabilities. Predictiveness curves show the risk for an event, as derived by a model with one or more predictors, depending on risk percentiles that represent the predictors' distribution in the underlying population.

METHODS

Connections between the degrees of necessity and of sufficiency and explained variation on the one hand and the predictiveness curve on the other hand are mathematically proved and exemplified using data of in-hospital death of Covid- 19 patients.

RESULTS

We show that the degrees of necessity and of sufficiency can be represented as proportions of areas easily identifiable in the plot of the predictiveness curve. In addition, we show that the proportion of explained variation, a common measure of predictiveness and relative importance of prognostic factors, is also closely connected to these areas.

CONCLUSION

Our investigations demonstrate that the predictiveness curve extended by these new interpretations of areas provides a comprehensive evaluation of markers or sets of markers for prediction.

TRIAL REGISTRATION

Austrian Coronavirus Adaptive Clinical Trial (ACOVACT); ClinicalTrials.gov, identifier NCT04351724.

Explained variation and degrees of necessity and of sufficiency for competing risks survival data

In this contribution, the Schemper-Henderson measure of explained variation for survival outcomes is extended to accommodate competing events (CEs) in addition to events of interest. The extension is achieved by moving from the unconditional and conditional survival functions of the original measure to unconditional and conditional cumulative incidence functions, the latter obtained, for example, from Fine and Gray models. In the absence of CEs, the original measure is obtained as a special case. We define explained variation on the population level and provide two different types of estimates. Recently, the authors have achieved a multiplicative decomposition of explained variation into degrees of necessity and degrees of sufficiency. These measures are also extended to the case of competing risks survival data. A SAS macro and an R function are provided to facilitate application. Interesting empirical properties of the measures are explored on the population level and by an extensive simulation study. Advantages of the approach are exemplified by an Austrian study of breast cancer with a high proportion of CEs.

Amyloid-like p53 as prognostic biomarker in serous ovarian cancer-a study of the OVCAD consortium

TP53 is the most commonly mutated gene in cancer and has been shown to form amyloid-like aggregates, similar to key proteins in neurodegenerative diseases. Nonetheless, the clinical implications of p53 aggregation remain unclear. Here, we investigated the presence and clinical relevance of p53 aggregates in serous ovarian cancer (OC). Using the p53-Seprion-ELISA, p53 aggregates were detected in 46 out of 81 patients, with a detection rate of 84.3% in patients with missense mutations. High p53 aggregation was associated with prolonged progression-free survival. We found associations of overall survival with p53 aggregates, but they did not reach statistical significance. Interestingly, p53 aggregation was significantly associated with elevated levels of p53 autoantibodies and increased apoptosis, suggesting that high levels of p53 aggregates may trigger an immune response and/or exert a cytotoxic effect. To conclude, for the first time, we demonstrated that p53 aggregates are an independent prognostic marker in serous OC. P53-targeted therapies based on the amount of these aggregates may improve the patient's prognosis.

Left atrial size predicts outcome in severe but asymptomatic mitral regurgitation

Patients with severe asymptomatic primary mitral regurgitation (MR) can be safely managed with an active surveillance strategy. Left atrial (LA) size is affected by MR severity, left ventricular function and is also associated with the risk of atrial fibrillation and may be an integrative parameter for risk stratification. The present study sought to determine the predictive value of LA size in a large series of asymptomatic patients with severe MR. 280 consecutive patients (88 female, median age 58 years) with severe primary MR and no guideline-based indications for surgery were included in a follow-up program until criteria for mitral surgery were reached. Event-free survival was determined and potential predictors of outcome were assessed. Survival free of any indication for surgery was 78% at 2 years, 52% at 6 years, 35% at 10 years and 19% at 15 years, respectively. Left atrial (LA) diameter was the strongest independent echocardiographic predictor of event-free survival with incremental predictive value for the thresholds of 50, 60 and 70 mm, respectively. In a multivariable analysis that encompassed age at baseline, previous history of atrial fibrillation, left ventricular end systolic diameter), LA diameter, sPAP > 50 mmHg and year of inclusion, LA diameter was the strongest independent echocardiographic predictor of event-free survival (adjusted HR = 1.039, p < 0.001). LA size is a simple and reproducible predictor of outcome in asymptomatic severe primary MR. In particular, it may help to identify patients who may benefit from early elective valve surgery in heart valve centers of excellence.

Serum parameters as prognostic biomarkers in a real world cancer patient population treated with anti PD-1/PD-L1 therapy

BACKGROUND

Immune checkpoint inhibitors (ICI) are regarded as a standard of care in multiple malignancies. We hypothesized that serum parameters are of prognostic value in ICI treated patients suffering from solid tumours.

METHODS

Data from 114 patients treated with ICIs for solid malignancies from 2015 to 2019 at the Medical University of Vienna were collected retrospectively. Data included baseline characteristics, cancer type, serum parameters such as lactate dehydrogenase (LDH), C-reactive protein (CRP), albumin (Alb) and lymphocyte counts as well as overall survival (OS) and progression free survival. Additionally, the Gustave Roussy Immune Score (GRIm score) and the Glasgow prognostic score (GPS) were calculated. Cox regression models including time-dependent effects and strata for tumour type were used. Prognostic factors were pre-selected using a relaxed LASSO approach.

RESULTS

The majority of patients were male (64.9%). The most common cancer types were non-small cell lung cancer (30.7%) and renal cell carcinoma (21.9%). Increased LDH and CRP were associated with poor 6-month OS (Hazard ratios (HR)=1.16 and 1.06 per 20% LDH/CRP increase; 95% CI 1.07-1.26 and 95% CI 1.03-1.09, respectively; p < .001). Both GRIm Score and GPS had a significant influence on OS (GRIm: HR = 2.84, 95% CI 1.72-4.69; p < .001 for high vs. low; GPS HR 3.57, 95% CI 1.76-7.25; p < .001 for poor vs. good). The proportion of explained variation (PEV) of our full multivariable model was significantly higher compared to the GRIm and GPS (PEV = 29.5% vs. 14.8% and 14.65%). When grouped into quartiles according to the individual 8-weeks change, both increased LDH and CRP correlated with poor OS (LDH (p=.001) and CRP (p < .001)).

CONCLUSION

The results of this analysis suggest that serum parameters might have prognostic value for the outcome of cancer patients treated with ICI, regardless of the tumour type.Key messagesIn this retrospective analysis, 114 patients with solid tumours were included. The results of this analysis point out that pre-treatment LDH, CRP and albumin levels are strongly prognostic for a poor 6-month OS.In addition to that, a high GRIm-score and poor GPS were associated with a worse OS (GRIm: HR = 2.84, 95% CI 1.72-4.69; p < .001 for high vs. low; GPS HR = 3.57, 95% CI 1.76-7.25; p < .001 for poor vs. good).Pre-treatment serum parameters might have prognostic value for the outcome of cancer patients treated with ICI, regardless of the tumour type.

Degrees of necessity and of sufficiency: Further results and extensions, with an application to covid-19 mortality in Austria

The purpose of this paper is to extend to ordinal and nominal outcomes the measures of degree of necessity and of sufficiency defined by the authors for dichotomous and survival outcomes in a previous paper. A cause, represented by certain values of prognostic factors, is considered necessary for an event if, without the cause, the event cannot develop. It is considered sufficient for an event if the event is unavoidable in the presence of the cause. The degrees of necessity and sufficiency, ranging from zero to one, are simple, intuitive functions of unconditional and conditional probabilities of an event such as disease or death. These probabilities often will be derived from logistic regression models; the measures, however, do not require any particular model. In addition, we study in detail the relationship between the proposed measures and the related explained variation summary for dichotomous outcomes, which are the common root for the developments for ordinal, nominal, and survival outcomes. We introduce and analyze the Austrian covid-19 data, with the aim of quantifying effects of age and other potentially prognostic factors on covid-19 mortality. This is achieved by standard regression methods but also in terms of the newly proposed measures. It is shown how they complement the toolbox of prognostic factor studies, in particular when comparing the importance of prognostic factors of different types. While the full model's degree of necessity is extremely high (0.933), its low degree of sufficiency (0.179) is responsible for the low proportion of explained variation (0.193).

UK Multicenter Prospective Evaluation of the Leibovich Score in Localized Renal Cell Carcinoma: Performance has Altered Over Time

Objective

To examine changes in outcome by the Leibovich score using contemporary and historic cohorts of patients presenting with renal cell carcinoma (RCC)

Patients and Methods

Prospective observational multicenter cohort study, recruiting patients with suspected newly diagnosed RCC. A historical cohort of patients was examined for comparison. Metastasis-free survival (MFS) formed the primary outcome measure. Model discrimination and calibration were evaluated using Cox proportional hazard regression and the Kaplan-Meier method. Overall performance of the Leibovich model was assessed by estimating explained variation.

Results

Seven hundred and six patients were recruited between 2011 and 2014 and RCC confirmed in 608 (86%) patients. Application of the Leibovich score to patients with localized clear cell RCC in this contemporary cohort demonstrated good model discrimination (c-index = 0.77) but suboptimal calibration, with improved MFS for intermediate- and high-risk patients (5-year MFS 85% and 50%, respectively) compared to the original Leibovich cohort (74% and 31%) and a historic (1998-2006) UK cohort (76% and 37%). The proportion of variation in outcome explained by the model is low and has declined over time (28% historic vs 22% contemporary UK cohort).

Conclusion

Prognostic models are widely employed in patients with localized RCC to guide surveillance intensity and clinical trial selection. However, the majority of the variation in outcome remains unexplained by the Leibovich model and, over time, MFS rates among intermediate- and high-risk classified patients have altered. These findings are likely to have implications for all such models used in this setting.

Prognostic Value of Right Ventricular Dysfunction and Tricuspid Regurgitation in Patients with Severe Low-Flow Low-Gradient Aortic Stenosis

Long and mid-term data in Low-Flow Low-Gradient Aortic Stenosis (LFLG-AS) are scarce. The present study sought to identify predictors of outcome in a sizeable cohort of patients with LFLG-AS. 76 consecutive patients with LFLG-AS (defined by a mean gradient <40 mmHg, an aortic valve area ≤1 cm² and an ejection fraction ≤50%) were prospectively enrolled and followed at regular intervals. Events defined as aortic valve replacement (AVR) and death were assessed and overall survival was determined. 44 patients underwent AVR (10 transcatheter and 34 surgical) whilst intervention was not performed in 32 patients, including 9 patients that died during a median waiting time of 4 months. Survival was significantly better after AVR with survival rates of 91.8% (CI 71.1–97.9%), 83.0% (CI 60.7–93.3%) and 56.3% (CI 32.1–74.8%) at 1,2 and 5 years as compared to 84.3% (CI 66.2–93.1%), 52.9% (CI 33.7–69.0%) and 30.3% (CI 14.6–47.5%), respectively, for patients managed conservatively (p = 0.017). The presence of right ventricular dysfunction (HR 3.47 [1.70–7.09]) and significant tricuspid regurgitation (TR) (HR 2.23 [1.13–4.39]) independently predicted overall mortality while the presence of significant TR (HR 3.40[1.38–8.35]) and higher aortic jet velocity (HR 0.91[0.82–1.00]) were independent predictors of mortality and survival after AVR. AVR is associated with improved long-term survival in patients with LFLG-AS. Treatment delays are associated with excessive mortality, warranting urgent treatment in eligible patients. Right ventricular involvement characterized by the presence of TR and/or right ventricular dysfunction, identifies patients at high risk of mortality under both conservative management and after AVR.

Quantifying degrees of necessity and of sufficiency in cause-effect relationships with dichotomous and survival outcomes

We suggest measures to quantify the degrees of necessity and of sufficiency of prognostic factors for dichotomous and for survival outcomes. A cause, represented by certain values of prognostic factors, is considered necessary for an event if, without the cause, the event cannot develop. It is considered sufficient for an event if the event is unavoidable in the presence of the cause. Necessity and sufficiency can be seen as the two faces of causation, and this symmetry and equal relevance are reflected by the suggested measures. The measures provide an approximate, in some cases an exact, multiplicative decomposition of explained variation as defined by Schemper and Henderson for censored survival and for dichotomous outcomes.

The measures, ranging from zero to one, are simple, intuitive functions of unconditional and conditional probabilities of an event such as disease or death. These probabilities often will be derived from logistic or Cox regression models; the measures, however, do not require any particular model.

The measures of the degree of necessity implicitly generalize the established attributable fraction or risk for dichotomous prognostic factors and dichotomous outcomes to continuous prognostic factors and to survival outcomes. In a setting with multiple prognostic factors, they provide marginal and partial results akin to marginal and partial odds and hazard ratios from multiple logistic and Cox regression.

Properties of the measures are explored by an extensive simulation study. Their application is demonstrated by three typical real data examples.

Explained variation in shared frailty models

Explained variation measures the relative gain in predictive accuracy when prediction based on prognostic factors replaces unconditional prediction. The factors may be measured on different scales or may be of different types (dichotomous, qualitative, or continuous). Thus, explained variation permits to establish a ranking of the importance of factors, even if predictive accuracy is too low to be helpful in clinical practice. In this contribution, the explained variation measure by Schemper and Henderson (2000) is extended to accommodate random factors, such as center effects in multicenter studies. This permits a direct comparison of the importance of centers and of other prognostic factors. We develop this extension for a shared frailty Cox model and provide an SAS macro and an R function to facilitate its application. Interesting empirical properties of the variation explained by a random factor are explored by a Monte Carlo study. Advantages of the approach are exemplified by an Austrian multicenter study of colon cancer.

Comparison of Inflammation-Based Prognostic Scores in a Cohort of Patients with Resectable Esophageal Cancer

Background

A number of studies have revealed that inflammation-based prognostic scores, such as Glasgow prognostic score (GPS), modified Glasgow prognostic score (mGPS), and C-reactive protein and albumin ratio (C/A ratio), are associated with poor outcome in cancer patients. However, until now, no study has investigated the role of these prognostic scores in a cohort of neoadjuvant-treated esophageal adenocarcinomas (nEAC) and squamous cell carcinomas (nESCC).

Methods

Patients had laboratory measurements within three days before resection. GPS, mGPS, and C/A ratio were tested together with established clinicopathological factors in simple and multiple Cox regression analysis of overall survival (OS) and disease-free survival (DFS).

Results

A total of 283 patients (201 EAC and 82 ESCC) with locally advanced esophageal cancer were enrolled. 167 patients received neoadjuvant treatment (59.0%). Simple analysis revealed that there were significant differences in cancer-specific survival in relation to elevated C-reactive protein (p = 0.011), lymph node status (p < 0.001), UICC stage (p < 0.001), and nEAC (p = 0.005). mGPS (p = 0.024) showed statistical significance in simple analysis. No statistical significance could be found for GPS (p = 0.29), mGPS (p = 0.16), and C/A ratio (p = 0.76) in multiple analysis.

Conclusion

The investigated prognostic scores should be used and interpreted carefully, and established factors like histology, including tumor size and differentiation, lymph node involvement, and status of resection margin remain the only reliable prognostic factors for patients suffering from resectable EC.