Non-parametric estimation of a time-dependent predictive accuracy curve

Curvature estimation techniques for advancing neurodegenerative disease analysis: a systematic review of machine learning and deep learning approaches

Neurodegenerative diseases present complex challenges that demand advanced analytical techniques to decode intricate brain structures and their changes over time. Curvature estimation within datasets has emerged as a critical tool in areas like neuroimaging and pattern recognition, with significant applications in diagnosing and understanding neurodegenerative diseases. This systematic review assesses state-of-the-art curvature estimation methodologies, covering classical mathematical techniques, machine learning, deep learning, and hybrid methods. Analysing 105 research papers from 2010 to 2023, we explore how each approach enhances our understanding of structural variations in neurodegenerative pathology. Our findings highlight a shift from classical methods to machine learning and deep learning, with neural network regression and convolutional neural networks gaining traction due to their precision in handling complex geometries and data-driven modelling. Hybrid methods further demonstrate the potential to merge classical and modern techniques for robust curvature estimation. This comprehensive review aims to equip researchers and clinicians with insights into effective curvature estimation methods, supporting the development of enhanced diagnostic tools and interventions for neurodegenerative diseases.

Monitoring peripheral blood data supports the prediction of immunotherapy response in advanced non-small cell lung cancer based on real-world data

The identification of non-small cell lung cancer (NSCLC) patients who will benefit from immunotherapy remains a clinical challenge. Monitoring real-world data (RWD) in the first cycles of therapy may provide a more accurate representation of response patterns in a real-world setting. We propose a multivariate Bayesian joint model using generalized linear mixed effects, trained and validated on RWD from 424 advanced NSCLC patients retrospectively collected from three clinical centers. Center1 was used as training ( N = 212 ), while Center2 and Center3 were used as independent testing sets ( N = 137 and N = 75 , respectively). Peripheral blood data (PBD) were collected at baseline and at three follow-up time points, alongside demographic and epidemiologic features. Six models were trained to predict progression-free survival at 6 months, PFS(6), using different number of longitudinal samples (baseline, two, or four time points) of the neutrophil-to-lymphocyte ratio (NLR) or a multivariate feature selection. Long-term predictions at 12 and 24 months were also evaluated. Prediction accuracy was measured using the area under the receiver operating characteristic curve (AUC). The proposed model significantly improved prediction performance, achieving AUCs of 0.870, 0.804 and 0.827 at 6, 12 and 24 months for Center2, and 0.824, 0.822 and 0.667 for Center3. There was also a significant difference in PFS and overall survival (OS) between predicted response groups, defined by a 6-month PFS cutoff (log-rank test p < 0.001 ). Our study suggests that the integration of multiple biomarkers and monitored PBD in an RWD-based Bayesian joint model framework significantly improves immunotherapy response prediction in advanced NSCLC compared to conventional approaches involving biomarker data at baseline only.

Validation of endoplasmic reticulum stress-related gene signature to predict prognosis and immune landscape of patients with non-small cell lung cancer

BACKGROUND

Lung cancer is one of the most common cancers worldwide, with the incidence increasing each year. It is crucial to improve the prognosis of patients who have lung cancer. Non-Small Cell Lung Cancer (NSCLC) accounts for the majority of lung cancer. Though its prognostic significance in NSCLC has not been often documented, Endoplasmic Reticulum (ER) stress has been identified to be implicated in tumour malignant behaviours and resistance to treatment.

OBJECTIVE

This work aimed to develop a gene profile linked to ER stress that could be applied to predictive and risk assessment for non-small cell lung cancer.

METHODS

Data from 1014 NSCLC patients were sourced from The Cancer Genome Atlas (TCGA) database, integrating clinical and Ribonucleic Acid (RNA) information. Diverse analytical techniques were utilized to identify ERS-associated genes associated with patients' prognoses. These techniques included Kaplan-Meier analysis, univariate Cox regression, Least Absolute Shrinkage and Selection Operator regression analysis (LASSO) regression, and Pearson correlation analysis. Using a risk score model obtained from multivariate Cox analysis, a nomogram was created and validated to classify patients into high- and low-risk groups. The study employed the CIBERSORT algorithm and Single-Sample Gene Set Eenrichment Analysis (ssGSEA) to investigate the tumour immune microenvironment. We used the Genomics of Drug Sensitivity in Cancer (GDSC) database and R tools to identify medicines that could be responsive.

RESULTS

Four genes - FABP5, C5AR1, CTSL, and LTA4H - were chosen to create the risk model. Overall Survival (OS) was considerably lower (P< 0.05) in the high-risk group. When it came to predictive accuracy, the risk model outperformed clinical considerations. Several medication types that are sensitive to high-risk groups were chosen.

CONCLUSION

Our study has produced a gene signature associated with ER stress that may be employed to forecast the prognosis and therapeutic response of non-small cell lung cancer patients.

Dynamic and concordance-assisted learning for risk stratification with application to Alzheimer's disease

Dynamic prediction models capable of retaining accuracy by evolving over time could play a significant role for monitoring disease progression in clinical practice. In biomedical studies with long-term follow up, participants are often monitored through periodic clinical visits with repeat measurements until an occurrence of the event of interest (e.g. disease onset) or the study end. Acknowledging the dynamic nature of disease risk and clinical information contained in the longitudinal markers, we propose an innovative concordance-assisted learning algorithm to derive a real-time risk stratification score. The proposed approach bypasses the need to fit regression models, such as joint models of the longitudinal markers and time-to-event outcome, and hence enjoys the desirable property of model robustness. Simulation studies confirmed that the proposed method has satisfactory performance in dynamically monitoring the risk of developing disease and differentiating high-risk and low-risk population over time. We apply the proposed method to the Alzheimer's Disease Neuroimaging Initiative data and develop a dynamic risk score of Alzheimer's Disease for patients with mild cognitive impairment using multiple longitudinal markers and baseline prognostic factors.

Evaluating dynamic and predictive discrimination for recurrent event models: use of a time-dependent C-index

Interest in analyzing recurrent event data has increased over the past few decades. One essential aspect of a risk prediction model for recurrent event data is to accurately distinguish individuals with different risks of developing a recurrent event. Although the concordance index (C-index) effectively evaluates the overall discriminative ability of a regression model for recurrent event data, a local measure is also desirable to capture dynamic performance of the regression model over time. Therefore, in this study, we propose a time-dependent C-index measure for inferring the model's discriminative ability locally. We formulated the C-index as a function of time using a flexible parametric model and constructed a concordance-based likelihood for estimation and inference. We adapted a perturbation-resampling procedure for variance estimation. Extensive simulations were conducted to investigate the proposed time-dependent C-index's finite-sample performance and estimation procedure. We applied the time-dependent C-index to three regression models of a study of re-hospitalization in patients with colorectal cancer to evaluate the models' discriminative capability.

Radiomics-based nomogram guides adaptive de-intensification in locoregionally advanced nasopharyngeal carcinoma following induction chemotherapy

OBJECTIVES

This study aimed to construct a radiomics-based model for prognosis and benefit prediction of concurrent chemoradiotherapy (CCRT) versus intensity-modulated radiotherapy (IMRT) in locoregionally advanced nasopharyngeal carcinoma (LANPC) following induction chemotherapy (IC).

MATERIALS AND METHODS

A cohort of 718 LANPC patients treated with IC + IMRT or IC + CCRT were retrospectively enrolled and assigned to a training set (n = 503) and a validation set (n = 215). Radiomic features were extracted from pre-IC and post-IC MRI. After feature selection, a delta-radiomics signature was built with LASSO-Cox regression. A nomogram incorporating independent clinical indicators and the delta-radiomics signature was then developed and evaluated for calibration and discrimination. Risk stratification by the nomogram was evaluated with Kaplan-Meier methods.

RESULTS

The delta-radiomics signature, which comprised 19 selected features, was independently associated with prognosis. The nomogram, composed of the delta-radiomics signature, age, T category, N category, treatment, and pre-treatment EBV DNA, showed great calibration and discrimination with an area under the receiver operator characteristic curve of 0.80 (95% CI 0.75-0.85) and 0.75 (95% CI 0.64-0.85) in the training and validation sets. Risk stratification by the nomogram, excluding the treatment factor, resulted in two groups with distinct overall survival. Significantly better outcomes were observed in the high-risk patients with IC + CCRT compared to those with IC + IMRT, while comparable outcomes between IC + IMRT and IC + CCRT were shown for low-risk patients.

CONCLUSION

The radiomics-based nomogram can predict prognosis and survival benefits from concurrent chemotherapy for LANPC following IC. Low-risk patients determined by the nomogram may be potential candidates for omitting concurrent chemotherapy during IMRT.

CLINICAL RELEVANCE STATEMENT

The radiomics-based nomogram was constructed for risk stratification and patient selection. It can help guide clinical decision-making for patients with locoregionally advanced nasopharyngeal carcinoma following induction chemotherapy, and avoid unnecessary toxicity caused by overtreatment.

KEY POINTS

• The benefits from concurrent chemotherapy remained controversial for locoregionally advanced nasopharyngeal carcinoma following induction chemotherapy. • Radiomics-based nomogram achieved prognosis and benefits prediction of concurrent chemotherapy. • Low-risk patients defined by the nomogram were candidates for de-intensification.

Analyzing heterogeneity in biomarker discriminative performance through partial time-dependent receiver operating characteristic curve modeling

This study investigates the heterogeneity of a biomarker's discriminative performance for predicting subsequent time-to-event outcomes across different patient subgroups. While the area under the curve (AUC) for the time-dependent receiver operating characteristic curve is commonly used to assess biomarker performance, the partial time-dependent AUC (PAUC) provides insights that are often more pertinent for population screening and diagnostic testing. To achieve this objective, we propose a regression model tailored for PAUC and develop two distinct estimation procedures for discrete and continuous covariates, employing a pseudo-partial likelihood method. Simulation studies are conducted to assess the performance of these procedures across various scenarios. We apply our model and inference procedure to the Alzheimer's Disease Neuroimaging Initiative data set to evaluate potential heterogeneities in the discriminative performance of biomarkers for early Alzheimer's disease diagnosis based on patients' characteristics.

Thoracic Aortic Volume as a Predictor of Cardiovascular Events: The Multi-Ethnic Study of Atherosclerosis

BACKGROUND

It is unclear whether thoracic aortic volume (TAV) is useful for cardiovascular (CV) disease prognosis and risk assessment.

PURPOSE

This study evaluated cross-sectional associations of TAV with CV risk factors, and longitudinal association with incident CV events in the multiethnic study of atherosclerosis.

STUDY TYPE

Retrospective cohort analysis of prospective data.

POPULATION

1182 participants (69 ± 9 years, 54% female, 37% Caucasian, 18% Chinese, 31% African American, 14% Hispanic, 60% hypertensive, and 20% diabetic) without prior CV disease.

FIELD STRENGTH AND SEQUENCES

Axial black-blood turbo spin echo or bright blood steady-state free precession images on 1.5T scanners.

ASSESSMENT

TAV was calculated using Simpson's method from axial images, and included the ascending arch and descending segments. Traditional CV risk factors were assessed at the time of MRI. CV outcomes over a 9-year follow-up period were recorded and represented a composite of stroke, stroke death, coronary heart disease (CHD), CHD death, atherosclerotic death, and CVD death.

STATISTICAL TESTS

Multivariable linear regression models adjusted for height and weight were used to determine the relationship (β coefficient) between TAV and CV risk factors. Cox regression models assessed the association of TAV and incident CV events. A P-value of <0.05 was deemed statistically significant.

RESULTS

Mean TAV was = 139 ± 41 mL. In multivariable regression, TAV was directly associated with age (β = 1.6), male gender (β = 23.9), systolic blood pressure (β = 0.1), and hypertension medication use (β = 7.9); and inversely associated with lipid medication use (β = -5.3) and treated diabetes (β = -8.9). Compared to Caucasians, Chinese Americans had higher TAV (β = 11.4), while African Americans had lower TAV (β = -7.0). Higher TAV was independently associated with incident CV events (HR: 1.057 per 10 mL).

CONCLUSION

Greater TAV is associated with incident CV events, increased age, and hypertension in a large multiethnic population while treated diabetes and lipid medication use were associated with lower TAV.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: Stage 2.

Addressing subject heterogeneity in time-dependent discrimination for biomarker evaluation

Accurate discrimination has been the central goal in identifying biomarkers for monitoring disease progression and early detection. Acknowledging the fact that discrimination accuracy of biomarkers for a time-to-event outcome often changes over time, local measures such as the time-dependent receiver operating characteristic curve and its area under the curve (AUC) are used to assess time-dependent predictive discrimination. However, such measures do not address subject heterogeneity, although the impact of covariates including demographics, disease-related characteristics, and other clinical information on the discriminatory performance of biomarkers needs to be investigated before their clinical use. We propose the covariate-specific time-dependent AUC, a measure for covariate-adjusted discrimination. We develop a regression model on the covariate-specific time-dependent AUC to understand how and in what magnitude the covariates influence biomarker performance. Then we construct a pseudo partial-likelihood for estimation and inference. This is followed by our establishing the asymptotic properties of the proposed estimators and provide variance estimation. The simulation studies and application to the AIDS Clinical Trials Group 175 data demonstrate that the proposed method offers an informative tool for inferring covariate-specific and time-dependent predictive discrimination.

Inference for covariate-adjusted time-dependent prognostic accuracy measures

Evaluating the prognostic performance of candidate markers for future disease onset or progression is one of the major goals in medical research. A marker's prognostic performance refers to how well it separates patients at the high or low risk of a future disease state. Often the discriminative performance of a marker is affected by the patient characteristics (covariates). Time-dependent receiver operating characteristic (ROC) curves that ignore the informativeness of the covariates will lead to biased estimates of the accuracy parameters. We propose a time-dependent ROC curve that accounts for the informativeness of the covariates in the case of censored data. We propose inverse probability weighted (IPW) estimators for estimating the proposed accuracy parameters. We investigate the performance of the IPW estimators through simulation studies and real-life data analysis.

A semi-parametric approach for time-dependent ROC curves with nonignorable missing biomarker

The main purpose of this paper is to survey the statistical inference for covariate-specific time-dependent receiver operating characteristic (ROC) curves with nonignorable missing continuous biomarker values. To construct time-dependent ROC curves, we consider a joint model which assumes that the failure time depends on the continuous biomarker and the covariates through a Cox proportional hazards model and that the continuous biomarker depends on the covariates through a semiparametric location model. Assuming a purely parametric model on the propensity score, we utilize instrumental variables to deal with the identifiable issue and estimate the unknown parameters of the propensity score by a simple and efficient method. In addition, when the propensity score is estimated, we develop HT and AIPW approaches to estimate our interested quantities. In the presence of nonignorable missing biomarker, our AIPW estimators of the interested quantities are still doubly robust when the true propensity score is a special parametric logistic model. At last, simulation studies are conducted to assess the performance of our proposed approaches, and a real data analysis is also carried out to illustrate its application.

Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes

Risk prediction models for survival outcomes are widely applied in medical research to predict future risk for the occurrence of the event. In many clinical studies, the biomarker data are measured repeatedly over time. To facilitate timely disease prognosis and decision making, many dynamic prediction models have been developed and generate predictions on a real-time basis. As a dynamic prediction model updates an individual's risk prediction over time based on new measurements, it is often important to examine how well the model performs at different measurement times and prediction times. In this article, we propose a two-dimensional area under curve (AUC) measure for dynamic prediction models and develop associated estimation and inference procedures. The estimation procedures are discussed under two types of biomarker measurement schedules: regular visits and irregular visits. The model parameters are estimated effectively by maximizing a pseudo-partial likelihood function. We apply the proposed method to a renal transplantation study to evaluate the discrimination performance of dynamic prediction models based on longitudinal biomarkers for graft failure.

Potential diagnostic of lymph node metastasis and prognostic values of TM4SFs in papillary thyroid carcinoma patients

Background: Although the prognosis of papillary thyroid carcinoma (PTC) is relatively good, it causes around 41,000 deaths per year, which is likely related to recurrence and metastasis. Lymph node metastasis (LNM) is an important indicator of PTC recurrence and transmembrane 4 superfamily (TM4SF) proteins regulate metastasis by modulating cell adhesion, migration, tissue differentiation, and tumor invasion. However, the diagnostic and prognostic values of TM4SF in PTC remain unclear. Methods: This study aimed to identify TM4SF genes with predictive value for LNM and prognostic value in PTC using bioinformatic analysis. We screened the differentially expressed genes (DEGs) of the TM4SF family in PTC using data from TCGA, constructed a PPI network using STRING, and evaluated the predictive role of TM4SF1 in LNM via a binary logistic regression analysis and ROC curve. We assessed the association between TM4SF1 expression and DNA methylation, and determined the functional and mechanistic role of TM4SF1 in promoting LNM via GSEA, KEGG, and GO. We estimated the relationship between each TM4SF gene and overall survival (OS, estimated by Kaplan-Meier analysis) in patients with PTC and established a predictive model of prognostic indicators using a LASSO penalized Cox analysis to identify hub genes. Finally, we explored the correlation between TM4SFs and TMB/MSI. Results: We identified 21 DEGs from the 41 TM4SFs between N0 (without LNM) and N1 (with LNM) patients, with TM4SF1, TM4SF4, UPK1B, and CD151 being highly expressed in the N1 group; several DEGs were observed in the TNM, T, and N cancer stages. The "integrins and other cell-surface receptors" pathway was the most significantly enriched functional category related to LNM and TM4SFs. TM4SF1 was identified as an indicator of LNM (AUC= 0.702). High levels of TM4SF1 might be related to Wnt/β-catenin pathway and epithelial-mesenchymal transition (EMT) process in PTC. The higher expression of TM4SF1 was also related to DNA promoter hypomethylation. CD9, TM4SF4, TSPAN2, and TSPAN16 were associated with OS in PTC patients and TSPAN2 has great potential to become a prognostic marker of PTC progression. For the prognostic model, the riskscore = (-0.0058)*CD82+(-0.4994)*+(0.1584)*TSPAN11+(1.7597)*TSPAN19+(0.2694)*TSPAN2 (lambda.min = 0.0149). The AUCs for 3-year, 5-year, and 10-year OS were 0.81, 0.851, and 0.804. TSPAN18, TSPAN31, and TSPAN32 were associated with both TMB and MSI in PTC patients. Conclusion: Our findings identified TM4SF1 as a potential diagnostic marker of LNM and TSPAN2 as a prognostic factor for patients with PTC. Our study provides a novel strategy to assess prognosis and predict effective treatments in PTC.

Comparison of a preoperative MR-based recurrence risk score versus the postoperative score and four clinical staging systems in hepatocellular carcinoma: a retrospective cohort study

OBJECTIVES

To establish a risk score integrating preoperative gadoxetic acid-enhanced magnetic resonance imaging (EOB-MRI) and clinical parameters to predict recurrence after hepatectomy for patients with hepatocellular carcinoma (HCC) and to compare its performance with that of a postoperative score and four clinical staging systems.

METHODS

Consecutive patients with surgically confirmed HCC who underwent preoperative EOB-MRI between July 2015 and November 2020 were retrospectively included. Two recurrence risk scores, one incorporating only preoperative variables and the other incorporating all preoperative and postoperative variables, were constructed via Cox regression models.

RESULTS

A total of 214 patients (derivation set, n = 150; test set, n = 64) were included. Six preoperative variables, namely tumor number, infiltrative appearance, corona enhancement, alpha-fetoprotein (AFP) level, aspartate aminotransferase (AST) level, and sex, were independently associated with recurrence. After adding postoperative features, microvascular invasion and tumor differentiation were additional significant variables in lieu of corona enhancement and AFP level. Using the above variables, the preoperative score achieved a C-index of 0.741 on the test set, which was comparable with that of the postoperative score (0.729; p = 0.235). The preoperative score yielded a larger time-dependent area under the receiver operating characteristic curve at 1 year (0.844) than three existing systems (0.734-0.742; p < 0.05 for all). Furthermore, the preoperative score stratified patients into two prognostically distinct risk strata with low and high risks of recurrence (p < 0.001).

CONCLUSION

The preoperative score integrating EOB-MRI features, AFP and AST levels, and sex improves recurrence risk estimation in HCC.

KEY POINTS

• The preoperative risk score incorporating three EOB-MRI findings, AFP and AST levels, and sex achieved comparable performance with that of the postoperative score for predicting recurrence after hepatectomy in patients with HCC. • Two risk strata with low and high risks of recurrence were obtained based on the preoperative score. • The preoperative score may help tailor pretreatment decision-making and facilitate candidate selection for adjuvant clinical trials.

Evaluation of log odds of positive lymph nodes in predicting the survival of patients with non-small cell lung cancer treated with neoadjuvant therapy and surgery: a SEER cohort-based study

Background

Log odds of positive lymph nodes (LODDS) is a novel lymph node (LN) descriptor that demonstrates promising prognostic value in many tumors. However, there is limited information regarding LODDS in patients with non-small cell lung cancer (NSCLC), especially those receiving neoadjuvant therapy followed by lung surgery.

Methods

A total of 2059 patients with NSCLC who received neoadjuvant therapy and surgery were identified from the Surveillance, Epidemiology, and End Results (SEER) database. We used the X-tile software to calculate the LODDS cutoff value. Kaplan–Meier survival analysis and receiver operating characteristic (ROC) curve analysis were performed to compare predictive values of the American Joint Committee on Cancer (AJCC) N staging descriptor and LODDS. Univariate and multivariate Cox regression and inverse probability of treatment weighting (IPTW) analyses were conducted to construct a model for predicting prognosis.

Results

According to the survival analysis, LODDS had better differentiating ability than the N staging descriptor (log-rank test, P < 0.0001 vs. P = 0.031). The ROC curve demonstrated that the AUC of LODDS was significantly higher than that of the N staging descriptor in the 1-, 3-, and 5-year survival analyses (all P < 0.05). Univariate and multivariate Cox regression analyses showed that LODDS was an independent risk factor for patients with NSCLC receiving neoadjuvant therapy followed by surgery both before and after IPTW (all P < 0.001). A clinicopathological model with LODDS, age, sex, T stage, and radiotherapy could better predict prognosis.

Conclusions

Compared with the AJCC N staging descriptor, LODDS exhibited better predictive ability for patients with NSCLC receiving neoadjuvant therapy followed by surgery. A multivariate clinicopathological model with LODDS demonstrated a sound performance in predicting prognosis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-022-09908-3.

Inflammation-Related LncRNAs Signature for Prognosis and Immune Response Evaluation in Uterine Corpus Endometrial Carcinoma

Backgrounds

Uterine corpus endometrial carcinoma (UCEC) is one of the greatest threats on the female reproductive system. The aim of this study is to explore the inflammation-related LncRNA (IRLs) signature predicting the clinical outcomes and response of UCEC patients to immunotherapy and chemotherapy.

Methods

Consensus clustering analysis was employed to determine inflammation-related subtype. Cox regression methods were used to unearth potential prognostic IRLs and set up a risk model. The prognostic value of the prognostic model was calculated by the Kaplan-Meier method, receiver operating characteristic (ROC) curves, and univariate and multivariate analyses. Differential abundance of immune cell infiltration, expression levels of immunomodulators, the status of tumor mutation burden (TMB), the response to immune checkpoint inhibitors (ICIs), drug sensitivity, and functional enrichment in different risk groups were also explored. Finally, we used quantitative real-time PCR (qRT-PCR) to confirm the expression patterns of model IRLs in clinical specimens.

Results

All UCEC cases were divided into two clusters (C1 = 454) and (C2 = 57) which had significant differences in prognosis and immune status. Five hub IRLs were selected to develop an IRL prognostic signature (IRLPS) which had value in forecasting the clinical outcome of UCEC patients. Biological processes related to tumor and immune response were screened. Function enrichment algorithm showed tumor signaling pathways (ERBB signaling, TGF-β signaling, and Wnt signaling) were remarkably activated in high-risk group scores. In addition, the high-risk group had a higher infiltration level of M2 macrophages and lower TMB value, suggesting patients with high risk were prone to a immunosuppressive status. Furthermore, we determined several potential molecular drugs for UCEC.

Conclusion

We successfully identified a novel molecular subtype and inflammation-related prognostic model for UCEC. Our constructed risk signature can be employed to assess the survival of UCEC patients and offer a valuable reference for clinical treatment regimens.

Arterial Stiffness and Long-Term Risk of Health Outcomes: The Framingham Heart Study

BACKGROUND

Arterial stiffness increases with age and is associated with an increased risk of adverse outcomes on short-term follow-up (typically <10 years). Data regarding associations of arterial stiffness with health outcomes on longer-term follow-up are lacking.

METHODS

We evaluated 7283 Framingham Study participants (mean age 50 years, 53% women) who underwent assessment of carotid-femoral pulse wave velocity (a marker of arterial stiffness) via applanation tonometry at one or more routine examinations. We used time-dependent Cox proportional hazards regression models to relate carotid-femoral pulse wave velocity to the incidence of health outcomes (updating carotid-femoral pulse wave velocity and all covariates at serial examinations).

RESULTS

On long-term follow-up (median 15 years; minimum-maximum, 0-20), participants developed cardiometabolic disease (hypertension [1255 events]; diabetes [381 events]), chronic kidney disease (529 events), dementia (235 events), cardiovascular disease (684 events) and its components (coronary heart disease [314 events], heart failure [191 events], transient ischemic attacks or stroke [250 events]), and death (1086 events). In multivariable-adjusted models, each SD increment in carotid-femoral pulse wave velocity was associated with increased risk of hypertension (hazard ratio [HR], 1.32 [95% CI, 1.21-1.44]), diabetes (HR, 1.32 [95% CI, 1.11-1.58]), chronic kidney disease (1.19 [95% CI, 1.05-1.34]), dementia (HR 1.27 [95% CI, 1.06-1.53]), cardiovascular disease (HR, 1.20 [95% CI, 1.06-1.36]) and its components (coronary heart disease, HR 1.37 [95% CI, 1.13-1.65]; transient ischemic attack/stroke, HR, 1.24 [95% CI, 1.00-1.53]), and death (HR, 1.29 [95% CI, 1.17-1.43]). The association with heart failure was borderline nonsignificant (HR, 1.21 [95% CI, 0.98-1.51], P=0.08).

CONCLUSIONS

Our prospective observations of a large community-based sample establish the long-term prognostic importance of arterial stiffness for multiple health outcomes.

Accurate Prediction of Metachronous Liver Metastasis in Stage I-III Colorectal Cancer Patients Using Deep Learning With Digital Pathological Images

Objectives

Metachronous liver metastasis (LM) significantly impacts the prognosis of stage I-III colorectal cancer (CRC) patients. An effective biomarker to predict LM after surgery is urgently needed. We aimed to develop deep learning-based models to assist in predicting LM in stage I-III CRC patients using digital pathological images.

Methods

Six-hundred eleven patients were retrospectively included in the study and randomly divided into training (428 patients) and validation (183 patients) cohorts according to the 7:3 ratio. Digital HE images from training cohort patients were used to construct the LM risk score based on a 50-layer residual convolutional neural network (ResNet-50). An LM prediction model was established by multivariable Cox analysis and confirmed in the validation cohort. The performance of the integrated nomogram was assessed with respect to its calibration, discrimination, and clinical application value.

Results

Patients were divided into low- and high-LM risk score groups according to the cutoff value and significant differences were observed in the LM of the different risk score groups in the training and validation cohorts (P&lt;0.001). Multivariable analysis revealed that the LM risk score, VELIPI, pT stage and pN stage were independent predictors of LM. Then, the prediction model was developed and presented as a nomogram to predict the 1-, 2-, and 3-year probability of LM. The integrated nomogram achieved satisfactory discrimination, with C-indexes of 0.807 (95% CI: 0.787, 0.827) and 0.812 (95% CI: 0.773, 0.850) and AUCs of 0.840 (95% CI: 0.795, 0.885) and 0.848 (95% CI: 0.766, 0.931) in the training and validation cohorts, respectively. Favorable calibration of the nomogram was confirmed in the training and validation cohorts. Integrated discrimination improvement and net reclassification index indicated that the integrated nomogram was superior to the traditional clinicopathological model. Decision curve analysis confirmed that the nomogram has clinical application value.

Conclusions

The LM risk score based on ResNet-50 and digital HE images was significantly associated with LM. The integrated nomogram could identify stage I-III CRC patients at high risk of LM after primary colectomy, so it may serve as a potential tool to choose the appropriate treatment to improve the prognosis of stage I-III CRC patients.

Estimation and inference of predictive discrimination for survival outcome risk prediction models

Accurate risk prediction has been the central goal in many studies of survival outcomes. In the presence of multiple risk factors, a censored regression model can be employed to estimate a risk prediction rule. Before the prediction tool can be popularized for practical use, it is crucial to rigorously assess its prediction performance. In our motivating example, researchers are interested in developing and validating a risk prediction tool to identify future lung cancer cases by integrating demographic information, disease characteristics and smoking-related data. Considering the long latency period of cancer, it is desirable for a prediction tool to achieve discriminative performance that does not weaken over time. We propose estimation and inferential procedures to comprehensively assess both the overall predictive discrimination and the temporal pattern of an estimated prediction rule. The proposed methods readily accommodate commonly used censored regression models, including the Cox proportional hazards model and the accelerated failure time model. The estimators are consistent and asymptotically normal, and reliable variance estimators are also developed. The proposed methods offer an informative tool for inferring time-dependent predictive discrimination, as well as for comparing the discrimination performance between candidate models. Applications of the proposed methods demonstrate enduring performance of the risk prediction tool in the PLCO study and detected decaying performance in a study of liver disease.

Association of baseline and change in global longitudinal strain by computed tomography with post-transcatheter aortic valve replacement outcomes

AIMS

Transcatheter aortic valve replacement (TAVR) procedural planning requires computed tomography angiography (CTA) which allows for the assessment of left ventricular global longitudinal strain (CTA-LVGLS). There is, however, limited data on the feasibility of CTA-LVGLS, and its prognostic value. This study sought to evaluate the incremental prognostic value of baseline CTA-LVGLS, change in CTA-LVGLS after TAVR, and their association with post-TAVR outcomes.

METHODS AND RESULTS

A total of 431 patients who underwent multiphasic gated CTA using dual-source system for TAVR planning at baseline and 1-month follow-up were included [median (interquartile range) age, 83 (77-87) years; 44% female, STS-PROM score: 3.3 (2.3-5.1)%, Echo-left ventricular ejection fraction (LVEF): 60 (55-65)%, CTA-LVGLS: -18.0 (-21.6 to -14.2)%, feasible in 97% of patients]. CTA-LVGLS was measured using dedicated feature-tracking software. Over a median follow-up of 19 (13-27) months, 99 endpoints of all-cause death or heart failure hospitalization occurred. The relative hazard of the endpoint increased as baseline CTA-LVGLS worsened with -18.2% as the threshold for higher events (P = 0.005). After adjustment for baseline characteristics, CTA-LVGLS remained associated with the endpoint [hazard ratio (HR) (95% confidence interval, CI), 1.08 (1.03-1.14); P = 0.005] and incrementally improved prognostication (C-index difference, 0.026). Although CTA-LVGLS improved after TAVR [-18.3 (-21.6 to -14.3)% vs. -18.7 (-21.9 to -15.4)%, P < 0.001], patients without CTA-LVGLS improvement had higher risk of the endpoint than those with improvement or preserved baseline global longitudinal strain [HR (95% CI), 1.92 (1.19-3.12); P = 0.008].

CONCLUSIONS

In this predominantly low-risk TAVR cohort of patients, mostly with normal LVEF, assessment of CTA-LVGLS is highly feasible improving risk stratification by providing independent and incremental prognostic value over clinical and echocardiographic characteristics.

Model-Based ROC Curve: Examining the Effect of Case Mix and Model Calibration on the ROC Plot

Background

The performance of risk prediction models is often characterized in terms of discrimination and calibration. The receiver-operating characteristic (ROC) curve is widely used for evaluating model discrimination. However, when comparing ROC curves across different samples, the effect of case mix makes the interpretation of discrepancies difficult. Further, compared with model discrimination, evaluating model calibration has not received the same level of attention. Current methods for examining model calibration require specification of smoothing or grouping factors.

Methods

We introduce the “model-based” ROC curve (mROC) to assess model calibration and the effect of case mix during external validation. The mROC curve is the ROC curve that should be observed if the prediction model is calibrated in the external population. We show that calibration-in-the-large and the equivalence of mROC and ROC curves are together sufficient conditions for the model to be calibrated. Based on this, we propose a novel statistical test for calibration that, unlike current methods, does not require any subjective specification of smoothing or grouping factors.

Results

Through a stylized example, we demonstrate how mROC separates the effect of case mix and model miscalibration when externally validating a risk prediction model. We present the results of simulation studies that confirm the properties of the new calibration test. A case study on predicting the risk of acute exacerbations of chronic obstructive pulmonary disease puts the developments in a practical context. R code for the implementation of this method is provided.

Conclusion

mROC can easily be constructed and used to interpret the effect of case mix and calibration on the ROC plot. Given the popularity of ROC curves among applied investigators, this framework can further promote assessment of model calibration.

Highlights

Xanthine dehydrogenase as a prognostic biomarker related to tumor immunology in hepatocellular carcinoma

Background

Xanthine dehydrogenase (XDH) is a critical enzyme involved in the oxidative metabolism of purines, pterin and aldehydes and a central component of the innate immune system. However, the prognostic value of XDH in predicting tumor-infiltrating lymphocyte abundance, the immune response, and survival in different cancers, including hepatocellular carcinoma (HCC), is still unclear.

Methods

XDH expression was analyzed in multiple databases, including Oncomine, the Tumor Immune Estimation Resource (TIMER), the Kaplan–Meier plotter database, the Gene Expression Profiling Interactive Analysis (GEPIA) database, and The Cancer Genome Atlas (TCGA). XDH-associated transcriptional profiles were detected with an mRNA array, and the levels of infiltrating immune cells were validated by immunohistochemistry (IHC) of HCC tissues. A predictive signature containing multiple XDH-associated immune genes was established using the Cox regression model.

Results

Decreased XDH mRNA expression was detected in human cancers originating from the liver, bladder, breast, colon, bile duct, kidney, and hematolymphoid system. The prognostic potential of XDH mRNA expression was also significant in certain other cancers, including HCC, breast cancer, kidney or bladder carcinoma, gastric cancer, mesothelioma, lung cancer, and ovarian cancer. In HCC, a low XDH mRNA level predicted poorer overall survival, disease-specific survival, disease-free survival, and progression-free survival. The prognostic value of XDH was independent of the clinical features of HCC patients. Indeed, XDH expression in HCC activated several immune-related pathways, including the T cell receptor, PI3K-AKT, and MAPK signaling pathways, which induced a cytotoxic immune response. Importantly, the microenvironment of XDH^high HCC tumors contained abundant infiltrating CD8 + T cells but not exhausted T cells. A risk prediction signature based on multiple XDH-associated immune genes was revealed as an independent predictor in the TCGA liver cancer cohort.

Conclusion

These findings suggest that XDH is a valuable prognostic biomarker in HCC and other cancers and indicate that it may function in tumor immunology. Loss of XDH expression may be an immune evasion mechanism for HCC.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12935-021-02173-7.

On the time-varying predictive performance of longitudinal biomarkers: Measure and estimation

In many biomedical studies, participants are monitored at periodic visits until the occurrence of the failure event. Biomarkers are often measured repeatedly during these visits, and such measurements can facilitate updated disease prediction. In this work, we propose a two-dimensional incident dynamic area under curve (AUC), to capture the variability due to both the biomarker assessment time and the prediction time to comprehensively quantify the predictive performance of a longitudinal biomarker. We propose a pseudo partial-likelihood to achieve consistent estimation of the AUC under two realistic scenarios of visit schedules. Variance estimation methods are designed to facilitate inferential procedures. We examine the finite-sample performance of our method through extensive simulations. The methods are applied to a study of chronic myeloid leukemia to evaluate the predictive performance of longitudinally collected gene expression levels.

Individual mortality risk predictive system of patients with acute-on-chronic liver failure based on a random survival forest model

Background:

The basis of individualized treatment should be individualized mortality risk predictive information. The present study aimed to develop an online individual mortality risk predictive tool for acute-on-chronic liver failure (ACLF) patients based on a random survival forest (RSF) algorithm.

Methods:

The current study retrospectively enrolled ACLF patients from the Department of Infectious Diseases of The First People's Hospital of Foshan, Shunde Hospital of Southern Medical University, and Jiangmen Central Hospital. Two hundred seventy-six consecutive ACLF patients were included in the present study as a model cohort (n = 276). Then the current study constructed a validation cohort by drawing patients from the model dataset based on the resampling method (n = 276). The RSF algorithm was used to develop an individual prognostic model for ACLF patients. The Brier score was used to evaluate the diagnostic accuracy of prognostic models. The weighted mean rank estimation method was used to compare the differences between the areas under the time-dependent ROC curves (AUROCs) of prognostic models.

Results:

Multivariate Cox regression identified hepatic encephalopathy (HE), age, serum sodium level, acute kidney injury (AKI), red cell distribution width (RDW), and international normalization index (INR) as independent risk factors for ACLF patients. A simplified RSF model was developed based on these previous risk factors. The AUROCs for predicting 3-, 6-, and 12-month mortality were 0.916, 0.916, and 0.905 for the RSF model and 0.872, 0.866, and 0.848 for the Cox model in the model cohort, respectively. The Brier scores were 0.119, 0.119, and 0.128 for the RSF model and 0.138, 0.146, and 0.156 for the Cox model, respectively. The nonparametric comparison suggested that the RSF model was superior to the Cox model for predicting the prognosis of ACLF patients.

Conclusions:

The current study developed a novel online individual mortality risk predictive tool that could predict individual mortality risk predictive curves for individual patients. Additionally, the current online individual mortality risk predictive tool could further provide predicted mortality percentages and 95% confidence intervals at user-defined time points.

A Prognostic Model Based on the Immune-Related lncRNAs in Colorectal Cancer

Background

Colorectal cancer (CRC) is one of the most common malignant tumors with a poor prognosis. At present, the pathogenesis is not completely clear. Therefore, finding reliable prognostic indicators for CRC is of important clinical significance. In this study, bioinformatics methods were used to screen the prognostic immune-related lncRNAs of CRC, and a prognostic risk scoring model based on immune-related lncRNAs signatures were constructed to provide a basis for prognostic evaluation and immunotherapy of CRC patients.

Methods

The clinical information and RNA-seq data of CRC patients were obtained from The Cancer Genome Atlas (TCGA) database. The information of immune-related lncRNA was downloaded from the immunology database and analysis portal. The differentially expressed immune-related lncRNAs (IRLs) were screened by the edgeR package of R software. The prognostic value of IRLs was studied. Based on Cox regression analysis, a prognostic index (IRLPI) based on IRLs was established, and the relationship between the risk score and the clinicopathological characteristics of CRC was analyzed to determine the effectiveness of the risk score model as an independent prognostic factor.

Results

A total of 240 differentially expressed IRLs were identified between normal colorectal cancer tissues and normal colorectal cancer tissues, in which 8 were significantly associated with the survival of CRC patients (P < 0.05), including LINC00461, LINC01055, ELFN1-AS1, LMO7-AS1, CYP4A22-AS1, AC079612.1, LINC01351, and MIR31HG. And most of the lncRNAs related to survival were risk factors for the prognosis of CRC. The index established based on the 7 survival-related IRLs found to be highly accurate in monitoring CRC prognosis. Besides, IRLPI was significantly correlated with a variety of pathological factors and immune cell infiltration.

Conclusion

Eight immune-related lncRNAs closely related to the prognosis of CRC patients were identified from the TCGA database. At the same time, an independent IRLPI was constructed, which may be helpful for clinicians to assess the prognosis of patients with CRC and to formulate individualized treatment plans.

Inference about time-dependent prognostic accuracy measures in the presence of competing risks

BACKGROUND

Evaluating a candidate marker or developing a model for predicting risk of future conditions is one of the major goals in medicine. However, model development and assessment for a time-to-event outcome may be complicated in the presence of competing risks. In this manuscript, we propose a local and a global estimators of cause-specific AUC for right-censored survival times in the presence of competing risks.

METHODS

The local estimator - cause-specific weighted mean rank (cWMR) - is a local average of time-specific observed cause-specific AUCs within a neighborhood of given time t. The global estimator - cause-specific fractional polynomials (cFPL) - is based on modelling the cause-specific AUC as a function of t through fractional polynomials.

RESULTS

We investigated the performance of the proposed cWMR and cFPL estimators through simulation studies and real-life data analysis. The estimators perform well in small samples, have minimal bias and appropriate coverage.

CONCLUSIONS

The local estimator cWMR and the global estimator cFPL will provide computationally efficient options for assessing the prognostic accuracy of markers for time-to-event outcome in the presence of competing risks in many practical settings.

Predictive accuracy of markers or risk scores for interval censored survival data

Methods for the evaluation of the predictive accuracy of biomarkers with respect to survival outcomes subject to right censoring have been discussed extensively in the literature. In cancer and other diseases, survival outcomes are commonly subject to interval censoring by design or due to the follow up schema. In this article, we present an estimator for the area under the time-dependent receiver operating characteristic (ROC) curve for interval censored data based on a nonparametric sieve maximum likelihood approach. We establish the asymptotic properties of the proposed estimator and illustrate its finite-sample properties using a simulation study. The application of our method is illustrated using data from a cancer clinical study. An open-source R package to implement the proposed method is available on Comprehensive R Archive Network.

Predicting Clinical Outcome in Expanded Criteria Donor Kidney Transplantation: A Retrospective Cohort Study

Background:

The gaps in organ supply and demand necessitate the use of expanded criteria donor (ECD) kidneys.

Objective:

To identify which pre-transplant and post-transplant predictors are most informative regarding short- and long-term ECD transplant outcomes.

Design:

Retrospective cohort study.

Setting:

Single center, Quebec, Canada.

Patients:

The patients were 163 consecutive first-time ECD kidney only transplant recipients who underwent transplantation at McGill University Health Centre (MUHC) between January 1, 2008 and December 31, 2014 and had frozen section wedge procurement biopsies.

Measurements:

Short-term graft outcomes, including delayed graft function and 1-year estimated glomerular filtration rate (eGFR), as well as long-term outcomes including all-cause graft loss (defined as return to dialysis, retransplantation, and death with function).

Methods:

Pre-transplant donor, recipient, and transplant characteristics were assessed as predictors of transplant outcomes. The added value of post-transplant predictors, including longitudinal eGFR, was also assessed using time-varying Cox proportional hazards models.

Results:

In univariate analyses, among the pre-transplant donor characteristics, histopathologic variables did not show evidence of association with delayed graft function, 1-year post-transplant eGFR or all cause graft loss. Recipient age was associated with all-cause graft loss (hazard ratio: 1.038 [95% confidence interval: 1.002-1.075] and the model produced only modest discrimination (C-index: 0.590; standard error [SE]: 0.045). Inclusion of time-dependent post-transplant eGFR improved the model’s prediction accuracy (C-index: 0.711; SE = 0.047). Pre-transplant ECD characteristics were not associated with long-term survival, whereas post-transplant characteristics allowed better model discrimination.

Limitations:

Single-center study, small sample size, and potential incomplete capture of all covariate data.

Conclusions:

Incorporation of dynamic prediction models into electronic health records may enable timely mitigation of ECD graft failure risk and/or facilitate planning for renal replacement therapies. Histopathologic findings on preimplantation biopsies have a limited role in predicting long-term ECD outcomes.

Trial registration:

Not applicable.

Serum Albumin at Partial Remission Predicts Outcomes in Membranous Nephropathy

Background

In primary membranous nephropathy (MN), partial remission (PR) (≥50% reduction of proteinuria to <3.5 g/d) is associated with a greater risk of relapse and end-stage kidney disease (ESKD) compared with complete remission (CR). We aimed to determine factors associated with relapse or renal failure in patients who attain the standard definition of PR.

Methods

We captured PR, CR, relapse, and the composite of doubling of serum creatinine or ESKD in a cohort of 267 patients with MN, nephrotic syndrome, and >12 months of follow-up. Characteristics at the time of PR associated with the composite outcome or relapse were evaluated using a time-to-event analysis.

Results

A total of 192 patients attained PR and 86 attained CR. Serum albumin at PR (hazard ratio [HR]: 1.58 per 0.5 g/dl decrease from 4.0 g/dl; 95% confidence interval [CI]: 1.03-2.43) and duration of nephrotic proteinuria (HR: 1.01 per month increase; 95% CI: 1.00-1.03) were independent risk factors for the composite endpoint. Serum albumin at PR was associated with an increased risk of relapse (HR: 1.58 per 0.5 g/dl decrease below 4.0 g/dl; 95% CI: 1.24-2.01). A cutoff for serum albumin ≤3.5 g/dl at PR performed best in predicting relapse and composite outcome.

Conclusions

Patients with serum albumin >3.5 g/dl at PR have decreased risk of composite outcome or relapse compared with PR with low albumin. A definition of PR that includes normalization of serum albumin may be a more robust surrogate endpoint in MN than the traditional definition of PR.

Prioritized concordance index for hierarchical survival outcomes

We propose an extension of Harrell's concordance (C) index to evaluate the prognostic utility of biomarkers for diseases with multiple measurable outcomes that can be prioritized. Our prioritized concordance index measures the probability that, given a random subject pair, the subject with the worst disease status as of a time τ has the higher predicted risk. Our prioritized concordance index uses the same approach as the win ratio, by basing generalized pairwise comparisons on the most severe or clinically important comparable outcome. We use an inverse probability weighting technique to correct for study-specific censoring. Asymptotic properties are derived using U-statistic properties. We apply the prioritized concordance index to two types of disease processes with a rare primary outcome and a more common secondary outcome. Our simulation studies show that when a predictor is predictive of both outcomes, the new concordance index can gain efficiency and power in identifying true prognostic variables compared to using the primary outcome alone. Using the prioritized concordance index, we examine whether novel clinical measures can be useful in predicting risk of type II diabetes in patients with impaired glucose resistance whose disease status can also regress to normal glucose resistance. We also examine the discrimination ability of four published risk models among ever smokers at risk of lung cancer incidence and subsequent death.

A 17-marker panel for global genomic instability in breast cancer

Genomic instability is a hallmark of cancer that plays a pivotal role in breast cancer development and evolution. A number of existing prognostic gene expression signatures for breast cancer are based on proliferation-related genes. Here, we identified a 17-marker panel associated with genome stability. A total of 136 primary breast carcinomas were stratified by genome stability. Matched gene expression profiles showed an innate segregation based on genome stability. We identified a 17-marker panel stratifying the training and validation cohorts into high- and low-risk patients. The 17 genes associated with genomic instability strongly impacted clinical outcome in breast cancer. Pathway analyses determined chromosome organisation, cell cycle regulation, and RNA processing as the underlying biological processes, thereby offering options for drug development and treatment tailoring. Our work supports the applicability of the 17-marker panel to improve clinical outcome prediction for breast cancer patients based on a signature accounting for genomic instability.

A Novel Tool to Evaluate the Accuracy of Predicting Survival and Guiding Lung Transplantation in Cystic Fibrosis

Background

Effective transplantation recommendations in cystic fibrosis (CF) require accurate survival predictions, so that high-risk patients may be prioritized for transplantation. In practice, decisions about transplantation are made dynamically, using routinely updated assessments. We present a novel tool for evaluating risk prediction models that, unlike traditional methods, captures classification accuracy in identifying high-risk patients in a dynamic fashion.

Methods

Predicted risk is used as a score to rank incident deaths versus patients who survive, with the goal of ranking the deaths higher. The mean rank across deaths at a given time measures time-specific predictive accuracy; when assessed over time, it reflects time-varying accuracy.

Results

Applying this approach to CF Registry data on patients followed from 1993-2011, we show that traditional methods do not capture the performance of models used dynamically in the clinical setting. Previously proposed multivariate risk scores perform no better than forced expiratory volume in 1 second as a percentage of predicted normal (FEV₁%) alone. Despite its value for survival prediction, FEV₁% has a low sensitivity of 45% over time (for fixed specificity of 95%), leaving room for improvement in prediction. Finally, prediction accuracy with annually-updated FEV₁% shows minor differences compared to FEV₁% updated every 2 years, which may have clinical implications regarding the optimal frequency of updating clinical information.

Conclusions

It is imperative to continue to develop models that accurately predict survival in CF. Our proposed approach can serve as the basis for evaluating the predictive ability of these models by better accounting for their dynamic clinical use.

A comparison of landmark methods and time-dependent ROC methods to evaluate the time-varying performance of prognostic markers for survival outcomes

BACKGROUND

Prognostic markers use an individual's characteristics at a given time to predict future disease events, with the ultimate goal of guiding medical decision-making. If an accurate prediction can be made, then a prognostic marker could be used clinically to identify those subjects at greatest risk for future adverse events and may be used to define populations appropriate for targeted therapeutic intervention. Often, a marker is measured at a single baseline time point such as disease diagnosis, and then used to guide decisions at multiple subsequent time points. However, the performance of candidate markers may vary over time as an individual's underlying clinical status changes.

METHODS

We provide an overview and comparison of modern statistical methods for evaluating the time-varying accuracy of a baseline prognostic marker. We compare approaches that consider cumulative versus incident events. Additionally, we compare the common approach of using hazard ratios obtained from Cox proportional hazards regression to more recently developed approaches using time-dependent receiver operating characteristic (ROC) curves. The alternative statistical summaries are illustrated using a multiple myeloma study of candidate biomarkers.

RESULTS

We found that time-varying HRs, HR (t), using local linear estimation revealed time trends more clearly by directly estimating the association at each time point t, compared to landmark analyses, which averaged across time ≥ t. Comparing area under the ROC curve (AUC) summaries, there was close agreement between AUC ^C/D (t,t+1) which defines cases cumulatively over 1-year intervals and AUC ^I/D (t) which defines cases as incident events. HR (t) was more consistent with AUC ^I/D (t), as estimation of these measures is localized at each time point.

CONCLUSIONS

We compared alternative summaries for quantifying a prognostic marker's time-varying performance. Although landmark-based predictions may be useful when patient predictions are needed at select times, a focus on incident events naturally facilitates evaluating trends in performance over time.

A Tutorial on Evaluating the Time-Varying Discrimination Accuracy of Survival Models Used in Dynamic Decision Making

Many medical decisions involve the use of dynamic information collected on individual patients toward predicting likely transitions in their future health status. If accurate predictions are developed, then a prognostic model can identify patients at greatest risk for future adverse events and may be used clinically to define populations appropriate for targeted intervention. In practice, a prognostic model is often used to guide decisions at multiple time points over the course of disease, and classification performance (i.e., sensitivity and specificity) for distinguishing high-risk v. low-risk individuals may vary over time as an individual's disease status and prognostic information change. In this tutorial, we detail contemporary statistical methods that can characterize the time-varying accuracy of prognostic survival models when used for dynamic decision making. Although statistical methods for evaluating prognostic models with simple binary outcomes are well established, methods appropriate for survival outcomes are less well known and require time-dependent extensions of sensitivity and specificity to fully characterize longitudinal biomarkers or models. The methods we review are particularly important in that they allow for appropriate handling of censored outcomes commonly encountered with event time data. We highlight the importance of determining whether clinical interest is in predicting cumulative (or prevalent) cases over a fixed future time interval v. predicting incident cases over a range of follow-up times and whether patient information is static or updated over time. We discuss implementation of time-dependent receiver operating characteristic approaches using relevant R statistical software packages. The statistical summaries are illustrated using a liver prognostic model to guide transplantation in primary biliary cirrhosis.

Dynamic thresholds and a summary ROC curve: Assessing prognostic accuracy of longitudinal markers

Cancer patients, chronic kidney disease patients, and subjects infected with HIV are routinely monitored over time using biomarkers that represent key health status indicators. Furthermore, biomarkers are frequently used to guide initiation of new treatments or to inform changes in intervention strategies. Since key medical decisions can be made on the basis of a longitudinal biomarker, it is important to evaluate the potential accuracy associated with longitudinal monitoring. To characterize the overall accuracy of a time-dependent marker, we introduce a summary ROC curve that displays the overall sensitivity associated with a time-dependent threshold that controls time-varying specificity. The proposed statistical methods are similar to concepts considered in disease screening, yet our methods are novel in choosing a potentially time-dependent threshold to define a positive test, and our methods allow time-specific control of the false-positive rate. The proposed summary ROC curve is a natural averaging of time-dependent incident/dynamic ROC curves and therefore provides a single summary of net error rates that can be achieved in the longitudinal setting.

A Novel 18-Marker Panel Predicting Clinical Outcome in Breast Cancer

Background: Gene expression profiling has made considerable contributions to our understanding of cancer biology and clinical care. This study describes a novel gene expression signature for breast cancer-specific survival that was validated using external datasets.Methods: Gene expression signatures for invasive breast carcinomas (mainly luminal B subtype) corresponding to 136 patients were analyzed using Cox regression, and the effect of each gene on disease-specific survival (DSS) was estimated. Iterative Bayesian model averaging was applied on multivariable Cox regression models resulting in an 18-marker panel, which was validated using three external validation datasets. The 18 genes were analyzed for common pathways and functions using the Ingenuity Pathway Analysis software. This study complied with the REMARK criteria.Results: The 18-gene multivariable model showed a high predictive power for DSS in the training and validation cohort and a clear stratification between high- and low-risk patients. The differentially expressed genes were predominantly involved in biological processes such as cell cycle, DNA replication, recombination, and repair. Furthermore, the majority of the 18 genes were found to play a pivotal role in cancer.Conclusions: Our findings demonstrated that the 18 molecular markers were strong predictors of breast cancer-specific mortality. The stable time-dependent area under the ROC curve function (AUC(t)) and high C-indices in the training and validation cohorts were further improved by fitting a combined model consisting of the 18-marker panel and established clinical markers.Impact: Our work supports the applicability of this 18-marker panel to improve clinical outcome prediction for breast cancer patients. Cancer Epidemiol Biomarkers Prev; 26(11); 1619-28. ©2017 AACR.

Time-dependent ROC curve analysis in medical research: current methods and applications

BACKGROUND

ROC (receiver operating characteristic) curve analysis is well established for assessing how well a marker is capable of discriminating between individuals who experience disease onset and individuals who do not. The classical (standard) approach of ROC curve analysis considers event (disease) status and marker value for an individual as fixed over time, however in practice, both the disease status and marker value change over time. Individuals who are disease-free earlier may develop the disease later due to longer study follow-up, and also their marker value may change from baseline during follow-up. Thus, an ROC curve as a function of time is more appropriate. However, many researchers still use the standard ROC curve approach to determine the marker capability ignoring the time dependency of the disease status or the marker.

METHODS

We comprehensively review currently proposed methodologies of time-dependent ROC curves which use single or longitudinal marker measurements, aiming to provide clarity in each methodology, identify software tools to carry out such analysis in practice and illustrate several applications of the methodology. We have also extended some methods to incorporate a longitudinal marker and illustrated the methodologies using a sequential dataset from the Mayo Clinic trial in primary biliary cirrhosis (PBC) of the liver.

RESULTS

From our methodological review, we have identified 18 estimation methods of time-dependent ROC curve analyses for censored event times and three other methods can only deal with non-censored event times. Despite the considerable numbers of estimation methods, applications of the methodology in clinical studies are still lacking.

CONCLUSIONS

The value of time-dependent ROC curve methods has been re-established. We have illustrated the methods in practice using currently available software and made some recommendations for future research.

Validation of tumor protein marker quantification by two independent automated immunofluorescence image analysis platforms

Protein marker levels in formalin-fixed, paraffin-embedded tissue sections traditionally have been assayed by chromogenic immunohistochemistry and evaluated visually by pathologists. Pathologist scoring of chromogen staining intensity is subjective and generates low-resolution ordinal or nominal data rather than continuous data. Emerging digital pathology platforms now allow quantification of chromogen or fluorescence signals by computer-assisted image analysis, providing continuous immunohistochemistry values. Fluorescence immunohistochemistry offers greater dynamic signal range than chromogen immunohistochemistry, and combined with image analysis holds the promise of enhanced sensitivity and analytic resolution, and consequently more robust quantification. However, commercial fluorescence scanners and image analysis software differ in features and capabilities, and claims of objective quantitative immunohistochemistry are difficult to validate as pathologist scoring is subjective and there is no accepted gold standard. Here we provide the first side-by-side validation of two technologically distinct commercial fluorescence immunohistochemistry analysis platforms. We document highly consistent results by (1) concordance analysis of fluorescence immunohistochemistry values and (2) agreement in outcome predictions both for objective, data-driven cutpoint dichotomization with Kaplan-Meier analyses or employment of continuous marker values to compute receiver-operating curves. The two platforms examined rely on distinct fluorescence immunohistochemistry imaging hardware, microscopy vs line scanning, and functionally distinct image analysis software. Fluorescence immunohistochemistry values for nuclear-localized and tyrosine-phosphorylated Stat5a/b computed by each platform on a cohort of 323 breast cancer cases revealed high concordance after linear calibration, a finding confirmed on an independent 382 case cohort, with concordance correlation coefficients >0.98. Data-driven optimal cutpoints for outcome prediction by either platform were reciprocally applicable to the data derived by the alternate platform, identifying patients with low Nuc-pYStat5 at ~3.5-fold increased risk of disease progression. Our analyses identified two highly concordant fluorescence immunohistochemistry platforms that may serve as benchmarks for testing of other platforms, and low interoperator variability supports the implementation of objective tumor marker quantification in pathology laboratories.

Time-dependent classification accuracy curve under marker-dependent sampling

Evaluating the classification accuracy of a candidate biomarker signaling the onset of disease or disease status is essential for medical decision making. A good biomarker would accurately identify the patients who are likely to progress or die at a particular time in the future or who are in urgent need for active treatments. To assess the performance of a candidate biomarker, the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) are commonly used. In many cases, the standard simple random sampling (SRS) design used for biomarker validation studies is costly and inefficient. In order to improve the efficiency and reduce the cost of biomarker validation, marker-dependent sampling (MDS) may be used. In a MDS design, the selection of patients to assess true survival time is dependent on the result of a biomarker assay. In this article, we introduce a nonparametric estimator for time-dependent AUC under a MDS design. The consistency and the asymptotic normality of the proposed estimator is established. Simulation shows the unbiasedness of the proposed estimator and a significant efficiency gain of the MDS design over the SRS design.

Which Biomarker is the Best for Predicting Mortality in Incident Peritoneal Dialysis Patients: NT-ProBNP, Cardiac TnT, or hsCRP?: A Prospective Observational Study

Although numerous previous studies have explored various biomarkers for their ability to predict mortality in end-stage renal disease (ESRD) patients, these studies have been limited by retrospective analyses, mostly prevalent dialysis patients, and the measurement of only 1 or 2 biomarkers. This prospective study was aimed to evaluate the association between 3 biomarkers and mortality in incident 335 ESRD patients starting continuous ambulatory peritoneal dialysis (CAPD) in Korea. According to the baseline NT-proBNP, cTnT, and hsCRP levels, the patients were stratified into tertiles, and cardiovascular (CV) and all-cause mortalities were compared. Additionally, time-dependent ROC curves were constructed, and the net reclassification index (NRI) and integrated discrimination improvement (IDI) of the models with various biomarkers were calculated. We found the upper tertile of NT-proBNP was significantly associated with increased risk of both CV and all-cause mortalities. However, the upper tertile of hsCRP was significantly related only to the high risk of all-cause mortality even after adjustment for age, sex, and white blood cell counts. Moreover, NT-proBNP had the highest predictive power for CV mortality, whereas hsCRP was the best prognostic marker for all-cause mortality among these biomarkers. In conclusions, NT-proBNP is a more significant prognostic factor for CV mortality than cTnT and hsCRP, whereas hsCRP is a more significant predictor than NT-proBNP and cTnT for all-cause mortality in incident peritoneal dialysis patients.

Lymph node ratio-based staging system for esophageal squamous cell carcinoma

AIM: To analyze a modified staging system utilizing lymph node ratio (LNR) in patients with esophageal squamous cell carcinoma (ESCC).

METHODS: Clinical data of 2011 patients with ESCC who underwent surgical resection alone between January 1995 and June 2010 at the Cancer Hospital of Shantou University Medical College were reviewed. The LNR, or node ratio (Nr) was defined as the ratio of metastatic LNs ompared to the total number of resected LNs. Overall survival between groups was compared with the log-rank test. The cutoff point of LNR was established by grouping patients with 10% increment in Nr, and then combining the neighborhood survival curves using the log-rank test. A new TNrM staging system, was constructed by replacing the American Joint Committee on Cancer (AJCC) N categories with the Nr categories in the new TNM staging system. The time-dependent receiver operating characteristic curves were used to evaluate the predictive performance of the seventh edition AJCC staging system and the TNrM staging system.

RESULTS: The median number of resected LNs was 12 (range: 4-44), and 25% and 75% interquartile rangeswere8 and 16. Patients were classified into four Nr categories with distinctive survival differences (Nr0: LNR = 0; Nr1: 0% < LNR ≤ 10%; Nr2: 10% < LNR ≤ 20%; and Nr3: LNR > 20%). From N categories to Nr categories, 557 patients changed their LN stage. The median survival time (MST) for the four Nr categories (Nr0-Nr3) was 155.0 mo, 39.0 mo, 28.0 mo, and 19.0 mo, respectively, and the 5-year overall survival was 61.1%, 41.1%, 33.0%, and 22.9%, respectively (P < 0.001). Overall survival was significantly different for the AJCC N categories when patients were subgrouped into 15 or more vs fewer than 15 examined nodes, except for the N3 category (P = 0.292). However, overall survival was similar when the patients in all four Nr categories were subgrouped into 15 or more vs fewer than 15 nodes. Using the time-dependent receiver operating characteristic, we found that the Nr category and TNrM stage had higher accuracy in predicting survival than the AJCC N category and TNM stage.

CONCLUSION: A staging system based on LNR may have better prognostic stratification of patients with ESCC than the current TNM system, especially for those undergoing limited lymphadenectomy.

A direct method to evaluate the time-dependent predictive accuracy for biomarkers

Time-dependent receiver operating characteristic (ROC) curves and their area under the curve (AUC) are important measures to evaluate the prediction accuracy of biomarkers for time-to-event endpoints (e.g., time to disease progression or death). In this article, we propose a direct method to estimate AUC(t) as a function of time t using a flexible fractional polynomials model, without the middle step of modeling the time-dependent ROC. We develop a pseudo partial-likelihood procedure for parameter estimation and provide a test procedure to compare the predictive performance between biomarkers. We establish the asymptotic properties of the proposed estimator and test statistics. A major advantage of the proposed method is its ease to make inference and to compare the prediction accuracy across biomarkers, rendering our method particularly appealing for studies that require comparing and screening a large number of candidate biomarkers. We evaluate the finite-sample performance of the proposed method through simulation studies and illustrate our method in an application to AIDS Clinical Trials Group 175 data.

NT-proBNP: is it a more significant risk factor for mortality than troponin T in incident hemodialysis patients?

Numerous studies have demonstrated that cardiac biomarkers are significant predictors of cardiovascular (CV) and all-cause mortality in ESRD patients, but most of the studies were retrospective or included small numbers of patients, only prevalent dialysis patients, or measured 1 or 2 biomarkers. This study was to analyze the association between 3 cardiac biomarkers and mortality in incident HD patients. A prospective cohort of 864 incident HD patients was followed for 30 months. Based on the median values of baseline NT-proBNP, cTnT, and hsCRP, the patients were divided into "high" and "low" groups, and CV and all-cause mortality were compared between each group. Additionally, time-dependent ROC curves were constructed, and the NRI and IDI of the models with various biomarkers were calculated. The CV survival rates were significantly lower in the "high" NT-proBNP and cTnT groups compared to the corresponding "low" groups, while there was no significant difference in CV survival rate between the 2 hsCRP groups. However, all-cause mortality rates were significantly higher in all 3 "high" groups compared to each lower group. In multivariate analyses, only Ln NT-proBNP was found to be an independent predictor of mortality. Moreover, NT-proBNP was a more prognostic marker for mortality compared to cTnT. In conclusion, NT-proBNP is the biomarker that results in the most added prognostic value on top of traditional risk factors for CV and all-cause mortality in incident HD patients.

Bayesian quantile regression for censored data

In this paper we propose a semiparametric quantile regression model for censored survival data. Quantile regression permits covariates to affect survival differently at different stages in the follow-up period, thus providing a comprehensive study of the survival distribution. We take a semiparametric approach, representing the quantile process as a linear combination of basis functions. The basis functions are chosen so that the prior for the quantile process is centered on a simple location-scale model, but flexible enough to accommodate a wide range of quantile processes. We show in a simulation study that this approach is competitive with existing methods. The method is illustrated using data from a drug treatment study, where we find that the Bayesian model often gives smaller measures of uncertainty than its competitors, and thus identifies more significant effects.