Comparison of predictive models to predict survival of cardiac surgery patients

Predictive Performance of Logistic Regression for Imbalanced Data with Categorical Covariate

Logistic regression is often used for the classification of a binary categorical dependent variable using various types of covariates (continuous or categorical). Imbalanced data will lead to biased parameter estimates and classification performance of the logistic regression model. Imbalanced data occurs when the number of cases in one category of the binary dependent variable is very much smaller than the other category. This simulation study investigates the effect of imbalanced data measured by imbalanced ratio on the parameter estimate of the binary logistic regression with a categorical covariate. Datasets were simulated with controlled different percentages of imbalance ratio (IR), from 1% to 50%, and for various sample sizes. The simulated datasets were then modeled using binary logistic regression. The bias in the estimates was measured using MSE (Mean Square Error). The simulation results provided evidence that the effect of imbalance ratio on the parameter estimate of the covariate decreased as sample size increased. The bias of the estimates depended on sample size whereby for sample size 100, 500, 1000 – 2000 and 2500 – 3500, the estimates were biased for IR below 30%, 10%, 5% and 2% respectively. Results also showed that parameter estimates were all biased at IR 1% for all sample size. An application using a real dataset supported the simulation results.

Predictive Performance of Logistic Regression for Imbalanced Data with Categorical Covariate

Logistic regression is often used for the classification of a binary categorical dependent variable using various types of covariates (continuous or categorical). Imbalanced data will lead to biased parameter estimates and classification performance of the logistic regression model. Imbalanced data occurs when the number of cases in one category of the binary dependent variable is very much smaller than the other category. This simulation study investigates the effect of imbalanced data measured by imbalanced ratio on the parameter estimate of the binary logistic regression with a categorical covariate. Datasets were simulated with controlled different percentages of imbalance ratio (IR), from 1% to 50%, and for various sample sizes. The simulated datasets were then modeled using binary logistic regression. The bias in the estimates was measured using Mean Square Error (MSE). The simulation results provided evidence that the effect of imbalance ratio on the parameter estimate of the covariate decreased as sample size increased. The bias of the estimated depends on sample size whereby for sample size 100, 500, 1000 - 2000 and 2500 - 3500, the estimated were biased for IR below 30%, 10%, 5% and 2% respectively. Results also showed that parameter estimates were all biased at IR 1% for all sample size. An application using a real dataset supported the simulation results.

Two new mathematical models for prediction of early mortality risk in coronary artery bypass graft surgery

OBJECTIVES

The aim of this study was to develop new models for prediction of short-term mortality risk in on-pump coronary artery bypass grafting (CABG) surgery using decision tree (DT) methods.

METHODS

Between September 2005 and April 2006, 948 consecutive patients underwent CABG surgery at Rajaie Heart Center. Potential risk factors were reviewed and univariate and multivariate analysis for short-term mortality were performed. The whole dataset was divided into mutually exclusive subsets. An entropy error fuzzy decision tree (EEFDT) and an entropy error crisp decision tree (EECDT) were implemented using 650 (68.6%) patient data and tested with 298 (31.4%) patient data. Ten times hold-out cross validation was done and the area under the receiver operative characteristic curve (AUC) was reported as model performance. The results were compared with the logistic regression (LR) model and EuroSCORE.

RESULTS

The overall short-term mortality rate was 3.8%, and was statistically higher in women than men (P<.001). The final EEFDT selected 19 variables and resulted in a tree with 39 nodes, 20 conditional rules, and AUC of 0.90±0.008. The final EECDT selected 15 variables and resulted in a tree with 35 nodes, 18 conditional rules, and AUC of 0.86±0.008. The LR model selected 10 variables and resulted in an AUC of 0.78±0.008; the AUC for EuroSCORE was 0.77±0.003. There were no differences in the discriminatory power of EEFDT and EECDT (P=.066) and their performance was superior to LR and EuroSCORE.

CONCLUSIONS

EEFDT, EECDT, LR, and EuroSCORE had clinical acceptance but the performance and accuracy of the DTs were superior to the other models.