1
|
Babaei Rikan S, Sorayaie Azar A, Naemi A, Bagherzadeh Mohasefi J, Pirnejad H, Wiil UK. Survival prediction of glioblastoma patients using modern deep learning and machine learning techniques. Sci Rep 2024; 14:2371. [PMID: 38287149 PMCID: PMC10824760 DOI: 10.1038/s41598-024-53006-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 01/25/2024] [Indexed: 01/31/2024] Open
Abstract
In this study, we utilized data from the Surveillance, Epidemiology, and End Results (SEER) database to predict the glioblastoma patients' survival outcomes. To assess dataset skewness and detect feature importance, we applied Pearson's second coefficient test of skewness and the Ordinary Least Squares method, respectively. Using two sampling strategies, holdout and five-fold cross-validation, we developed five machine learning (ML) models alongside a feed-forward deep neural network (DNN) for the multiclass classification and regression prediction of glioblastoma patient survival. After balancing the classification and regression datasets, we obtained 46,340 and 28,573 samples, respectively. Shapley additive explanations (SHAP) were then used to explain the decision-making process of the best model. In both classification and regression tasks, as well as across holdout and cross-validation sampling strategies, the DNN consistently outperformed the ML models. Notably, the accuracy were 90.25% and 90.22% for holdout and five-fold cross-validation, respectively, while the corresponding R2 values were 0.6565 and 0.6622. SHAP analysis revealed the importance of age at diagnosis as the most influential feature in the DNN's survival predictions. These findings suggest that the DNN holds promise as a practical auxiliary tool for clinicians, aiding them in optimal decision-making concerning the treatment and care trajectories for glioblastoma patients.
Collapse
Affiliation(s)
| | | | - Amin Naemi
- SDU Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| | | | - Habibollah Pirnejad
- Erasmus School of Health Policy and Management (ESHPM), Erasmus University Rotterdam, Rotterdam, The Netherlands.
- Patient Safety Research Center, Clinical Research Institute, Urmia University of Medical Sciences, Urmia, Iran.
| | - Uffe Kock Wiil
- SDU Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
2
|
Ghafari R, Azar AS, Ghafari A, Aghdam FM, Valizadeh M, Khalili N, Hatamkhani S. Prediction of the Fatal Acute Complications of Myocardial Infarction via Machine Learning Algorithms. J Tehran Heart Cent 2023; 18:278-287. [PMID: 38680646 PMCID: PMC11053239 DOI: 10.18502/jthc.v18i4.14827] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 06/05/2023] [Indexed: 05/01/2024] Open
Abstract
Background Myocardial infarction (MI) is a major cause of death, particularly during the first year. The avoidance of potentially fatal outcomes requires expeditious preventative steps. Machine learning (ML) is a subfield of artificial intelligence science that detects the underlying patterns of available big data for modeling them. This study aimed to establish an ML model with numerous features to predict the fatal complications of MI during the first 72 hours of hospital admission. Methods We applied an MI complications database that contains the demographic and clinical records of patients during the 3 days of admission based on 2 output classes: dead due to the known complications of MI and alive. We utilized the recursive feature elimination (RFE) method to apply feature selection. Thus, after applying this method, we reduced the number of features to 50. The performance of 4 common ML classifier algorithms, namely logistic regression, support vector machine, random forest, and extreme gradient boosting (XGBoost), was evaluated using 8 classification metrics (sensitivity, specificity, precision, false-positive rate, false-negative rate, accuracy, F1-score, and AUC). Results In this study of 1699 patients with confirmed MI, 15.94% experienced fatal complications, and the rest remained alive. The XGBoost model achieved more desirable results based on the accuracy and F1-score metrics and distinguished patients with fatal complications from surviving ones (AUC=78.65%, sensitivity=94.35%, accuracy=91.47%, and F1-score=95.14%). Cardiogenic shock was the most significant feature influencing the prediction of the XGBoost algorithm. Conclusion XGBoost algorithms can be a promising model for predicting fatal complications following MI.
Collapse
Affiliation(s)
- Reza Ghafari
- Pharmacy Faculty, Urmia University of Medical Sciences, Urmia, Iran
| | | | - Ali Ghafari
- Medical Physics and Biomedical Engineering Department, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
- Research Center for Evidence-Based Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
| | | | - Morteza Valizadeh
- Faculty of Electrical and Computer Engineering, Urmia University, Urmia, Iran
| | - Naser Khalili
- Department of Cardiology, School of Medicine, Urmia University of Medical Sciences, Urmia, Iran
| | - Shima Hatamkhani
- Experimental and Applied Pharmaceutical Sciences Research Center, Urmia University of Medical Sciences, Urmia, Iran
- Department of Clinical Pharmacy, Urmia University of Medical Sciences, Urmia, Iran
| |
Collapse
|
3
|
Sorayaie Azar A, Naemi A, Babaei Rikan S, Bagherzadeh Mohasefi J, Pirnejad H, Wiil UK. Monkeypox detection using deep neural networks. BMC Infect Dis 2023; 23:438. [PMID: 37370031 DOI: 10.1186/s12879-023-08408-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Accepted: 06/20/2023] [Indexed: 06/29/2023] Open
Abstract
BACKGROUND In May 2022, the World Health Organization (WHO) European Region announced an atypical Monkeypox epidemic in response to reports of numerous cases in some member countries unrelated to those where the illness is endemic. This issue has raised concerns about the widespread nature of this disease around the world. The experience with Coronavirus Disease 2019 (COVID-19) has increased awareness about pandemics among researchers and health authorities. METHODS Deep Neural Networks (DNNs) have shown promising performance in detecting COVID-19 and predicting its outcomes. As a result, researchers have begun applying similar methods to detect Monkeypox disease. In this study, we utilize a dataset comprising skin images of three diseases: Monkeypox, Chickenpox, Measles, and Normal cases. We develop seven DNN models to identify Monkeypox from these images. Two scenarios of including two classes and four classes are implemented. RESULTS The results show that our proposed DenseNet201-based architecture has the best performance, with Accuracy = 97.63%, F1-Score = 90.51%, and Area Under Curve (AUC) = 94.27% in two-class scenario; and Accuracy = 95.18%, F1-Score = 89.61%, AUC = 92.06% for four-class scenario. Comparing our study with previous studies with similar scenarios, shows that our proposed model demonstrates superior performance, particularly in terms of the F1-Score metric. For the sake of transparency and explainability, Local Interpretable Model-Agnostic Explanations (LIME) and Gradient-weighted Class Activation Mapping (Grad-Cam) were developed to interpret the results. These techniques aim to provide insights into the decision-making process, thereby increasing the trust of clinicians. CONCLUSION The DenseNet201 model outperforms the other models in terms of the confusion metrics, regardless of the scenario. One significant accomplishment of this study is the utilization of LIME and Grad-Cam to identify the affected areas and assess their significance in diagnosing diseases based on skin images. By incorporating these techniques, we enhance our understanding of the infected regions and their relevance in distinguishing Monkeypox from other similar diseases. Our proposed model can serve as a valuable auxiliary tool for diagnosing Monkeypox and distinguishing it from other related conditions.
Collapse
Affiliation(s)
| | - Amin Naemi
- Center for Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| | | | | | - Habibollah Pirnejad
- Patient Safety Research Center, Clinical Research Institute, Urmia University of Medical Sciences, Urmia, Iran
- Erasmus School of Health Policy & Management (ESHPM), Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Uffe Kock Wiil
- Center for Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
4
|
Sorayaie Azar A, Babaei Rikan S, Naemi A, Bagherzadeh Mohasefi J, Pirnejad H, Bagherzadeh Mohasefi M, Wiil UK. Application of machine learning techniques for predicting survival in ovarian cancer. BMC Med Inform Decis Mak 2022; 22:345. [PMID: 36585641 PMCID: PMC9801354 DOI: 10.1186/s12911-022-02087-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Accepted: 12/15/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Ovarian cancer is the fifth leading cause of mortality among women in the United States. Ovarian cancer is also known as forgotten cancer or silent disease. The survival of ovarian cancer patients depends on several factors, including the treatment process and the prognosis. METHODS The ovarian cancer patients' dataset is compiled from the Surveillance, Epidemiology, and End Results (SEER) database. With the help of a clinician, the dataset is curated, and the most relevant features are selected. Pearson's second coefficient of skewness test is used to evaluate the skewness of the dataset. Pearson correlation coefficient is also used to investigate the associations between features. Statistical test is utilized to evaluate the significance of the features. Six Machine Learning (ML) models, including K-Nearest Neighbors , Support Vector Machine (SVM), Decision Tree (DT), Random Forest (RF), Adaptive Boosting (AdaBoost), and Extreme Gradient Boosting (XGBoost), are implemented for survival prediction in both classification and regression approaches. An interpretable method, Shapley Additive Explanations (SHAP), is applied to clarify the decision-making process and determine the importance of each feature in prediction. Additionally, DTs of the RF model are displayed to show how the model predicts the survival intervals. RESULTS Our results show that RF (Accuracy = 88.72%, AUC = 82.38%) and XGBoost (Root Mean Squad Error (RMSE)) = 20.61%, R2 = 0.4667) have the best performance for classification and regression approaches, respectively. Furthermore, using the SHAP method along with extracted DTs of the RF model, the most important features in the dataset are identified. Histologic type ICD-O-3, chemotherapy recode, year of diagnosis, age at diagnosis, tumor stage, and grade are the most important determinant factors in survival prediction. CONCLUSION To the best of our knowledge, our study is the first study that develops various ML models to predict ovarian cancer patients' survival on the SEER database in both classification and regression approaches. These ML algorithms also achieve more accurate results and outperform statistical methods. Furthermore, our study is the first study to use the SHAP method to increase confidence and transparency of the proposed models' prediction for clinicians. Moreover, our developed models, as an automated auxiliary tool, can help clinicians to have a better understanding of the estimated survival as well as important features that affect survival.
Collapse
Affiliation(s)
- Amir Sorayaie Azar
- grid.412763.50000 0004 0442 8645Department of Computer Engineering, Urmia University, Urmia, Iran
| | - Samin Babaei Rikan
- grid.412763.50000 0004 0442 8645Department of Computer Engineering, Urmia University, Urmia, Iran
| | - Amin Naemi
- grid.10825.3e0000 0001 0728 0170Center for Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| | | | - Habibollah Pirnejad
- grid.412763.50000 0004 0442 8645Patient Safety Research Center, Clinical Research Institute, Urmia University of Medical Sciences, Urmia, Iran ,grid.6906.90000000092621349Erasmus School of Health Policy and Management (ESHPM), Erasmus University Rotterdam, Rotterdam, The Netherlands
| | | | - Uffe Kock Wiil
- grid.10825.3e0000 0001 0728 0170Center for Health Informatics and Technology, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
5
|
Babaei Rikan S, Sorayaie Azar A, Ghafari A, Bagherzadeh Mohasefi J, Pirnejad H. COVID-19 Diagnosis from Routine Blood Tests using Artificial Intelligence Techniques. Biomed Signal Process Control 2021; 72:103263. [PMID: 34745318 PMCID: PMC8559794 DOI: 10.1016/j.bspc.2021.103263] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Revised: 09/30/2021] [Accepted: 10/15/2021] [Indexed: 12/21/2022]
Abstract
Coronavirus disease (COVID-19) is a
unique worldwide pandemic. With new mutations of the virus with higher
transmission rates, it is imperative to diagnose positive cases as
quickly and accurately as possible. Therefore, a fast, accurate, and
automatic system for COVID-19 diagnosis can be very useful for
clinicians. In this study, seven machine learning and four deep learning
models were presented to diagnose positive cases of COVID-19 from three
routine laboratory blood tests datasets. Three correlation coefficient
methods, i.e., Pearson, Spearman, and Kendall, were used to demonstrate
the relevance among samples. A four-fold cross-validation method was used
to train, validate, and test the proposed models. In all three datasets,
the proposed deep neural network (DNN) model achieved the highest values
of accuracy, precision, recall or sensitivity, specificity, F1-Score,
AUC, and MCC. On average, accuracy 92.11%, specificity 84.56%, and AUC
92.20% values have been obtained in the first dataset. In the second
dataset, on average, accuracy 93.16%, specificity 93.02%, and AUC 93.20%
values have been obtained. Finally, in the third dataset, on average, the
values of accuracy 92.5%, specificity 85%, and AUC 92.20% have been
obtained. In this study, we used a statistical t-test to validate the
results. Finally, using artificial intelligence interpretation methods,
important and impactful features in the developed model were presented.
The proposed DNN model can be used as a supplementary tool for diagnosing
COVID-19, which can quickly provide clinicians with highly accurate
diagnoses of positive cases in a timely manner.
Collapse
Affiliation(s)
| | | | - Ali Ghafari
- Medical Physics and Biomedical Engineering Department, Medical Faculty, Tehran University of Medical Sciences, Tehran, Iran
| | | | - Habibollah Pirnejad
- Patient Safety Research Center, Clinical Research Institute, Urmia University of Medical Sciences, Urmia, Iran.,Erasmus School of Health Policy & Management (ESHPM), Erasmus University Rotterdam, Rotterdam, The Netherlands
| |
Collapse
|
6
|
Sorayaie Azar A, Ghafari A, Ostadi Najar M, Babaei Rikan S, Ghafari R, Farajpouri Khamene M, Sheikhzadeh P. Covidense: Providing a Suitable Solution for Diagnosing Covid-19 Lung Infection Based on Deep Learning from Chest X-Ray Images of Patients. fbt 2021. [DOI: 10.18502/fbt.v8i2.6517] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Purpose: Coronavirus disease 2019 (Covid-19), first reported in December 2019 in Wuhan, China, has become a pandemic. Chest imaging is used for the diagnosis of Covid-19 patients and can address problems concerning Reverse Transcription-Polymerase Chain Reaction (RT-PCR) shortcomings. Chest X-ray images can act as an appropriate alternative to Computed Tomography (CT) for diagnosing Covid-19. The purpose of this study is to use a Deep Learning method for diagnosing Covid-19 cases using chest X-ray images. Thus, we propose Covidense based on the pre-trained Densenet-201 model and is trained on a dataset comprising chest X-ray images of Covid-19, normal, bacterial pneumonia, and viral pneumonia cases.
Materials and Methods: In this study, a total number of 1280 chest X-ray images of Covid-19, normal, bacterial and viral pneumonia cases were collected from open access repositories. Covidense, a convolutional neural network model, is based on the pre-trained DenseNet-201 architecture, and after pre-processing the images, it has been trained and tested on the images using the 5-fold cross-validation method.
Results: The accuracy of different classifications including classification of two classes (Covid-19, normal), three classes 1 (Covid-19, normal and bacterial pneumonia), three classes 2 (Covid-19, normal and viral pneumonia), and four classes (Covid-19, normal, bacterial pneumonia and viral pneumonia) are 99.46%, 92.86%, 93.91 %, and 91.01% respectively.
Conclusion: This model can differentiate pneumonia caused by Covid-19 from other types of pneumonia, including bacterial and viral. The proposed model offers high accuracy and can be of great help for effective screening. Thus, reducing the rate of infection spread. Also, it can act as a complementary tool for the detection and diagnosis of Covid-19.
Collapse
|