1
|
Ou C, Liu J, Qian Y, Chong W, Liu D, He X, Zhang X, Duan CZ. Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study. Front Neurol 2021; 12:735142. [PMID: 34912282 PMCID: PMC8666475 DOI: 10.3389/fneur.2021.735142] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Accepted: 11/01/2021] [Indexed: 11/30/2022] Open
Abstract
Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons. Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction. Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC). Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models. Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers.
Collapse
Affiliation(s)
- Chubin Ou
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China.,Department of Biomedical Sciences, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, NSW, Australia
| | - Jiahui Liu
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Yi Qian
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Winston Chong
- Monash Medical Centre, Monash University, Clayton, VIC, Australia
| | - Dangqi Liu
- Department of Neurosurgery, The First People's Hospital of Foshan, Foshan, China
| | - Xuying He
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Xin Zhang
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Chuan-Zhi Duan
- Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| |
Collapse
|
2
|
Liu H, Guo W, Xiang S, Hu P, Sun F, Gao J, Zhang X, Wang P, Jing W, Zhang L, Yang X, Duan C, He M, Zhang H, Qu Y. The natural course of unruptured intracranial aneurysms in a Chinese cohort: protocol of a multi-center registration study in CIAP. J Transl Med 2019; 17:349. [PMID: 31640726 PMCID: PMC6805494 DOI: 10.1186/s12967-019-2092-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2019] [Accepted: 10/05/2019] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND Subarachnoid hemorrhage (SAH) accounts for 4.4% of cerebral vascular disease, which is one of the leading causes of death in China. Rupture of intracranial aneurysms (IAs) is the most common cause of SAH. The natural history of unruptured IAs (UIAs) and the risk factors for rupture are among the key issues regarding the pathogenesis of IA and SAH that remain unclear in the Chinese population. METHODS The China Intracranial Aneurysm Project (CIAP) is a prospective, observational, multicenter registry study of the natural courses, risk factors for the onset and rupture, treatment methods, comorbidity management and other aspects of intracranial aneurysms. To date, there are five studies in the CIAP. CIAP-1 is a prospective observational cohort study of UIAs. More than 5000 patients who will be followed for at least 1 year are expected to be enrolled in this cohort. These participants come from more than 20 centers that represent different regions in China. Enrollment began on May 1, 2017, and will take approximately 5 years. A nationwide online database of UIAs will be built. Participants' basic, lifestyle, clinical and follow-up information will be collected. The blood samples will be stored in the Central Biological Specimen Bank. Strict standards have been established and will be followed in this study to ensure efficient implementation. DISCUSSION The natural course of UIAs in the Chinese population will be explored in this registry study. In addition, the risk factors for the rupture of the UIAs and the joint effect of those factors will be analyzed. The present study aims to create a nationwide database of UIAs and investigate the natural course of UIAs in China. Trial registration The Natural Course of Unruptured Intracranial Aneurysms in a Chinese Cohort (ClinicalTrials.gov Identifier: NCT03117803). Registered: July 5, 2017.
Collapse
Affiliation(s)
- Haixiao Liu
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Wei Guo
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Sishi Xiang
- Department of Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing, China
| | - Peng Hu
- Department of Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing, China
| | - Feifei Sun
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Junmei Gao
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Xiaoyang Zhang
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Ping Wang
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Wenting Jing
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Lei Zhang
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China
| | - Xinjian Yang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute and Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Chuanzhi Duan
- Department of Neurosurgery, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Min He
- Department of Neurosurgery, West China Hospital, Sichuan University, Chengdu, China
| | - Hongqi Zhang
- Department of Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing, China.
| | - Yan Qu
- Department of Neurosurgery, Tangdu Hospital, The Fourth Military Medical University, Xi'an, China.
| |
Collapse
|
3
|
Chen J, Liu J, Zhang Y, Tian Z, Wang K, Zhang Y, Mu S, Lv M, Jiang P, Duan C, Zhang H, Qu Y, He M, Yang X. China Intracranial Aneurysm Project (CIAP): protocol for a registry study on a multidimensional prediction model for rupture risk of unruptured intracranial aneurysms. J Transl Med 2018; 16:263. [PMID: 30257699 PMCID: PMC6158879 DOI: 10.1186/s12967-018-1641-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Accepted: 09/20/2018] [Indexed: 02/05/2023] Open
Abstract
BACKGROUND Ruptured aneurysms, the commonest cause of nontraumatic subarachnoid hemorrhage, can be catastrophic; the mortality and morbidity of affected patients being very high. Some risk factors, such as smoking, hypertension and female sex have been identified, whereas others, such as hemodynamics, imaging, and genomics, remain unclear. Currently, no accurate model that includes all factors for predicting such rupture is available. We plan to use data from a large cohort of Chinese individuals to set up a multidimensional model for predicting risk of rupture of unruptured intracranial aneurysms (UIAs). METHODS The China Intracranial Aneurysm Project-2 (CIAP-2) will comprise screening of a cohort of 500 patients with UIA (From CIAP-1) and focus on hemodynamic factors, high resolution magnetic resonance imaging (HRMRI) findings, genetic factors, and biomarkers. Possible risk factors for rupture of UIA, including genetic factors, biomarkers, HRMRI, and hemodynamic factors, will be analyzed. The first project of the China Intracranial Aneurysm Project (CIAP-1; chaired by the Department of Neurosurgery, Tangdu Hospital, Fourth Military Medical University, Xi'an, Shaanxi, China) will prospectively collect a cohort of 5000 patients with UIA from 20 centers in China, and collect baseline information for each patient. Multidimensional data will be acquired in follow-up assessments. Statistically significant clinical features in the UIA cohort will also be analyzed and integrated into the model for predicting risk of UIA rupture. After the model has been set up, the resultant evidence-based prediction will provide a preliminary theoretical basis for treating aneurysms at high risk of rupture. DISCUSSION This study will explore the risk of rupture of aneurysms and develop a scientific multidimensional model for predicting rupture of unruptured intracranial aneurysms. Clinical Trials registration A Study on a Multidimensional Prediction Model for Rupture Risk of Unruptured Intracranial Aneurysms (CIAP-2), NCT03133624. Registered: 16 April 2017. https://clinicaltrials.gov/ct2/show/NCT03133624.
Collapse
Affiliation(s)
- Junfan Chen
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Jian Liu
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Yisen Zhang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Zhongbin Tian
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Kun Wang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Ying Zhang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Shiqing Mu
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Ming Lv
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - Peng Jiang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China
| | - ChuanZhi Duan
- Department of Neurosurgery, Zhujiang Hospital, Southern Medical University, Guangzhou, Guangdong, China
| | - Hongqi Zhang
- Department of Neurosurgery, Xuanwu Hospital, Capital Medical University, Beijing, China
| | - Yan Qu
- Department of Neurosurgery, Tangdu Hospital, Fourth Military Medical University, Xi'an, Shanxi, China
| | - Min He
- Department of Neurosurgery, West China Hospital, Sichuan University, Chengdu, Sichuan, China
| | - Xinjian Yang
- Department of Interventional Neuroradiology, Beijing Neurosurgical Institute, Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China.
| |
Collapse
|