1
|
Shi P, Xin J, Wu J, Deng Y, Cai Z, Du S, Zheng N. Detection of thin-cap fibroatheroma in IVOCT images based on weakly supervised learning and domain knowledge. JOURNAL OF BIOPHOTONICS 2023; 16:e202200343. [PMID: 36635865 DOI: 10.1002/jbio.202200343] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 12/28/2022] [Accepted: 01/09/2023] [Indexed: 05/17/2023]
Abstract
Automatic detection of thin-cap fibroatheroma (TCFA) on intravascular optical coherence tomography images is essential for the prevention of acute coronary syndrome. However, existing methods need to mark the exact location of TCFAs on each frame as supervision, which is extremely time-consuming and expensive. Hence, a new weakly supervised framework is proposed to detect TCFAs using only image-level tags as supervision. The framework comprises cut, feature extraction, relation, and detection modules. First, based on prior knowledge, a cut module was designed to generate a small number of specific region proposals. Then, to learn global information, a relation module was designed to learn the spatial adjacency and order relationships at the feature level, and an attention-based strategy was introduced in the detection module to effectively aggregate the classification results of region proposals as the image-level predicted score. The results demonstrate that the proposed method surpassed the state-of-the-art weakly supervised detection methods.
Collapse
Affiliation(s)
- Peiwen Shi
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Jingmin Xin
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Jiayi Wu
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Yangyang Deng
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
- Cardiovascular Department, First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, China
| | - Zhuotong Cai
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Shaoyi Du
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| | - Nanning Zheng
- Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an, China
| |
Collapse
|
2
|
Liu S, Xin J, Wu J, Deng Y, Su R, Niessen WJ, Zheng N, van Walsum T. Multi-view Contour-constrained Transformer Network for Thin-cap Fibroatheroma Identification. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.12.041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
|
3
|
Shi P, Xin J, Zheng N. A-line-based thin-cap fibroatheroma detection with multi-view IVOCT images using multi-task learning and contrastive learning. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:2298-2306. [PMID: 36520750 DOI: 10.1364/josaa.464303] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 10/21/2022] [Indexed: 06/17/2023]
Abstract
Automatic detection of thin-cap fibroatheroma (TCFA) is essential to prevent acute coronary syndrome. Hence, in this paper, a method is proposed to detect TCFAs by directly classifying each A-line using multi-view intravascular optical coherence tomography (IVOCT) images. To solve the problem of false positives, a multi-input-output network was developed to implement image-level classification and A-line-based classification at the same time, and a contrastive consistency term was designed to ensure consistency between two tasks. In addition, to learn spatial and global information and obtain the complete extent of TCFAs, an architecture and a regional connectivity constraint term are proposed to classify each A-line of IVOCT images. Experimental results obtained on the 2017 China Computer Vision Conference IVOCT dataset show that the proposed method achieved state-of-art performance with a total score of 88.7±0.88%, overlap rate of 88.64±0.26%, precision rate of 84.34±0.86%, and recall rate of 93.67±2.29%.
Collapse
|
4
|
Zhang R, Fan Y, Qi W, Wang A, Tang X, Gao T. Current research and future prospects of IVOCT imaging-based detection of the vascular lumen and vulnerable plaque. JOURNAL OF BIOPHOTONICS 2022; 15:e202100376. [PMID: 35139263 DOI: 10.1002/jbio.202100376] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2021] [Revised: 01/17/2022] [Accepted: 02/07/2022] [Indexed: 06/14/2023]
Abstract
Intravascular optical coherence tomography (IVOCT) is an imaging method that has developed rapidly in recent years and is useful in coronary atherosclerosis diagnosis. It is widely used in the assessment of vulnerable plaque. This review summarizes the main research methods used in recent years for blood vessel lumen boundary detection and segmentation and vulnerable plaque segmentation and classification. This article aims to comprehensively and systematically introduce the research progress on internal tissues of blood vessels based on IVOCT images. The characteristics and advantages of various methods have been summarized to provide theoretical ideas and methods for the reference of relevant researchers and scholars.
Collapse
Affiliation(s)
- Ruolin Zhang
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Yingwei Fan
- School of Medical Technology, Beijing Institute of Technology, Beijing, China
| | - Wenliu Qi
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Ancong Wang
- School of Life Science, Beijing Institute of Technology, Beijing, China
| | - Xiaoying Tang
- School of Life Science, Beijing Institute of Technology, Beijing, China
- School of Medical Technology, Beijing Institute of Technology, Beijing, China
| | - Tianxin Gao
- School of Life Science, Beijing Institute of Technology, Beijing, China
| |
Collapse
|
5
|
Zhang J, Han R, Shao G, Lv B, Sun K. Artificial Intelligence in Cardiovascular Atherosclerosis Imaging. J Pers Med 2022; 12:jpm12030420. [PMID: 35330420 PMCID: PMC8952318 DOI: 10.3390/jpm12030420] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Revised: 02/15/2022] [Accepted: 03/04/2022] [Indexed: 12/22/2022] Open
Abstract
At present, artificial intelligence (AI) has already been applied in cardiovascular imaging (e.g., image segmentation, automated measurements, and eventually, automated diagnosis) and it has been propelled to the forefront of cardiovascular medical imaging research. In this review, we presented the current status of artificial intelligence applied to image analysis of coronary atherosclerotic plaques, covering multiple areas from plaque component analysis (e.g., identification of plaque properties, identification of vulnerable plaque, detection of myocardial function, and risk prediction) to risk prediction. Additionally, we discuss the current evidence, strengths, limitations, and future directions for AI in cardiac imaging of atherosclerotic plaques, as well as lessons that can be learned from other areas. The continuous development of computer science and technology may further promote the development of this field.
Collapse
Affiliation(s)
- Jia Zhang
- Hohhot Health Committee, Hohhot 010000, China;
| | - Ruijuan Han
- The People’s Hospital of Longgang District, Shenzhen 518172, China;
| | - Guo Shao
- The Third People’s Hospital of Longgang District, Shenzhen 518100, China;
| | - Bin Lv
- Fuwai Hospital, National Center for Cardiovascular Diseases, Beijing 100037, China;
| | - Kai Sun
- The Third People’s Hospital of Longgang District, Shenzhen 518100, China;
- Correspondence:
| |
Collapse
|
6
|
Wu X, Zhang Y, Zhang P, Hui H, Jing J, Tian F, Jiang J, Yang X, Chen Y, Tian J. Structure attention co-training neural network for neovascularization segmentation in intravascular optical coherence tomography. Med Phys 2022; 49:1723-1738. [PMID: 35061247 DOI: 10.1002/mp.15477] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 01/09/2022] [Accepted: 01/10/2022] [Indexed: 11/11/2022] Open
Abstract
PURPOSE To development and validate a Neovascularization (NV) segmentation model in intravascular optical coherence tomography (IVOCT) through deep learning methods. METHODS AND MATERIALS A total of 1950 2D slices of 70 IVOCT pullbacks were used in our study. We randomly selected 1273 2D slices from 44 patients as the training set, 379 2D slices from 11 patients as the validation set, and 298 2D slices from the last 15 patients as the testing set. Automatic NV segmentation is quite challenging, as it must address issues of speckle noise, shadow artifacts, high distribution variation, etc. To meet these challenges, a new deep learning-based segmentation method is developed based on a co-training architecture with an integrated structural attention mechanism. Co-training is developed to exploit the features of three consecutive slices. The structural attention mechanism comprises spatial and channel attention modules and is integrated into the co-training architecture at each up-sampling step. A cascaded fixed network is further incorporated to achieve segmentation at the image level in a coarse-to-fine manner. RESULTS Extensive experiments were performed involving a comparison with several state-of-the-art deep learning-based segmentation methods. Moreover, the consistency of the results with those of manual segmentation was also investigated. Our proposed NV automatic segmentation method achieved the highest correlation with the manual delineation by interventional cardiologists (the Pearson correlation coefficient is 0.825). CONCLUSION In this work, we proposed a co-training architecture with an integrated structural attention mechanism to segment NV in IVOCT images. The good agreement between our segmentation results and manual segmentation indicates that the proposed method has great potential for application in the clinical investigation of NV-related plaque diagnosis and treatment. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Xiangjun Wu
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine and Engineering, Beihang University, Beijing, 100083, China.,CAS Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Beijing, 100190, China.,Beijing Key Laboratory of Molecular Imaging, Beijing, 100190, China
| | - Yingqian Zhang
- Senior Department of Cardiology, the Sixth Medical Center of PLA General Hospital, Beijing, 100853, China
| | - Peng Zhang
- Department of Biomedical Engineering, School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China
| | - Hui Hui
- CAS Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Beijing, 100190, China.,Beijing Key Laboratory of Molecular Imaging, Beijing, 100190, China.,University of Chinese Academy of Sciences, Beijing, 100190, China
| | - Jing Jing
- Senior Department of Cardiology, the Sixth Medical Center of PLA General Hospital, Beijing, 100853, China
| | - Feng Tian
- Senior Department of Cardiology, the Sixth Medical Center of PLA General Hospital, Beijing, 100853, China
| | - Jingying Jiang
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine and Engineering, Beihang University, Beijing, 100083, China
| | - Xin Yang
- CAS Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Beijing, 100190, China.,Beijing Key Laboratory of Molecular Imaging, Beijing, 100190, China
| | - Yundai Chen
- Senior Department of Cardiology, the Sixth Medical Center of PLA General Hospital, Beijing, 100853, China.,Southern Medical University, Guangzhou, 510515, China
| | - Jie Tian
- Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Medicine and Engineering, Beihang University, Beijing, 100083, China.,CAS Key Laboratory of Molecular Imaging, The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Beijing, 100190, China.,Beijing Key Laboratory of Molecular Imaging, Beijing, 100190, China.,Zhuhai Precision Medical Center, Zhuhai People's Hospital, affiliated with Jinan University, Zhuhai, 519000, China
| |
Collapse
|
7
|
Fedewa R, Puri R, Fleischman E, Lee J, Prabhu D, Wilson DL, Vince DG, Fleischman A. Artificial Intelligence in Intracoronary Imaging. Curr Cardiol Rep 2020; 22:46. [DOI: 10.1007/s11886-020-01299-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
8
|
Yan Q, Xu M, Wong DWK, Taruya A, Tanaka A, Liu J, Wong P, Cheng J. Automatic fibroatheroma identification in intravascular optical coherence tomography volumes. JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING 2019. [DOI: 10.1007/s12652-019-01549-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2018] [Accepted: 10/14/2019] [Indexed: 08/30/2023]
|
9
|
Gharaibeh Y, Prabhu D, Kolluru C, Lee J, Zimin V, Bezerra H, Wilson D. Coronary calcification segmentation in intravascular OCT images using deep learning: application to calcification scoring. J Med Imaging (Bellingham) 2019; 6:045002. [PMID: 31903407 PMCID: PMC6934132 DOI: 10.1117/1.jmi.6.4.045002] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Accepted: 12/05/2019] [Indexed: 01/18/2023] Open
Abstract
Major calcifications are of great concern when performing percutaneous coronary interventions because they inhibit proper stent deployment. We created a comprehensive software to segment calcifications in intravascular optical coherence tomography (IVOCT) images and to calculate their impact using the stent-deployment calcification score, as reported by Fujino et al. We segmented the vascular lumen and calcifications using the pretrained SegNet, convolutional neural network, which was refined for our task. We cleaned segmentation results using conditional random field processing. We evaluated the method on manually annotated IVOCT volumes of interest (VOIs) without lesions and with calcifications, lipidous, or mixed lesions. The dataset included 48 VOIs taken from 34 clinical pullbacks, giving a total of 2640 in vivo images. Annotations were determined from consensus between two expert analysts. Keeping VOIs intact, we performed 10-fold cross-validation over all data. Following segmentation noise cleaning, we obtained sensitivities of 0.85 ± 0.04 , 0.99 ± 0.01 , and 0.97 ± 0.01 for calcified, lumen, and other tissue classes, respectively. From segmented regions, we automatically determined calcification depth, angle, and thickness attributes. Bland-Altman analysis suggested strong correlation between manually and automatically obtained lumen and calcification attributes. Agreement between manually and automatically obtained stent-deployment calcification scores was good (four of five lesions gave exact agreement). Results are encouraging and suggest our classification approach could be applied clinically for assessment and treatment planning of coronary calcification lesions.
Collapse
Affiliation(s)
- Yazan Gharaibeh
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - David Prabhu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Chaitanya Kolluru
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Juhwan Lee
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Vladislav Zimin
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Hiram Bezerra
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - David Wilson
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
- Case Western Reserve University, Department of Radiology, Cleveland, Ohio, United States
| |
Collapse
|
10
|
Prabhu D, Bezerra HG, Kolluru C, Gharaibeh Y, Mehanna E, Wu H, Wilson DL. Automated A-line coronary plaque classification of intravascular optical coherence tomography images using handcrafted features and large datasets. JOURNAL OF BIOMEDICAL OPTICS 2019; 24:1-15. [PMID: 31586357 PMCID: PMC6784787 DOI: 10.1117/1.jbo.24.10.106002] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2019] [Accepted: 08/20/2019] [Indexed: 05/31/2023]
Abstract
We developed machine learning methods to identify fibrolipidic and fibrocalcific A-lines in intravascular optical coherence tomography (IVOCT) images using a comprehensive set of handcrafted features. We incorporated features developed in previous studies (e.g., optical attenuation and A-line peaks). In addition, we included vascular lumen morphology and three-dimensional (3-D) digital edge and texture features. Classification methods were developed using expansive datasets (∼7000 images), consisting of both clinical in-vivo images and an ex-vivo dataset, which was validated using 3-D cryo-imaging/histology. Conditional random field was used to perform 3-D classification noise cleaning of classification results. We tested various multiclass approaches, classifiers, and feature selection schemes and found that a three-class support vector machine with minimal-redundancy-maximal-relevance feature selection gave the best performance. We found that inclusion of our morphological and 3-D features improved overall classification accuracy. On a held-out test set consisting of >1700 images, we obtained an overall accuracy of 81.58%, with the following (sensitivity/specificity) for each class: other (81.43/89.59), fibrolipidic (94.48/87.32), and fibrocalcific (74.82/95.28). The en-face views of classification results showed that automated classification easily captured the preponderance of a disease segment (e.g., a calcified segment had large regions of fibrocalcific classifications). Finally, we demonstrated proof-of-concept for streamlining A-line classification output with existing fibrolipidic and fibrocalcific boundary segmentation methods, to enable fully automated plaque quantification. The results suggest that our classification approach is a viable step toward fully automated IVOCT plaque classification and segmentation for live-time treatment planning and for offline assessment of drug and biologic therapeutics.
Collapse
Affiliation(s)
- David Prabhu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Hiram G. Bezerra
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Chaitanya Kolluru
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Yazan Gharaibeh
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - Emile Mehanna
- University Hospitals Cleveland Medical Center, Harrington Heart and Vascular Institute, Cardiovascular Imaging Core Laboratory, Cleveland, Ohio, United States
| | - Hao Wu
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
| | - David L. Wilson
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, Ohio, United States
- Case Western Reserve University, Department of Radiology, Cleveland, Ohio, United States
| |
Collapse
|