1
|
Toto L, Romano A, Pavan M, Degl'Innocenti D, Olivotto V, Formenti F, Viggiano P, Midena E, Mastropasqua R. A deep learning approach to hard exudates detection and disorganization of retinal inner layers identification on OCT images. Sci Rep 2024; 14:16652. [PMID: 39030181 PMCID: PMC11271624 DOI: 10.1038/s41598-024-63844-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 06/03/2024] [Indexed: 07/21/2024] Open
Abstract
The purpose of the study was to detect Hard Exudates (HE) and classify Disorganization of Retinal Inner Layers (DRIL) implementing a Deep Learning (DL) system on optical coherence tomography (OCT) images of eyes with diabetic macular edema (DME). We collected a dataset composed of 442 OCT images on which we annotated 6847 HE and the presence of DRIL. A complex operational pipeline was defined to implement data cleaning and image transformations, and train two DL models. The state-of-the-art neural network architectures (Yolov7, ConvNeXt, RegNetX) and advanced techniques were exploited to aggregate the results (Ensemble learning, Edge detection) and obtain a final model. The DL approach reached good performance in detecting HE and classifying DRIL. Regarding HE detection the model got an AP@0.5 score equal to 34.4% with Precision of 48.7% and Recall of 43.1%; while for DRIL classification an Accuracy of 91.1% with Sensitivity and Specificity both of 91.1% and AUC and AUPR values equal to 91% were obtained. The P-value was lower than 0.05 and the Kappa coefficient was 0.82. The DL models proved to be able to identify HE and DRIL in eyes with DME with a very good accuracy and all the metrics calculated confirmed the system performance. Our DL approach demonstrated to be a good candidate as a supporting tool for ophthalmologists in OCT images analysis.
Collapse
Affiliation(s)
- Lisa Toto
- Ophthalmology Clinic, Department of Medicine and Ageing Science, University "G. D'Annunzio" of Chieti-Pescara, Via Dei Vestini Snc, 66100, Chieti, Italy
| | - Anna Romano
- Ophthalmology Clinic, Department of Medicine and Ageing Science, University "G. D'Annunzio" of Chieti-Pescara, Via Dei Vestini Snc, 66100, Chieti, Italy.
| | - Marco Pavan
- Datamantix S.R.L. Artificial Intelligence Company, Via Paolo Sarpi, 14/15, 33100, Udine, Italy
| | - Dante Degl'Innocenti
- Datamantix S.R.L. Artificial Intelligence Company, Via Paolo Sarpi, 14/15, 33100, Udine, Italy
| | - Valentina Olivotto
- Datamantix S.R.L. Artificial Intelligence Company, Via Paolo Sarpi, 14/15, 33100, Udine, Italy
| | - Federico Formenti
- Ophthalmology Clinic, Department of Medicine and Ageing Science, University "G. D'Annunzio" of Chieti-Pescara, Via Dei Vestini Snc, 66100, Chieti, Italy
| | - Pasquale Viggiano
- Ophthalmology Clinic, Department of Translational Biomedicine Neuroscience, University of Bari "Aldo Moro", Bari, Italy
| | - Edoardo Midena
- Department of Ophthalmology, University of Padova, 35128, Padova, Italy
- IRCCS- Fondazione Bietti, 00198, Roma, Italy
| | - Rodolfo Mastropasqua
- Ophthalmology Clinic, Department of Neuroscience, Imaging and Clinical Science, "G. D'Annunzio" University of Chieti-Pescara, Via Dei Vestini Snc, 66100, Chieti, Italy
| |
Collapse
|
2
|
Yan J, Zeng Y, Lin J, Pei Z, Fan J, Fang C, Cai Y. Enhanced object detection in pediatric bronchoscopy images using YOLO-based algorithms with CBAM attention mechanism. Heliyon 2024; 10:e32678. [PMID: 39021922 PMCID: PMC11252869 DOI: 10.1016/j.heliyon.2024.e32678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 06/05/2024] [Accepted: 06/06/2024] [Indexed: 07/20/2024] Open
Abstract
Background and Objective Bronchoscopy is a widely used diagnostic and therapeutic procedure for respiratory disorders such as infections and tumors. However, visualizing the bronchial tubes and lungs can be challenging due to the presence of various objects, such as mucus, blood, and foreign bodies. Accurately identifying the anatomical location of the bronchi can be quite challenging, especially for medical professionals who are new to the field. Deep learning-based object detection algorithms can assist doctors in analyzing images or videos of the bronchial tubes to identify key features such as the epiglottis, vocal cord, and right basal bronchus. This study aims to improve the accuracy of object detection in bronchoscopy images by integrating a YOLO-based algorithm with a CBAM attention mechanism. Methods The CBAM attention module is implemented in the YOLO-V7 and YOLO-V8 object detection models to improve their object identification and classification capabilities in bronchoscopy images. Various YOLO-based object detection algorithms, such as YOLO-V5, YOLO-V7, and YOLO-V8 are compared on this dataset. Experiments are conducted to evaluate the performance of the proposed method and different algorithms. Results The proposed method significantly improves the accuracy and reliability of object detection for bronchoscopy images. This approach demonstrates the potential benefits of incorporating an attention mechanism in medical imaging and the benefits of utilizing object detection algorithms in bronchoscopy. In the experiments, the YOLO-V8-based model achieved a mean Average Precision (mAP) of 87.09% on the given dataset with an Intersection over Union (IoU) threshold of 0.5. After incorporating the Convolutional Block Attention Module (CBAM) into the YOLO-V8 architecture, the proposed method achieved a significantly enhanced m A P 0.5 and m A P 0.5 : 0.95 of 88.27% and 55.39%, respectively. Conclusions Our findings indicate that by incorporating a CBAM attention mechanism with a YOLO-based algorithm, there is a noticeable improvement in object detection performance in bronchoscopy images. This study provides valuable insights into enhancing the performance of attention mechanisms for object detection in medical imaging.
Collapse
Affiliation(s)
- Jianqi Yan
- Faculty of Innovation Engineering, Macau University of Science and Technology, Avenida Wai Long, Taipa, 999078, Macau
- R&D Department, Quanbao Technologies Co. Ltd, Hagongda Road, Xiangzhou District, Zhuhai, 519087, China
| | - Yifan Zeng
- R&D Department, Quanbao Technologies Co. Ltd, Hagongda Road, Xiangzhou District, Zhuhai, 519087, China
| | - Junhong Lin
- Pediatric Respiratory Department, M-Healtcare, Zhujiang New Town Clinic 2/F, No. 11 Xiancun Road, Tianhe District, Guangzhou, 510623, China
| | - Zhiyuan Pei
- Faculty of Innovation Engineering, Macau University of Science and Technology, Avenida Wai Long, Taipa, 999078, Macau
- R&D Department, Quanbao Technologies Co. Ltd, Hagongda Road, Xiangzhou District, Zhuhai, 519087, China
| | - Jinrui Fan
- General Surgery, Zhuhai People's Hospital, Kangning Road, Xiangzhou District, Zhuhai, 519000, China
| | - Chuanyu Fang
- R&D Department, Quanbao Technologies Co. Ltd, Hagongda Road, Xiangzhou District, Zhuhai, 519087, China
| | - Yong Cai
- Advanced Institute of Natural Sciences, Beijing Normal University, Jinfeng Road, Xiangzhou District, Zhuhai, 519087, China
| |
Collapse
|
3
|
Zhao T, Guan Y, Tu D, Yuan L, Lu G. Neighbored-attention U-net (NAU-net) for diabetic retinopathy image segmentation. Front Med (Lausanne) 2023; 10:1309795. [PMID: 38131040 PMCID: PMC10733532 DOI: 10.3389/fmed.2023.1309795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Accepted: 11/22/2023] [Indexed: 12/23/2023] Open
Abstract
Background Diabetic retinopathy-related (DR-related) diseases are posing an increasing threat to eye health as the number of patients with diabetes mellitus that are young increases significantly. The automatic diagnosis of DR-related diseases has benefited from the rapid development of image semantic segmentation and other deep learning technology. Methods Inspired by the architecture of U-Net family, a neighbored attention U-Net (NAU-Net) is designed to balance the identification performance and computational cost for DR fundus image segmentation. In the new network, only the neighboring high- and low-dimensional feature maps of the encoder and decoder are fused by using four attention gates. With the help of this improvement, the common target features in the high-dimensional feature maps of encoder are enhanced, and they are also fused with the low-dimensional feature map of decoder. Moreover, this network fuses only neighboring layers and does not include the inner layers commonly used in U-Net++. Consequently, the proposed network incurs a better identification performance with a lower computational cost. Results The experimental results of three open datasets of DR fundus images, including DRIVE, HRF, and CHASEDB, indicate that the NAU-Net outperforms FCN, SegNet, attention U-Net, and U-Net++ in terms of Dice score, IoU, accuracy, and precision, while its computation cost is between attention U-Net and U-Net++. Conclusion The proposed NAU-Net exhibits better performance at a relatively low computational cost and provides an efficient novel approach for DR fundus image segmentation and a new automatic tool for DR-related eye disease diagnosis.
Collapse
Affiliation(s)
- Tingting Zhao
- The Second Department of Internal Medicine, Donghu Hospital of Wuhan, Wuhan, China
| | - Yawen Guan
- The Second Department of Internal Medicine, Donghu Hospital of Wuhan, Wuhan, China
| | - Dan Tu
- The Second Department of Internal Medicine, Donghu Hospital of Wuhan, Wuhan, China
| | - Lixia Yuan
- The Department of Ophthalmology, Donghu Hospital of Wuhan, Wuhan, China
| | - Guangtao Lu
- Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan, China
| |
Collapse
|
4
|
Wu H, Zhao J, Li J, Zeng Y, Wu W, Zhou Z, Wu S, Xu L, Song M, Yu Q, Song Z, Chen L. One-Stage Detection without Segmentation for Multi-Type Coronary Lesions in Angiography Images Using Deep Learning. Diagnostics (Basel) 2023; 13:3011. [PMID: 37761378 PMCID: PMC10528585 DOI: 10.3390/diagnostics13183011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 09/12/2023] [Accepted: 09/18/2023] [Indexed: 09/29/2023] Open
Abstract
It is rare to use the one-stage model without segmentation for the automatic detection of coronary lesions. This study sequentially enrolled 200 patients with significant stenoses and occlusions of the right coronary and categorized their angiography images into two angle views: The CRA (cranial) view of 98 patients with 2453 images and the LAO (left anterior oblique) view of 176 patients with 3338 images. Randomization was performed at the patient level to the training set and test set using a 7:3 ratio. YOLOv5 was adopted as the key model for direct detection. Four types of lesions were studied: Local Stenosis (LS), Diffuse Stenosis (DS), Bifurcation Stenosis (BS), and Chronic Total Occlusion (CTO). At the image level, the precision, recall, mAP@0.1, and mAP@0.5 predicted by the model were 0.64, 0.68, 0.66, and 0.49 in the CRA view and 0.68, 0.73, 0.70, and 0.56 in the LAO view, respectively. At the patient level, the precision, recall, and F1scores predicted by the model were 0.52, 0.91, and 0.65 in the CRA view and 0.50, 0.94, and 0.64 in the LAO view, respectively. YOLOv5 performed the best for lesions of CTO and LS at both the image level and the patient level. In conclusion, the one-stage model without segmentation as YOLOv5 is feasible to be used in automatic coronary lesion detection, with the most suitable types of lesions as LS and CTO.
Collapse
Affiliation(s)
- Hui Wu
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Jing Zhao
- Department of Geriatrics, The Third Medical Center of Chinese PLA General Hospital, Beijing 100039, China
| | - Jiehui Li
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Yan Zeng
- Department of Research Center, Shanghai United Imaging Intelligence Co., Ltd., Shanghai 201807, China
| | - Weiwei Wu
- College of Biomedical Engineering, Capital Medical University, Beijing 100069, China
| | - Zhuhuang Zhou
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Shuicai Wu
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Liang Xu
- State Key Laboratory of Cardiovascular Disease, Department of Structural Heart Disease, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Min Song
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Qibin Yu
- State Key Laboratory of Cardiovascular Disease, Department of Cardiac Surgery, National Center for Cardiovascular Diseases, Fuwai Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing 100037, China
| | - Ziwei Song
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| | - Lin Chen
- Department of Biomedical Engineering, Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
5
|
Raudonis V, Kairys A, Verkauskiene R, Sokolovska J, Petrovski G, Balciuniene VJ, Volke V. Automatic Detection of Microaneurysms in Fundus Images Using an Ensemble-Based Segmentation Method. SENSORS (BASEL, SWITZERLAND) 2023; 23:3431. [PMID: 37050491 PMCID: PMC10099354 DOI: 10.3390/s23073431] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 03/10/2023] [Accepted: 03/21/2023] [Indexed: 06/19/2023]
Abstract
In this study, a novel method for automatic microaneurysm detection in color fundus images is presented. The proposed method is based on three main steps: (1) image breakdown to smaller image patches, (2) inference to segmentation models, and (3) reconstruction of the predicted segmentation map from output patches. The proposed segmentation method is based on an ensemble of three individual deep networks, such as U-Net, ResNet34-UNet and UNet++. The performance evaluation is based on the calculation of the Dice score and IoU values. The ensemble-based model achieved higher Dice score (0.95) and IoU (0.91) values compared to other network architectures. The proposed ensemble-based model demonstrates the high practical application potential for detection of early-stage diabetic retinopathy in color fundus images.
Collapse
Affiliation(s)
- Vidas Raudonis
- Automation Department, Kaunas University of Technology, 51368 Kaunas, Lithuania
| | - Arturas Kairys
- Automation Department, Kaunas University of Technology, 51368 Kaunas, Lithuania
| | - Rasa Verkauskiene
- Institute of Endocrinology, Lithuanian University of Health Sciences, 50140 Kaunas, Lithuania
| | | | - Goran Petrovski
- Center of Eye Research and Innovative Diagnostics, Department of Ophthalmology, Oslo University Hospital and Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, 0372 Oslo, Norway
- Department of Ophthalmology, University of Split School of Medicine and University Hospital Centre, 21000 Split, Croatia
| | | | - Vallo Volke
- Faculty of Medicine, Tartu University, 50411 Tartu, Estonia
| |
Collapse
|
6
|
Lin G, Liu K, Xia X, Yan R. An Efficient and Intelligent Detection Method for Fabric Defects based on Improved YOLOv5. SENSORS (BASEL, SWITZERLAND) 2022; 23:97. [PMID: 36616696 PMCID: PMC9824629 DOI: 10.3390/s23010097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 12/15/2022] [Accepted: 12/19/2022] [Indexed: 06/17/2023]
Abstract
Limited by computing resources of embedded devices, there are problems in the field of fabric defect detection, including small defect size, extremely unbalanced aspect ratio of defect size, and slow detection speed. To address these problems, a sliding window multihead self-attention mechanism is proposed for the detection of small targets, and the Swin Transformer module is introduced to replace the main module in the original YOLOv5 algorithm. First, to reduce the distance between several scales, the weighted bidirectional feature network is employed on embedded devices. In addition, it is helpful to improve the perception of small-target faults by incorporating a detection layer to achieve four-scale detection. At last, to improve the learning of positive sample instances and lower the missed detection rate, the generalized focal loss function is finally implemented on YOLOv5. Experimental results show that the accuracy of the improved algorithm on the fabric dataset reaches 85.6%, and the mAP is increased by 4.2% to 76.5%, which meets the requirements for real-time detection on embedded devices.
Collapse
Affiliation(s)
- Guijuan Lin
- School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen 361024, China
| | - Keyu Liu
- School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen 361024, China
| | - Xuke Xia
- Quanzhou Institute of Equipment Manufacturing, Haixi Institute, Chinese Academy of Sciences, Jinjiang 362216, China
| | - Ruopeng Yan
- School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen 361024, China
| |
Collapse
|
7
|
Yan B, Li J, Yang Z, Zhang X, Hao X. AIE-YOLO: Auxiliary Information Enhanced YOLO for Small Object Detection. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22218221. [PMID: 36365919 PMCID: PMC9658690 DOI: 10.3390/s22218221] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 10/21/2022] [Accepted: 10/24/2022] [Indexed: 05/14/2023]
Abstract
Small object detection is one of the key challenges in the current computer vision field due to the low amount of information carried and the information loss caused by feature extraction. You Only Look Once v5 (YOLOv5) adopts the Path Aggregation Network to alleviate the problem of information loss, but it cannot restore the information that has been lost. To this end, an auxiliary information-enhanced YOLO is proposed to improve the sensitivity and detection performance of YOLOv5 to small objects. Firstly, a context enhancement module containing a receptive field size of 21×21 is proposed, which captures the global and local information of the image by fusing multi-scale receptive fields, and introduces an attention branch to enhance the expressive ability of key features and suppress background noise. To further enhance the feature expression ability of small objects, we introduce the high- and low-frequency information decomposed by wavelet transform into PANet to participate in multi-scale feature fusion, so as to solve the problem that the features of small objects gradually disappear after multiple downsampling and pooling operations. Experiments on the challenging dataset Tsinghua-Tencent 100 K show that the mean average precision of the proposed model is 9.5% higher than that of the original YOLOv5 while maintaining the real-time speed, which is better than the mainstream object detection models.
Collapse
|
8
|
El-Baz A, Giridharan GA, Shalaby A, Mahmoud AH, Ghazal M. Special Issue "Computer Aided Diagnosis Sensors". SENSORS (BASEL, SWITZERLAND) 2022; 22:8052. [PMID: 36298403 PMCID: PMC9610085 DOI: 10.3390/s22208052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Accepted: 10/19/2022] [Indexed: 06/16/2023]
Abstract
Sensors used to diagnose, monitor or treat diseases in the medical domain are known as medical sensors [...].
Collapse
Affiliation(s)
- Ayman El-Baz
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA
| | | | - Ahmed Shalaby
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA
| | - Ali H. Mahmoud
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA
| | - Mohammed Ghazal
- Electrical, Computer, and Biomedical Engineering Department, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates
| |
Collapse
|