1
|
Tahir AM, Guo L, Ward RK, Yu X, Rideout A, Hore M, Wang ZJ. Explainable machine learning for assessing upper respiratory tract of racehorses from endoscopy videos. Comput Biol Med 2024; 181:109030. [PMID: 39173488 DOI: 10.1016/j.compbiomed.2024.109030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Revised: 06/20/2024] [Accepted: 08/13/2024] [Indexed: 08/24/2024]
Abstract
Laryngeal hemiplegia (LH) is a major upper respiratory tract (URT) complication in racehorses. Endoscopy imaging of horse throat is a gold standard for URT assessment. However, current manual assessment faces several challenges, stemming from the poor quality of endoscopy videos and subjectivity of manual grading. To overcome such limitations, we propose an explainable machine learning (ML)-based solution for efficient URT assessment. Specifically, a cascaded YOLOv8 architecture is utilized to segment the key semantic regions and landmarks per frame. Several spatiotemporal features are then extracted from key landmarks points and fed to a decision tree (DT) model to classify LH as Grade 1,2,3 or 4 denoting absence of LH, mild, moderate, and severe LH, respectively. The proposed method, validated through 5-fold cross-validation on 107 videos, showed promising performance in classifying different LH grades with 100%, 91.18%, 94.74% and 100% sensitivity values for Grade 1 to 4, respectively. Further validation on an external dataset of 72 cases confirmed its generalization capability with 90%, 80.95%, 100%, and 100% sensitivity values for Grade 1 to 4, respectively. We introduced several explainability related assessment functions, including: (i) visualization of YOLOv8 output to detect landmark estimation errors which can affect the final classification, (ii) time-series visualization to assess video quality, and (iii) backtracking of the DT output to identify borderline cases. We incorporated domain knowledge (e.g., veterinarian diagnostic procedures) into the proposed ML framework. This provides an assistive tool with clinical-relevance and explainability that can ease and speed up the URT assessment by veterinarians.
Collapse
Affiliation(s)
- Anas Mohammed Tahir
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Li Guo
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Rabab K Ward
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Xinhui Yu
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| | - Andrew Rideout
- Point To Point Research & Development, Vancouver, BC, Canada.
| | - Michael Hore
- Hagyard Equine Medical Institute, Lexington, KY, USA.
| | - Z Jane Wang
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada.
| |
Collapse
|
2
|
Oukdach Y, Garbaz A, Kerkaou Z, El Ansari M, Koutti L, El Ouafdi AF, Salihoun M. UViT-Seg: An Efficient ViT and U-Net-Based Framework for Accurate Colorectal Polyp Segmentation in Colonoscopy and WCE Images. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01124-8. [PMID: 38671336 DOI: 10.1007/s10278-024-01124-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 04/01/2024] [Accepted: 04/13/2024] [Indexed: 04/28/2024]
Abstract
Colorectal cancer (CRC) stands out as one of the most prevalent global cancers. The accurate localization of colorectal polyps in endoscopy images is pivotal for timely detection and removal, contributing significantly to CRC prevention. The manual analysis of images generated by gastrointestinal screening technologies poses a tedious task for doctors. Therefore, computer vision-assisted cancer detection could serve as an efficient tool for polyp segmentation. Numerous efforts have been dedicated to automating polyp localization, with the majority of studies relying on convolutional neural networks (CNNs) to learn features from polyp images. Despite their success in polyp segmentation tasks, CNNs exhibit significant limitations in precisely determining polyp location and shape due to their sole reliance on learning local features from images. While gastrointestinal images manifest significant variation in their features, encompassing both high- and low-level ones, a framework that combines the ability to learn both features of polyps is desired. This paper introduces UViT-Seg, a framework designed for polyp segmentation in gastrointestinal images. Operating on an encoder-decoder architecture, UViT-Seg employs two distinct feature extraction methods. A vision transformer in the encoder section captures long-range semantic information, while a CNN module, integrating squeeze-excitation and dual attention mechanisms, captures low-level features, focusing on critical image regions. Experimental evaluations conducted on five public datasets, including CVC clinic, ColonDB, Kvasir-SEG, ETIS LaribDB, and Kvasir Capsule-SEG, demonstrate UViT-Seg's effectiveness in polyp localization. To confirm its generalization performance, the model is tested on datasets not used in training. Benchmarking against common segmentation methods and state-of-the-art polyp segmentation approaches, the proposed model yields promising results. For instance, it achieves a mean Dice coefficient of 0.915 and a mean intersection over union of 0.902 on the CVC Colon dataset. Furthermore, UViT-Seg has the advantage of being efficient, requiring fewer computational resources for both training and testing. This feature positions it as an optimal choice for real-world deployment scenarios.
Collapse
Affiliation(s)
- Yassine Oukdach
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco.
| | - Anass Garbaz
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Zakaria Kerkaou
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Mohamed El Ansari
- Informatics and Applications Laboratory, Department of Computer Sciences, Faculty of Science, Moulay Ismail University, B.P 11201, Meknès, 52000, Morocco
| | - Lahcen Koutti
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Ahmed Fouad El Ouafdi
- LabSIV, Department of Computer Science, Faculty of Sciences, Ibnou Zohr University, Agadir, 80000, Morocco
| | - Mouna Salihoun
- Faculty of Medicine and Pharmacy of Rabat, Mohammed V University of Rabat, Rabat, 10000, Morocco
| |
Collapse
|
3
|
Hsu CM, Chen TH, Hsu CC, Wu CH, Lin CJ, Le PH, Lin CY, Kuo T. Two-stage deep-learning-based colonoscopy polyp detection incorporating fisheye and reflection correction. J Gastroenterol Hepatol 2024; 39:733-739. [PMID: 38225761 DOI: 10.1111/jgh.16470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Revised: 11/28/2023] [Accepted: 12/14/2023] [Indexed: 01/17/2024]
Abstract
BACKGROUND AND AIM Colonoscopy is a useful method for the diagnosis and management of colorectal diseases. Many computer-aided systems have been developed to assist clinicians in detecting colorectal lesions by analyzing colonoscopy images. However, fisheye-lens distortion and light reflection in colonoscopy images can substantially affect the clarity of these images and their utility in detecting polyps. This study proposed a two-stage deep-learning model to correct distortion and reflections in colonoscopy images and thus facilitate polyp detection. METHODS Images were collected from the PolypSet dataset, the Kvasir-SEG dataset, and one medical center's patient archiving and communication system. The training, validation, and testing datasets comprised 808, 202, and 1100 images, respectively. The first stage involved the correction of fisheye-related distortion in colonoscopy images and polyp detection, which was performed using a convolutional neural network. The second stage involved the use of generative and adversarial networks for correcting reflective colonoscopy images before the convolutional neural network was used for polyp detection. RESULTS The model had higher accuracy when it was validated using corrected images than when it was validated using uncorrected images (96.8% vs 90.8%, P < 0.001). The model's accuracy in detecting polyps in the Kvasir-SEG dataset reached 96%, and the area under the receiver operating characteristic curve was 0.94. CONCLUSION The proposed model can facilitate the clinical diagnosis of colorectal polyps and improve the quality of colonoscopy.
Collapse
Affiliation(s)
- Chen-Ming Hsu
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Taoyuan Branch, Taoyuan, Taiwan
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
- Chang Gung University College of Medicine, Taoyuan, Taiwan
| | - Tsung-Hsing Chen
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
- Chang Gung University College of Medicine, Taoyuan, Taiwan
| | - Chien-Chang Hsu
- Department of Computer Science and Information Engineering, Fu Jen Catholic University, Taipei, Taiwan
| | - Che-Hao Wu
- Department of Computer Science and Information Engineering, Fu Jen Catholic University, Taipei, Taiwan
| | - Chun-Jung Lin
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
- Chang Gung University College of Medicine, Taoyuan, Taiwan
| | - Puo-Hsien Le
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
- Chang Gung University College of Medicine, Taoyuan, Taiwan
| | - Cheng-Yu Lin
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
| | - Tony Kuo
- Department of Gastroenterology and Hepatology, Chang Gung Memorial Hospital Linkou Main Branch, Taoyuan, Taiwan
| |
Collapse
|
4
|
Jain S, Atale R, Gupta A, Mishra U, Seal A, Ojha A, Jaworek-Korjakowska J, Krejcar O. CoInNet: A Convolution-Involution Network With a Novel Statistical Attention for Automatic Polyp Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3987-4000. [PMID: 37768798 DOI: 10.1109/tmi.2023.3320151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/30/2023]
Abstract
Polyps are very common abnormalities in human gastrointestinal regions. Their early diagnosis may help in reducing the risk of colorectal cancer. Vision-based computer-aided diagnostic systems automatically identify polyp regions to assist surgeons in their removal. Due to their varying shape, color, size, texture, and unclear boundaries, polyp segmentation in images is a challenging problem. Existing deep learning segmentation models mostly rely on convolutional neural networks that have certain limitations in learning the diversity in visual patterns at different spatial locations. Further, they fail to capture inter-feature dependencies. Vision transformer models have also been deployed for polyp segmentation due to their powerful global feature extraction capabilities. But they too are supplemented by convolution layers for learning contextual local information. In the present paper, a polyp segmentation model CoInNet is proposed with a novel feature extraction mechanism that leverages the strengths of convolution and involution operations and learns to highlight polyp regions in images by considering the relationship between different feature maps through a statistical feature attention unit. To further aid the network in learning polyp boundaries, an anomaly boundary approximation module is introduced that uses recursively fed feature fusion to refine segmentation results. It is indeed remarkable that even tiny-sized polyps with only 0.01% of an image area can be precisely segmented by CoInNet. It is crucial for clinical applications, as small polyps can be easily overlooked even in the manual examination due to the voluminous size of wireless capsule endoscopy videos. CoInNet outperforms thirteen state-of-the-art methods on five benchmark polyp segmentation datasets.
Collapse
|
5
|
Alam MJ, Fattah SA. SR-AttNet: An Interpretable Stretch-Relax Attention based Deep Neural Network for Polyp Segmentation in Colonoscopy Images. Comput Biol Med 2023; 160:106945. [PMID: 37163966 DOI: 10.1016/j.compbiomed.2023.106945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2022] [Revised: 04/07/2023] [Accepted: 04/14/2023] [Indexed: 05/12/2023]
Abstract
BACKGROUND Colorectal polyp is a common structural gastrointestinal (GI) anomaly, which can in certain cases turn malignant. Colonoscopic image inspection is, thereby, an important step for isolating the polyps as well as removing them if necessary. However, the process is around 30-60 min long and inspecting each image for polyps can prove to be a tedious task. Hence, an automatic computerized process for efficient and accurate polyp isolation can be a useful tool. METHODS In this study, a deep learning network is introduced for colorectal polyp segmentation. The network is based on an encoder-decoder architecture, however, having both un-dilated and dilated filtering in order to extract both near and far local information as well as perceive image depth. Four-fold skip-connections exist between each spatial encoder-decoder due to both type of filtering and a 'Feature-to-Mask' pipeline processes the decoded dilated and un-dilated features for final prediction. The proposed network implements a 'Stretch-Relax' based attention system, SR-Attention, to generate high variance spatial features in order to obtain useful attention masks for cognitive feature selection. From this 'Stretch-Relax' attention based operation, the network is termed as 'SR-AttNet'. RESULTS Training and optimization is performed on four different datasets, and inference has been done on five (Kvasir-SEG, CVC-ClinicDB, CVC-Colon, ETIS-Larib, EndoCV2020); all of which output higher Dice-score compared to state-of-the-art and existing networks. The efficacy and interpretability of SR-Attention is also demonstrated based on quantitative variance. CONCLUSION In consequence, the proposed SR-AttNet can be considered for an automated and general approach for polyp segmentation during colonoscopy.
Collapse
Affiliation(s)
- Md Jahin Alam
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh.
| | - Shaikh Anowarul Fattah
- Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1205, Bangladesh.
| |
Collapse
|
6
|
Ali S, Jha D, Ghatwary N, Realdon S, Cannizzaro R, Salem OE, Lamarque D, Daul C, Riegler MA, Anonsen KV, Petlund A, Halvorsen P, Rittscher J, de Lange T, East JE. A multi-centre polyp detection and segmentation dataset for generalisability assessment. Sci Data 2023; 10:75. [PMID: 36746950 PMCID: PMC9902556 DOI: 10.1038/s41597-023-01981-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 01/23/2023] [Indexed: 02/08/2023] Open
Abstract
Polyps in the colon are widely known cancer precursors identified by colonoscopy. Whilst most polyps are benign, the polyp's number, size and surface structure are linked to the risk of colon cancer. Several methods have been developed to automate polyp detection and segmentation. However, the main issue is that they are not tested rigorously on a large multicentre purpose-built dataset, one reason being the lack of a comprehensive public dataset. As a result, the developed methods may not generalise to different population datasets. To this extent, we have curated a dataset from six unique centres incorporating more than 300 patients. The dataset includes both single frame and sequence data with 3762 annotated polyp labels with precise delineation of polyp boundaries verified by six senior gastroenterologists. To our knowledge, this is the most comprehensive detection and pixel-level segmentation dataset (referred to as PolypGen) curated by a team of computational scientists and expert gastroenterologists. The paper provides insight into data construction and annotation strategies, quality assurance, and technical validation.
Collapse
Affiliation(s)
- Sharib Ali
- School of Computing, University of Leeds, LS2 9JT, Leeds, United Kingdom.
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, OX3 7DQ, Oxford, United Kingdom.
- Oxford National Institute for Health Research Biomedical Research centre, OX4 2PG, Oxford, United Kingdom.
| | - Debesh Jha
- SimulaMet, Pilestredet 52, 0167, Oslo, Norway
- Department of Computer Science, UiT The Arctic University of Norway, Hansine Hansens veg 18, 9019, Tromsø, Norway
- Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, USA
| | - Noha Ghatwary
- Computer Engineering Department, Arab Academy for Science and Technology,Smart Village, Giza, Egypt
| | - Stefano Realdon
- Oncological Gastroenterology - Centro di Riferimento Oncologico di Aviano (CRO), IRCCS, 2, 33081, Aviano, PN, Italy
| | - Renato Cannizzaro
- Oncological Gastroenterology - Centro di Riferimento Oncologico di Aviano (CRO), IRCCS, 2, 33081, Aviano, PN, Italy
- Department of Medical, Surgical and Health Sciences, University of Trieste, 34127, Trieste, Italy
| | - Osama E Salem
- Faculty of Medicine, University of Alexandria, 21131, Alexandria, Egypt
| | - Dominique Lamarque
- Université de Versailles St-Quentin en Yvelines, Hôpital Ambroise Paré, 9 Av. Charles de Gaulle, 92100, Boulogne-Billancourt, France
| | - Christian Daul
- CRAN UMR 7039, Université de Lorraine and CNRS, F-54010, Vandœuvre-Lès-Nancy, France
| | - Michael A Riegler
- SimulaMet, Pilestredet 52, 0167, Oslo, Norway
- Department of Computer Science, UiT The Arctic University of Norway, Hansine Hansens veg 18, 9019, Tromsø, Norway
| | - Kim V Anonsen
- Oslo University Hospital Ullevål, Kirkeveien 166, 0450, Oslo, Norway
| | | | - Pål Halvorsen
- SimulaMet, Pilestredet 52, 0167, Oslo, Norway
- Oslo Metropolitan University, Pilestredet 46, 0167, Oslo, Norway
| | - Jens Rittscher
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, OX3 7DQ, Oxford, United Kingdom
- Oxford National Institute for Health Research Biomedical Research centre, OX4 2PG, Oxford, United Kingdom
| | - Thomas de Lange
- Augere Medical, Nedre Vaskegang 6, 0186, Oslo, Norway
- Medical Department, Sahlgrenska University Hospital-Mölndal, Blå stråket 5, 413 45, Göteborg, Sweden
- Department of Molecular and Clinical Medicine, Sahlgrenska Academy, University of Gothenburg, 41345, Göteborg, Sweden
| | - James E East
- Oxford National Institute for Health Research Biomedical Research centre, OX4 2PG, Oxford, United Kingdom
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, OX3 9DU, Oxford, United Kingdom
| |
Collapse
|
7
|
Qureshi I, Yan J, Abbas Q, Shaheed K, Riaz AB, Wahid A, Khan MWJ, Szczuko P. Medical image segmentation using deep semantic-based methods: A review of techniques, applications and emerging trends. INFORMATION FUSION 2023. [DOI: 10.1016/j.inffus.2022.09.031] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2023]
|
8
|
Sadagopan R, Ravi S, Adithya SV, Vivekanandhan S. PolyEffNetV1: A CNN based colorectal polyp detection in colonoscopy images. Proc Inst Mech Eng H 2023; 237:406-418. [PMID: 36683465 DOI: 10.1177/09544119221149233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
Abstract
Presence of polyps is the root cause of colorectal cancer, hence identification of such polyps at an early stage can help in advance treatments to avoid complications to the patient. Since there are variations in the size and shape of polyps, the task of detecting them in colonoscopy images becomes challenging. Hence our work is to leverage an algorithm for segmentation and classification of the polyp of colonoscopy images using Deep learning algorithms. In this work, we propose PolypEffNetV1, a U-Net to segment the different pathologies present in the colonoscopy frame and EfficientNetB5 to classify the detected pathologies. The colonoscopy images for the segmentation process are taken from the open-source dataset KVASIR, it consists of 1000 images with "ground truth" labeling. For classification, combination of KVASIR and CVC datasets are incorporated, which consists of 1612 images with 1696 polyp regions and 760 non-polyp inflamed regions. The proposed PolypEffNetV1 produced testing accuracy of 97.1%, Jaccard index of 0.84, dice coefficient of 0.91, and F1-score of 0.89. Subsequently, for classification to evidence whether the segmented region is polyp or non-polyp inflammation, the developed classifier produced validation accuracy of 99%, specificity of 98%, and sensitivity of 99%. Hence the proposed system could be used by gastroenterologists to identify the presence of polyp in the colonoscopy images/videos which will in turn increase healthcare quality. These developed models can be either deployed on the edge of the device to enable real-time aidance or can be integrated with existing software-application for offline review and treatment planning.
Collapse
Affiliation(s)
- Rajkumar Sadagopan
- Department of Biomedical Engineering, Rajalakshmi Engineering College, Chennai, India.,Centre of Excellence in Medical Imaging, Rajalakshmi Engineering College, Chennai, India
| | - Saravanan Ravi
- Department of Biomedical Engineering, Rajalakshmi Engineering College, Chennai, India.,Centre of Excellence in Medical Imaging, Rajalakshmi Engineering College, Chennai, India
| | - Sairam Vuppala Adithya
- Department of Biomedical Engineering, Rajalakshmi Engineering College, Chennai, India.,Centre of Excellence in Medical Imaging, Rajalakshmi Engineering College, Chennai, India
| | - Sapthagirivasan Vivekanandhan
- Department of Biomedical Engineering, Rajalakshmi Engineering College, Chennai, India.,Medical and Life Sciences Department, Engineering R&D Division, IT Services Company, Bengaluru, India
| |
Collapse
|
9
|
Niu K, Guo Z, Peng X, Pei S. P-ResUnet: Segmentation of brain tissue with Purified Residual Unet. Comput Biol Med 2022; 151:106294. [PMID: 36435055 DOI: 10.1016/j.compbiomed.2022.106294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Revised: 10/14/2022] [Accepted: 11/06/2022] [Indexed: 11/13/2022]
Abstract
Brain tissue of Magnetic Resonance Imaging is precisely segmented and quantified, which aids in the diagnosis of neurological diseases such as epilepsy, Alzheimer's, and multiple sclerosis. Recently, UNet-like architectures are widely used for medical image segmentation, which achieved promising performance by using the skip connection to fuse the low-level and high-level information. However, In the process of integrating the low-level and high-level information, the non-object information (noise) will be added, which reduces the accuracy of medical image segmentation. Likewise, the same problem also exists in the residual unit. Since the output and input of the residual unit are fused, the non-object information (noise) of the input of the residual unit will be in the integration. To address this challenging problem, in this paper we propose a Purified Residual U-net for the segmentation of brain tissue. This model encodes the image to obtain deep semantic information and purifies the information of low-level features and the residual unit from the image, and acquires the result through a decoder at last. We use the Dilated Pyramid Separate Block (DPSB) as the first block to purify the features for each layer in the encoder without the first layer, which expands the receptive field of the convolution kernel with only a few parameters added. In the first layer, we have explored the best performance achieved with DPB. We find the most non-object information (noise) in the initial image, so it is good for the accuracy to exchange the information to the max degree. We have conducted experiments with the widely used IBSR-18 dataset composed of T-1 weighted MRI volumes from 18 subjects. The results show that compared with some of the cutting-edge methods, our method enhances segmentation performance with the mean dice score reaching 91.093% and the mean Hausdorff distance decreasing to 3.2606.
Collapse
Affiliation(s)
- Ke Niu
- Beijing Information Science and Technology University, Beijing, China.
| | - Zhongmin Guo
- Beijing Information Science and Technology University, Beijing, China.
| | - Xueping Peng
- Australian Artificial Intelligence Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Australia.
| | - Su Pei
- Beijing Information Science and Technology University, Beijing, China.
| |
Collapse
|
10
|
Muacevic A, Adler JR, Landge S, Pundkar A, Chandanwale R. Unusual Solitary Neurofibroma of Common Peroneal Nerve in a Child. Cureus 2022; 14:e33039. [PMID: 36721607 PMCID: PMC9881391 DOI: 10.7759/cureus.33039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 12/28/2022] [Indexed: 12/29/2022] Open
Abstract
Neurofibroma (NF) is a tumour of peripheral nerves, which would be seldom seen in the limbs, particularly in children's limbs. Soft, skin-coloured papules or small sub-mucosal nodules appear as these lesions. Neurofibroma is classified into three types: localized, diffuse, and plexiform. The vast majority of nerve injury is sporadic and localized, with an incredibly low risk of tumour formation. Neurofibromatosis can present as multiple skin lesions along with bone deformities in which a full investigation is critical where an undiscovered widespread illness may arise. This case study describes a neurofibroma on the common peroneal nerve of the left lower limb in a 6-year-old child who visited our hospital with chief complaints of pain and swelling around the left proximal leg.
Collapse
|
11
|
Tavanapong W, Oh J, Riegler MA, Khaleel M, Mittal B, de Groen PC. Artificial Intelligence for Colonoscopy: Past, Present, and Future. IEEE J Biomed Health Inform 2022; 26:3950-3965. [PMID: 35316197 PMCID: PMC9478992 DOI: 10.1109/jbhi.2022.3160098] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
During the past decades, many automated image analysis methods have been developed for colonoscopy. Real-time implementation of the most promising methods during colonoscopy has been tested in clinical trials, including several recent multi-center studies. All trials have shown results that may contribute to prevention of colorectal cancer. We summarize the past and present development of colonoscopy video analysis methods, focusing on two categories of artificial intelligence (AI) technologies used in clinical trials. These are (1) analysis and feedback for improving colonoscopy quality and (2) detection of abnormalities. Our survey includes methods that use traditional machine learning algorithms on carefully designed hand-crafted features as well as recent deep-learning methods. Lastly, we present the gap between current state-of-the-art technology and desirable clinical features and conclude with future directions of endoscopic AI technology development that will bridge the current gap.
Collapse
|
12
|
Ashkani Chenarlogh V, Shabanzadeh A, Ghelich Oghli M, Sirjani N, Farzin Moghadam S, Akhavan A, Arabi H, Shiri I, Shabanzadeh Z, Sanei Taheri M, Kazem Tarzamni M. Clinical target segmentation using a novel deep neural network: double attention Res-U-Net. Sci Rep 2022; 12:6717. [PMID: 35468984 PMCID: PMC9038725 DOI: 10.1038/s41598-022-10429-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 03/24/2022] [Indexed: 01/10/2023] Open
Abstract
We introduced Double Attention Res-U-Net architecture to address medical image segmentation problem in different medical imaging system. Accurate medical image segmentation suffers from some challenges including, difficulty of different interest object modeling, presence of noise, and signal dropout throughout the measurement. The base line image segmentation approaches are not sufficient for complex target segmentation throughout the various medical image types. To overcome the issues, a novel U-Net-based model proposed that consists of two consecutive networks with five and four encoding and decoding levels respectively. In each of networks, there are four residual blocks between the encoder-decoder path and skip connections that help the networks to tackle the vanishing gradient problem, followed by the multi-scale attention gates to generate richer contextual information. To evaluate our architecture, we investigated three distinct data-sets, (i.e., CVC-ClinicDB dataset, Multi-site MRI dataset, and a collected ultrasound dataset). The proposed algorithm achieved Dice and Jaccard coefficients of 95.79%, 91.62%, respectively for CRL, and 93.84% and 89.08% for fetal foot segmentation. Moreover, the proposed model outperformed the state-of-the-art U-Net based model on the external CVC-ClinicDB, and multi-site MRI datasets with Dice and Jaccard coefficients of 83%, 75.31% for CVC-ClinicDB, and 92.07% and 87.14% for multi-site MRI dataset, respectively.
Collapse
Affiliation(s)
- Vahid Ashkani Chenarlogh
- Research and Development Department, Med Fanavaran Plus Co., Karaj, Iran
- Department of Electrical and Computer Engineering, National Center for Audiology, Western University, London, Canada
| | - Ali Shabanzadeh
- Research and Development Department, Med Fanavaran Plus Co., Karaj, Iran
| | - Mostafa Ghelich Oghli
- Research and Development Department, Med Fanavaran Plus Co., Karaj, Iran.
- Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium.
| | - Nasim Sirjani
- Research and Development Department, Med Fanavaran Plus Co., Karaj, Iran
| | | | - Ardavan Akhavan
- Research and Development Department, Med Fanavaran Plus Co., Karaj, Iran
| | - Hossein Arabi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211, Geneva 4, Switzerland
| | - Isaac Shiri
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211, Geneva 4, Switzerland
| | - Zahra Shabanzadeh
- School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Morteza Sanei Taheri
- Department of Radiology, Shohada-e-Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mohammad Kazem Tarzamni
- Department of Radiology, Imam Reza Hospital, Tabriz University of Medical Sciences, Tabriz, Iran
| |
Collapse
|
13
|
Chen S, Urban G, Baldi P. Weakly Supervised Polyp Segmentation in Colonoscopy Images Using Deep Neural Networks. J Imaging 2022; 8:jimaging8050121. [PMID: 35621885 PMCID: PMC9144698 DOI: 10.3390/jimaging8050121] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2022] [Revised: 04/15/2022] [Accepted: 04/19/2022] [Indexed: 02/01/2023] Open
Abstract
Colorectal cancer (CRC) is a leading cause of mortality worldwide, and preventive screening modalities such as colonoscopy have been shown to noticeably decrease CRC incidence and mortality. Improving colonoscopy quality remains a challenging task due to limiting factors including the training levels of colonoscopists and the variability in polyp sizes, morphologies, and locations. Deep learning methods have led to state-of-the-art systems for the identification of polyps in colonoscopy videos. In this study, we show that deep learning can also be applied to the segmentation of polyps in real time, and the underlying models can be trained using mostly weakly labeled data, in the form of bounding box annotations that do not contain precise contour information. A novel dataset, Polyp-Box-Seg of 4070 colonoscopy images with polyps from over 2000 patients, is collected, and a subset of 1300 images is manually annotated with segmentation masks. A series of models is trained to evaluate various strategies that utilize bounding box annotations for segmentation tasks. A model trained on the 1300 polyp images with segmentation masks achieves a dice coefficient of 81.52%, which improves significantly to 85.53% when using a weakly supervised strategy leveraging bounding box images. The Polyp-Box-Seg dataset, together with a real-time video demonstration of the segmentation system, are publicly available.
Collapse
Affiliation(s)
- Siwei Chen
- Department of Computer Science, University of California, Irvine, CA 92697, USA; (S.C.); (G.U.)
- Institute for Genomics and Bioinformatics, University of California, Irvine, CA 92697, USA
| | - Gregor Urban
- Department of Computer Science, University of California, Irvine, CA 92697, USA; (S.C.); (G.U.)
- Institute for Genomics and Bioinformatics, University of California, Irvine, CA 92697, USA
| | - Pierre Baldi
- Department of Computer Science, University of California, Irvine, CA 92697, USA; (S.C.); (G.U.)
- Institute for Genomics and Bioinformatics, University of California, Irvine, CA 92697, USA
- Center for Machine Learning and Intelligent Systems, University of California, Irvine, CA 92697, USA
- Correspondence: ; Tel.: +1-949-824-5809
| |
Collapse
|
14
|
Srivastava A, Jha D, Chanda S, Pal U, Johansen H, Johansen D, Riegler M, Ali S, Halvorsen P. MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation. IEEE J Biomed Health Inform 2021; 26:2252-2263. [PMID: 34941539 DOI: 10.1109/jbhi.2021.3138024] [Citation(s) in RCA: 51] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Methods based on convolutional neural networks have improved the performance of biomedical image segmentation. However, most of these methods cannot efficiently segment objects of variable sizes and train on small and biased datasets, which are common for biomedical use cases. While methods exist that incorporate multi-scale fusion approaches to address the challenges arising with variable sizes, they usually use complex models that are more suitable for general semantic segmentation problems. In this paper, we propose a novel architecture called Multi-Scale Residual Fusion Network (MSRF-Net), which is specially designed for medical image segmentation. The proposed MSRF-Net is able to exchange multi-scale features of varying receptive fields using a Dual-Scale Dense Fusion (DSDF) block. Our DSDF block can exchange information rigorously across two different resolution scales, and our MSRF sub-network uses multiple DSDF blocks in sequence to perform multi-scale fusion. This allows the preservation of resolution, improved information flow and propagation of both high- and low-level features to obtain accurate segmentation maps. The proposed MSRF-Net allows to capture object variabilities and provides improved results on different biomedical datasets. Extensive experiments on MSRF-Net demonstrate that the proposed method outperforms the cutting edge medical image segmentation methods on four publicly available datasets. We achieve the Dice Coefficient (DSC) of 0.9217, 0.9420, and 0.9224, 0.8824 on Kvasir-SEG, CVC-ClinicDB, 2018 Data Science Bowl dataset, and ISIC-2018 skin lesion segmentation challenge dataset respectively. We further conducted generalizability tests that also achieved the highest DSC score with 0.7921 and 0.7575 on CVC-ClinicDB and Kvasir-SEG, respectively.
Collapse
|
15
|
Ali S, Dmitrieva M, Ghatwary N, Bano S, Polat G, Temizel A, Krenzer A, Hekalo A, Guo YB, Matuszewski B, Gridach M, Voiculescu I, Yoganand V, Chavan A, Raj A, Nguyen NT, Tran DQ, Huynh LD, Boutry N, Rezvy S, Chen H, Choi YH, Subramanian A, Balasubramanian V, Gao XW, Hu H, Liao Y, Stoyanov D, Daul C, Realdon S, Cannizzaro R, Lamarque D, Tran-Nguyen T, Bailey A, Braden B, East JE, Rittscher J. Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy. Med Image Anal 2021; 70:102002. [PMID: 33657508 DOI: 10.1016/j.media.2021.102002] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 02/04/2021] [Accepted: 02/11/2021] [Indexed: 12/12/2022]
Abstract
The Endoscopy Computer Vision Challenge (EndoCV) is a crowd-sourcing initiative to address eminent problems in developing reliable computer aided detection and diagnosis endoscopy systems and suggest a pathway for clinical translation of technologies. Whilst endoscopy is a widely used diagnostic and treatment tool for hollow-organs, there are several core challenges often faced by endoscopists, mainly: 1) presence of multi-class artefacts that hinder their visual interpretation, and 2) difficulty in identifying subtle precancerous precursors and cancer abnormalities. Artefacts often affect the robustness of deep learning methods applied to the gastrointestinal tract organs as they can be confused with tissue of interest. EndoCV2020 challenges are designed to address research questions in these remits. In this paper, we present a summary of methods developed by the top 17 teams and provide an objective comparison of state-of-the-art methods and methods designed by the participants for two sub-challenges: i) artefact detection and segmentation (EAD2020), and ii) disease detection and segmentation (EDD2020). Multi-center, multi-organ, multi-class, and multi-modal clinical endoscopy datasets were compiled for both EAD2020 and EDD2020 sub-challenges. The out-of-sample generalization ability of detection algorithms was also evaluated. Whilst most teams focused on accuracy improvements, only a few methods hold credibility for clinical usability. The best performing teams provided solutions to tackle class imbalance, and variabilities in size, origin, modality and occurrences by exploring data augmentation, data fusion, and optimal class thresholding techniques.
Collapse
Affiliation(s)
- Sharib Ali
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK.
| | - Mariia Dmitrieva
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK
| | - Noha Ghatwary
- Computer Engineering Department, Arab Academy for Science and Technology, Alexandria, Egypt
| | - Sophia Bano
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences(WEISS) and Department of Computer Science, University College London, London, UK
| | - Gorkem Polat
- Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
| | - Alptekin Temizel
- Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
| | - Adrian Krenzer
- Department of Artificial Intelligence and Knowledge Systems, University of Würzburg, Germany
| | - Amar Hekalo
- Department of Artificial Intelligence and Knowledge Systems, University of Würzburg, Germany
| | - Yun Bo Guo
- School of Engineering, University of Central Lancashire, UK
| | | | - Mourad Gridach
- Ibn Zohr University, Computer Science HIT, Agadir, Morocco
| | | | - Vishnusai Yoganand
- Mimyk Medical Simulations Pvt Ltd, Indian Institute of Science, Bengaluru, India
| | - Arnav Chavan
- Indian Institute of Technology (ISM), Dhanbad, India
| | - Aryan Raj
- Indian Institute of Technology (ISM), Dhanbad, India
| | - Nhan T Nguyen
- Medical Imaging Department, Vingroup Big Data Institute (VinBDI), Hanoi, Vietnam
| | - Dat Q Tran
- Medical Imaging Department, Vingroup Big Data Institute (VinBDI), Hanoi, Vietnam
| | - Le Duy Huynh
- EPITA Research and Development Laboratory (LRDE), F-94270 Le Kremlin-Bicêtre, France
| | - Nicolas Boutry
- EPITA Research and Development Laboratory (LRDE), F-94270 Le Kremlin-Bicêtre, France
| | - Shahadate Rezvy
- School of Science and Technology, Middlesex University London, UK
| | - Haijian Chen
- Department of Computer Science, School of Informatics, Xiamen University, China
| | - Yoon Ho Choi
- Dept. of Health Sciences & Tech., Samsung Advanced Institute for Health Sciences & Tech. (SAIHST), Sungkyunkwan University, Seoul, Republic of Korea
| | | | | | - Xiaohong W Gao
- School of Science and Technology, Middlesex University London, UK
| | - Hongyu Hu
- Shanghai Jiaotong University, Shanghai, China
| | | | - Danail Stoyanov
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences(WEISS) and Department of Computer Science, University College London, London, UK
| | - Christian Daul
- CRAN UMR 7039, University of Lorraine, CNRS, Nancy, France
| | | | | | - Dominique Lamarque
- Université de Versailles St-Quentin en Yvelines, Hôpital Ambroise Paré, France
| | - Terry Tran-Nguyen
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Adam Bailey
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - Barbara Braden
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - James E East
- Translational Gastroenterology Unit, Experimental Medicine Div., John Radcliffe Hospital, University of Oxford, Oxford, UK; Oxford NIHR Biomedical Research Centre, Oxford, UK
| | - Jens Rittscher
- Institute of Biomedical Engineering and Big Data Institute, Old Road Campus, University of Oxford, Oxford, UK
| |
Collapse
|
16
|
Mahmud T, Paul B, Fattah SA. PolypSegNet: A modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images. Comput Biol Med 2020; 128:104119. [PMID: 33254083 DOI: 10.1016/j.compbiomed.2020.104119] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2020] [Revised: 11/07/2020] [Accepted: 11/08/2020] [Indexed: 12/21/2022]
Abstract
Colorectal cancer has become one of the major causes of death throughout the world. Early detection of Polyp, an early symptom of colorectal cancer, can increase the survival rate to 90%. Segmentation of Polyp regions from colonoscopy images can facilitate the faster diagnosis. Due to varying sizes, shapes, and textures of polyps with subtle visible differences with the background, automated segmentation of polyps still poses a major challenge towards traditional diagnostic methods. Conventional Unet architecture and some of its variants have gained much popularity for its automated segmentation though having several architectural limitations that result in sub-optimal performance. In this paper, an encoder-decoder based modified deep neural network architecture is proposed, named as PolypSegNet, to overcome several limitations of traditional architectures for very precise automated segmentation of polyp regions from colonoscopy images. For achieving more generalized representation at each scale of both the encoder and decoder module, several sequential depth dilated inception (DDI) blocks are integrated into each unit layer for aggregating features from different receptive areas utilizing depthwise dilated convolutions. Different scales of contextual information from all encoder unit layers pass through the proposed deep fusion skip module (DFSM) to generate skip interconnection with each decoder layer rather than separately connecting different levels of encoder and decoder. For more efficient reconstruction in the decoder module, multi-scale decoded feature maps generated at various levels of the decoder are jointly optimized in the proposed deep reconstruction module (DRM) instead of only considering the decoded feature map from final decoder layer. Extensive experimentations on four publicly available databases provide very satisfactory performance with mean five-fold cross-validation dice scores of 91.52% in CVC-ClinicDB database, 92.8% in CVC-ColonDB database, 88.72% in Kvasir-SEG database, and 84.79% in ETIS-Larib database. The proposed network provides very accurate segmented polyp regions that will expedite the diagnosis of polyps even in challenging conditions.
Collapse
Affiliation(s)
- Tanvir Mahmud
- Department of EEE, BUET, ECE Building, West Palashi, Dhaka, 1205, Bangladesh.
| | - Bishmoy Paul
- Department of EEE, BUET, ECE Building, West Palashi, Dhaka, 1205, Bangladesh.
| | | |
Collapse
|