Iterative feedback-based models for image and video polyp segmentation

Accurate segmentation of polyps in colonoscopy images has gained significant attention in recent years, given its crucial role in automated colorectal cancer diagnosis. Many existing deep learning-based methods follow a one-stage processing pipeline, often involving feature fusion across different levels or utilizing boundary-related attention mechanisms. Drawing on the success of applying Iterative Feedback Units (IFU) in image polyp segmentation, this paper proposes FlowICBNet by extending the IFU to the domain of video polyp segmentation. By harnessing the unique capabilities of IFU to propagate and refine past segmentation results, our method proves effective in mitigating challenges linked to the inherent limitations of endoscopic imaging, notably the presence of frequent camera shake and frame defocusing. Furthermore, in FlowICBNet, we introduce two pivotal modules: Reference Frame Selection (RFS) and Flow Guided Warping (FGW). These modules play a crucial role in filtering and selecting the most suitable historical reference frames for the task at hand. The experimental results on a large video polyp segmentation dataset demonstrate that our method can significantly outperform state-of-the-art methods by notable margins achieving an average metrics improvement of 7.5% on SUN-SEG-Easy and 7.4% on SUN-SEG-Hard. Our code is available at https://github.com/eraserNut/ICBNet.

Self-supervised out-of-distribution detection in wireless capsule endoscopy images

While deep learning has displayed excellent performance in a broad spectrum of application areas, neural networks still struggle to recognize what they have not seen, i.e., out-of-distribution (OOD) inputs. In the medical field, building robust models that are able to detect OOD images is highly critical, as these rare images could show diseases or anomalies that should be detected. In this study, we use wireless capsule endoscopy (WCE) images to present a novel patch-based self-supervised approach comprising three stages. First, we train a triplet network to learn vector representations of WCE image patches. Second, we cluster the patch embeddings to group patches in terms of visual similarity. Third, we use the cluster assignments as pseudolabels to train a patch classifier and use the Out-of-Distribution Detector for Neural Networks (ODIN) for OOD detection. The system has been tested on the Kvasir-capsule, a publicly released WCE dataset. Empirical results show an OOD detection improvement compared to baseline methods. Our method can detect unseen pathologies and anomalies such as lymphangiectasia, foreign bodies and blood with AUROC>0.6. This work presents an effective solution for OOD detection models without needing labeled images.

EfficientNetB0 cum FPN Based Semantic Segmentation of Gastrointestinal Tract Organs in MRI Scans

The segmentation of gastrointestinal (GI) organs is crucial in radiation therapy for treating GI cancer. It allows for developing a targeted radiation therapy plan while minimizing radiation exposure to healthy tissue, improving treatment success, and decreasing side effects. Medical diagnostics in GI tract organ segmentation is essential for accurate disease detection, precise differential diagnosis, optimal treatment planning, and efficient disease monitoring. This research presents a hybrid encoder-decoder-based model for segmenting healthy organs in the GI tract in biomedical images of cancer patients, which might help radiation oncologists treat cancer more quickly. Here, EfficientNet B0 is used as a bottom-up encoder architecture for downsampling to capture contextual information by extracting meaningful and discriminative features from input images. The performance of the EfficientNet B0 encoder is compared with that of three encoders: ResNet 50, MobileNet V2, and Timm Gernet. The Feature Pyramid Network (FPN) is a top-down decoder architecture used for upsampling to recover spatial information. The performance of the FPN decoder was compared with that of three decoders: PAN, Linknet, and MAnet. This paper proposes a segmentation model named as the Feature Pyramid Network (FPN), with EfficientNet B0 as the encoder. Furthermore, the proposed hybrid model is analyzed using Adam, Adadelta, SGD, and RMSprop optimizers. Four performance criteria are used to assess the models: the Jaccard and Dice coefficients, model loss, and processing time. The proposed model can achieve Dice coefficient and Jaccard index values of 0.8975 and 0.8832, respectively. The proposed method can assist radiation oncologists in precisely targeting areas hosting cancer cells in the gastrointestinal tract, allowing for more efficient and timely cancer treatment.

PPNet: Pyramid pooling based network for polyp segmentation

Colonoscopy is the gold standard method for investigating the gastrointestinal tract. Localizing the polyps in colonoscopy images plays a vital role when doing a colonoscopy screening, and it is also quite important for the following treatment, e.g., polyp resection. Many deep learning-based methods have been applied for solving the polyp segmentation issue. However, precisely polyp segmentation is still an open issue. Considering the effectiveness of the Pyramid Pooling Transformer (P2T) in modeling long-range dependencies and capturing robust contextual features, as well as the power of pyramid pooling in extracting features, we propose a pyramid pooling based network for polyp segmentation, namely PPNet. We first adopt the P2T as the encoder for extracting more powerful features. Next, a pyramid feature fusion module (PFFM) combining the channel attention scheme is utilized for learning a global contextual feature, in order to guide the information transition in the decoder branch. Aiming to enhance the effectiveness of PPNet on feature extraction during the decoder stage layer by layer, we introduce the memory-keeping pyramid pooling module (MPPM) into each side branch of the encoder, and transmit the corresponding feature to each lower-level side branch. Experimental results conducted on five public colorectal polyp segmentation datasets are given and discussed. Our method performs better compared with several state-of-the-art polyp extraction networks, which demonstrate the effectiveness of the mechanism of pyramid pooling for colorectal polyp segmentation.

Comprehensive review of publicly available colonoscopic imaging databases for artificial intelligence research: availability, accessibility, and usability

BACKGROUND AND AIMS

Publicly available databases containing colonoscopic imaging data are valuable resources for artificial intelligence (AI) research. Currently, little is known regarding the available number and content of these databases. This review aimed to describe the availability, accessibility, and usability of publicly available colonoscopic imaging databases, focusing on polyp detection, polyp characterization, and quality of colonoscopy.

METHODS

A systematic literature search was performed in MEDLINE and Embase to identify AI studies describing publicly available colonoscopic imaging databases published after 2010. Second, a targeted search using Google's Dataset Search, Google Search, GitHub, and Figshare was done to identify databases directly. Databases were included if they contained data about polyp detection, polyp characterization, or quality of colonoscopy. To assess accessibility of databases, the following categories were defined: open access, open access with barriers, and regulated access. To assess the potential usability of the included databases, essential details of each database were extracted using a checklist derived from the Checklist for Artificial Intelligence in Medical Imaging.

RESULTS

We identified 22 databases with open access, 3 databases with open access with barriers, and 15 databases with regulated access. The 22 open access databases contained 19,463 images and 952 videos. Nineteen of these databases focused on polyp detection, localization, and/or segmentation; 6 on polyp characterization, and 3 on quality of colonoscopy. Only half of these databases have been used by other researcher to develop, train, or benchmark their AI system. Although technical details were in general well reported, important details such as polyp and patient demographics and the annotation process were under-reported in almost all databases.

CONCLUSIONS

This review provides greater insight on public availability of colonoscopic imaging databases for AI research. Incomplete reporting of important details limits the ability of researchers to assess the usability of current databases.

COLORECTAL POLYP DETECTION USING IMAGE ENHANCEMENT AND SCALED YOLOv4 ALGORITHM

Colorectal cancer (CRC) is the common cancer-related cause of death globally. It is now the third leading cause of cancer-related mortality worldwide. As the number of instances of colorectal polyps rises, it is more important than ever to identify and diagnose them early. Object detection models have recently become popular for extracting highly representative features. Colonoscopy is shown to be a useful diagnostic procedure for examining anomalies in the digestive system’s bottom half. This research presents a novel image-enhancing approach followed by a Scaled YOLOv4 Network for the early diagnosis of polyps, lowering the high risk of CRC therapy. The proposed network is trained using the CVC ClinicDB and the CVC ColonDB and the Etis Larib database are used for testing. On the CVC ColonDB database, the performance metrics are precision (95.13%), recall (74.92%), F1-score (83.19%), and F2-score (89.89%). On the ETIS Larib database, the performance metrics are precision (94.30%), recall (77.30%), F1-score (84.90%), and F2-score (80.20%). On both the databases, the suggested methodology outperforms the present one in terms of F1-score, F2-score, and precision compared to the futuristic method. The proposed Yolo object identification model provides an accurate polyp detection strategy in a real-time application.

UPolySeg: A U-Net-Based Polyp Segmentation Network Using Colonoscopy Images

Colonoscopy is a gold standard procedure for tracking the lower gastrointestinal region. A colorectal polyp is one such condition that is detected through colonoscopy. Even though technical advancements have improved the early detection of colorectal polyps, there is still a high percentage of misses due to various factors. Polyp segmentation can play a significant role in the detection of polyps at the early stage and can thus help reduce the severity of the disease. In this work, the authors implemented several image pre-processing techniques such as coherence transport and contrast limited adaptive histogram equalization (CLAHE) to handle different challenges in colonoscopy images. The processed image was then segmented into a polyp and normal pixel using a U-Net-based deep learning segmentation model named UPolySeg. The main framework of UPolySeg has an encoder–decoder section with feature concatenation in the same layer as the encoder–decoder along with the use of dilated convolution. The model was experimentally verified using the publicly available Kvasir-SEG dataset, which gives a global accuracy of 96.77%, a dice coefficient of 96.86%, an IoU of 87.91%, a recall of 95.57%, and a precision of 92.29%. The new framework for the polyp segmentation implementing UPolySeg improved the performance by 1.93% compared with prior work.

AUTOMATED POLYP DETECTION IN COLONOSCOPY VIDEOS USING IMAGE ENHANCEMENT AND SALIENCY DETECTION ALGORITHM

Colonoscopy has proven to be an active diagnostic tool that examines the lower half of the digestive system’s anomalies. This paper confers a Computer-Aided Detection (CAD) method for polyps from colonoscopy images that helps to diagnose the early stage of Colorectal Cancer (CRC). The proposed method consists primarily of image enhancement, followed by the creation of a saliency map, feature extraction using the Histogram of Oriented-Gradients (HOG) feature extractor, and classification using the Support Vector Machine (SVM). We present an efficient image enhancement algorithm for highlighting clinically significant features in colonoscopy images. The proposed enhancement approach can improve the overall contrast and brightness by minimizing the effects of inconsistent illumination conditions. Detailed experiments have been conducted using the publicly available colonoscopy databases CVC ClinicDB, CVC ColonDB and the ETIS Larib. The performance measures are found to be in terms of precision (91.69%), recall (81.53%), F1-score (86.31%) and F2-score (89.45%) for the CVC ColonDB database and precision (90.29%), recall (61.73%), F1-score (73.32%) and F2-score (82.64%) for the ETIS Larib database. Comparison with the futuristic method shows that the proposed approach surpasses the existing one in terms of precision, F1-score, and F2-score. The proposed enhancement with saliency-based selection significantly reduced the number of search windows, resulting in an efficient polyp detection algorithm.

Visual saliency detection approach for long-term ECG analysis

BACKGROUND AND OBJECTIVE

Detection and analysis of QRS-complex as well as the processing of electrocardiogram (ECG) signal using computers are being practiced for over the last fifty-eight years, approximately, and yet the thirst of designing superior ECG processing and recognition algorithms still captures researchers' attention around the globe. A saliency detection-based technique for the processing of one-dimensional biomedical signals such as ECG is proposed here for the first time, to the best or our knowledge.

METHODS AND RESULTS

In this proposed research work, first, a trigonometric threshold-based technique is used to identify the QRS-complexes from the ECG signal. Motion-artifact (MA) and sudden-change-in-baseline (SCB) types of noises are considered to be the toughest among others to filter out from the ECG signals as the bandwidths of these two types of noises overlap with that of the ECG. Only one feature is extracted from each of the QRS-complex-intervals, and the normalised values of this feature are arranged in the form of a gray-scale image. Then, a saliency detection-based technique is applied iteratively on the gray-scale image to detect those regions of the ECG signals, which are highly corrupted with MA and (or) SCB noises. Next, three unique geometric-features are extracted from the rest of the QRS-complexes, which are not corrupted with MA or SCB noises, and the normalised values of these three features are arranged in the form of an Red-Green-Blue (RGB) image. Again, the saliency detection-based technique is applied to identify the abnormal QRS-complexes from the RGB image.

CONCLUSIONS

The technique is tested on long-term ECG signals; totaling a duration of 17.54 days, and its performance is evaluated through both quantitative and qualitative measures. The applicability, scope of implement in real-time scenarios, advantage of the proposed technique over the existing ones are discussed with a group of clinicians and cardiologists, and very affirmative and encouraging responses are received from them.

Automatic Acetowhite Lesion Segmentation via Specular Reflection Removal and Deep Attention Network

Automatic acetowhite lesion segmentation in colposcopy images (cervigrams) is essential in assisting gynecologists for the diagnosis of cervical intraepithelial neoplasia grades and cervical cancer. It can also help gynecologists determine the correct lesion areas for further pathological examination. Existing computer-aided diagnosis algorithms show poor segmentation performance because of specular reflections, insufficient training data and the inability to focus on semantically meaningful lesion parts. In this paper, a novel computer-aided diagnosis algorithm is proposed to segment acetowhite lesions in cervigrams automatically. To reduce the interference of specularities on segmentation performance, a specular reflection removal mechanism is presented to detect and inpaint these areas with precision. Moreover, we design a cervigram image classification network to classify pathology results and generate lesion attention maps, which are subsequently leveraged to guide a more accurate lesion segmentation task by the proposed lesion-aware convolutional neural network. We conducted comprehensive experiments to evaluate the proposed approaches on 3045 clinical cervigrams. Our results show that our method outperforms state-of-the-art approaches and achieves better Dice similarity coefficient and Hausdorff Distance values in acetowhite legion segmentation.

Learn to Threshold: ThresholdNet With Confidence-Guided Manifold Mixup for Polyp Segmentation

The automatic segmentation of polyp in endoscopy images is crucial for early diagnosis and cure of colorectal cancer. Existing deep learning-based methods for polyp segmentation, however, are inadequate due to the limited annotated dataset and the class imbalance problems. Moreover, these methods obtained the final polyp segmentation results by simply thresholding the likelihood maps at an eclectic and equivalent value (often set to 0.5). In this paper, we propose a novel ThresholdNet with a confidence-guided manifold mixup (CGMMix) data augmentation method, mainly for addressing the aforementioned issues in polyp segmentation. The CGMMix conducts manifold mixup at the image and feature levels, and adaptively lures the decision boundary away from the under-represented polyp class with the confidence guidance to alleviate the limited training dataset and the class imbalance problems. Two consistency regularizations, mixup feature map consistency (MFMC) loss and mixup confidence map consistency (MCMC) loss, are devised to exploit the consistent constraints in the training of the augmented mixup data. We then propose a two-branch approach, termed ThresholdNet, to collaborate the segmentation and threshold learning in an alternative training strategy. The threshold map supervision generator (TMSG) is embedded to provide supervision for the threshold map, thereby inducing better optimization of the threshold branch. As a consequence, ThresholdNet is able to calibrate the segmentation result with the learned threshold map. We illustrate the effectiveness of the proposed method on two polyp segmentation datasets, and our methods achieved the state-of-the-art result with 87.307% and 87.879% dice score on the EndoScene dataset and the WCE polyp dataset. The source code is available at https://github.com/Guo-Xiaoqing/ThresholdNet.

PolypSegNet: A modified encoder-decoder architecture for automated polyp segmentation from colonoscopy images

Colorectal cancer has become one of the major causes of death throughout the world. Early detection of Polyp, an early symptom of colorectal cancer, can increase the survival rate to 90%. Segmentation of Polyp regions from colonoscopy images can facilitate the faster diagnosis. Due to varying sizes, shapes, and textures of polyps with subtle visible differences with the background, automated segmentation of polyps still poses a major challenge towards traditional diagnostic methods. Conventional Unet architecture and some of its variants have gained much popularity for its automated segmentation though having several architectural limitations that result in sub-optimal performance. In this paper, an encoder-decoder based modified deep neural network architecture is proposed, named as PolypSegNet, to overcome several limitations of traditional architectures for very precise automated segmentation of polyp regions from colonoscopy images. For achieving more generalized representation at each scale of both the encoder and decoder module, several sequential depth dilated inception (DDI) blocks are integrated into each unit layer for aggregating features from different receptive areas utilizing depthwise dilated convolutions. Different scales of contextual information from all encoder unit layers pass through the proposed deep fusion skip module (DFSM) to generate skip interconnection with each decoder layer rather than separately connecting different levels of encoder and decoder. For more efficient reconstruction in the decoder module, multi-scale decoded feature maps generated at various levels of the decoder are jointly optimized in the proposed deep reconstruction module (DRM) instead of only considering the decoded feature map from final decoder layer. Extensive experimentations on four publicly available databases provide very satisfactory performance with mean five-fold cross-validation dice scores of 91.52% in CVC-ClinicDB database, 92.8% in CVC-ColonDB database, 88.72% in Kvasir-SEG database, and 84.79% in ETIS-Larib database. The proposed network provides very accurate segmented polyp regions that will expedite the diagnosis of polyps even in challenging conditions.

An Extensive Study on Cross-Dataset Bias and Evaluation Metrics Interpretation for Machine Learning Applied to Gastrointestinal Tract Abnormality Classification

Precise and efficient automated identification of gastrointestinal (GI) tract diseases can help doctors treat more patients and improve the rate of disease detection and identification. Currently, automatic analysis of diseases in the GI tract is a hot topic in both computer science and medical-related journals. Nevertheless, the evaluation of such an automatic analysis is often incomplete or simply wrong. Algorithms are often only tested on small and biased datasets, and cross-dataset evaluations are rarely performed. A clear understanding of evaluation metrics and machine learning models with cross datasets is crucial to bring research in the field to a new quality level. Toward this goal, we present comprehensive evaluations of five distinct machine learning models using global features and deep neural networks that can classify 16 different key types of GI tract conditions, including pathological findings, anatomical landmarks, polyp removal conditions, and normal findings from images captured by common GI tract examination instruments. In our evaluation, we introduce performance hexagons using six performance metrics, such as recall, precision, specificity, accuracy, F1-score, and the Matthews correlation coefficient to demonstrate how to determine the real capabilities of models rather than evaluating them shallowly. Furthermore, we perform cross-dataset evaluations using different datasets for training and testing. With these cross-dataset evaluations, we demonstrate the challenge of actually building a generalizable model that could be used across different hospitals. Our experiments clearly show that more sophisticated performance metrics and evaluation methods need to be applied to get reliable models rather than depending on evaluations of the splits of the same dataset—that is, the performance metrics should always be interpreted together rather than relying on a single metric.

Semi-supervised WCE image classification with adaptive aggregated attention

Accurate abnormality classification in Wireless Capsule Endoscopy (WCE) images is crucial for early gastrointestinal (GI) tract cancer diagnosis and treatment, while it remains challenging due to the limited annotated dataset, the huge intra-class variances and the high degree of inter-class similarities. To tackle these dilemmas, we propose a novel semi-supervised learning method with Adaptive Aggregated Attention (AAA) module for automatic WCE image classification. Firstly, a novel deformation field based image preprocessing strategy is proposed to remove the black background and circular boundaries in WCE images. Then we propose a synergic network to learn discriminative image features, consisting of two branches: an abnormal regions estimator (the first branch) and an abnormal information distiller (the second branch). The first branch utilizes the proposed AAA module to capture global dependencies and incorporate context information to highlight the most meaningful regions, while the second branch mainly focuses on these calculated attention regions for accurate and robust abnormality classification. Finally, these two branches are jointly optimized by minimizing the proposed discriminative angular (DA) loss and Jensen-Shannon divergence (JS) loss with labeled data as well as unlabeled data. Comprehensive experiments have been conducted on the public CAD-CAP WCE dataset. The proposed method achieves 93.17% overall accuracy in a fourfold cross-validation, verifying its effectiveness for WCE image classification. The source code is available at https://github.com/Guo-Xiaoqing/SSL_WCE.

Optimized Rate Control Algorithm of High-Efficiency Video Coding Based on Region of Interest

Aiming at the problems that the strategy of target bit allocation at the CTU layer has deviations from the human subjective observation mechanism, and the update phase of parametric model has a higher complexity in the JCTVC-K0103 rate control algorithm of ITU-T H.265/high efficiency video coding (HEVC) standard. Optimized rate control (ORC) algorithm of ITU-T H.265/HEVC based on region of interest (ROI) is proposed. Firstly, the algorithm extracts the region of interest of video frames based on time and space domains by using the improved Itti model. Then, the weight of target bits w is recalculated based on space-time domains to improve the rate control accuracy, and the target bits are distributed based on ROI by the adaptive weight algorithm once again to make the output videos more attuned with the human visual attention mechanism. Finally, the quasi-Newton algorithm is used to update the rate distortion model, which reduces the computational complexity in the update phase of the parametric model. The experimental results show that the ORC algorithm can obtain a better subjective quality in the compressed results with less bit error compared with the other two algorithms. Meanwhile, the rate distortion performance of the ORC algorithm is better on the premise of guaranteeing rate control performance.

Polyp Segmentation using Generative Adversarial Network

Colorectal cancer is one of the highest causes of cancer-related death and the patient's survival rate depends on the stage at which polyps are detected. Polyp segmentation is a challenging research task due to variations in the size and shape of polyps leading to necessitate robust approaches for diagnosis. This paper studies the deep generative convolutional framework for the task of polyp segmentation. Here, the analysis of polyp segmentation has been explored with the pix2pix conditional generative adversarial network. On CVC- Clinic dataset, the proposed network achieves Jaccard index of 81.27% and Dice index of 88.48%.

Machine learning for computer-aided polyp detection using wavelets and content-based image

The continuous growing of machine learning techniques, their capabilities improvements and the availability of data being continuously collected, recorded and updated, can enhance diagnosis stages by making it faster and more accurate than human diagnosis. In lower endoscopies procedures, most of the diagnosis relies on the capabilities and expertise of the physician. During medical training, physicians can be benefited from the assistance of algorithms able to automatically detect polyps, thus enhancing their diagnosis. In this paper, we propose a machine learning approach trained to detect polyps in lower endoscopies recordings with high accuracy and sensitivity, previously processed using wavelet transform for feature extraction. The propose system is validated using available datasets. From a set of 1132 images, our system showed a 97.9% of accuracy in diagnosing polyps, around 10% more efficient than other approaches using techniques with a low computational requirement previously published. In addition, the false positive rate was 0.03. This encouraging result can be also extended to other diagnosis.