A comprehensive survey of image segmentation: clustering methods, performance parameters, and benchmark datasets

Histogram matching-enhanced adversarial learning for unsupervised domain adaptation in medical image segmentation

BACKGROUND

Unsupervised domain adaptation (UDA) seeks to mitigate the performance degradation of deep neural networks when applied to new, unlabeled domains by leveraging knowledge from source domains. In medical image segmentation, prevailing UDA techniques often utilize adversarial learning to address domain shifts for cross-modality adaptation. Current research on adversarial learning tends to adopt increasingly complex models and loss functions, making the training process highly intricate and less stable/robust. Furthermore, most methods primarily focused on segmentation accuracy while neglecting the associated confidence levels and uncertainties.

PURPOSE

To develop a simple yet effective UDA method based on histogram matching-enhanced adversarial learning (HMeAL-UDA), and provide comprehensive uncertainty estimations of the model predictions.

METHODS

Aiming to bridge the domain gap while reducing the model complexity, we developed a novel adversarial learning approach to align multi-modality features. The method, termed HMeAL-UDA, integrates a plug-and-play histogram matching strategy to mitigate domain-specific image style biases across modalities. We employed adversarial learning to constrain the model in the prediction space, enabling it to focus on domain-invariant features during segmentation. Moreover, we quantified the model's prediction confidence using Monte Carlo (MC) dropouts to assess two voxel-level uncertainty estimates of the segmentation results, which were subsequently aggregated into a volume-level uncertainty score, providing an overall measure of the model's reliability. The proposed method was evaluated on three public datasets (Combined Healthy Abdominal Organ Segmentation [CHAOS], Beyond the Cranial Vault [BTCV], and Abdominal Multi-Organ Segmentation Challenge [AMOS]) and one in-house clinical dataset (UTSW). We used 30 MRI scans (20 from the CHAOS dataset and 10 from the in-house dataset) and 30 CT scans from the BTCV dataset for UDA-based, cross-modality liver segmentation. Additionally, 240 CT scans and 60 MRI scans from the AMOS dataset were utilized for cross-modality multi-organ segmentation. The training and testing sets for each modality were split with ratios of approximately 4:1-3:1.

RESULTS

Extensive experiments on cross-modality medical image segmentation demonstrated the superiority of HMeAL-UDA over two state-of-the-art approaches. HMeAL-UDA achieved a mean (± s.d.) Dice similarity coefficient (DSC) of 91.34% ± 1.23% and an HD95 of 6.18 ± 2.93 mm for cross-modality (from CT to MRI) adaptation of abdominal multi-organ segmentation, and a DSC of 87.13% ± 3.67% with an HD95 of 2.48 ± 1.56 mm for segmentation adaptation in the opposite direction (MRI to CT). The results are approaching or even outperforming those of supervised methods trained with "ground-truth" labels in the target domain. In addition, we provide a comprehensive assessment of the model's uncertainty, which can help with the understanding of segmentation reliability to guide clinical decisions.

CONCLUSION

HMeAL-UDA provides a powerful segmentation tool to address cross-modality domain shifts, with the potential to generalize to other deep learning applications in medical imaging.

Automatic computed tomography image segmentation method for liver tumor based on a modified tokenized multilayer perceptron and attention mechanism

Background

The automatic medical image segmentation of liver and tumor plays a pivotal role in the clinical diagnosis of liver diseases. A number of effective methods based on deep neural networks, including convolutional neural networks (CNNs) and vision transformer (ViT) have been developed. However, these networks primarily focus on enhancing segmentation accuracy while often overlooking the segmentation speed, which is vital for rapid diagnosis in clinical settings. Therefore, we aimed to develop an automatic computed tomography (CT) image segmentation method for liver tumors that reduces inference time while maintaining accuracy, as rigorously validated through experimental studies.

Methods

We developed a U-shaped network enhanced by a multiscale attention module and attention gates, aimed at efficient CT image segmentation of liver tumors. In this network, a modified tokenized multilayer perceptron (MLP) block is first leveraged to reduce the feature dimensions and facilitate information interaction between adjacent patches so that the network can learn the key features of tumors with less computational complexity. Second, attention gates are added into the skip connections between the encoder and decoder, emphasizing feature expression in relevant regions and enabling the network to focus more on liver tumor features. Finally, a multiscale attention mechanism autonomously adjusts weights for each scale, allowing the network to adapt effectively to varying sizes of liver tumors. Our methodology was validated via the Liver Tumor Segmentation 2017 (LiTS17) public dataset. The data from this database are from seven global clinical sites. All data are anonymized, and the images have been prescreened to ensure the absence of personal identifiers. Standard metrics were used to evaluate the performance of the model.

Results

The 21 cases were included for testing. The proposed network attained a Dice score of 0.713 [95% confidence interval (CI): 0.592-0.834], a volumetric overlap error of 0.39 (95% CI: 0.17-0.61), a relative volume difference score of 0.19 (95% CI: -0.37 to 0.31), an average symmetric surface distance of 2.04 mm (95% CI: 0.89-4.19), a maximum surface distance of 9.42 mm (95% CI: 6.97-19.87), and an inference time of 26 ms on average for liver tumor segmentation.

Conclusions

The proposed network demonstrated efficient liver tumor segmentation performance with less inference time. Our findings contribute to the application of neural networks in rapid clinical diagnosis and treatment.

Understanding the Impact of Evaluation Metrics in Kinetic Models for Consensus-Based Segmentation

In this article, we extend a recently introduced kinetic model for consensus-based segmentation of images. In particular, we will interpret the set of pixels of a 2D image as an interacting particle system that evolves in time in view of a consensus-type process obtained by interactions between pixels and external noise. Thanks to a kinetic formulation of the introduced model, we derive the large time solution of the model. We will show that the parameters defining the segmentation task can be chosen from a plurality of loss functions that characterize the evaluation metrics.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Medical image segmentation plays a critical role in accurate diagnosis and treatment planning, enabling precise analysis across a wide range of clinical tasks. This review begins by offering a comprehensive overview of traditional segmentation techniques, including thresholding, edge-based methods, region-based approaches, clustering, and graph-based segmentation. While these methods are computationally efficient and interpretable, they often face significant challenges when applied to complex, noisy, or variable medical images. The central focus of this review is the transformative impact of deep learning on medical image segmentation. We delve into prominent deep learning architectures such as Convolutional Neural Networks (CNNs), Fully Convolutional Networks (FCNs), U-Net, Recurrent Neural Networks (RNNs), Adversarial Networks (GANs), and Autoencoders (AEs). Each architecture is analyzed in terms of its structural foundation and specific application to medical image segmentation, illustrating how these models have enhanced segmentation accuracy across various clinical contexts. Finally, the review examines the integration of deep learning with traditional segmentation methods, addressing the limitations of both approaches. These hybrid strategies offer improved segmentation performance, particularly in challenging scenarios involving weak edges, noise, or inconsistent intensities. By synthesizing recent advancements, this review provides a detailed resource for researchers and practitioners, offering valuable insights into the current landscape and future directions of medical image segmentation.

Brain Tumor Detection and Categorization with Segmentation of Improved Unsupervised Clustering Approach and Machine Learning Classifier

There is no doubt that brain tumors are one of the leading causes of death in the world. A biopsy is considered the most important procedure in cancer diagnosis, but it comes with drawbacks, including low sensitivity, risks during biopsy treatment, and a lengthy wait for results. Early identification provides patients with a better prognosis and reduces treatment costs. The conventional methods of identifying brain tumors are based on medical professional skills, so there is a possibility of human error. The labor-intensive nature of traditional approaches makes healthcare resources expensive. A variety of imaging methods are available to detect brain tumors, including magnetic resonance imaging (MRI) and computed tomography (CT). Medical imaging research is being advanced by computer-aided diagnostic processes that enable visualization. Using clustering, automatic tumor segmentation leads to accurate tumor detection that reduces risk and helps with effective treatment. This study proposed a better Fuzzy C-Means segmentation algorithm for MRI images. To reduce complexity, the most relevant shape, texture, and color features are selected. The improved Extreme Learning machine classifies the tumors with 98.56% accuracy, 99.14% precision, and 99.25% recall. The proposed classifier consistently demonstrates higher accuracy across all tumor classes compared to existing models. Specifically, the proposed model exhibits accuracy improvements ranging from 1.21% to 6.23% when compared to other models. This consistent enhancement in accuracy emphasizes the robust performance of the proposed classifier, suggesting its potential for more accurate and reliable brain tumor classification. The improved algorithm achieved accuracy, precision, and recall rates of 98.47%, 98.59%, and 98.74% on the Fig share dataset and 99.42%, 99.75%, and 99.28% on the Kaggle dataset, respectively, which surpasses competing algorithms, particularly in detecting glioma grades. The proposed algorithm shows an improvement in accuracy, of approximately 5.39%, in the Fig share dataset and of 6.22% in the Kaggle dataset when compared to existing models. Despite challenges, including artifacts and computational complexity, the study's commitment to refining the technique and addressing limitations positions the improved FCM model as a noteworthy advancement in the realm of precise and efficient brain tumor identification.

Deep learning-based algorithm for postoperative glioblastoma MRI segmentation: a promising new tool for tumor burden assessment

OBJECTIVE

Clinical and surgical decisions for glioblastoma patients depend on a tumor imaging-based evaluation. Artificial Intelligence (AI) can be applied to magnetic resonance imaging (MRI) assessment to support clinical practice, surgery planning and prognostic predictions. In a real-world context, the current obstacles for AI are low-quality imaging and postoperative reliability. The aim of this study is to train an automatic algorithm for glioblastoma segmentation on a clinical MRI dataset and to obtain reliable results both pre- and post-operatively.

METHODS

The dataset used for this study comprises 237 (71 preoperative and 166 postoperative) MRIs from 71 patients affected by a histologically confirmed Grade IV Glioma. The implemented U-Net architecture was trained by transfer learning to perform the segmentation task on postoperative MRIs. The training was carried out first on BraTS2021 dataset for preoperative segmentation. Performance is evaluated using DICE score (DS) and Hausdorff 95% (H95).

RESULTS

In preoperative scenario, overall DS is 91.09 (± 0.60) and H95 is 8.35 (± 1.12), considering tumor core, enhancing tumor and whole tumor (ET and edema). In postoperative context, overall DS is 72.31 (± 2.88) and H95 is 23.43 (± 7.24), considering resection cavity (RC), gross tumor volume (GTV) and whole tumor (WT). Remarkably, the RC segmentation obtained a mean DS of 63.52 (± 8.90) in postoperative MRIs.

CONCLUSIONS

The performances achieved by the algorithm are consistent with previous literature for both pre-operative and post-operative glioblastoma's MRI evaluation. Through the proposed algorithm, it is possible to reduce the impact of low-quality images and missing sequences.

A Two-Stage Automatic Color Thresholding Technique

Thresholding is a prerequisite for many computer vision algorithms. By suppressing the background in an image, one can remove unnecessary information and shift one's focus to the object of inspection. We propose a two-stage histogram-based background suppression technique based on the chromaticity of the image pixels. The method is unsupervised, fully automated, and does not need any training or ground-truth data. The performance of the proposed method was evaluated using a printed circuit assembly (PCA) board dataset and the University of Waterloo skin cancer dataset. Accurately performing background suppression in PCA boards facilitates the inspection of digital images with small objects of interest, such as text or microcontrollers on a PCA board. The segmentation of skin cancer lesions will help doctors to automate skin cancer detection. The results showed a clear and robust background-foreground separation across various sample images under different camera or lighting conditions, which the naked implementation of existing state-of-the-art thresholding methods could not achieve.

A survey on the utilization of Superpixel image for clustering based image segmentation

Superpixel become increasingly popular in image segmentation field as it greatly helps image segmentation techniques to segment the region of interest accurately in noisy environment and also reduces the computation effort to a great extent. However, selection of proper superpixel generation techniques and superpixel image segmentation techniques play a very crucial role in the domain of different kinds of image segmentation. Clustering is a well-accepted image segmentation technique and proved their effective performance over various image segmentation field. Therefore, this study presents an up-to-date survey on the employment of superpixel image in combined with clustering techniques for the various image segmentation. The contribution of the survey has four parts namely (i) overview of superpixel image generation techniques, (ii) clustering techniques especially efficient partitional clustering techniques, their issues and overcoming strategies, (iii) Review of superpixel combined with clustering strategies exist in literature for various image segmentation, (iv) lastly, the comparative study among superpixel combined with partitional clustering techniques has been performed over oral pathology and leaf images to find out the efficacy of the combination of superpixel and partitional clustering approaches. Our evaluations and observation provide in-depth understanding of several superpixel generation strategies and how they apply to the partitional clustering method.

A fully automatic method for segmentation of soccer playing fields

This paper proposes a strategy to segment the playing field in soccer images, suitable for integration in many soccer image analysis applications. The combination of a green chromaticity-based analysis and an analysis of the chromatic distortion using full-color information, both at the pixel-level, allows segmenting the green areas of the images. Then, a fully automatic post-processing block at the region-level discards the green areas that do not belong to the playing field. The strategy has been evaluated with hundreds of annotated images from matches in several stadiums with different grass shades and light conditions. The results obtained have been of great quality in all the images, even in those with the most complex lighting conditions (e.g., high contrast between sunlit and shadowed areas). In addition, these results have improved those obtained with leading state-of-the-art playing field segmentation strategies.

A New Suppression-based Possibilistic Fuzzy c-means Clustering Algorithm

Possibilistic fuzzy c-means (PFCM) is one of the most widely used clustering algorithm that solves the noise sensitivity problem of Fuzzy c-means (FCM) and coincident clusters problem of possibilistic c-means (PCM). Though PFCM is a highly reliable clustering algorithm but  the efficiency of the algorithm can be further improved by introducing the concept of suppression. Suppression-based algorithms employ the winner and non-winner based suppression technique on the datasets, helping in performing better classification of real-world datasets into clusters. In this paper, we propose a suppression-based possibilistic fuzzy c-means clustering algorithm (SPFCM) for the process of clustering. The paper explores the performance of the proposed methodology based on number of misclassifications for various real datasets and synthetic datasets and it is found to perform better than other clustering techniques in the sequel, i.e., normal as well as suppression-based algorithms. The SPFCM is found to perform more efficiently and converges faster as compared to other clustering techniques.

Two- and Three-Dimensional Benchmarks for Particle Detection from an Industrial Rotary Kiln Combustion Chamber Based on Light-Field-Camera Recording

This paper describes a benchmark dataset for the detection of fuel particles in 2D and 3D image data in a rotary kiln combustion chamber. The specific challenges of detecting the small particles under demanding environmental conditions allows for the performance of existing and new particle detection techniques to be evaluated. The data set includes a classification of burning and non-burning particles, which can be in the air but also on the rotary kiln wall. The light-field camera used for data generation offers the potential to develop and objectively evaluate new advanced particle detection methods due to the additional 3D information. Besides explanations of the data set and the contained ground truth, an evaluation procedure of the particle detection based on the ground truth and results for an own particle detection procedure for the data set are presented.

Multi Level Approach for Segmentation of Interstitial Lung Disease (ILD) Patterns Classification Based on Superpixel Processing and Fusion of K-Means Clusters: SPFKMC

During the COVID-19 pandemic, huge interstitial lung disease (ILD) lung images have been captured. It is high time to develop the efficient segmentation techniques utilized to separate the anatomical structures and ILD patterns for disease and infection level identification. The effectiveness of disease classification directly depends on the accuracy of initial stages like preprocessing and segmentation. This paper proposed a hybrid segmentation algorithm designed for ILD images by taking advantage of superpixel and K-means clustering approaches. Segmented superpixel images adapt the better irregular local and spatial neighborhoods that are helpful to improving the performance of K-means clustering-based ILD image segmentation. To overcome the limitations of multiclass belongings, semiadaptive wavelet-based fusion is applied over selected K-means clusters. The performance of the proposed SPFKMC was compared with that of 3-class Fuzzy C-Means clustering (FCM) and K-Means clustering in terms of accuracy, Jaccard similarity index, and Dice similarity coefficient. The SPFKMC algorithm gives an accuracy of 99.28%, DSC 98.72%, and JSI 97.87%. The proposed Fused Clustering gives better results as compared to traditional K-means clustering segmentation with wavelet-based fused cluster results.

A new firefly algorithm-based superpixel clustering method for vehicle segmentation

The vehicle segmentation in the images of a crowded and unstructured road traffic, having inconsistent driving patterns and vivid attributes like colour, shapes, and size, is a complex task. For the same, this paper presents a new firefly algorithm-based superpixel clustering method for vehicle segmentation. The proposed method introduces a modified firefly algorithm by incorporating the best solution for enhancing the exploitation behaviour and solution precision. The modified firefly algorithm is further used to obtain the optimal superpixel clusters. The modified firefly algorithm is compared against state-of-the-art meta-heuristic algorithms on IEEE CEC 2015 benchmark problems in terms of mean fitness value, Wilcoxon rank-sum test, convergence behaviour, and box plot. The proposed meta-heuristic algorithm performed superior on more than 80% of the considered benchmark problems. Moreover, the modified firefly algorithm is statistically better on more than 92% of the total problems during Wilcoxon test. Further, the proposed segmentation method is analysed on a traffic dataset to segment the auto-rickshaw. The performance of the proposed method has been compared with kmeans-based superpixel clustering method. The proposed method shows the highest mean value of 0.6242 for Dice coefficient. Both qualitative and quantitative results affirm the efficacy of the proposed method.

Graphical Image Region Extraction with K-Means Clustering and Watershed

With a wide range of applications, image segmentation is a complex and difficult preprocessing step that plays an important role in automatic visual systems, which accuracy impacts, not only on segmentation results, but directly affects the effectiveness of the follow-up tasks. Despite the many advances achieved in the last decades, image segmentation remains a challenging problem, particularly, the segmenting of color images due to the diverse inhomogeneities of color, textures and shapes present in the descriptive features of the images. In trademark graphic images segmentation, beyond these difficulties, we must also take into account the high noise and low resolution, which are often present. Trademark graphic images can also be very heterogeneous with regard to the elements that make them up, which can be overlapping and with varying lighting conditions. Due to the immense variation encountered in corporate logos and trademark graphic images, it is often difficult to select a single method for extracting relevant image regions in a way that produces satisfactory results. Many of the hybrid approaches that integrate the Watershed and K-Means algorithms involve processing very high quality and visually similar images, such as medical images, meaning that either approach can be tweaked to work on images that follow a certain pattern. Trademark images are totally different from each other and are usually fully colored. Our system solves this difficulty given it is a generalized implementation designed to work in most scenarios, through the use of customizable parameters and completely unbiased for an image type. In this paper, we propose a hybrid approach to Image Region Extraction that focuses on automated region proposal and segmentation techniques. In particular, we analyze popular techniques such as K-Means Clustering and Watershedding and their effectiveness when deployed in a hybrid environment to be applied to a highly variable dataset. The proposed system consists of a multi-stage algorithm that takes as input an RGB image and produces multiple outputs, corresponding to the extracted regions. After preprocessing steps, a K-Means function with random initial centroids and a user-defined value for k is executed over the RGB image, generating a gray-scale segmented image, to which a threshold method is applied to generate a binary mask, containing the necessary information to generate a distance map. Then, the Watershed function is performed over the distance map, using the markers defined by the Connected Component Analysis function that labels regions on 8-way pixel connectivity, ensuring that all regions are correctly found. Finally, individual objects are labelled for extraction through a contour method, based on border following. The achieved results show adequate region extraction capabilities when processing graphical images from different datasets, where the system correctly distinguishes the most relevant visual elements of images with minimal tweaking.

Application of Improved Satin Bowerbird Optimizer in Image Segmentation

Aiming at the problems of low optimization accuracy and slow convergence speed of Satin Bowerbird Optimizer (SBO), an improved Satin Bowerbird Optimizer (ISBO) based on chaotic initialization and Cauchy mutation strategy is proposed. In order to improve the value of the proposed algorithm in engineering and practical applications, we apply it to the segmentation of medical and plant images. To improve the optimization accuracy, convergence speed and pertinence of the initial population, the population is initialized by introducing the Logistic chaotic map. To avoid the algorithm falling into local optimum (prematurity), the search performance of the algorithm is improved through Cauchy mutation strategy. Based on extensive visual and quantitative data analysis, this paper conducts a comparative analysis of the ISBO with the SBO, the fuzzy Gray Wolf Optimizer (FGWO), and the Fuzzy Coyote Optimization Algorithm (FCOA). The results show that the ISBO achieves better segmentation effects in both medical and plant disease images.

Optimal keyframe selection-based lossless video-watermarking technique using IGSA in LWT domain for copyright protection

Video piracy is a challenging issue in the modern world. Approximately $$90\%$$ 90 % of newly released films were illegally distributed around the world via the Internet. To overcome this issue, video watermarking is an effective process that integrates a logo in video frames as a watermark. Therefore, this paper presents an efficient lossless video-watermarking scheme based on optimal keyframe selection using an intelligent gravitational search algorithm in linear wavelet transform. This technique obtains color motion and motionless frames from the cover video by the histogram difference method. One-level linear wavelet transform is performed on the chrominance channel of motion frames and a low-frequency sub-band LL opts for watermark embedding. The performance of the proposed technique has been evaluated against 12 video processing attacks in terms of imperceptibility and robustness. Experiments demonstrate that the proposed technique outperforms five state-of-the-art schemes on the considered attacks.

Exploratory Analysis on Pixelwise Image Segmentation Metrics with an Application in Proximal Sensing

A considerable number of metrics can be used to evaluate the performance of machine learning algorithms. While much work is dedicated to the study and improvement of data quality and models’ performance, much less research is focused on the study of these evaluation metrics, their intrinsic relationship, the interplay of the influence among the metrics, the models, the data, and the environments and conditions in which they are to be applied. While some works have been conducted on general machine learning tasks such as classification, fewer efforts have been dedicated to more complex problems such as object detection and image segmentation, in which the evaluation of performance can vary drastically depending on the objectives and domains of application. Working in an agricultural context, specifically on the problem of the automatic detection of plants in proximal sensing images, we studied twelve evaluation metrics that we used to evaluate three image segmentation models recently presented in the literature. After a unified presentation of these metrics, we carried out an exploratory analysis of their relationships using a correlation analysis, a clustering of variables, and two factorial analyses (namely principal component analysis and multiple factorial analysis). We distinguished three groups of highly linked metrics and, through visual inspection of the representative images of each group, identified the aspects of segmentation that each group evaluates. The aim of this exploratory analysis was to provide some clues to practitioners for understanding and choosing the metrics that are most relevant to their agricultural task.