Recent advances on image edge detection: A comprehensive review

Zero-shot incremental learning using spatial-frequency feature representations

Zero-shot incremental learning aims to enable a model to generalize to new classes without forgetting previously learned classes. However, the semantic gap between old and new sample classes can lead to catastrophic forgetting. Additionally, existing algorithms lack the ability to capture significant information from each sample image domain. Therefore, this paper proposes a novel spatial-frequency feature representation network (SFFRNet) that contains a spatial feature extraction (SFE) module and a frequency feature extraction (FFE) module to improve the zero-shot translation for the class incremental learning algorithm. The proposed SFFRNet has the ability to effectively extract spatial-frequency feature representation from input images, improve the accuracy of image classification, and fundamentally alleviate catastrophic forgetting. Extensive experiments on the CUB 200-2011 and CIFAR-100 datasets demonstrate that our proposed algorithm outperforms state-of-the-art incremental learning algorithms.

A Multiscale Gradient Fusion Method for Color Image Edge Detection Using CBM3D Filtering

In this paper, we present a novel color edge detection method that integrates collaborative filtering with multiscale gradient fusion. The Block-Matching and 3D (BM3D) filter is utilized to enhance sparse representations in the transform domain, effectively reducing noise. The multiscale gradient fusion technique compensates for the loss of detail in single-scale edge detection, thereby improving both edge resolution and overall quality. RGB images from the dataset are converted into the XYZ color space through mathematical transformations. The Colored Block-Matching and 3D (CBM3D) filter is applied to the sparse images to reduce noise. Next, the vector gradients of the color image and anisotropic Gaussian directional derivatives for two scale parameters are computed. These are then averaged pixel-by-pixel to generate a refined edge strength map. To enhance the edge features, the image undergoes normalization and non-maximum suppression. This is followed by edge contour extraction using double-thresholding and a novel morphological refinement technique. Experimental results on the edge detection dataset demonstrate that the proposed method offers robust noise resistance and superior edge quality, outperforming traditional methods such as Color Sobel, Color Canny, SE, and Color AGDD, as evidenced by performance metrics including the PR curve, AUC, PSNR, MSE, and FOM.

Quantitative Analysis of Perovskite Morphologies Employing Deep Learning Framework Enables Accurate Solar Cell Performance Prediction

In perovskite solar cells, grain boundaries are considered one of the major structural defect sites, and consequently affect solar cell performance. Therefore, a precise edge detection of perovskite grains may enable to predict resulting solar cell performance. Herein, a deep learning model, Self-UNet, is developed to extract and quantify morphological information such as grain boundary length (GBL), the number of grains (NG), and average grain surface area (AGSA) from scanning elecron microscope (SEM) images. The Self-UNet excels conventional Canny and UNet models in edge extraction; the Dice coefficient and F1-score exhibit as high as 91.22% and 93.58%, respectively. The high edge detection accuracy of Self-UNet allows for not only identifying tiny grains stuck between relatively large grains, but also distinguishing actual grain boundaries from grooves on grain surface from low quality SEM images, avoiding under- or over-estimation of grain information. Moreover, the gradient boosted decision tree (GBDT) regression integrated to the Self-UNet exhibits high accuracy in predicting solar cell efficiency with relative errors of less than 10% compared to the experimentally measured efficiencies, which is corroborated by results from the literature and the experiments. Additionally, the GBL can be verified in multiple ways as a new morphological feature.

A Two stage deep learning network for automated femoral segmentation in bilateral lower limb CT scans

This study presents the development of a deep learning-based two-stage network designed for the efficient and precise segmentation of the femur in full lower limb CT images. The proposed network incorporates a dual-phase approach: rapid delineation of regions of interest followed by semantic segmentation of the femur. The experimental dataset comprises 100 samples obtained from a hospital, partitioned into 85 for training, 8 for validation, and 7 for testing. In the first stage, the model achieves an average Intersection over Union of 0.9671 and a mean Average Precision of 0.9656, effectively delineating the femoral region with high accuracy. During the second stage, the network attains an average Dice coefficient of 0.953, sensitivity of 0.965, specificity of 0.998, and pixel accuracy of 0.996, ensuring precise segmentation of the femur. When compared to the single-stage SegResNet architecture, the proposed two-stage model demonstrates faster convergence during training, reduced inference times, higher segmentation accuracy, and overall superior performance. Comparative evaluations against the TransUnet model further highlight the network's notable advantages in accuracy and robustness. In summary, the proposed two-stage network offers an efficient, accurate, and autonomous solution for femur segmentation in large-scale and complex medical imaging datasets. Requiring relatively modest training and computational resources, the model exhibits significant potential for scalability and clinical applicability, making it a valuable tool for advancing femoral image segmentation and supporting diagnostic workflows.

Local cues enable classification of image patches as surfaces, object boundaries, or illumination changes

To correctly parse the visual scene, one must detect edges and determine their underlying cause. Previous work has demonstrated that image-computable neural networks trained to differentiate natural shadow and occlusion edges exhibited sensitivity to boundary sharpness and texture differences. Although these models showed a strong correlation with human performance on an edge classification task, this previous study did not directly investigate whether humans actually make use of boundary sharpness and texture cues when classifying edges as shadows or occlusions. Here we directly investigated this using synthetic image patch stimuli formed by quilting together two different natural textures, allowing us to parametrically manipulate boundary sharpness, texture modulation, and luminance modulation. In a series of initial "training" experiments, observers were trained to correctly identify the cause of natural image patches taken from one of three categories (occlusion, shadow, uniform texture). In a subsequent series of "test" experiments, these same observers then classified 5 sets of synthetic boundary images defined by varying boundary sharpness, luminance modulation, and texture modulation cues using a set of novel parametric stimuli. These three visual cues exhibited strong interactions to determine categorization probabilities. For sharp edges, increasing luminance modulation made it less likely the patch would be classified as a texture and more likely it would be classified as an occlusion, whereas for blurred edges, increasing luminance modulation made it more likely the patch would be classified as a shadow. Boundary sharpness had a profound effect, so that in the presence of luminance modulation increasing sharpness decreased the likelihood of classification as a shadow and increased the likelihood of classification as an occlusion. Texture modulation had little effect on categorization, except in the case of a sharp boundary with zero luminance modulation. Results were consistent across all 5 stimulus sets, showing these effects are not due to the idiosyncrasies of the particular texture pairs. Human performance was found to be well explained by a simple linear multinomial logistic regression model defined on luminance, texture and sharpness cues, with slightly improved performance for a more complicated nonlinear model taking multiplicative parameter combinations into account. Our results demonstrate that human observers make use of the same cues as our previous machine learning models when detecting edges and determining their cause, helping us to better understand the neural and perceptual mechanisms of scene parsing.

Innovative adaptive edge detection for noisy images using wavelet and Gaussian method

Edge detection is a crucial task in image processing and remote sensing, particularly for accurately identifying and separating shapes in noisy digital images. To enhance robustness and detail in edge detection, this study presents an innovative edge detection method, which integrates a denoising module and an adaptive thresholding technique to effectively address challenges associated with Gaussian noise in images. The proposed denoising module employs wavelet and Gaussian denoising functions to decompose, filter, and reconstruct the image, thereby reducing the impact of noise and enhancing image quality. For edge detection, an adaptive thresholding method based on a modified OTSU method is utilized. Comprehensive experiments validate the proposed framework by comparing detected edges against ground truth across various levels of Gaussian noise (0.1%, 10%, 20%, and 30%). The median thresholding function is chosen for its stability and convenience, while hard thresholding is avoided due to its tendency to introduce artifacts. Objective metrics, including Mean Squared Error (MSE), Accuracy, and Peak Signal-to-Noise Ratio (PSNR), are employed for evaluation. Comparative results indicate that the proposed method outperforms traditional methods, such as Canny and Roberts, showcasing its effectiveness in edge detection.

Unmanned roller lateral positioning method for asphalt road construction

Unmanned rollers are typically equipped with satellite-based positioning systems for positional monitoring. However, satellite-based positioning systems may result in unmanned rollers driving out of the specified compaction areas during asphalt road construction, which affects the compaction quality and has potential safety hazards. Additionally, satellite-based positioning systems may encounter signal interference and cannot locate unmanned rollers. To solve this problem, a lateral positioning method for unmanned rollers is proposed to realize the positioning of unmanned rollers relative to asphalt road. First, we captured images from different perspectives and developed a dataset for asphalt road construction. Second, a method for boundary extraction of asphalt road is proposed to accurately locate pixels of asphalt road boundary. Subsequently, the lateral distances are measured by the designed lateral positioning methods. Finally, field validation experiments are conducted to evaluate the effectiveness of the proposed lateral positioning method. The results indicate that the method excels in extracting the asphalt road boundary. Furthermore, the proposed lateral positioning method shows excellent performance, with a mean relative error of 3.40% and a frequency of 6.25 Hz. The proposed lateral positioning method meets the performance requirements for lateral positioning in both accuracy and real-time in asphalt road construction for unmanned rollers.

Material Visual Perception and Discharging Robot Control for Baijiu Fermented Grains in Underground Tank

Addressing the issue of excessive manual intervention in discharging fermented grains from underground tanks in traditional brewing technology, this paper proposes an intelligent grains-out strategy based on a multi-degree-of-freedom hybrid robot. The robot's structure and control system are introduced, along with analyses of kinematics solutions for its parallel components and end-effector speeds. According to its structural characteristics and working conditions, a visual-perception-based motion control method of discharging fermented grains is determined. The enhanced perception of underground tanks' positions is achieved through improved Canny edge detection algorithms, and a YOLO-v7 neural network is employed to train an image segmentation model for fermented grains' surface, integrating depth information to synthesize point clouds. We then carry out the downsampling and three-dimensional reconstruction of these point clouds, then match the underground tank model with the fermented grain surface model to replicate the tank's interior space. Finally, a digging motion control method is proposed and experimentally validated for feasibility and operational efficiency.

An Unbiased Feature Estimation Network for Few-Shot Fine-Grained Image Classification

Few-shot fine-grained image classification (FSFGIC) aims to classify subspecies with similar appearances under conditions of very limited data. In this paper, we observe an interesting phenomenon: different types of image data augmentation techniques have varying effects on the performance of FSFGIC methods. This indicates that there may be biases in the features extracted from the input images. The bias of the acquired feature may cause deviation in the calculation of similarity, which is particularly detrimental to FSFGIC tasks characterized by low inter-class variation and high intra-class variation, thus affecting the classification accuracy. To address the problems mentioned, we propose an unbiased feature estimation network. The designed network has the capability to significantly optimize the quality of the obtained feature representations and effectively reduce the feature bias from input images. Furthermore, our proposed architecture can be easily integrated into any contextual training mechanism. Extensive experiments on the FSFGIC tasks demonstrate the effectiveness of the proposed algorithm, showing a notable improvement in classification accuracy.

DRA-UNet for Coal Mining Ground Surface Crack Delineation with UAV High-Resolution Images

Coal mining in the Loess Plateau can very easily generate ground cracks, and these cracks can immediately result in ventilation trouble under the mine shaft, runoff disturbance, and vegetation destruction. Advanced UAV (Unmanned Aerial Vehicle) high-resolution mapping and DL (Deep Learning) are introduced as the key methods to quickly delineate coal mining ground surface cracks for disaster prevention. Firstly, the dataset named the Ground Cracks of Coal Mining Area Unmanned Aerial Vehicle (GCCMA-UAV) is built, with a ground resolution of 3 cm, which is suitable to make a 1:500 thematic map of the ground crack. This GCCMA-UAV dataset includes 6280 images of ground cracks, and the size of the imagery is 256 × 256 pixels. Secondly, the DRA-UNet model is built effectively for coal mining ground surface crack delineation. This DRA-UNet model is an improved UNet DL model, which mainly includes the DAM (Dual Dttention Dechanism) module, the RN (residual network) module, and the ASPP (Atrous Spatial Pyramid Pooling) module. The DRA-UNet model shows the highest recall rate of 77.29% when the DRA-UNet was compared with other similar DL models, such as DeepLabV3+, SegNet, PSPNet, and so on. DRA-UNet also has other relatively reliable indicators; the precision rate is 84.92% and the F1 score is 78.87%. Finally, DRA-UNet is applied to delineate cracks on a DOM (Digital Orthophoto Map) of 3 km2 in the mining workface area, with a ground resolution of 3 cm. There were 4903 cracks that were delineated from the DOM in the Huojitu Coal Mine Shaft. This DRA-UNet model effectively improves the efficiency of crack delineation.

Image reconstruction in graphic design based on Global residual Network optimized compressed sensing model

The article aims to address the challenges of information degradation and distortion in graphic design, focusing on optimizing the traditional compressed sensing (CS) model. This optimization involves creating a co-reconstruction group derived from compressed observations of local image blocks. Following an initial reconstruction of compressed observations within similar groups, an initially reconstructed image block co-reconstruction group is obtained, featuring degraded reconstructed images. These images undergo channel stitching and are input into a global residual network. This network is composed of a non-local feature adaptive interaction module stacked with the aim of fusion to enhance local feature reconstruction. Results indicate that the solution space constraint for reconstructed images is achieved at a low sampling rate. Moreover, high-frequency information within the images is effectively reconstructed, improving image reconstruction accuracy.

A Rapid Nanofocusing Method for a Deep-Sea Gene Sequencing Microscope Based on Critical Illumination

In the deep-sea environment, the volume available for an in-situ gene sequencer is severely limited. In addition, optical imaging systems are subject to real-time, large-scale defocusing problems caused by ambient temperature fluctuations and vibrational perturbations. To address these challenges, we propose an edge detection algorithm for defocused images based on grayscale gradients and establish a defocus state detection model with nanometer resolution capabilities by relying on the inherent critical illumination light field. The model has been applied to a prototype deep-sea gene sequencing microscope with a 20× objective. It has demonstrated the ability to focus within a dynamic range of ±40 μm with an accuracy of 200 nm by a single iteration within 160 ms. By increasing the number of iterations and exposures, the focusing accuracy can be refined to 78 nm within a dynamic range of ±100 μm within 1.2 s. Notably, unlike conventional photoelectric hill-climbing, this method requires no additional hardware and meets the wide dynamic range, speed, and high-accuracy autofocusing requirements of deep-sea gene sequencing in a compact form factor.

Overlapping Shoeprint Detection by Edge Detection and Deep Learning

In the field of 2-D image processing and computer vision, accurately detecting and segmenting objects in scenarios where they overlap or are obscured remains a challenge. This difficulty is worse in the analysis of shoeprints used in forensic investigations because they are embedded in noisy environments such as the ground and can be indistinct. Traditional convolutional neural networks (CNNs), despite their success in various image analysis tasks, struggle with accurately delineating overlapping objects due to the complexity of segmenting intertwined textures and boundaries against a background of noise. This study introduces and employs the YOLO (You Only Look Once) model enhanced by edge detection and image segmentation techniques to improve the detection of overlapping shoeprints. By focusing on the critical boundary information between shoeprint textures and the ground, our method demonstrates improvements in sensitivity and precision, achieving confidence levels above 85% for minimally overlapped images and maintaining above 70% for extensively overlapped instances. Heatmaps of convolution layers were generated to show how the network converges towards successful detection using these enhancements. This research may provide a potential methodology for addressing the broader challenge of detecting multiple overlapping objects against noisy backgrounds.

Image edge detection enhancement using coefficients of Sakaguchi type functions mapped onto petal shaped domain

This research introduces a new approach to elevate the precision of image edge detection through a new algorithm rooted in the coefficients derived from the subclass SC t , ρ (CSKP model). Our method employs convolution operations on input image pixels, utilizing the CSKP mask window in eight distinct directions, fostering a comprehensive and multi-directional analysis of edge features. To gauge the efficacy of our algorithm, image quality is assessed through perceptually significant metrics, including contrast, correlation, energy, homogeneity, and entropy. The study aims to contribute a valuable tool for diverse applications such as computer vision and medical imaging by presenting a robust and innovative solution to enhance image edge detection. The results demonstrate notable improvements, affirming the potential of the proposed algorithm to advance the current state-of-the-art in image processing.

Optimizing support vector machine (SVM) by social spider optimization (SSO) for edge detection in colored images

Edge detection in images is a vital application of image processing in fields such as object detection and identification of lesion regions in medical images. This problem is more complex in the domain of color images due to the combination of color layer information and the need to achieve a unified edge boundary across these layers, which increases the complexity of the problem. In this paper, a simple and effective method for edge detection in color images is proposed using a combination of support vector machine (SVM) and the social spider optimization (SSO) algorithm. In the proposed method, the input color image is first converted to a grayscale image, and an initial estimation of the image edges is performed based on it. To this end, the proposed method utilizes an SVM with a Radial Basis Function (RBF) kernel, in which the model's hyperparameters are tuned using the SSO algorithm. After the formation of initial image edges, the resulting edges are compared with pairwise combinations of color layers, and an attempt is made to improve the edge localization using the SSO algorithm. In this step, the optimization algorithm's task is to refine the image edges in a way that maximizes the compatibility with pairwise combinations of color layers. This process leads to the formation of prominent image edges and reduces the adverse effects of noise on the final result. The performance of the proposed method in edge detection of various color images has been evaluated and compared with similar previous strategies. According to the obtained results, the proposed method can successfully identify image edges more accurately, as the edges identified by the proposed method have an average accuracy of 93.11% for the BSDS500 database, which is an increase of at least 0.74% compared to other methods.

Image Data-Centric Visual Feature Selection on Roll-to-Roll Slot-Die Coating Systems for Edge Wave Coating Defect Detection

Roll-to-roll (R2R) manufacturing depends on a system's capability to deposit high-quality coatings with precise thickness, width, and uniformity. Therefore, consistent maintenance requires the immediate and accurate detection of coating defects. This study proposes a primary color selection (PCS) method to detect edge defects in R2R systems. This method addresses challenges associated with training data demands, complexity, and defect adaptability through a vision data-centric approach, ensuring precise edge coating defect detection. Using color information, high accuracy was achieved while minimizing data capacity requirements and processing time. Precise edge detection was facilitated by accurately distinguishing coated and noncoated regions by selecting the primary color channel based on color variability. The PCS method achieved superior accuracy (95.8%), outperforming the traditional weighted sum method (78.3%). This method is suitable for real-time detection in manufacturing systems and mitigates edge coating defects, thus facilitating quality control and production optimization.

Transformer with difference convolutional network for lightweight universal boundary detection

Although deep-learning methods can achieve human-level performance in boundary detection, their improvements mostly rely on larger models and specific datasets, leading to significant computational power consumption. As a fundamental low-level vision task, a single model with fewer parameters to achieve cross-dataset boundary detection merits further investigation. In this study, a lightweight universal boundary detection method was developed based on convolution and a transformer. The network is called a "transformer with difference convolutional network" (TDCN), which implies the introduction of a difference convolutional network rather than a pure transformer. The TDCN structure consists of three parts: convolution, transformer, and head function. First, a convolution network fused with edge operators is used to extract multiscale difference features. These pixel difference features are then fed to the hierarchical transformer as tokens. Considering the intrinsic characteristics of the boundary detection task, a new boundary-aware self-attention structure was designed in the transformer to provide inductive bias. By incorporating the proposed attention loss function, it introduces the direction of the boundary as strongly supervised information to improve the detection ability of the model. Finally, several head functions with multiscale feature inputs were trained using a bidirectional additive strategy. In the experiments, the proposed method achieved competitive performance on multiple public datasets with fewer model parameters. A single model was obtained to realize universal prediction even for different datasets without retraining, demonstrating the effectiveness of the method. The code is available at https://github.com/neulmc/TDCN.

Stochastic neuro-fuzzy system implemented in memristor crossbar arrays

Neuro-symbolic artificial intelligence has garnered considerable attention amid increasing industry demands for high-performance neural networks that are interpretable and adaptable to previously unknown problem domains with minimal reconfiguration. However, implementing neuro-symbolic hardware is challenging due to the complexity in symbolic knowledge representation and calculation. We experimentally demonstrated a memristor-based neuro-fuzzy hardware based on TiN/TaOx/HfOx/TiN chips that is superior to its silicon-based counterpart in terms of throughput and energy efficiency by using array topological structure for knowledge representation and physical laws for computing. Intrinsic memristor variability is fully exploited to increase robustness in knowledge representation. A hybrid in situ training strategy is proposed for error minimizing in training. The hardware adapts easier to a previously unknown environment, achieving ~6.6 times faster convergence and ~6 times lower error than deep learning. The hardware energy efficiency is over two orders of magnitude greater than field-programmable gate arrays. This research greatly extends the capability of memristor-based neuromorphic computing systems in artificial intelligence.

Intelligent Tapping Machine: Tap Geometry Inspection

Currently, the majority of industrial metal processing involves the use of taps for cutting. However, existing tap machines require relocation to specialized inspection stations and only assess the condition of the cutting edges for defects. They do not evaluate the quality of the cutting angles and the amount of removed material. Machine vision, a key component of smart manufacturing, is commonly used for visual inspection. Taps are employed for processing various materials. Traditional tap replacement relies on the technician's accumulated empirical experience to determine the service life of the tap. Therefore, we propose the use of visual inspection of the tap's external features to determine whether replacement or regrinding is needed. We examined the bearing surface of the tap and utilized single images to identify the cutting angle, clearance angle, and cone angles. By inspecting the side of the tap, we calculated the wear of each cusp. This inspection process can facilitate the development of a tap life system, allowing for the estimation of the durability and wear of taps and nuts made of different materials. Statistical analysis can be employed to predict the lifespan of taps in production lines. Experimental error is 16 μm. Wear from tapping 60 times is equivalent to 8 s of electric grinding. We have introduced a parameter, thread removal quantity, which has not been proposed by anyone else.

Accurate Detection for Zirconium Sheet Surface Scratches Based on Visible Light Images

Zirconium sheet has been widely used in various fields, e.g., chemistry and aerospace. The surface scratches on the zirconium sheets caused by complex processing environment have a negative impact on the performance, e.g., working life and fatigue fracture resistance. Therefore, it is necessary to detect the defect of zirconium sheets. However, it is difficult to detect such scratch images due to lots of scattered additive noise and complex interlaced structural texture. Hence, we propose a framework for adaptively detecting scratches on the surface images of zirconium sheets, including noise removing and texture suppressing. First, the noise removal algorithm, i.e., an optimized threshold function based on dual-tree complex wavelet transform, uses selected parameters to remove scattered and numerous noise. Second, the texture suppression algorithm, i.e., an optimized relative total variation enhancement model, employs selected parameters to suppress interlaced texture. Finally, by connecting disconnection based on two types of connection algorithms and replacing the Gaussian filter in the standard Canny edge detection algorithm with our proposed framework, we can more robustly detect the scratches. The experimental results show that the proposed framework is of higher accuracy.

Image local structure information learning for fine-grained visual classification

Learning discriminative visual patterns from image local salient regions is widely used for fine-grained visual classification (FGVC) tasks such as plant or animal species classification. A large number of complex networks have been designed for learning discriminative feature representations. In this paper, we propose a novel local structure information (LSI) learning method for FGVC. Firstly, we indicate that the existing FGVC methods have not properly considered how to extract LSI from an input image for FGVC. Then an LSI extraction technique is introduced which has the ability to properly depict the properties of different local structure features in images. Secondly, a novel LSI learning module is proposed to be added into a given backbone network for enhancing the ability of the network to find salient regions. Thirdly, extensive experiments show that our proposed method achieves better performance on six image datasets. Particularly, the proposed method performs far better on datasets with a limited number of images.