Two-Phase Object-Based Deep Learning for Multi-Temporal SAR Image Change Detection

ESCNet: An End-to-End Superpixel-Enhanced Change Detection Network for Very-High-Resolution Remote Sensing Images

Change detection (CD), as one of the central problems in Earth observation, has attracted a lot of research interest over recent decades. Due to the rapid development of satellite sensors in recent years, we have witnessed an enrichment of the CD source data with the availability of very-high-resolution (VHR) multispectral imagery, which provides abundant change clues. However, precisely locating real changed areas still remains a challenge. In this article, we propose an end-to-end superpixel-enhanced CD network (ESCNet) for VHR images, which combines differentiable superpixel segmentation and a deep convolutional neural network (DCNN). Two weight-sharing superpixel sampling networks (SSNs) are tailored for the feature extraction and superpixel segmentation of bitemporal image pairs. A UNet-based Siamese neural network is then employed to mine the different information. The superpixels are then leveraged to reduce the latent noise in the pixel-level feature maps while preserving the edges, where a novel superpixelation module is used to serve this purpose. Furthermore, to compensate for the dependence on the number of superpixels, we propose an innovative adaptive superpixel merging (ASM) module, which has a concise form and is fully differentiable. A pixel-level refinement module making use of the multilevel decoded features is also appended to the end of the framework. Experiments on two public datasets confirmed the superiority of ESCNet compared to the traditional and state-of-the-art (SOTA) deep learning-based CD (DLCD) methods.

Azimuth-Sensitive Object Detection of High-Resolution SAR Images in Complex Scenes by Using a Spatial Orientation Attention Enhancement Network

The scattering features of objects in synthetic aperture radar (SAR) imagery are highly sensitive to different azimuth angles, and detecting azimuth-sensitive objects in complex scenes becomes a challenging task. To address this issue, we propose a novel framework called the spatial orientation attention enhancement network (SOAEN) by using aircraft detection in complex scenes of SAR imagery as a case study. Taking YOLOX as the basic framework, this framework introduces the inverted pyramid ConvMixer network (IPCN), the spatial-orientation-enhanced path aggregation feature pyramid network (SOEPAFPN), and the anchor-free decoupled head (AFDH) to achieve performance improvement. A spatial orientation attention module is proposed and introduced into the path aggregation feature pyramid network to form a new structure, the SOEPAFPN, for capturing feature transformations in different directions, highlighting object features and suppressing background effects; the IPCN is adapted to replace the backbone network of YOLOX for enhancing the multiscale feature extraction capability and reducing the computational complexity, while the AFDH is used to decouple object localization and classification to improve the efficiency and accuracy of object localization and classification. The experimental results of the multiple real complex scenes on Gaofen-3 1 m images show that the proposed method achieves the highest detection accuracy, with an average detection rate of 91.22% compared with the YOLO series networks.

Hierarchical Transmission Tower Detection from High-Resolution SAR Image

The small scale of transmission towers and the environmental diversity around their situations make their detection in Synthetic Aperture Radar (SAR) images a challenging task. This paper presents a new hierarchical detection algorithm for transmission towers. First, Signal-to-Clutter Ratios (SCRs) of pixels are calculated. Afterwards, a SCR threshold is set. Since transmission towers possess strong scattering characteristics, pixels with SCRs above the threshold are considered as potential transmission tower pixels. Second, spatial densities of potential transmission tower pixels are calculated. According to the aggregation characteristics of transmission tower pixels, some potential transmission tower pixels with small spatial densities are removed. The remained potential transmission tower pixels are considered as candidate transmission tower pixels. The candidate transmission tower pixels are grouped by the nearest neighbour scheme such that in each group the distance between pixels is under a given threshold. Thus, each of the groups is viewed as a quasi-transmission tower. Convex-hulls of quasi-transmission towers are built, and then Minimum Bounding Rectangle (MBR) for each convex-hull is generated. According to the rectangle aspect ratios of MBRs, the real transmission towers are extracted. C-band HH-polarization GaoFen-3 (GF-3) amplitude images are used for experiments and four of the most popular transmission tower detection algorithms are selected as comparing algorithms to validate the proposed algorithms. The detection performance of transmission towers is evaluated with detection rate and quality factor. Experimental results verify that the proposed algorithm can efficiently and accurately detect transmission towers while maintaining the transmission tower geometry to a certain extent, which indicates that the proposed algorithm is efficient and promising.

SAR Target Detection Based on Improved SSD with Saliency Map and Residual Network

A target detection method based on an improved single shot multibox detector (SSD) is proposed to solve insufficient training samples for synthetic aperture radar (SAR) target detection. We propose two strategies to improve the SSD: model structure optimization and small sample augmentation. For model structure optimization, the first approach is to extract deep features of the target with residual networks instead of with VGGNet. Then, the aspect ratios of the default boxes are redesigned to match the different targets’ sizes. For small sample augmentation, besides the routine image processing methods, such as rotating, translating, and mirroring, enough training samples are obtained based on the saliency map theory in machine vision. Lastly, a simulated SAR image dataset called Geometric Objects (GO) is constructed, which contains dihedral angles, surface plates and cylinders. The experimental results on the GO-simulated image dataset and the MSTAR real image dataset demonstrate that the proposed method has better performance in SAR target detection than other detection methods.

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 1: Literature Review

Convolutional neural network (CNN)-based deep learning (DL) is a powerful, recently developed image classification approach. With origins in the computer vision and image processing communities, the accuracy assessment methods developed for CNN-based DL use a wide range of metrics that may be unfamiliar to the remote sensing (RS) community. To explore the differences between traditional RS and DL RS methods, we surveyed a random selection of 100 papers from the RS DL literature. The results show that RS DL studies have largely abandoned traditional RS accuracy assessment terminology, though some of the accuracy measures typically used in DL papers, most notably precision and recall, have direct equivalents in traditional RS terminology. Some of the DL accuracy terms have multiple names, or are equivalent to another measure. In our sample, DL studies only rarely reported a complete confusion matrix, and when they did so, it was even more rare that the confusion matrix estimated population properties. On the other hand, some DL studies are increasingly paying attention to the role of class prevalence in designing accuracy assessment approaches. DL studies that evaluate the decision boundary threshold over a range of values tend to use the precision-recall (P-R) curve, the associated area under the curve (AUC) measures of average precision (AP) and mean average precision (mAP), rather than the traditional receiver operating characteristic (ROC) curve and its AUC. DL studies are also notable for testing the generalization of their models on entirely new datasets, including data from new areas, new acquisition times, or even new sensors.

Patch-Based Change Detection Method for SAR Images with Label Updating Strategy

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.

A Classified Adversarial Network for Multi-Spectral Remote Sensing Image Change Detection

Adversarial training has demonstrated advanced capabilities for generating image models. In this paper, we propose a deep neural network, named a classified adversarial network (CAN), for multi-spectral image change detection. This network is based on generative adversarial networks (GANs). The generator captures the distribution of the bitemporal multi-spectral image data and transforms it into change detection results, and these change detection results (as the fake data) are input into the discriminator to train the discriminator. The results obtained by pre-classification are also input into the discriminator as the real data. The adversarial training can facilitate the generator learning the transformation from a bitemporal image to a change map. When the generator is trained well, the generator has the ability to generate the final result. The bitemporal multi-spectral images are input into the generator, and then the final change detection results are obtained from the generator. The proposed method is completely unsupervised, and we only need to input the preprocessed data that were obtained from the pre-classification and training sample selection. Through adversarial training, the generator can better learn the relationship between the bitemporal multi-spectral image data and the corresponding labels. Finally, the well-trained generator can be applied to process the raw bitemporal multi-spectral images to obtain the final change map (CM). The effectiveness and robustness of the proposed method were verified by the experimental results on the real high-resolution multi-spectral image data sets.