Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks

Multiview High Dynamic Range Image Synthesis Using Fuzzy Broad Learning System

Compared with the normal low dynamic range (LDR) images, the high dynamic range (HDR) images provide more dynamic range and image details. Although the existing techniques for generating the HDR images have a good effect for static scenes, they usually produce artifacts on the HDR images for dynamic scenes. In recent years, some learning-based approaches are used to synthesize the HDR images and obtain good results. However, there are also many problems, including the deficiency of explaining and the time-consuming training process. In this article, we propose a novel approach to synthesize multiview HDR images through fuzzy broad learning system (FBLS). We use a set of multiview LDR images with different exposure as input and transfer corresponding Takagi-Sugeno (TS) fuzzy subsystems; then, the structure is expanded in a wide sense in the "enhancement groups" which transfer from the TS fuzzy rules with nonlinear transformation. After integrating fuzzy subsystems and enhancement groups with the trained-well weight, the HDR image is generated. In FBLS, applying the incremental learning algorithm and the pseudoinverse method to compute the weights can greatly reduce the training time. In addition, the fuzzy system has better interpretability. In the learning process, IF-THEN fuzzy rules can effectively help the model to detect the artifacts and reject them in the final HDR result. These advantages solve the problem of existing deep-learning methods. Furthermore, we set up a new dataset of multiview LDR images with corresponding HDR ground truth to train our system. Our experimental results show that our system can synthesize high-quality multiview HDR images, which has a higher training speed than other learning methods.

Patch-Based Change Detection Method for SAR Images with Label Updating Strategy

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.

Rain Streak Removal for Single Images Using Conditional Generative Adversarial Networks

Rapid developments in urbanization and smart city environments have accelerated the need to deliver safe, sustainable, and effective resource utilization and service provision and have thereby enhanced the need for intelligent, real-time video surveillance. Recent advances in machine learning and deep learning have the capability to detect and localize salient objects in surveillance video streams; however, several practical issues remain unaddressed, such as diverse weather conditions, recording conditions, and motion blur. In this context, image de-raining is an important issue that has been investigated extensively in recent years to provide accurate and quality surveillance in the smart city domain. Existing deep convolutional neural networks have obtained great success in image translation and other computer vision tasks; however, image de-raining is ill posed and has not been addressed in real-time, intelligent video surveillance systems. In this work, we propose to utilize the generative capabilities of recently introduced conditional generative adversarial networks (cGANs) as an image de-raining approach. We utilize the adversarial loss in GANs that provides an additional component to the loss function, which in turn regulates the final output and helps to yield better results. Experiments on both real and synthetic data show that the proposed method outperforms most of the existing state-of-the-art models in terms of quantitative evaluations and visual appearance.

Earth Environmental Monitoring Using Multi-Temporal Synthetic Aperture Radar: A Critical Review of Selected Applications

Microwave remote sensing has widely demonstrated its potential in the continuous monitoring of our rapidly changing planet. This review provides an overview of state-of-the-art methodologies for multi-temporal synthetic aperture radar change detection and its applications to biosphere and hydrosphere monitoring, with special focus on topics like forestry, water resources management in semi-arid environments and floods. The analyzed literature is categorized on the base of the approach adopted and the data exploited and discussed in light of the downstream remote sensing market. The purpose is to highlight the main issues and limitations preventing the diffusion of synthetic aperture radar data in both industrial and multidisciplinary research contexts and the possible solutions for boosting their usage among end-users.

A Novel Squeeze-and-Excitation W-Net for 2D and 3D Building Change Detection with Multi-Source and Multi-Feature Remote Sensing Data

Building Change Detection (BCD) is one of the core issues in earth observation and has received extensive attention in recent years. With the rapid development of earth observation technology, the data source of remote sensing change detection is continuously enriched, which provides the possibility to describe the spatial details of the ground objects more finely and to characterize the ground objects with multiple perspectives and levels. However, due to the different physical mechanisms of multi-source remote sensing data, BCD based on heterogeneous data is a challenge. Previous studies mostly focused on the BCD of homogeneous remote sensing data, while the use of multi-source remote sensing data and considering multiple features to conduct 2D and 3D BCD research is sporadic. In this article, we propose a novel and general squeeze-and-excitation W-Net, which is developed from U-Net and SE-Net. Its unique advantage is that it can not only be used for BCD of homogeneous and heterogeneous remote sensing data respectively but also can input both homogeneous and heterogeneous remote sensing data for 2D or 3D BCD by relying on its bidirectional symmetric end-to-end network architecture. Moreover, from a unique perspective, we use image features that are stable in performance and less affected by radiation differences and temporal changes. We innovatively introduced the squeeze-and-excitation module to explicitly model the interdependence between feature channels so that the response between the feature channels is adaptively recalibrated to improve the information mining ability and detection accuracy of the model. As far as we know, this is the first proposed network architecture that can simultaneously use multi-source and multi-feature remote sensing data for 2D and 3D BCD. The experimental results in two 2D data sets and two challenging 3D data sets demonstrate that the promising performances of the squeeze-and-excitation W-Net outperform several traditional and state-of-the-art approaches. Moreover, both visual and quantitative analyses of the experimental results demonstrate competitive performance in the proposed network. This demonstrates that the proposed network and method are practical, physically justified, and have great potential application value in large-scale 2D and 3D BCD and qualitative and quantitative research.

Learning Student Networks via Feature Embedding

Deep convolutional neural networks have been widely used in numerous applications, but their demanding storage and computational resource requirements prevent their applications on mobile devices. Knowledge distillation aims to optimize a portable student network by taking the knowledge from a well-trained heavy teacher network. Traditional teacher-student-based methods used to rely on additional fully connected layers to bridge intermediate layers of teacher and student networks, which brings in a large number of auxiliary parameters. In contrast, this article aims to propagate information from teacher to student without introducing new variables that need to be optimized. We regard the teacher-student paradigm from a new perspective of feature embedding. By introducing the locality preserving loss, the student network is encouraged to generate the low-dimensional features that could inherit intrinsic properties of their corresponding high-dimensional features from the teacher network. The resulting portable network, thus, can naturally maintain the performance as that of the teacher network. Theoretical analysis is provided to justify the lower computation complexity of the proposed method. Experiments on benchmark data sets and well-trained networks suggest that the proposed algorithm is superior to state-of-the-art teacher-student learning methods in terms of computational and storage complexity.

Optimizing Near Real-Time Detection of Deforestation on Tropical Rainforests Using Sentinel-1 Data

Early Warning Systems (EWS) for near real-time detection of deforestation are a fundamental component of public policies focusing on the reduction in forest biomass loss and associated CO2 emissions. Most of the operational EWS are based on optical data, which are severely limited by the cloud cover in tropical environments. Synthetic Aperture Radar (SAR) data can help to overcome this observational gap. SAR measurements, however, can be altered by atmospheric effects on and variations in surface moisture. Different techniques of time series (TS) stabilization have been used to mitigate the instability of C-band SAR measurements. Here, we evaluate the performance of two different approaches to SAR TS stabilization, harmonic deseasonalization and spatial stabilization, as well as two deforestation detection techniques, Adaptive Linear Thresholding (ALT) and maximum likelihood classification (MLC). We set up a rigorous, Amazon-wide validation experiment using the Google Earth Engine platform to sample and process Sentinel-1A data of nearly 6000 locations in the whole Brazilian Amazonian basin, generating more than 8M processed samples. Half of those locations correspond to non-degraded forest areas, while the other half pertained to 2019 deforested areas. The detection results showed that the spatial stabilization algorithm improved the results of the MLC approach, reaching 94.36% global accuracy. The ALT detection algorithm performed better, reaching 95.91% global accuracy, regardless of the use of any stabilization method. The results of this experiment are being used to develop an operational EWS in the Brazilian Amazon.

Weakly Supervised Change Detection Based on Edge Mapping and SDAE Network in High-Resolution Remote Sensing Images

Change detection for high-resolution remote sensing images is more and more widespread in the application of monitoring the Earth’s surface. However, on the one hand, the ground truth could facilitate the distinction between changed and unchanged areas, but it is hard to acquire them. On the other hand, due to the complexity of remote sensing images, it is difficult to extract features of difference, let alone the construction of the classification model that performs change detection based on the features of difference in each pixel pair. Aiming at these challenges, this paper proposes a weakly supervised change detection method based on edge mapping and Stacked Denoising Auto-Encoders (SDAE) network called EM-SDAE. We analyze the difference in edge maps of bi-temporal remote sensing images to acquire part of the ground truth at a relatively low cost. Moreover, we design a neural network based on SDAE with a deep structure, which extracts the features of difference so as to efficiently classify changed and unchanged regions after being trained with the ground truth. In our experiments, three real sets of high-resolution remote sensing images are employed to validate the high efficiency of our proposed method. The results show that accuracy can even reach up to 91.18% with our method. In particular, compared with the state-of-the-art work (e.g., IR-MAD, PCA-k-means, CaffeNet, USFA, and DSFA), it improves the Kappa coefficient by 27.19% on average.

Performance Analysis of Deep Convolutional Autoencoders with Different Patch Sizes for Change Detection from Burnt Areas

Fire is one of the primary sources of damages to natural environments globally. Estimates show that approximately 4 million km2 of land burns yearly. Studies have shown that such estimates often underestimate the real extent of burnt land, which highlights the need to find better, state-of-the-art methods to detect and classify these areas. This study aimed to analyze the use of deep convolutional Autoencoders in the classification of burnt areas, considering different sample patch sizes. A simple Autoencoder and the U-Net and ResUnet architectures were evaluated. We collected Landsat 8 OLI+ data from three scenes in four consecutive dates to detect the changes specifically in the form of burnt land. The data were sampled according to four different sampling strategies to evaluate possible performance changes related to sampling window sizes. The training stage used two scenes, while the validation stage used the remaining scene. The ground truth change mask was created using the Normalized Burn Ratio (NBR) spectral index through a thresholding approach. The classifications were evaluated according to the F1 index, Kappa index, and mean Intersection over Union (mIoU) value. Results have shown that the U-Net and ResUnet architectures offered the best classifications with average F1, Kappa, and mIoU values of approximately 0.96, representing excellent classification results. We have also verified that a sampling window size of 256 by 256 pixels offered the best results.

Weighted Broad Learning System and Its Application in Nonlinear Industrial Process Modeling

Broad learning system (BLS) is a novel neural network with effective and efficient learning ability. BLS has attracted increasing attention from many scholars owing to its excellent performance. This article proposes a weighted BLS (WBLS) based on BLS to tackle the noise and outliers in an industrial process. WBLS provides a unified framework for easily using different methods of calculating the weighted penalty factor. Using the weighted penalty factor to constrain the contribution of each sample to modeling, the normal and abnormal samples were allocated higher and lower weights to increase and decrease their contributions, respectively. Hence, the WBLS can eliminate the bad effect of noise and outliers on the modeling. The weighted ridge regression algorithm is used to compute the algorithm solution. Weighted incremental learning algorithms are also developed using the weighted penalty factor to tackle the noise and outliers in the additional samples and quickly increase nodes or samples without retraining. The proposed weighted incremental learning algorithms provide a unified framework for using different methods of computing weights. We test the feasibility of the proposed algorithms on some public data sets and a real-world application. Experiment results show that our method has better generalization and robustness.

Sparse Manifold-Regularized Neural Networks for Polarimetric SAR Terrain Classification

In this article, a new deep neural network based on sparse filtering and manifold regularization (DSMR) is proposed for feature extraction and classification of polarimetric synthetic aperture radar (PolSAR) data. DSMR uses a novel deep neural network (DNN) to automatically learn features from raw SAR data. During preprocessing, the spatial information between pixels on PolSAR images is exploited to weight each data sample. Then, in the pretraining and fine-tuning, DSMR uses the population sparsity and the lifetime sparsity (dual sparsity) to learn the global features and preserves the local structure of data by neighborhood-based manifold regularization. The dual sparsity only needs to tune a few parameters, and the manifold regularization cuts down the number of training samples. Experimental results on synthesized and real PolSAR data sets from different SAR systems show that DSMR can improve classification accuracy compared with conventional DNNs, even for data sets with a large angle of incidence.

A Survey of Change Detection Methods Based on Remote Sensing Images for Multi-Source and Multi-Objective Scenarios

Quantities of multi-temporal remote sensing (RS) images create favorable conditions for exploring the urban change in the long term. However, diverse multi-source features and change patterns bring challenges to the change detection in urban cases. In order to sort out the development venation of urban change detection, we make an observation of the literatures on change detection in the last five years, which focuses on the disparate multi-source RS images and multi-objective scenarios determined according to scene category. Based on the survey, a general change detection framework, including change information extraction, data fusion, and analysis of multi-objective scenarios modules, is summarized. Owing to the attributes of input RS images affect the technical selection of each module, data characteristics and application domains across different categories of RS images are discussed firstly. On this basis, not only the evolution process and relationship of the representative solutions are elaborated in the module description, through emphasizing the feasibility of fusing diverse data and the manifold application scenarios, we also advocate a complete change detection pipeline. At the end of the paper, we conclude the current development situation and put forward possible research direction of urban change detection, in the hope of providing insights to the following research.

A Deep One-Class Neural Network for Anomalous Event Detection in Complex Scenes

How to build a generic deep one-class (DeepOC) model to solve one-class classification problems for anomaly detection, such as anomalous event detection in complex scenes? The characteristics of existing one-class labels lead to a dilemma: it is hard to directly use a multiple classifier based on deep neural networks to solve one-class classification problems. Therefore, in this article, we propose a novel DeepOC neural network, termed as DeepOC, which can simultaneously learn compact feature representations and train a DeepOC classifier. Only with the given normal samples, we use the stacked convolutional encoder to generate their low-dimensional high-level features and train a one-class classifier to make these features as compact as possible. Meanwhile, for the sake of the correct mapping relation and the feature representations' diversity, we utilize a decoder in order to reconstruct raw samples from these low-dimensional feature representations. This structure is gradually established using an adversarial mechanism during the training stage. This mechanism is the key to our model. It organically combines two seemingly contradictory components and allows them to take advantage of each other, thus making the model robust and effective. Unlike methods that use handcrafted features or those that are separated into two stages (extracting features and training classifiers), DeepOC is a one-stage model using reliable features that are automatically extracted by neural networks. Experiments on various benchmark data sets show that DeepOC is feasible and achieves the state-of-the-art anomaly detection results compared with a dozen existing methods.

A Classified Adversarial Network for Multi-Spectral Remote Sensing Image Change Detection

Adversarial training has demonstrated advanced capabilities for generating image models. In this paper, we propose a deep neural network, named a classified adversarial network (CAN), for multi-spectral image change detection. This network is based on generative adversarial networks (GANs). The generator captures the distribution of the bitemporal multi-spectral image data and transforms it into change detection results, and these change detection results (as the fake data) are input into the discriminator to train the discriminator. The results obtained by pre-classification are also input into the discriminator as the real data. The adversarial training can facilitate the generator learning the transformation from a bitemporal image to a change map. When the generator is trained well, the generator has the ability to generate the final result. The bitemporal multi-spectral images are input into the generator, and then the final change detection results are obtained from the generator. The proposed method is completely unsupervised, and we only need to input the preprocessed data that were obtained from the pre-classification and training sample selection. Through adversarial training, the generator can better learn the relationship between the bitemporal multi-spectral image data and the corresponding labels. Finally, the well-trained generator can be applied to process the raw bitemporal multi-spectral images to obtain the final change map (CM). The effectiveness and robustness of the proposed method were verified by the experimental results on the real high-resolution multi-spectral image data sets.

Self-Supervised Representation Learning for Remote Sensing Image Change Detection Based on Temporal Prediction

Traditional change detection (CD) methods operate in the simple image domain or hand-crafted features, which has less robustness to the inconsistencies (e.g., brightness and noise distribution, etc.) between bitemporal satellite images. Recently, deep learning techniques have reported compelling performance on robust feature learning. However, generating accurate semantic supervision that reveals real change information in satellite images still remains challenging, especially for manual annotation. To solve this problem, we propose a novel self-supervised representation learning method based on temporal prediction for remote sensing image CD. The main idea of our algorithm is to transform two satellite images into more consistent feature representations through a self-supervised mechanism without semantic supervision and any additional computations. Based on the transformed feature representations, a better difference image (DI) can be obtained, which reduces the propagated error of DI on the final detection result. In the self-supervised mechanism, the network is asked to identify different sample patches between two temporal images, namely, temporal prediction. By designing the network for the temporal prediction task to imitate the discriminator of generative adversarial networks, the distribution-aware feature representations are automatically captured and the result with powerful robustness can be acquired. Experimental results on real remote sensing data sets show the effectiveness and superiority of our method, improving the detection precision up to 0.94–35.49%.

Lightweight Pyramid Networks for Image Deraining

Existing deep convolutional neural networks (CNNs) have found major success in image deraining, but at the expense of an enormous number of parameters. This limits their potential applications, e.g., in mobile devices. In this paper, we propose a lightweight pyramid networt (LPNet) for single-image deraining. Instead of designing a complex network structure, we use domain-specific knowledge to simplify the learning process. In particular, we find that by introducing the mature Gaussian-Laplacian image pyramid decomposition technology to the neural network, the learning problem at each pyramid level is greatly simplified and can be handled by a relatively shallow network with few parameters. We adopt recursive and residual network structures to build the proposed LPNet, which has less than 8K parameters while still achieving the state-of-the-art performance on rain removal. We also discuss the potential value of LPNet for other low- and high-level vision tasks.

Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges

Change detection based on remote sensing (RS) data is an important method of detecting changes on the Earth’s surface and has a wide range of applications in urban planning, environmental monitoring, agriculture investigation, disaster assessment, and map revision. In recent years, integrated artificial intelligence (AI) technology has become a research focus in developing new change detection methods. Although some researchers claim that AI-based change detection approaches outperform traditional change detection approaches, it is not immediately obvious how and to what extent AI can improve the performance of change detection. This review focuses on the state-of-the-art methods, applications, and challenges of AI for change detection. Specifically, the implementation process of AI-based change detection is first introduced. Then, the data from different sensors used for change detection, including optical RS data, synthetic aperture radar (SAR) data, street view images, and combined heterogeneous data, are presented, and the available open datasets are also listed. The general frameworks of AI-based change detection methods are reviewed and analyzed systematically, and the unsupervised schemes used in AI-based change detection are further analyzed. Subsequently, the commonly used networks in AI for change detection are described. From a practical point of view, the application domains of AI-based change detection methods are classified based on their applicability. Finally, the major challenges and prospects of AI for change detection are discussed and delineated, including (a) heterogeneous big data processing, (b) unsupervised AI, and (c) the reliability of AI. This review will be beneficial for researchers in understanding this field.

A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection

Remote sensing image change detection (CD) is done to identify desired significant changes between bitemporal images. Given two co-registered images taken at different times, the illumination variations and misregistration errors overwhelm the real object changes. Exploring the relationships among different spatial–temporal pixels may improve the performances of CD methods. In our work, we propose a novel Siamese-based spatial–temporal attention neural network. In contrast to previous methods that separately encode the bitemporal images without referring to any useful spatial–temporal dependency, we design a CD self-attention mechanism to model the spatial–temporal relationships. We integrate a new CD self-attention module in the procedure of feature extraction. Our self-attention module calculates the attention weights between any two pixels at different times and positions and uses them to generate more discriminative features. Considering that the object may have different scales, we partition the image into multi-scale subregions and introduce the self-attention in each subregion. In this way, we could capture spatial–temporal dependencies at various scales, thereby generating better representations to accommodate objects of various sizes. We also introduce a CD dataset LEVIR-CD, which is two orders of magnitude larger than other public datasets of this field. LEVIR-CD consists of a large set of bitemporal Google Earth images, with 637 image pairs (1024 × 1024) and over 31 k independently labeled change instances. Our proposed attention module improves the F1-score of our baseline model from 83.9 to 87.3 with acceptable computational overhead. Experimental results on a public remote sensing image CD dataset show our method outperforms several other state-of-the-art methods.

A Convolutional Neural Network with Parallel Multi-Scale Spatial Pooling to Detect Temporal Changes in SAR Images

In synthetic aperture radar (SAR) image change detection, it is quite challenging to exploit the changing information from the noisy difference image subject to the speckle. In this paper, we propose a multi-scale spatial pooling (MSSP) network to exploit the changed information from the noisy difference image. Being different from the traditional convolutional network with only mono-scale pooling kernels, in the proposed method, multi-scale pooling kernels are equipped in a convolutional network to exploit the spatial context information on changed regions from the difference image. Furthermore, to verify the generalization of the proposed method, we apply our proposed method to the cross-dataset bitemporal SAR image change detection, where the MSSP network (MSSP-Net) is trained on a dataset and then applied to an unknown testing dataset. We compare the proposed method with other state-of-arts and the comparisons are performed on four challenging datasets of bitemporal SAR images. Experimental results demonstrate that our proposed method obtain comparable results with S-PCA-Net on YR-A and YR-B dataset and outperforms other state-of-art methods, especially on the Sendai-A and Sendai-B datasets with more complex scenes. More important, MSSP-Net is more efficient than S-PCA-Net and convolutional neural networks (CNN) with less executing time in both training and testing phases.

Outdoor Shadow Estimating Using Multiclass Geometric Decomposition Based on BLS

Illumination is a significant component of an image, and illumination estimation of an outdoor scene from given images is still challenging yet it has wide applications. Most of the traditional illumination estimating methods require prior knowledge or fixed objects within the scene, which makes them often limited by the scene of a given image. We propose an optimization approach that integrates the multiclass cues of the image(s) [a main input image and optional auxiliary input image(s)]. First, Sun visibility is estimated by the efficient broad learning system. And then for the scene with visible Sun, we classify the information in the image by the proposed classification algorithm, which combines the geometric information and shadow information to make the most of the information. And we apply a respective algorithm for every class to estimate the illumination parameters. Finally, our approach integrates all of the estimating results by the Markov random field. We make full use of the cues in the given image instead of an extra requirement for the scene, and the qualitative results are presented and show that our approach outperformed other methods with similar conditions.

Skip-Connected Covariance Network for Remote Sensing Scene Classification

This paper proposes a novel end-to-end learning model, called skip-connected covariance (SCCov) network, for remote sensing scene classification (RSSC). The innovative contribution of this paper is to embed two novel modules into the traditional convolutional neural network (CNN) model, i.e., skip connections and covariance pooling. The advantages of newly developed SCCov are twofold. First, by means of the skip connections, the multi-resolution feature maps produced by the CNN are combined together, which provides important benefits to address the presence of large-scale variance in RSSC data sets. Second, by using covariance pooling, we can fully exploit the second-order information contained in such multi-resolution feature maps. This allows the CNN to achieve more representative feature learning when dealing with RSSC problems. Experimental results, conducted using three large-scale benchmark data sets, demonstrate that our newly proposed SCCov network exhibits very competitive or superior classification performance when compared with the current state-of-the-art RSSC techniques, using a much lower amount of parameters. Specifically, our SCCov only needs 10% of the parameters used by its counterparts.

Downsizing and enhancing broad learning systems by feature augmentation and residuals boosting

Recently, a broad learning system (BLS) has been theoretically and experimentally confirmed to be an efficient incremental learning system. To get rid of deep architecture, BLS shares the same architecture and learning mechanism of the well-known functional link neural networks (FLNN), but works in broad learning way on both the randomly mapped features of original features of data and their randomly generated enhancement nodes. As such, BLS often requires a huge heap of hidden nodes to achieve the prescribed or satisfactory performance, which may inevitably cause both overwhelming storage requirement and overfitting phenomenon. In this study, a stacked architecture of broad learning systems called D&BLS is proposed to achieve enhanced performance and simultaneously downsize the system architecture. By boosting the residuals between previous and current layers and simultaneously augmenting the original input space with the outputs of the previous layer as the inputs of current layer, D&BLS stacks several lightweight BLS sub-systems to guarantee stronger feature representation capability and better classification/regression performance. Three fast incremental learning algorithms of D&BLS are also developed, without the need for the whole re-training. Experimental results on some popular datasets demonstrate the effectiveness of D&BLS in the sense of both enhanced performance and reduced system architecture.

Transfer Change Rules from Recurrent Fully Convolutional Networks for Hyperspectral Unmanned Aerial Vehicle Images without Ground Truth Data

Change detection (CD) networks based on supervised learning have been used in diverse CD tasks. However, such supervised CD networks require a large amount of data and only use information from current images. In addition, it is time consuming to manually acquire the ground truth data for newly obtained images. Here, we proposed a novel method for CD in case of a lack of training data in an area near by another one with the available ground truth data. The proposed method automatically entails generating training data and fine-tuning the CD network. To detect changes in target images without ground truth data, the difference images were generated using spectral similarity measure, and the training data were selected via fuzzy c-means clustering. Recurrent fully convolutional networks with multiscale three-dimensional filters were used to extract objects of various sizes from unmanned aerial vehicle (UAV) images. The CD network was pre-trained on labeled source domain data; then, the network was fine-tuned on target images using generated training data. Two further CD networks were trained with a combined weighted loss function. The training data in the target domain were iteratively updated using he prediction map of the CD network. Experiments on two hyperspectral UAV datasets confirmed that the proposed method is capable of transferring change rules and improving CD results based on training data extracted in an unsupervised way.

Bipartite Differential Neural Network for Unsupervised Image Change Detection

Image change detection detects the regions of change in multiple images of the same scene taken at different times, which plays a crucial role in many applications. The two most popular image change detection techniques are as follows: pixel-based methods heavily rely on accurate image coregistration while object-based approaches can tolerate coregistration errors to some extent but are sensitive to image segmentation or classification errors. To address these issues, we propose an unsupervised image change detection approach based on a novel bipartite differential neural network (BDNN). The BDNN is a deep neural network with two input ends, which can extract the holistic features from the unchanged regions in the two input images, where two learnable change disguise maps (CDMs) are used to disguise the changed regions in the two input images, respectively, and thus demarcate the unchanged regions therein. The network parameters and CDMs will be learned by optimizing an objective function, which combines a loss function defined as the likelihood of the given input image pair over all possible input image pairs and two constraints imposed on CDMs. Compared with the pixel-based and object-based techniques, the BDNN is less sensitive to inaccurate image coregistration and does not involve image segmentation or classification. In fact, it can even skip over coregistration if the degree of transformation (due to the different view angles and/or positions of the camera) between the two input images is not that large. We compare the proposed approach with several state-of-the-art image change detection methods on various homogeneous and heterogeneous image pairs with and without coregistration. The results demonstrate the superiority of the proposed approach.

Two-Phase Object-Based Deep Learning for Multi-Temporal SAR Image Change Detection

Change detection is one of the fundamental applications of synthetic aperture radar (SAR) images. However, speckle noise presented in SAR images has a negative effect on change detection, leading to frequent false alarms in the mapping products. In this research, a novel two-phase object-based deep learning approach is proposed for multi-temporal SAR image change detection. Compared with traditional methods, the proposed approach brings two main innovations. One is to classify all pixels into three categories rather than two categories: unchanged pixels, changed pixels caused by strong speckle (false changes), and changed pixels formed by real terrain variation (real changes). The other is to group neighbouring pixels into superpixel objects such as to exploit local spatial context. Two phases are designed in the methodology: (1) Generate objects based on the simple linear iterative clustering (SLIC) algorithm, and discriminate these objects into changed and unchanged classes using fuzzy c-means (FCM) clustering and a deep PCANet. The prediction of this Phase is the set of changed and unchanged superpixels. (2) Deep learning on the pixel sets over the changed superpixels only, obtained in the first phase, to discriminate real changes from false changes. SLIC is employed again to achieve new superpixels in the second phase. Low rank and sparse decomposition are applied to these new superpixels to suppress speckle noise significantly. A further clustering step is applied to these new superpixels via FCM. A new PCANet is then trained to classify two kinds of changed superpixels to achieve the final change maps. Numerical experiments demonstrate that, compared with benchmark methods, the proposed approach can distinguish real changes from false changes effectively with significantly reduced false alarm rates, and achieve up to 99.71% change detection accuracy using multi-temporal SAR imagery.

ℓ1-gain filter design of discrete-time positive neural networks with mixed delays

This paper mainly focuses on the filter design with ℓ₁-gain disturbance attenuation performance for a class of discrete-time positive neural networks. Discrete and distributed time-varying delays occurring in neuron transmission are taken into account. Especially, the probabilistic distribution of distributed delays is described by a Bernoulli random process in the system model. First, criteria on the positiveness and the unique equilibrium of discrete-time neural networks are presented. Second, through linear Lyapunov method, sufficient conditions for globally asymptotic stability with ℓ₁-gain disturbance attenuation performance of positive neural networks are proposed. Third, using the results obtained above, criteria on ℓ₁-gain stability of the established filtering error system are presented, based on which a linear programming (LP) approach is put forward to design the desired positive filter. Finally, two examples of applications to water distribution network and genetic regulatory network are given to demonstrate the effectiveness and applicability of the derived results.

Near Real-Time Wildfire Progression Monitoring with Sentinel-1 SAR Time Series and Deep Learning

In recent years, the world witnessed many devastating wildfires that resulted in destructive human and environmental impacts across the globe. Emergency response and rapid response for mitigation calls for effective approaches for near real-time wildfire monitoring. Capable of penetrating clouds and smoke, and imaging day and night, Synthetic Aperture Radar (SAR) can play a critical role in wildfire monitoring. In this communication, we investigated and demonstrated the potential of Sentinel-1 SAR time series with a deep learning framework for near real-time wildfire progression monitoring. The deep learning framework, based on a Convolutional Neural Network (CNN), is developed to detect burnt areas automatically using every new SAR image acquired during the wildfires and by exploiting all available pre-fire SAR time series to characterize the temporal backscatter variations. The results show that Sentinel-1 SAR backscatter can detect wildfires and capture their temporal progression as demonstrated for three large and impactful wildfires: the 2017 Elephant Hill Fire in British Columbia, Canada, the 2018 Camp Fire in California, USA, and the 2019 Chuckegg Creek Fire in northern Alberta, Canada. Compared to the traditional log-ratio operator, CNN-based deep learning framework can better distinguish burnt areas with higher accuracy. These findings demonstrate that spaceborne SAR time series with deep learning can play a significant role for near real-time wildfire monitoring when the data becomes available at daily and hourly intervals with the launches of RADARSAT Constellation Missions in 2019, and SAR CubeSat constellations.

PolSAR Image Classification with Lightweight 3D Convolutional Networks

Convolutional neural networks (CNNs) have become the state-of-the-art in optical image processing. Recently, CNNs have been used in polarimetric synthetic aperture radar (PolSAR) image classification and obtained promising results. Unlike optical images, the unique phase information of PolSAR data expresses the structure information of objects. This special data representation makes 3D convolution which explicitly modeling the relationship between polarimetric channels perform better in the task of PolSAR image classification. However, the development of deep 3D-CNNs will cause a huge number of model parameters and expensive computational costs, which not only leads to the decrease of the interpretation speed during testing, but also greatly increases the risk of over-fitting. To alleviate this problem, a lightweight 3D-CNN framework that compresses 3D-CNNs from two aspects is proposed in this paper. Lightweight convolution operations, i.e., pseudo-3D and 3D-depthwise separable convolutions, are considered as low-latency replacements for vanilla 3D convolution. Further, fully connected layers are replaced by global average pooling to reduce the number of model parameters so as to save the memory. Under the specific classification task, the proposed methods can reduce up to 69.83% of the model parameters in convolution layers of the 3D-CNN as well as almost all the model parameters in fully connected layers, which ensures the fast PolSAR interpretation. Experiments on three PolSAR benchmark datasets, i.e., AIRSAR Flevoland, ESAR Oberpfaffenhofen, EMISAR Foulum, show that the proposed lightweight architectures can not only maintain but also slightly improve the accuracy under various criteria.

A Novel Neural Network for Remote Sensing Image Matching

Rapid development of remote sensing (RS) imaging technology makes the acquired images have larger size, higher resolution, and more complex structure, which goes beyond the reach of classical hand-crafted feature-based matching. In this paper, we propose a feature learning approach based on two-branch networks to transform the image matching task into a two-class classification problem. To match two key points, two image patches centered at the key points are entered into the proposed network. The network aims to learn discriminative feature representations for patch matching, so that more matching pairs can be obtained on the premise of maintaining higher subpixel matching accuracy. The proposed network adopts a two-stage training mode to deal with the complex characteristics of RS images. An adaptive sample selection strategy is proposed to determine the size of each patch by the scale of its central key point. Thus, each patch can preserve the texture structure around its key point rather than all patches have a predetermined size. In the matching prediction stage, two strategies, namely, superpixel-based sample graded strategy and superpixel-based ordered spatial matching, are designed to improve the matching efficiency and matching accuracy, respectively. The experimental results and theoretical analysis demonstrate the feasibility, robustness, and effectiveness of the proposed method.

Newly Built Construction Detection in SAR Images Using Deep Learning

Remote sensing data can be utilized to help developing countries monitor the use of land. However, the problem of constant cloud coverage prevents us from taking full advantage of satellite optical images. Therefore, we instead opt to use data from synthetic-aperture radar (SAR), which can capture images of the Earth’s surface regardless of the weather conditions. In this study, we use SAR data to identify newly built constructions. Most studies on change detection tend to detect all of the changes that have a similar temporal change characteristic occurring on two occasions, while we want to identify only the constructions and avoid detecting other changes such as the seasonal change of vegetation. To do so, we study various deep learning network techniques and have decided to propose the fully convolutional network with a skip connection. We train this network with pairs of SAR data acquired on two different occasions from Bangkok and the ground truth, which we manually create from optical images available from Google Earth for all of the SAR pairs. Experiments to assign the most suitable patch size, loss weighting, and epoch number to the network are discussed in this paper. The trained model can be used to generate a binary map that indicates the position of these newly built constructions precisely with the Bangkok dataset, as well as with the Hanoi and Xiamen datasets with acceptable results. The proposed model can even be used with SAR images of the same specific satellite from another orbit direction and still give promising results.

End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++

Change detection (CD) is essential to the accurate understanding of land surface changes using available Earth observation data. Due to the great advantages in deep feature representation and nonlinear problem modeling, deep learning is becoming increasingly popular to solve CD tasks in remote-sensing community. However, most existing deep learning-based CD methods are implemented by either generating difference images using deep features or learning change relations between pixel patches, which leads to error accumulation problems since many intermediate processing steps are needed to obtain final change maps. To address the above-mentioned issues, a novel end-to-end CD method is proposed based on an effective encoder-decoder architecture for semantic segmentation named UNet++, where change maps could be learned from scratch using available annotated datasets. Firstly, co-registered image pairs are concatenated as an input for the improved UNet++ network, where both global and fine-grained information can be utilized to generate feature maps with high spatial accuracy. Then, the fusion strategy of multiple side outputs is adopted to combine change maps from different semantic levels, thereby generating a final change map with high accuracy. The effectiveness and reliability of our proposed CD method are verified on very-high-resolution (VHR) satellite image datasets. Extensive experimental results have shown that our proposed approach outperforms the other state-of-the-art CD methods.

Building Instance Change Detection from Large-Scale Aerial Images using Convolutional Neural Networks and Simulated Samples

We present a novel convolutional neural network (CNN)-based change detection framework for locating changed building instances as well as changed building pixels from very high resolution (VHR) aerial images. The distinctive advantage of the framework is the self-training ability, which is highly important in deep-learning-based change detection in practice, as high-quality samples of changes are always lacking for training a successful deep learning model. The framework consists two parts: a building extraction network to produce a binary building map and a building change detection network to produce a building change map. The building extraction network is implemented with two widely used structures: a Mask R-CNN for object-based instance segmentation, and a multi-scale full convolutional network for pixel-based semantic segmentation. The building change detection network takes bi-temporal building maps produced from the building extraction network as input and outputs a building change map at the object and pixel levels. By simulating arbitrary building changes and various building parallaxes in the binary building map, the building change detection network is well trained without real-life samples. This greatly lowers the requirements of labeled changed buildings, and guarantees the algorithm’s robustness to registration errors caused by parallaxes. To evaluate the proposed method, we chose a wide range of urban areas from an open-source dataset as training and testing areas, and both pixel-based and object-based model evaluation measures were used. Experiments demonstrated our approach was vastly superior: without using any real change samples, it reached 63% average precision (AP) at the object (building instance) level. In contrast, with adequate training samples, other methods—including the most recent CNN-based and generative adversarial network (GAN)-based ones—have only reached 25% AP in their best cases.

An Unsupervised SAR Change Detection Method Based on Stochastic Subspace Ensemble Learning

As synthetic aperture radar (SAR) is playing an increasingly important role in Earth observations, many new methods and technologies have been proposed for change detection using multi-temporal SAR images. Especially with the development of deep learning, numerous methods have been proposed in recent years. However, the requirement to have a certain number of high-quality samples has become one of the main reasons for the limited development of these methods. Thus, in this paper, we propose an unsupervised SAR change detection method that is based on stochastic subspace ensemble learning. The proposed method consists of two stages: The first stage involves the automatic determination of high-confidence samples, which includes a fusion strategy and a refinement process; and the second stage entails using the stochastic subspace ensemble learning module, which contains three steps: obtaining the subsample sets, establishing and training a two-channel network, and applying the prediction results and an ensemble strategy. The subsample sets are used to solve the problem of imbalanced samples. The two-channel networks are used to extract high-dimensional features and learn the relationship between the neighborhood of the pixels in the original images and the labels. Finally, by using an ensemble strategy, the results predicted by all patches reclassified in each network are integrated as the detection result. The experimental results of different SAR datasets prove the effectiveness and the feasibility of the proposed method.

Dual Learning-Based Siamese Framework for Change Detection Using Bi-Temporal VHR Optical Remote Sensing Images

As a fundamental and profound task in remote sensing, change detection from very-high-resolution (VHR) images plays a vital role in a wide range of applications and attracts considerable attention. Current methods generally focus on the research of simultaneously modeling and discriminating the changed and unchanged features. In practice, for bi-temporal VHR optical remote sensing images, the temporal spectral variability tends to exist in all bands throughout the entire paired images, making it difficult to distinguish none-changes and changes with a single model. In this paper, motivated by this observation, we propose a novel hybrid end-to-end framework named dual learning-based Siamese framework (DLSF) for change detection. The framework comprises two parallel streams which are dual learning-based domain transfer and Siamese-based change decision. The former stream is aimed at reducing the domain differences of two paired images and retaining the intrinsic information by translating them into each other’s domain. While the latter stream is aimed at learning a decision strategy to decide the changes in two domains, respectively. By training our proposed framework with certain change map references, this method learns a cross-domain translation in order to suppress the differences of unchanged regions and highlight the differences of changed regions in two domains, respectively, then focus on the detection of changed regions. To the best of our knowledge, the idea of incorporating dual learning framework and Siamese network for change detection is novel. The experimental results on two datasets and the comparison with other state-of-the-art methods verify the efficiency and superiority of our proposed DLSF.

Urban Building Change Detection in SAR Images Using Combined Differential Image and Residual U-Net Network

With the rapid development of urbanization in China, monitoring urban changes is of great significance to city management, urban planning, and cadastral map updating. Spaceborne synthetic aperture radar (SAR) sensors can capture a large area of radar images quickly with fine spatiotemporal resolution and are not affected by weather conditions, making multi-temporal SAR images suitable for change detection. In this paper, a new urban building change detection method based on an improved difference image and residual U-Net network is proposed. In order to overcome the intensity compression problem of the traditional log-ratio method, the spatial distance and intensity similarity are combined to generate a weighting function to obtain a weighted difference image. By fusing the weighted difference image and the bitemporal original images, the three-channel color difference image is generated for building change detection. Due to the complexity of urban environments and the small scale of building changes, the residual U-Net network is used instead of fixed statistical models and the construction and classifier of the network are modified to distinguish between different building changes. Three scenes of Sentinel-1 interferometric wide swath data are used to validate the proposed method. The experimental results and comparative analysis show that our proposed method is effective for urban building change detection and is superior to the original U-Net and SVM method.

Change Detection of Optical Remote Sensing Image Disturbed by Thin Cloud Using Wavelet Coefficient Substitution Algorithm

The detection of changes in optical remote sensing images under the interference of thin clouds is studied for the first time in this paper. First, the optical remote sensing image is subjected to thin cloud removal processing, and then the processed remote sensing image is subjected to image change detection. Based on the analysis of the characteristics of thin cloud images, a method for removing thin clouds based on wavelet coefficient substitution is proposed in this paper. Based on the change in the wavelet coefficient, the high- and low-frequency parts of the remote sensing image are replaced separately, and the low-frequency clouds are suppressed while maintaining the high-frequency detail of the image, which achieves good results. Then, an unsupervised change detection algorithm based on a combined difference graph and fuzzy c-means clustering algorithm (FCM) clustering is applied. First, the image is transformed into a logarithmic domain, and the image is denoised using Frost filtering. Then, the mean ratio method and the difference method are used to obtain two graph difference maps, and the combined difference graph method is used to obtain the final difference image. The experimental results show that the algorithm can effectively solve the problem of image change detection under thin cloud interference.

Change Analysis in Urban Areas Based on Statistical Features and Temporal Clustering Using TerraSAR-X Time-Series Images

The existing unsupervised multitemporal change detection approaches for synthetic aperture radar (SAR) images based on the pixel level usually suffer from the serious influence of speckle noise, and the classification accuracy of temporal change patterns is liable to be affected by the generation method of similarity matrices and the pre-specified cluster number. To address these issues, a novel time-series change detection method with high efficiency is proposed in this paper. Firstly, spatial feature extraction using local statistical information on patches is conducted to reduce the noise and for subsequent temporal grouping. Secondly, a density-based clustering method is adopted to categorize the pixel series in the temporal dimension, in view of its efficiency and robustness. Change detection and classification results are then obtained by a fast differential strategy in the final step. The experimental results and analysis of synthetic and realistic time-series SAR images acquired by TerraSAR-X in urban areas demonstrate the effectiveness of the proposed method, which outperforms other approaches in terms of both qualitative results and quantitative indices of macro F1-scores and micro F1-scores. Furthermore, we make the case that more temporal change information for buildings can be obtained, which includes when the first and last detected change occurred and the frequency of changes.

Fusion Network for Change Detection of High-Resolution Panchromatic Imagery

This paper proposes a fusion network for detecting changes between two high-resolution panchromatic images. The proposed fusion network consists of front- and back-end neural network architectures to generate dual outputs for change detection. Two networks for change detection were applied to handle image- and high-level changes of information, respectively. The fusion network employs single-path and dual-path networks to accomplish low-level and high-level differential detection, respectively. Based on two dual outputs, a two-stage decision algorithm was proposed to efficiently yield the final change detection results. The dual outputs were incorporated into the two-stage decision by operating logical operations. The proposed algorithm was designed to incorporate not only dual network outputs but also neighboring information. In this paper, a new fused loss function was presented to estimate the errors and optimize the proposed network during the learning stage. Based on our experimental evaluation, the proposed method yields a better detection performance than conventional neural network algorithms, with an average area under the curve of 0.9709, percentage correct classification of 99%, and Kappa of 75 for many test datasets.

Change Detection in Remote Sensing Images Based on Image Mapping and a Deep Capsule Network

Homogeneous image change detection research has been well developed, and many methods have been proposed. However, change detection between heterogeneous images is challenging since heterogeneous images are in different domains. Therefore, direct heterogeneous image comparison in the way that we do it is difficult. In this paper, a method for heterogeneous synthetic aperture radar (SAR) image and optical image change detection is proposed, which is based on a pixel-level mapping method and a capsule network with a deep structure. The mapping method proposed transforms an image from one feature space to another feature space. Then, the images can be compared directly in a similarly transformed space. In the mapping process, some image blocks in unchanged areas are selected, and these blocks are only a small part of the image. Then, the weighted parameters are acquired by calculating the Euclidean distances between the pixel to be transformed and the pixels in these blocks. The Euclidean distance calculated according to the weighted coordinates is taken as the pixel gray value in another feature space. The other image is transformed in a similar manner. In the transformed feature space, these images are compared, and the fusion of the two different images is achieved. The two experimental images are input to a capsule network, which has a deep structure. The image fusion result is taken as the training labels. The training samples are selected according to the ratio of the center pixel label and its neighboring pixels’ labels. The capsule network can improve the detection result and suppress noise. Experiments on remote sensing datasets show the final detection results, and the proposed method obtains a satisfactory performance.

Local Restricted Convolutional Neural Network for Change Detection in Polarimetric SAR Images

To detect changed areas in multitemporal polarimetric synthetic aperture radar (SAR) images, this paper presents a novel version of convolutional neural network (CNN), which is named local restricted CNN (LRCNN). CNN with only convolutional layers is employed for change detection first, and then LRCNN is formed by imposing a spatial constraint called local restriction on the output layer of CNN. In the training of CNN/LRCNN, the polarimetric property of SAR image is fully used instead of manual labeled pixels. As a preparation, a similarity measure for polarimetric SAR data is proposed, and several layered difference images (LDIs) of polarimetric SAR images are produced. Next, the LDIs are transformed into discriminative enhanced LDIs (DELDIs). CNN/LRCNN is trained to model these DELDIs by a regression pretraining, and then a classification fine-tuning is conducted with some pseudolabeled pixels obtained from DELDIs. Finally, the change detection result showing changed areas is directly generated from the output of the trained CNN/LRCNN. The relation of LRCNN to the traditional way for change detection is also discussed to illustrate our method from an overall point of view. Tested on one simulated data set and two real data sets, the effectiveness of LRCNN is certified and it outperforms various traditional algorithms. In fact, the experimental results demonstrate that the proposed LRCNN for change detection not only recognizes different types of changed/unchanged data, but also ensures noise insensitivity without losing details in changed areas.