Change Detection in Synthetic Aperture Radar Images Based on Deep Neural Networks

Locality Preservation for Unsupervised Multimodal Change Detection in Remote Sensing Imagery

Multimodal change detection (MCD) is a topic of increasing interest in remote sensing. Due to different imaging mechanisms, the multimodal images cannot be directly compared to detect the changes. In this article, we explore the topological structure of multimodal images and construct the links between class relationships (same/different) and change labels (changed/unchanged) of pairwise superpixels, which are imaging modality-invariant. With these links, we formulate the MCD problem within a mathematical framework termed the locality-preserving energy model (LPEM), which is used to maintain the local consistency constraints embedded in the links: the structure consistency based on feature similarity and the label consistency based on spatial continuity. Because the foundation of LPEM, i.e., the links, is intuitively explainable and universal, the proposed method is very robust across different MCD situations. Noteworthy, LPEM is built directly on the label of each superpixel, so it is a paradigm that outputs the change map (CM) directly without the need to generate intermediate difference image (DI) as most previous algorithms have done. Experiments on different real datasets demonstrate the effectiveness of the proposed method. Source code of the proposed method is made available at https://github.com/yulisun/LPEM.

Commonality Feature Representation Learning for Unsupervised Multimodal Change Detection

The main challenge of multimodal change detection (MCD) is that multimodal bitemporal images (MBIs) cannot be compared directly to identify changes. To overcome this problem, this paper proposes a novel commonality feature representation learning (CFRL) and constructs a CFRL-based unsupervised MCD framework. The CFRL is composed of a Siamese-based encoder and two decoders. First, the Siamese-based encoder can map original MBIs in the same feature space for extracting the representative features of each modality. Then, the two decoders are used to reconstruct the original MBIs by regressing themselves, respectively. Meanwhile, we swap the decoders to reconstruct the pseudo-MBIs to conduct modality alignment. Subsequently, all reconstructed images are input to the Siamese-based encoder again to map them in a same feature space, by which representative features are obtained. On this basis, latent commonality features between MBIs can be extracted by minimizing the distance between these representative features. These latent commonality features are comparable and can be used to identify changes. Notably, the proposed CFRL can be performed simultaneously in two modalities corresponding to MBIs. Therefore, two change magnitude images (CMIs) can be generated simultaneously by measuring the difference between the commonality features of MBIs. Finally, a simple threshold algorithm or a clustering algorithm can be employed to divide CMIs into binary change maps. Extensive experiments on six publicly available MCD datasets show that the proposed CFRL-based framework can achieve superior performance compared with other state-of-the-art approaches.

A semantic segmentation framework with UNet-pyramid for landslide prediction using remote sensing data

Landslides are frequent all over the world, posing serious threats to human life, infrastructure, and economic operations, making them chronic disasters. This study proposes a novel landslide detection methodology that is automated and based on a hybrid deep learning approach. Currently, Deep Learning is constrained by the lack of applicability, lack of data, and low efficiency in landslide detection but with recent advancement in deep learning-based solutions for landslide detection has sparked considerable advantages over traditional techniques. In order to prevent and mitigate disaster, we introduced a hybrid model based on remote sensing technologies such as satellite images. Specifically, the proposed approach consists hybrid U-Net model integrated with a pyramid pooling layer for landslide detection, which uses high-resolution landslide images from the Landslide4Sense dataset. The UNet-Pyramid model has the following modifications: To improve feature acquisition and advancements to strengthen the model's attention U-Net architecture is integrated with the pyramid pooling layers and OBIA technique. The UNet-Pyramid model was trained and validated using labeled images taken from the Landslide4Sense dataset and the validated set using OBIA to improve its efficacy. The overall Precision, Recall, and F1 Score of the UNet-pyramid model for landslide detection are 91%, 84%, and 87%, respectively.

Semisupervised Change Detection Based on Bihierarchical Feature Aggregation and Extraction Network

With the rapid development of remote sensing (RS) technology, high-resolution RS image change detection (CD) has been widely used in many applications. Pixel-based CD techniques are maneuverable and widely used, but vulnerable to noise interference. Object-based CD techniques can effectively utilize the abundant spectrum, texture, shape, and spatial information but easy-to-ignore details of RS images. How to combine the advantages of pixel-based methods and object-based methods remains a challenging problem. Besides, although supervised methods have the capability to learn from data, the true labels representing changed information of RS images are often hard to obtain. To address these issues, this article proposes a novel semisupervised CD framework for high-resolution RS images, which employs small amounts of true labeled data and a lot of unlabeled data to train the CD network. A bihierarchical feature aggregation and extraction network (BFAEN) is designed to achieve the pixelwise together with objectwise feature concatenation feature representation for the comprehensive utilization of the two-level features. In order to alleviate the coarseness and insufficiency of labeled samples, a confident learning algorithm is used to eliminate noisy labels and a novel loss function is designed for training the model using true- and pseudo-labels in a semisupervised fashion. Experimental results on real datasets demonstrate the effectiveness and superiority of the proposed method.

Fusion PCAM R-CNN of Automatic Segmentation for Magnetic Flux Leakage Defects

Magnetic leakage detection technology plays an important role in the long-oil pipeline. Automatic segmentation of defecting images is crucial for the detection of magnetic flux leakage (MFL) works. At present, accurate segmentation for small defects has always been a difficult problem. In contrast to the state-of-the-art MFL detection methodologies based on convolution neural network (CNN), an optimization method is devised in our study by integrating mask region-based CNN (Mask R-CNN) and information entropy constraint (IEC). To be precise, the principal component analysis (PCA) is utilized to improve the feature learning and network segmentation ability of the convolution kernel. The similarity constraint rule of information entropy is proposed to be inserted into the convolution layer in the Mask R-CNN network. The Mask R-CNN optimizes the convolutional kernel with similar weights or higher similarity, meanwhile, the PCA network reduces the dimension of the feature image to reconstruct the original feature vector. As such, the feature extraction of MFL defects is optimized in the convolution check. The research results can be applied in the field of MFL detection.

Filter Pruning by Switching to Neighboring CNNs With Good Attributes

Filter pruning is effective to reduce the computational costs of neural networks. Existing methods show that updating the previous pruned filter would enable large model capacity and achieve better performance. However, during the iterative pruning process, even if the network weights are updated to new values, the pruning criterion remains the same. In addition, when evaluating the filter importance, only the magnitude information of the filters is considered. However, in neural networks, filters do not work individually, but they would affect other filters. As a result, the magnitude information of each filter, which merely reflects the information of an individual filter itself, is not enough to judge the filter importance. To solve the above problems, we propose meta-attribute-based filter pruning (MFP). First, to expand the existing magnitude information-based pruning criteria, we introduce a new set of criteria to consider the geometric distance of filters. Additionally, to explicitly assess the current state of the network, we adaptively select the most suitable criteria for pruning via a meta-attribute, a property of the neural network at the current state. Experiments on two image classification benchmarks validate our method. For ResNet-50 on ILSVRC-2012, we could reduce more than 50% FLOPs with only 0.44% top-5 accuracy loss.

Evaluating Urban Building Damage of 2023 Kahramanmaras, Turkey Earthquake Sequence Using SAR Change Detection

On February 6, 2023 (local time), two earthquakes (Mw7.8 and Mw7.7) struck central and southern Turkey, causing extensive damage to several cities and claiming a toll of 40,000 lives. In this study, we propose a method for seismic building damage assessment and analysis by combining SAR amplitude and phase coherence change detection. We determined building damage in five severely impacted urban areas and calculated the damage ratio by measuring the urban area and the damaged area. The largest damage ratio of 18.93% is observed in Nurdagi, and the smallest ratio of 7.59% is found in Islahiye. We verified the results by comparing them with high-resolution optical images and AI recognition results from the Microsoft team. We also used pixel offset tracking (POT) technology and D-InSAR technology to obtain surface deformation using Sentinel-1A images and analyzed the relationship between surface deformation and post-earthquake urban building damage. The results show that Nurdagi has the largest urban average surface deformation of 0.48 m and Antakya has the smallest deformation of 0.09 m. We found that buildings in the areas with steeper slopes or closer to earthquake faults have higher risk of collapse. We also discussed the influence of SAR image parameters on building change recognition. Image resolution and observation geometry have a great influence on the change detection results, and the resolution can be improved by various means to raise the recognition accuracy. Our research findings can guide earthquake disaster assessment and analysis and identify influential factors of earthquake damage.

Comparative Analyses of Unsupervised PCA K-Means Change Detection Algorithm from the Viewpoint of Follow-Up Plan

In this study, principal component analysis and k-means clustering (PCAKM) methods for synthetic aperture radar (SAR) data are analyzed to reduce the sensitivity caused by changes in the parameters and input images of the algorithm, increase the accuracy, and make an improvement in the computation time, which are advantageous for scoring in the follow-up plan. Although there are many supervised methods described in the literature, unsupervised methods may be more appropriate in terms of computing time, data scarcity, and explainability in order to supply a trustworthy system. We consider the PCAKM algorithm, which is used as a benchmark method in many studies when making comparisons. Error metrics, computing times, and utility functions are calculated for 22 modified PCAKM regarding difference images and filtering methods. Various images with different characteristics affect the results of the configurations. However, it is evident that the PCAKM becomes less sensitive and more accurate for both the overall results and image results. Scoring by utilizing these results and other map information is a gap and innovation. Obtaining a change map in a fast, explainable, more robust and less sensitive way is one of the aims of our studies on scoring points in the follow-up plan.

Unsupervised Change Detection in Multitemporal VHR Images Based on Deep Kernel PCA Convolutional Mapping Network

With the development of Earth observation technology, a very-high-resolution (VHR) image has become an important data source of change detection (CD). These days, deep learning (DL) methods have achieved conspicuous performance in the CD of VHR images. Nonetheless, most of the existing CD models based on DL require annotated training samples. In this article, a novel unsupervised model, called kernel principal component analysis (KPCA) convolution, is proposed for extracting representative features from multitemporal VHR images. Based on the KPCA convolution, an unsupervised deep siamese KPCA convolutional mapping network (KPCA-MNet) is designed for binary and multiclass CD. In the KPCA-MNet, the high-level spatial-spectral feature maps are extracted by a deep siamese network consisting of weight-shared KPCA convolutional layers. Then, the change information in the feature difference map is mapped into a 2-D polar domain. Finally, the CD results are generated by threshold segmentation and clustering algorithms. All procedures of KPCA-MNet do not require labeled data. The theoretical analysis and experimental results in two binary CD datasets and one multiclass CD datasets demonstrate the validity, robustness, and potential of the proposed method.

DPFL-Nets: Deep Pyramid Feature Learning Networks for Multiscale Change Detection

Due to the complementary properties of different types of sensors, change detection between heterogeneous images receives increasing attention from researchers. However, change detection cannot be handled by directly comparing two heterogeneous images since they demonstrate different image appearances and statistics. In this article, we propose a deep pyramid feature learning network (DPFL-Net) for change detection, especially between heterogeneous images. DPFL-Net can learn a series of hierarchical features in an unsupervised fashion, containing both spatial details and multiscale contextual information. The learned pyramid features from two input images make unchanged pixels matched exactly and changed ones dissimilar and after transformed into the same space for each scale successively. We further propose fusion blocks to aggregate multiscale difference images (DIs), generating an enhanced DI with strong separability. Based on the enhanced DI, unchanged areas are predicted and used to train DPFL-Net in the next iteration. In this article, pyramid features and unchanged areas are updated alternately, leading to an unsupervised change detection method. In the feature transformation process, local consistency is introduced to constrain the learned pyramid features, modeling the correlations between the neighboring pixels and reducing the false alarms. Experimental results demonstrate that the proposed approach achieves superior or at least comparable results to the existing state-of-the-art change detection methods in both homogeneous and heterogeneous cases.

Commonality Autoencoder: Learning Common Features for Change Detection From Heterogeneous Images

Change detection based on heterogeneous images, such as optical images and synthetic aperture radar images, is a challenging problem because of their huge appearance differences. To combat this problem, we propose an unsupervised change detection method that contains only a convolutional autoencoder (CAE) for feature extraction and the commonality autoencoder for commonalities exploration. The CAE can eliminate a large part of redundancies in two heterogeneous images and obtain more consistent feature representations. The proposed commonality autoencoder has the ability to discover common features of ground objects between two heterogeneous images by transforming one heterogeneous image representation into another. The unchanged regions with the same ground objects share much more common features than the changed regions. Therefore, the number of common features can indicate changed regions and unchanged regions, and then a difference map can be calculated. At last, the change detection result is generated by applying a segmentation algorithm to the difference map. In our method, the network parameters of the commonality autoencoder are learned by the relevance of unchanged regions instead of the labels. Our experimental results on five real data sets demonstrate the promising performance of the proposed framework compared with several existing approaches.

Effects of Anesthetics on Proliferation and Apoptosis of Drug-Resistant Human Colon Cancer Cells

In recent years, people's living standards are getting higher and higher, and life pressure is also increasing, and there are also many problems in eating habits. This is also the direct cause of colon cancer. The aim of this paper was to investigate whether anesthetic drugs could positively affect the proliferation and apoptosis of colon cancer cells. In this paper, the significance of anesthetic drugs is proposed, and an artificial neural network algorithm based on artificial intelligence is proposed. It is well known that artificial neural networks play an important role in medicine. The experimental results of this paper show that the incidence of colon cancer in 2020 will be in the range of 5%-35%, and the incidence of colon cancer in 2021 will be in the range of 7%-30%. While colon cancer rates in 2021 do not appear to be as high as colon cancer rates in 2020, they are generally much higher than colon cancer rates in 2020. It can be seen that as the population ages, the number of colon cancer patients is increasing due to the lack of emphasis on health. This also means that the incidence of colon cancer is getting higher and higher, and traditional drug chemotherapy has been unable to play a good role in inhibiting the proliferation of colon cancer cells. Therefore, this paper investigated the effects of anesthetic drugs on the proliferation and apoptosis of human colon cancer cells.

Changed Detection Based on Patch Robust Principal Component Analysis

Change detection on retinal fundus image pairs mainly seeks to compare the important differences between a pair of images obtained at two different time points such as in anatomical structures or lesions. Illumination variation usually challenges the change detection methods in many cases. Robust principal component analysis (RPCA) takes intensity normalization and linear interpolation to greatly reduce the illumination variation between the continuous frames and then decomposes the image matrix to obtain the robust background model. The matrix-RPCA can obtain clear change regions, but when there are local bright spots on the image, the background model is vulnerable to illumination, and the change detection results are inaccurate. In this paper, a patch-based RPCA (P-RPCA) is proposed to detect the change of fundus image pairs, where a pair of fundus images is normalized and linearly interpolated to expand a low-rank image sequence; then, images are divided into many patches to obtain an image-patch matrix, and finally, the change regions are obtained by the low-rank decomposition. The proposed method is validated on a set of large lesion image pairs in clinical data. The area under curve (AUC) and mean average precision (mAP) of the method proposed in this paper are 0.9832 and 0.8641, respectively. For a group of small lesion image pairs with obvious local illumination changes in clinical data, the AUC and mAP obtained by the P-RPCA method are 0.9893 and 0.9401, respectively. The results show that the P-RPCA method is more robust to local illumination changes than the RPCA method, and has stronger performance in change detection than the RPCA method.

Change Detection Based on Fusion Difference Image and Multi-Scale Morphological Reconstruction for SAR Images

Synthetic aperture radar (SAR) image-change detection is widely used in various fields, such as environmental monitoring and ecological monitoring. There is too much noise and insufficient information utilization, which make the results of change detection inaccurate. Thus, we propose an SAR image-change-detection method based on multiplicative fusion difference image (DI), saliency detection (SD), multi-scale morphological reconstruction (MSMR), and fuzzy c-means (FCM) clustering. Firstly, a new fusion DI method is proposed by multiplying the ratio (R) method based on the ratio of the image before and after the change and the mean ratio (MR) method based on the ratio of the image neighborhood mean value. The new DI operator ratio–mean ratio (RMR) enlarges the characteristics of unchanged areas and changed areas. Secondly, saliency detection is used in DI, which is conducive to the subsequent sub-area processing. Thirdly, we propose an improved FCM clustering-change-detection method based on MSMR. The proposed method has high computational efficiency, and the neighborhood information obtained by morphological reconstruction is fully used. Six real SAR data sets are used in different experiments to demonstrate the effectiveness of the proposed saliency ratio–mean ratio with multi-scale morphological reconstruction fuzzy c-means (SRMR-MSMRFCM). Finally, four classical noise-sensitive methods are used to detect our DI method and demonstrate the strong denoising and detail-preserving ability.

Multi-scale feature progressive fusion network for remote sensing image change detection

Presently, research on deep learning-based change detection (CD) methods has become a hot topic. In particular, feature pyramid networks (FPNs) are widely used in CD tasks to gradually fuse semantic features. However, existing FPN-based CD methods do not correctly detect the complete change region and cannot accurately locate the boundaries of the change region. To solve these problems, a new Multi-Scale Feature Progressive Fusion Network (MFPF-Net) is proposed, which consists of three innovative modules: Layer Feature Fusion Module (LFFM), Multi-Scale Feature Aggregation Module (MSFA), and Multi-Scale Feature Distribution Module (MSFD). Specifically, we first concatenate the features of each layer extracted from the bi-temporal images with their difference maps, and the resulting change maps fuse richer semantic information while effectively representing change regions. Then, the obtained change maps of each layer are directly aggregated, which improves the effective communication and full fusion of feature maps in CD while avoiding the interference of indirect information. Finally, the aggregated feature maps are layered again by pooling and convolution operations, and then a feature fusion strategy with a pyramid structure is used, with layers fused from low to high, to obtain richer contextual information, so that each layer of the layered feature maps has original semantic information and semantic features of other layers. We conducted comprehensive experiments on three publicly available benchmark datasets, CDD, LEVIR-CD, and WHU-CD to verify the effectiveness of the method, and the experimental results show that the method in this paper outperforms other comparative methods.

Effect of Bodybuilding and Fitness Exercise on Physical Fitness Based on Deep Learning

With the rapid development of society and economy, people's living standards are improving day by day, and increasingly attention is paid to physical health, which has set off a fitness upsurge. The purpose of this paper was to analyze the impact of bodybuilding exercise on physical fitness based on deep learning. It provides a reference for fitness enthusiasts to choose scientific and targeted exercise methods, and provides a theoretical basis for the promotion of bodybuilding and fitness. This paper first gives a general introduction to deep learning and adds image segmentation technology to design experiments for bodybuilding and fitness. The experiment was divided into groups A and B, and control group C. In this paper, recurrent neural network and gated recurrent neural network are introduced to compare and analyze the data, and the stability of data processing with different activation functions is compared. The data results show that under the scientific and reasonable arrangement of exercise conditions, bodybuilding and fitness exercises have a corresponding positive effect on the body shape and posture of the subjects. It is more practical to choose a combination of aerobic and anaerobic exercise. In this paper, based on the deep learning algorithm, compared with the recurrent neural network, the gated recurrent neural network is more suitable for processing sequence problems. In the experimental analysis part, this paper compares and analyzes the experimental results of the data under different activation functions, sigmoid function, and tanh function. It is found that the tanh activation function and the gated recurrent neural network are more stable for data processing. The highest AUC value of the traditional recurrent neural network differs by 0.78 from the highest AUC value of the gated recurrent neural network. The data analysis results are in line with the actual situation.

An end-to-end laser-induced damage change detection approach for optical elements via siamese network and multi-layer perceptrons

With the presence of complex background noise, parasitic light, and dust attachment, it is still a challenging issue to perform high-precision laser-induced damage change detection of optical elements in the captured optical images. For resolving this problem, this paper presents an end-to-end damage change detection model based on siamese network and multi-layer perceptrons (SiamMLP). Firstly, representative features of bi-temporal damage images are efficiently extracted by the cascaded multi-layer perceptron modules in the siamese network. After that, the extracted features are concatenated and then classified into changed and unchanged classes. Due to its concise architecture and strong feature representation ability, the proposed method obtains excellent damage change detection results efficiently and effectively. To address the unbalanced distribution of hard and easy samples, a novel metric called hard metric is introduced in this paper for quantitatively evaluating the classification difficulty degree of the samples. The hard metric assigns a classification difficulty for each individual sample to precisely adjust the loss assigned to the sample. In the training stage, a novel hard loss is presented to train the proposed model. Cooperating with the hard metric, the hard loss can up-weight the loss of hard samples and down-weight the loss of easy samples, which results in a more powerful online hard sample mining ability of the proposed model. The experimental results on two real datasets validate the effectiveness and superiority of the proposed method.

Change detection based on unsupervised sparse representation for fundus image pair

Detecting changes is an important issue for ophthalmology to compare longitudinal fundus images at different stages and obtain change regions. Illumination variations bring distractions on the change regions by the pixel-by-pixel comparison. In this paper, a new unsupervised change detection method based on sparse representation classification (SRC) is proposed for the fundus image pair. First, the local neighborhood patches are extracted from the reference image to build a dictionary of the local background. Then the current image patch is represented sparsely and its background is reconstructed by the obtained dictionary. Finally, change regions are given through background subtracting. The SRC method can correct automatically illumination variations through the representation coefficients and filter local contrast and global intensity effectively. In experiments of this paper, the AUC and mAP values of SRC method are 0.9858 and 0.8647 respectively for the image pair with small lesions; the AUC and mAP values of the fusion method of IRHSF and SRC are 0.9892 and 0.9692 separately for the image pair with the big change region. Experiments show that the proposed method in this paper is more robust than RPCA for the illumination variations and can detect change regions more effectively than pixel-wised image differencing.

Intelligent Performance Evaluation of Urban Subway PPP Project Based on Deep Neural Network Model

Neural network refers to an algorithmic mathematical model that imitates the behavioral characteristics of animal neural networks and performs distributed information processing. This kind of network depends on the complexity of the system and needs to adjust the internal node relationship, so as to achieve the purpose of processing information. With the continuous development of the economy, the transportation problem needs to be solved urgently, and the urban subway has emerged at the historic moment. The subway is a fast, large-capacity, electric-driven rail transit built in the city. The advantages of the subway provide conditions for the mitigation of urban traffic, due to the large number of cars, traffic jams, frequent accidents, and serious environmental pollution. In the city center, there are more cars and less space, and the parking lot is not commensurate with the number of cars, making parking difficult. This paper aims to study the intelligent performance evaluation of urban subway PPP projects based on deep neural network models. The subway project has a large investment, a long period, and a wide range, but the development time of the subway in China is relatively short. In order to promote the stable progress of subway projects, it is very necessary to conduct intelligent performance evaluation on subway projects. This paper compares and analyzes the basic characteristics of the PPP model and verifies the applicability and necessity of the PPP model in urban subway transportation projects. Through the investigation of relevant literature, this article puts forward the research content of the social impact assessment of subway projects. The experimental results of this paper show that, from the perspective of whether it is necessary to evaluate the performance of PPP projects, 65% of people think it is very necessary, and 22% think it is more necessary. 3% of people think it is unnecessary, and 10% of people hold an indifferent attitude. These data show that the intelligent performance evaluation of urban subway PPP projects has exploratory significance for urban infrastructure design and construction.

Biophysical Model: A Promising Method in the Study of the Mechanism of Propofol: A Narrative Review

The physiological and neuroregulatory mechanism of propofol is largely based on very limited knowledge. It is one of the important puzzling issues in anesthesiology and is of great value in both scientific and clinical fields. It is acknowledged that neural networks which are comprised of a number of neural circuits might be involved in the anesthetic mechanism. However, the mechanism of this hypothesis needs to be further elucidated. With the progress of artificial intelligence, it is more likely to solve this problem through using artificial neural networks to perform temporal waveform data analysis and to construct biophysical computational models. This review focuses on current knowledge regarding the anesthetic mechanism of propofol, an intravenous general anesthetic, by constructing biophysical computational models.

Parallel Bookkeeping Path of Accounting in Government Accounting System Based on Deep Neural Network

“Parallel bookkeeping” is a key technical arrangement to achieve the goal of moderately separating and connecting the financial accounting system and budget accounting system established by the government accounting system. It is still a new thing for the majority of financial personnel in the government accounting subject. A deep neural network is the basis of deep learning. Up to now, the neural network has been applied in many fields, and its application in the financial field is more in-depth. The neural network is of great help to financial accounting. Integrating it into parallel bookkeeping in accounting can improve the work efficiency and accuracy of financial personnel. Through experimental analysis, it is found that its efficiency and accuracy are improved by 45% and 21.34% compared with the previous parallel bookkeeping path. The accounting parallel bookkeeping path based on the deep neural network studied in this paper not only has great practical significance for the work of financial personnel but also has far-reaching significance for the research of accounting paths in the future.

MSGATN: A Superpixel-Based Multi-Scale Siamese Graph Attention Network for Change Detection in Remote Sensing Images

With the rapid development of Earth observation technology, how to effectively and efficiently detect changes in multi-temporal images has become an important but challenging problem. Relying on the advantages of high performance and robustness, object-based change detection (CD) has become increasingly popular. By analyzing the similarity of local pixels, object-based CD aggregates similar pixels into one object and takes it as the basic processing unit. However, object-based approaches often have difficulty capturing discriminative features, as irregular objects make processing difficult. To address this problem, in this paper, we propose a novel superpixel-based multi-scale Siamese graph attention network (MSGATN) which can process unstructured data natively and extract valuable features. First, a difference image (DI) is generated by Euclidean distance between bitemporal images. Second, superpixel segmentation is employed based on DI to divide each image into many homogeneous regions. Then, these superpixels are used to model the problem by graph theory to construct a series of nodes with the adjacency between them. Subsequently, the multi-scale neighborhood features of the nodes are extracted through applying a graph convolutional network and concatenated by an attention mechanism. Finally, the binary change map can be obtained by classifying each node by some fully connected layers. The novel features of MSGATN can be summarized as follows: (1) Training in multi-scale constructed graphs improves the recognition over changed land cover of varied sizes and shapes. (2) Spectral and spatial self-attention mechanisms are exploited for a better change detection performance. The experimental results on several real datasets show the effectiveness and superiority of the proposed method. In addition, compared to other recent methods, the proposed can demonstrate very high processing efficiency and greatly reduce the dependence on labeled training samples in a semisupervised training fashion.

Design of Financial Risk Control Model Based on Deep Learning Neural Network

In recent years, with the continuous increase of financial business, the risk of business is on the rise. Among them, major risk cases are frequent, the cases are increasingly complex, and the means of committing crimes are concealed. The main research contents of this paper include the preprocessing of internal and external financial data and the structure design of recurrent NNs. Its purpose is to design a financial risk control model based on a deep learning NNs, thereby reducing financial risk. The Borderline-SMOTE algorithm is used first to preprocess the sample data, and the oversampling method is used to eliminate the imbalance of the data, and then, the long short-term memory deep NNs algorithm is introduced to process the sample data with time series characteristics. The final experiment shows that LSTM has a better accuracy, reaching 0.9715, compared with traditional methods; the sample preprocessing method and risk control model proposed in this paper have better ability to identify fraudulent customers, and the model itself has faster iteration efficiency.

Detection of Flood Extent Using Sentinel-1A/B Synthetic Aperture Radar: An Application for Hurricane Harvey, Houston, TX

The increasing number of flood events combined with coastal urbanization has contributed to significant economic losses and damage to buildings and infrastructure. Development of higher resolution SAR flood mapping that accurately identifies flood features at all scales can be incorporated into operational flood forecasting tools, improving response and resilience to large flood events. Here, we present a comparison of several methods for characterizing flood inundation using a combination of synthetic aperture radar (SAR) remote sensing data and machine learning methods. We implement two applications with SAR GRD data, an amplitude thresholding technique applied, for the first time, to Sentinel-1A/B SAR data, and a machine learning technique, DeepLabv3+. We also apply DeepLabv3+ to a false color RGB characterization of dual polarization SAR data. Analyses at 10 m pixel spacing are performed for the major flood event associated with Hurricane Harvey and associated inundation in Houston, TX in August of 2017. We compare these results with high-resolution aerial optical images over this time period, acquired by the NOAA Remote Sensing Division. We compare the results with NDWI produced from Sentinel-2 images, also at 10 m pixel spacing, and statistical testing suggests that the amplitude thresholding technique is the most effective, although the machine learning analysis is successful at reproducing the inundation shape and extent. These results demonstrate the effectiveness of flood inundation mapping at unprecedented resolutions and its potential for use in operational emergency hazard response to large flood events.

R-IMNet: Spatial-Temporal Evolution Analysis of Resource-Exhausted Urban Land Based on Residual-Intelligent Module Network

The transformation of resource-exhausted urban land is an urgent problem for sustainable urban development in the world today. Obtaining the urban land use type and analyzing the changes in their land use can lead to better management of the relationship between economic development and resource utilization. In this paper, a residual-intelligent module network was proposed to solve the problems of low classification accuracy and missing objects edge information in traditional computer classification methods. The classification of four Landsat-TM/OLI images from 1993–2020 for Jiaozuo city (the first batch of resource-exhausted cities in China) was realized by this method. The results (overall accuracy was 98.61%, in 2020 images) were better than the comparison models (support vector machine, 2D-convolutional neural network, hybrid convolution networks; overall accuracy was 87.12%, 96.16%, 98.46%, respectively) and effectively reduced the loss of information on the edge of the ground objects. On this basis, six main land use types were constructed by combining field surveys and other methods. The characteristics and driving forces of spatial-temporal change in land use were explored from the aspect of social, economic and policy factors. The results showed that from 1993 to 2020 the cultivated land, forest land, water body and other land types in the study area decreased by 690.97 km2, 57.54 km2, 47.04 km2 and 59.43 km2, respectively. The construction land and bare land increased by 839.38 km2 and 15.57 km2, respectively. The transfer of land use types was mainly from cultivated land to construction land, with a cumulative conversion of 920.95 km2 within 27 years. The driving forces of land use in the study area were analyzed by principal component analysis (PCA) and regression analysis. The spatial-temporal evolution of land use types was affected by policy changes, the level of social development and the adjustment in the economy, industry and agriculture structure. The investment in fixed assets and per capita net income in rural areas were the top two influencing factors and their cumulative contribution rate was 94.62%. The findings of this study can provide scientific reference and theoretical support for land use planning, land reclamation in mining areas, ecological protection and sustainable development in Jiaozuo and other resource-exhausted cities in the world.

A Survey on Deep Learning-Based Change Detection from High-Resolution Remote Sensing Images

Change detection based on remote sensing images plays an important role in the field of remote sensing analysis, and it has been widely used in many areas, such as resources monitoring, urban planning, disaster assessment, etc. In recent years, it has aroused widespread interest due to the explosive development of artificial intelligence (AI) technology, and change detection algorithms based on deep learning frameworks have made it possible to detect more delicate changes (such as the alteration of small buildings) with the help of huge amounts of remote sensing data, especially high-resolution (HR) data. Although there are many methods, we still lack a deep review of the recent progress concerning the latest deep learning methods in change detection. To this end, the main purpose of this paper is to provide a review of the available deep learning-based change detection algorithms using HR remote sensing images. The paper first describes the change detection framework and classifies the methods from the perspective of the deep network architectures adopted. Then, we review the latest progress in the application of deep learning in various granularity structures for change detection. Further, the paper provides a summary of HR datasets derived from different sensors, along with information related to change detection, for the potential use of researchers. Simultaneously, representative evaluation metrics for this task are investigated. Finally, a conclusion of the challenges for change detection using HR remote sensing images, which must be dealt with in order to improve the model’s performance, is presented. In addition, we put forward promising directions for future research in this area.

Adaptive Contourlet Fusion Clustering for SAR Image Change Detection

In this paper, a novel unsupervised change detection method called adaptive Contourlet fusion clustering based on adaptive Contourlet fusion and fast non-local clustering is proposed for multi-temporal synthetic aperture radar (SAR) images. A binary image indicating changed regions is generated by a novel fuzzy clustering algorithm from a Contourlet fused difference image. Contourlet fusion uses complementary information from different types of difference images. For unchanged regions, the details should be restrained while highlighted for changed regions. Different fusion rules are designed for low frequency band and high frequency directional bands of Contourlet coefficients. Then a fast non-local clustering algorithm (FNLC) is proposed to classify the fused image to generate changed and unchanged regions. In order to reduce the impact of noise while preserve details of changed regions, not only local but also non-local information are incorporated into the FNLC in a fuzzy way. Experiments on both small and large scale datasets demonstrate the state-of-the-art performance of the proposed method in real applications.

Application of an Improved Clustering Algorithm of Neural Networks in Performance Appraisal Systems

With the development of economic globalization, human resource competition has long become the key core of enterprise development and peer competition. Reasonably Formulating an enterprise’s employee performance appraisal management system and conducting standardized, fair, and just appraisal management are the basic requirements for the survival and development of an enterprise. This paper studies the application of an improved clustering algorithm based on neural network in an employee performance appraisal management system and explores its application value in the employee performance appraisal management system by using the improved ART2 clustering method that draws on leakage competition and Hebb rules. The experimental results of this paper show that the satisfaction of this system in the four aspects of integrated data management, system stability and convenience, and transparency in performance appraisal are all above 66%. This shows that this system has superior performance and good reference value.

A Mutual Teaching Framework with Momentum Correction for Unsupervised Hyperspectral Image Change Detection

Deep-learning methods rely on massive labeled data, which has become one of the main impediments in hyperspectral image change detection (HSI-CD). To resolve this problem, pseudo-labels generated by traditional methods are widely used to drive model learning. In this paper, we propose a mutual teaching approach with momentum correction for unsupervised HSI-CD to cope with noise in pseudo-labels, which is harmful for model training. First, we adopt two structurally identical models simultaneously, allowing them to select high-confidence samples for each other to suppress self-confidence bias, and continuously update pseudo-labels during iterations to fine-tune the models. Furthermore, a new group confidence-based sample filtering method is designed to obtain reliable training samples for HSI. This method considers both the quality and diversity of the selected samples by determining the confidence of each group instead of single instances. Finally, to better extract the spatial–temporal spectral features of bitemporal HSIs, a 3D convolutional neural network (3DCNN) is designed as an HSI-CD classifier and the basic network of our framework. Due to mutual teaching and dynamic label learning, pseudo-labels can be continuously updated and refined in iterations, and thus, the proposed method can achieve a better performance compared with those with fixed pseudo-labels. Experimental results on several HSI datasets demonstrate the effectiveness of our method.

Deep Learning-Based Change Detection in Remote Sensing Images: A Review

Images gathered from different satellites are vastly available these days due to the fast development of remote sensing (RS) technology. These images significantly enhance the data sources of change detection (CD). CD is a technique of recognizing the dissimilarities in the images acquired at distinct intervals and are used for numerous applications, such as urban area development, disaster management, land cover object identification, etc. In recent years, deep learning (DL) techniques have been used tremendously in change detection processes, where it has achieved great success because of their practical applications. Some researchers have even claimed that DL approaches outperform traditional approaches and enhance change detection accuracy. Therefore, this review focuses on deep learning techniques, such as supervised, unsupervised, and semi-supervised for different change detection datasets, such as SAR, multispectral, hyperspectral, VHR, and heterogeneous images, and their advantages and disadvantages will be highlighted. In the end, some significant challenges are discussed to understand the context of improvements in change detection datasets and deep learning models. Overall, this review will be beneficial for the future development of CD methods.

Dual-Path Deep Fusion Network for Face Image Hallucination

Along with the performance improvement of deep-learning-based face hallucination methods, various face priors (facial shape, facial landmark heatmaps, or parsing maps) have been used to describe holistic and partial facial features, making the cost of generating super-resolved face images expensive and laborious. To deal with this problem, we present a simple yet effective dual-path deep fusion network (DPDFN) for face image super-resolution (SR) without requiring additional face prior, which learns the global facial shape and local facial components through two individual branches. The proposed DPDFN is composed of three components: a global memory subnetwork (GMN), a local reinforcement subnetwork (LRN), and a fusion and reconstruction module (FRM). In particular, GMN characterize the holistic facial shape by employing recurrent dense residual learning to excavate wide-range context across spatial series. Meanwhile, LRN is committed to learning local facial components, which focuses on the patch-wise mapping relations between low-resolution (LR) and high-resolution (HR) space on local regions rather than the entire image. Furthermore, by aggregating the global and local facial information from the preceding dual-path subnetworks, FRM can generate the corresponding high-quality face image. Experimental results of face hallucination on public face data sets and face recognition on real-world data sets (VGGface and SCFace) show the superiority both on visual effect and objective indicators over the previous state-of-the-art methods.

Change Detection in Synthetic Aperture Radar Images Based on a Generalized Gamma Deep Belief Networks

Change detection from synthetic aperture radar (SAR) images is of great significance for natural environmental protection and human societal activity, which can be regarded as the process of assigning a class label (changed or unchanged) to each of the image pixels. This paper presents a novel classification technique to address the SAR change-detection task that employs a generalized Gamma deep belief network (gΓ-DBN) to learn features from difference images. We aim to develop a robust change detection method that can adapt to different types of scenarios for bitemporal co-registered Yellow River SAR image data set. This data set characterized by different looks, which means that the two images are affected by different levels of speckle. Widely used probability distributions offer limited accuracy for describing the opposite class pixels of difference images, making change detection entail greater difficulties. To address the issue, first, a gΓ-DBN can be constructed to extract the hierarchical features from raw data and fit the distribution of the difference images by means of a generalized Gamma distribution. Next, we propose learning the stacked spatial and temporal information extracted from various difference images by the gΓ-DBN. Consequently, a joint high-level representation can be effectively learned for the final change map. The visual and quantitative analysis results obtained on the Yellow River SAR image data set demonstrate the effectiveness and robustness of the proposed method.

Attention-Guided Siamese Fusion Network for Change Detection of Remote Sensing Images

Change detection for remote sensing images is an indispensable procedure for many remote sensing applications, such as geological disaster assessment, environmental monitoring, and urban development monitoring. Through this technique, the difference in certain areas after some emergencies can be determined to estimate their influence. Additionally, by analyzing the sequential difference maps, the change tendency can be found to help to predict future changes, such as urban development and environmental pollution. The complex variety of changes and interferential changes caused by imaging processing, such as season, weather and sensors, are critical factors that affect the effectiveness of change detection methods. Recently, there have been many research achievements surrounding this topic, but a perfect solution to all the problems in change detection has not yet been achieved. In this paper, we mainly focus on reducing the influence of imaging processing through the deep neural network technique with limited labeled samples. The attention-guided Siamese fusion network is constructed based on one basic Siamese network for change detection. In contrast to common processing, besides high-level feature fusion, feature fusion is operated during the whole feature extraction process by using an attention information fusion module. This module can not only realize the information fusion of two feature extraction network branches, but also guide the feature learning network to focus on feature channels with high importance. Finally, extensive experiments were performed on three public datasets, which could verify the significance of information fusion and the guidance of the attention mechanism during feature learning in comparison with related methods.

CANet: A Combined Attention Network for Remote Sensing Image Change Detection

Change detection (CD) is one of the essential tasks in remote sensing image processing and analysis. Remote sensing CD is a process of determining and evaluating changes in various surface objects over time. The impressive achievements of deep learning in image processing and computer vision provide an innovative concept for the task of CD. However, existing methods based on deep learning still have problems detecting small changed regions correctly and distinguishing the boundaries of the changed regions. To solve the above shortcomings and improve the efficiency of CD networks, inspired by the fact that an attention mechanism can refine features effectively, we propose an attention-based network for remote sensing CD, which has two important components: an asymmetric convolution block (ACB) and a combined attention mechanism. First, the proposed method extracts the features of bi-temporal images, which contain two parallel encoders with shared weights and structures. Then, the feature maps are fed into the combined attention module to reconstruct the change maps and obtain refined feature maps. The proposed CANet is evaluated on the two publicly available datasets for challenging remote sensing image CD. Extensive empirical results with four popular metrics show that the designed framework yields a robust CD detector with good generalization performance. In the CDD and LEVIR-CD datasets, the F1 values of the CANet are 3.3% and 1.3% higher than those of advanced CD methods, respectively. A quantitative analysis and qualitative comparison indicate that our method outperforms competitive baselines in terms of both effectiveness and robustness.

Deep Siamese Networks Based Change Detection with Remote Sensing Images

Although considerable success has been achieved in change detection on optical remote sensing images, accurate detection of specific changes is still challenging. Due to the diversity and complexity of the ground surface changes and the increasing demand for detecting changes that require high-level semantics, we have to resort to deep learning techniques to extract the intrinsic representations of changed areas. However, one key problem for developing deep learning metho for detecting specific change areas is the limitation of annotated data. In this paper, we collect a change detection dataset with 862 labeled image pairs, where the urban construction-related changes are labeled. Further, we propose a supervised change detection method based on a deep siamese semantic segmentation network to handle the proposed data effectively. The novelty of the method is that the proposed siamese network treats the change detection problem as a binary semantic segmentation task and learns to extract features from the image pairs directly. The siamese architecture as well as the elaborately designed semantic segmentation networks significantly improve the performance on change detection tasks. Experimental results demonstrate the promising performance of the proposed network compared to existing approaches.

Individualized Glaucoma Change Detection Using Deep Learning Auto Encoder-Based Regions of Interest

Purpose

To compare change over time in eye-specific optical coherence tomography (OCT) retinal nerve fiber layer (RNFL)-based region-of-interest (ROI) maps developed using unsupervised deep-learning auto-encoders (DL-AE) to circumpapillary RNFL (cpRNFL) thickness for the detection of glaucomatous progression.

Methods

Forty-four progressing glaucoma eyes (by stereophotograph assessment), 189 nonprogressing glaucoma eyes (by stereophotograph assessment), and 109 healthy eyes were followed for ≥3 years with ≥4 visits using OCT. The San Diego Automated Layer Segmentation Algorithm was used to automatically segment the RNFL layer from raw three-dimensional OCT images. For each longitudinal series, DL-AEs were used to generate individualized eye-based ROI maps by identifying RNFL regions of likely progression and no change. Sensitivities and specificities for detecting change over time and rates of change over time were compared for the DL-AE ROI and global cpRNFL thickness measurements derived from a 2.22-mm to 3.45-mm annulus centered on the optic disc.

Results

The sensitivity for detecting change in progressing eyes was greater for DL-AE ROIs than for global cpRNFL annulus thicknesses (0.90 and 0.63, respectively). The specificity for detecting not likely progression in nonprogressing eyes was similar (0.92 and 0.93, respectively). The mean rates of change in DL-AE ROI were significantly faster than for cpRNFL annulus thickness in progressing eyes (-1.28 µm/y vs. -0.83 µm/y) and nonprogressing eyes (-1.03 µm/y vs. -0.78 µm/y).

Conclusions

Eye-specific ROIs identified using DL-AE analysis of OCT images show promise for improving assessment of glaucomatous progression.

Translational Relevance

The detection and monitoring of structural glaucomatous progression can be improved by considering eye-specific regions of likely progression identified using deep learning.

Semantic segmentation of PolSAR image data using advanced deep learning model

Urban area mapping is an important application of remote sensing which aims at both estimation and change in land cover under the urban area. A major challenge being faced while analyzing Synthetic Aperture Radar (SAR) based remote sensing data is that there is a lot of similarity between highly vegetated urban areas and oriented urban targets with that of actual vegetation. This similarity between some urban areas and vegetation leads to misclassification of the urban area into forest cover. The present work is a precursor study for the dual-frequency L and S-band NASA-ISRO Synthetic Aperture Radar (NISAR) mission and aims at minimizing the misclassification of such highly vegetated and oriented urban targets into vegetation class with the help of deep learning. In this study, three machine learning algorithms Random Forest (RF), K-Nearest Neighbour (KNN), and Support Vector Machine (SVM) have been implemented along with a deep learning model DeepLabv3+ for semantic segmentation of Polarimetric SAR (PolSAR) data. It is a general perception that a large dataset is required for the successful implementation of any deep learning model but in the field of SAR based remote sensing, a major issue is the unavailability of a large benchmark labeled dataset for the implementation of deep learning algorithms from scratch. In current work, it has been shown that a pre-trained deep learning model DeepLabv3+ outperforms the machine learning algorithms for land use and land cover (LULC) classification task even with a small dataset using transfer learning. The highest pixel accuracy of 87.78% and overall pixel accuracy of 85.65% have been achieved with DeepLabv3+ and Random Forest performs best among the machine learning algorithms with overall pixel accuracy of 77.91% while SVM and KNN trail with an overall accuracy of 77.01% and 76.47% respectively. The highest precision of 0.9228 is recorded for the urban class for semantic segmentation task with DeepLabv3+ while machine learning algorithms SVM and RF gave comparable results with a precision of 0.8977 and 0.8958 respectively.

Change Capsule Network for Optical Remote Sensing Image Change Detection

Change detection based on deep learning has made great progress recently, but there are still some challenges, such as the small data size in open-labeled datasets, the different viewpoints in image pairs, and the poor similarity measures in feature pairs. To alleviate these problems, this paper presents a novel change capsule network by taking advantage of a capsule network that can better deal with the different viewpoints and can achieve satisfactory performance with small training data for optical remote sensing image change detection. First, two identical non-shared weight capsule networks are designed to extract the vector-based features of image pairs. Second, the unchanged region reconstruction module is adopted to keep the feature space of the unchanged region more consistent. Third, vector cosine and vector difference are utilized to compare the vector-based features in a capsule network efficiently, which can enlarge the separability between the changed pixels and the unchanged pixels. Finally, a binary change map can be produced by analyzing both the vector cosine and vector difference. From the unchanged region reconstruction module and the vector cosine and vector difference module, the extracted feature pairs in a change capsule network are more comparable and separable. Moreover, to test the effectiveness of the proposed change capsule network in dealing with the different viewpoints in multi-temporal images, we collect a new change detection dataset from a taken-over Al Udeid Air Basee (AUAB) using Google Earth. The results of the experiments carried out on the AUAB dataset show that a change capsule network can better deal with the different viewpoints and can improve the comparability and separability of feature pairs. Furthermore, a comparison of the experimental results carried out on the AUAB dataset and SZTAKI AirChange Benchmark Set demonstrates the effectiveness and superiority of the proposed method.

Deep Multiscale Detail Networks for Multiband Spectral Image Sharpening

We introduce a new deep detail network architecture with grouped multiscale dilated convolutions to sharpen images contain multiband spectral information. Specifically, our end-to-end network directly fuses low-resolution multispectral and panchromatic inputs to produce high-resolution multispectral results, which is the same goal of the pansharpening in remote sensing. The proposed network architecture is designed by utilizing our domain knowledge and considering the two aims of the pansharpening: spectral and spatial preservations. For spectral preservation, the up-sampled multispectral images are directly added to the output for lossless spectral information propagation. For spatial preservation, we train the proposed network in the high-frequency domain instead of the commonly used image domain. Different from conventional network structures, we remove pooling and batch normalization layers to preserve spatial information and improve generalization to new satellites, respectively. To effectively and efficiently obtain multiscale contextual features at a fine-grained level, we propose a grouped multiscale dilated network structure to enlarge the receptive fields for each network layer. This structure allows the network to capture multiscale representations without increasing the parameter burden and network complexity. These representations are finally utilized to reconstruct the residual images which contain spatial details of PAN. Our trained network is able to generalize different satellite images without the need for parameter tuning. Moreover, our model is a general framework, which can be directly used for other kinds of multiband spectral image sharpening, e.g., hyperspectral image sharpening. Experiments show that our model performs favorably against compared methods in terms of both qualitative and quantitative qualities.