Remote Sensing Image Classification Based on Stacked Denoising Autoencoder

Decision Fusion at Pixel Level of Multi-Band Data for Land Cover Classification-A Review

According to existing signatures for various kinds of land cover coming from different spectral bands, i.e., optical, thermal infrared and PolSAR, it is possible to infer about the land cover type having a single decision from each of the spectral bands. Fusing these decisions, it is possible to radically improve the reliability of the decision regarding each pixel, taking into consideration the correlation of the individual decisions of the specific pixel as well as additional information transferred from the pixels' neighborhood. Different remotely sensed data contribute their own information regarding the characteristics of the materials lying in each separate pixel. Hyperspectral and multispectral images give analytic information regarding the reflectance of each pixel in a very detailed manner. Thermal infrared images give valuable information regarding the temperature of the surface covered by each pixel, which is very important for recording thermal locations in urban regions. Finally, SAR data provide structural and electrical characteristics of each pixel. Combining information from some of these sources further improves the capability for reliable categorization of each pixel. The necessary mathematical background regarding pixel-based classification and decision fusion methods is analytically presented.

Endoscopic Image Classification Based on Explainable Deep Learning

Deep learning has achieved remarkably positive results and impacts on medical diagnostics in recent years. Due to its use in several proposals, deep learning has reached sufficient accuracy to implement; however, the algorithms are black boxes that are hard to understand, and model decisions are often made without reason or explanation. To reduce this gap, explainable artificial intelligence (XAI) offers a huge opportunity to receive informed decision support from deep learning models and opens the black box of the method. We conducted an explainable deep learning method based on ResNet152 combined with Grad-CAM for endoscopy image classification. We used an open-source KVASIR dataset that consisted of a total of 8000 wireless capsule images. The heat map of the classification results and an efficient augmentation method achieved a high positive result with 98.28% training and 93.46% validation accuracy in terms of medical image classification.

NDVI Threshold-Based Urban Green Space Mapping from Sentinel-2A at the Local Governmental Area (LGA) Level of Victoria, Australia

Obtaining accurate, precise and timely spatial information on the distribution and dynamics of urban green space is crucial in understanding livability of the cities and urban dwellers. Inspired from the importance of spatial information in planning urban lives, and availability of state-of-the-art remote sensing data and technologies in open access forms, in this work, we develop a simple three-level hierarchical mapping of urban green space with multiple usability to various stakeholders. We utilize the established Normalized Difference Vegetation Index (NDVI) threshold on Sentinel-2A Earth Observation image data to classify the urban vegetation of each Victorian Local Government Area (LGA). Firstly, we categorize each LGA region into two broad classes as vegetation and non-vegetation; secondly, we further categorize the vegetation regions of each LGA into two sub-classes as shrub (including grassland) and trees; thirdly, for both shrub and trees classes, we further classify them as stressed and healthy. We not only map the urban vegetation in hierarchy but also develop Urban Green Space Index (UGSI) and Per Capita Green Space (PCGS) for the Victorian Local Government Areas (LGAs) to provide insights on the association of demography with urban green infrastructure using urban spatial analytics. To show the efficacy of the applied method, we evaluate our results using a Google Earth Engine (GEE) platform across different NDVI threshold ranges. The evaluation result shows that our method produces excellent performance metrics such as mean precision, recall, f-score and accuracy. In addition to this, we also prepare a recent Sentinel-2A dataset and derived products of urban green space coverage of the Victorian LGAs that are useful for multiple stakeholders ranging from bushfire modellers to biodiversity conservationists in contributing to sustainable and resilient urban lives.

Unsupervised feature learning and clustering of particles imaged in raw holograms using an autoencoder

Digital holography is a useful tool to image microscopic particles. Reconstructed holograms give high-resolution shape information that can be used to identify the types of particles. However, the process of reconstructing holograms is computationally intensive and cannot easily keep up with the rate of data acquisition on low-power sensor platforms. In this work, we explore the possibility of performing object clustering on holograms that have not been reconstructed, i.e., images of raw interference patterns, using the latent representations of a deep-learning autoencoder and a self-organizing mapping network in a fully unsupervised manner. We demonstrate this concept on synthetically generated holograms of different shapes, where clustering of raw holograms achieves an accuracy of 94.4%. This is comparable to the 97.4% accuracy achieved using the reconstructed holograms of the same targets. Directly clustering raw holograms takes less than 0.1 s per image using a low-power CPU board. This represents a three-order of magnitude reduction in processing time compared to clustering of reconstructed holograms and makes it possible to interpret targets in real time on low-power sensor platforms. Experiments on real holograms demonstrate significant gains in clustering accuracy through the use of synthetic holograms to train models. Clustering accuracy increased from 47.1% when the models were trained only on the real raw holograms, to 64.1% when the models were entirely trained on the synthetic raw holograms, and further increased to 75.9% when models were trained on the both synthetic and real datasets using transfer learning. These results are broadly comparable to those achieved when reconstructed holograms are used, where the highest accuracy of 70% achieved when clustering raw holograms outperforms the highest accuracy achieved when clustering reconstructed holograms by a significant margin for our datasets.

HFO Detection in Epilepsy: A Stacked Denoising Autoencoder and Sample Weight Adjusting Factors-Based Method

High-frequency oscillations (HFOs) recorded by the intracranial electroencephalography (iEEG) are the promising biomarkers of epileptogenic zones. Accurate detection of HFOs is the key to pre-operative assessment for epilepsy. Due to the subjective bias caused by manual features and the class imbalance between HFOs and false HFOs, it is difficult to obtain satisfactory detection performance by the existing methods. To solve these problems, we put forward a novel method to accurately detect HFOs based on the stacked denoising autoencoder (SDAE) and the ensemble classifier with sample weight adjusting factors. First, the adjustable threshold of Hilbert envelopes is proposed to isolate the events of interest (EoIs) from background activities. Then, the SDAE network is utilized to automatically extract features of EoIs in the time-frequency domain. Finally, the AdaBoost-based support vector machine ensemble classifier with sample weight adjusting factors is devised to separate HFOs from EoIs by using the extracted features. These adjusting factors are used to solve the class imbalance problem by adjusting sample weights when learning the base classifiers. Our HFO detection method is evaluated by using clinical iEEG data recorded from 20 patients with medically refractory epilepsy. The experimental results show that our detection method outperforms some existing methods in terms of sensitivity and false discovery rate. In addition, the HFOs detected by our method are effective for localizing seizure onset zones.

Unsupervised Multi-Level Feature Extraction for Improvement of Hyperspectral Classification

Deep learning models have strong abilities in learning features and they have been successfully applied in hyperspectral images (HSIs). However, the training of most deep learning models requires labeled samples and the collection of labeled samples are labor-consuming in HSI. In addition, single-level features from a single layer are usually considered, which may result in the loss of some important information. Using multiple networks to obtain multi-level features is a solution, but at the cost of longer training time and computational complexity. To solve these problems, a novel unsupervised multi-level feature extraction framework that is based on a three dimensional convolutional autoencoder (3D-CAE) is proposed in this paper. The designed 3D-CAE is stacked by fully 3D convolutional layers and 3D deconvolutional layers, which allows for the spectral-spatial information of targets to be mined simultaneously. Besides, the 3D-CAE can be trained in an unsupervised way without involving labeled samples. Moreover, the multi-level features are directly obtained from the encoded layers with different scales and resolutions, which is more efficient than using multiple networks to get them. The effectiveness of the proposed multi-level features is verified on two hyperspectral data sets. The results demonstrate that the proposed method has great promise in unsupervised feature learning and can help us to further improve the hyperspectral classification when compared with single-level features.

Stacked Autoencoders Driven by Semi-Supervised Learning for Building Extraction from near Infrared Remote Sensing Imagery

In this paper, we propose a Stack Auto-encoder (SAE)-Driven and Semi-Supervised (SSL)-Based Deep Neural Network (DNN) to extract buildings from relatively low-cost satellite near infrared images. The novelty of our scheme is that we employ only an extremely small portion of labeled data for training the deep model which constitutes less than 0.08% of the total data. This way, we significantly reduce the manual effort needed to complete an annotation process, and thus the time required for creating a reliable labeled dataset. On the contrary, we apply novel semi-supervised techniques to estimate soft labels (targets) of the vast amount of existing unlabeled data and then we utilize these soft estimates to improve model training. Overall, four SSL schemes are employed, the Anchor Graph, the Safe Semi-Supervised Regression (SAFER), the Squared-loss Mutual Information Regularization (SMIR), and an equal importance Weighted Average of them (WeiAve). To retain only the most meaning information of the input data, labeled and unlabeled ones, we also employ a Stack Autoencoder (SAE) trained under an unsupervised manner. This way, we handle noise in the input signals, attributed to dimensionality redundancy, without sacrificing meaningful information. Experimental results on the benchmarked dataset of Vaihingen city in Germany indicate that our approach outperforms all state-of-the-art methods in the field using the same type of color orthoimages, though the fact that a limited dataset is utilized (10 times less data or better, compared to other approaches), while our performance is close to the one achieved by high expensive and much more precise input information like the one derived from Light Detection and Ranging (LiDAR) sensors. In addition, the proposed approach can be easily expanded to handle any number of classes, including buildings, vegetation, and ground.

An Adversarial Generative Network for Crop Classification from Remote Sensing Timeseries Images

Due to the increasing demand for the monitoring of crop conditions and food production, it is a challenging and meaningful task to identify crops from remote sensing images. The state-of the-art crop classification models are mostly built on supervised classification models such as support vector machines (SVM), convolutional neural networks (CNN), and long- and short-term memory neural networks (LSTM). Meanwhile, as an unsupervised generative model, the adversarial generative network (GAN) is rarely used to complete classification tasks for agricultural applications. In this work, we propose a new method that combines GAN, CNN, and LSTM models to classify crops of corn and soybeans from remote sensing time-series images, in which GAN’s discriminator was used as the final classifier. The method is feasible on the condition that the training samples are small, and it fully takes advantage of spectral, spatial, and phenology features of crops from satellite data. The classification experiments were conducted on crops of corn, soybeans, and others. To verify the effectiveness of the proposed method, comparisons with models of SVM, SegNet, CNN, LSTM, and different combinations were also conducted. The results show that our method achieved the best classification results, with the Kappa coefficient of 0.7933 and overall accuracy of 0.86. Experiments in other study areas also demonstrate the extensibility of the proposed method.

Deep Spatio-Temporal Representation and Ensemble Classification for Attention Deficit/Hyperactivity Disorder

Attention deficit/Hyperactivity disorder (ADHD) is a complex, universal and heterogeneous neurodevelopmental disease. The traditional diagnosis of ADHD relies on the long-term analysis of complex information such as clinical data (electroencephalogram, etc.), patients' behavior and psychological tests by professional doctors. In recent years, functional magnetic resonance imaging (fMRI) has been developing rapidly and is widely employed in the study of brain cognition due to its non-invasive and non-radiation characteristics. We propose an algorithm based on convolutional denoising autoencoder (CDAE) and adaptive boosting decision trees (AdaDT) to improve the results of ADHD classification. Firstly, combining the advantages of convolutional neural networks (CNNs) and the denoising autoencoder (DAE), we developed a convolutional denoising autoencoder to extract the spatial features of fMRI data and obtain spatial features sorted by time. Then, AdaDT was exploited to classify the features extracted by CDAE. Finally, we validate the algorithm on the ADHD-200 test dataset. The experimental results show that our method offers improved classification compared with state-of-the-art methods in terms of the average accuracy of each individual site and all sites, meanwhile, our algorithm can maintain a certain balance between specificity and sensitivity.

Unsupervised Satellite Image Time Series Clustering Using Object-Based Approaches and 3D Convolutional Autoencoder

Nowadays, satellite image time series (SITS) analysis has become an indispensable part of many research projects as the quantity of freely available remote sensed data increases every day. However, with the growing image resolution, pixel-level SITS analysis approaches have been replaced by more efficient ones leveraging object-based data representations. Unfortunately, the segmentation of a full time series may be a complicated task as some objects undergo important variations from one image to another and can also appear and disappear. In this paper, we propose an algorithm that performs both segmentation and clustering of SITS. It is achieved by using a compressed SITS representation obtained with a multi-view 3D convolutional autoencoder. First, a unique segmentation map is computed for the whole SITS. Then, the extracted spatio-temporal objects are clustered using their encoded descriptors. The proposed approach was evaluated on two real-life datasets and outperformed the state-of-the-art methods.

Assessment of hotspots using sparse autoencoder in industrial zones

Remote sensing satellite systems can be used to detect industrial zones by means of thermal infrared bands. There are several satellite systems loaded with thermal infrared sensors such as Landsat and Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER). In this study, ASTER thermal infrared data were converted to land surface temperature (LST) in order to determine hotspots caused by industrial zones. High LST values surrounded by low LST values are called hotspots here. These hotspots can be determined by applying different methodologies. One of these methods of sparse autoencoder can be used to indicate hotspots using different sizes of hidden layers. The principle of sparse autoencoder depends on unlabeled data in unsupervised learning. It does not need any information about labeled data as in supervised learning. The autoencoder reproduces its output with the same dimensions as the input image by managing the size of the hidden layer. The reconstruction of the image depends on the minimization of a cost function. The size of the hidden layer sets the fitting degree of the function for the reproduced image. A low-order reproduced image is the main target for hotspot detection. In this study, the difference between the original image and the reproduced image was analyzed for hotspot detection. Sparse autoencoder was successfully applied to ASTER thermal band 10 for hotspot detection in 7 pre-defined sites of a region known for steel industry for the two different days.

Segmentation of Oil Spills on Side-Looking Airborne Radar Imagery with Autoencoders

In this work, we use deep neural autoencoders to segment oil spills from Side-Looking Airborne Radar (SLAR) imagery. Synthetic Aperture Radar (SAR) has been much exploited for ocean surface monitoring, especially for oil pollution detection, but few approaches in the literature use SLAR. Our sensor consists of two SAR antennas mounted on an aircraft, enabling a quicker response than satellite sensors for emergency services when an oil spill occurs. Experiments on TERMA radar were carried out to detect oil spills on Spanish coasts using deep selectional autoencoders and RED-nets (very deep Residual Encoder-Decoder Networks). Different configurations of these networks were evaluated and the best topology significantly outperformed previous approaches, correctly detecting 100% of the spills and obtaining an F1 score of 93.01% at the pixel level. The proposed autoencoders perform accurately in SLAR imagery that has artifacts and noise caused by the aircraft maneuvers, in different weather conditions and with the presence of look-alikes due to natural phenomena such as shoals of fish and seaweed.