Nonconvex-sparsity and Nonlocal-smoothness Based Blind Hyperspectral Unmixing

Deep Learning and Hyperspectral Imaging for Liver Cancer Staging and Cirrhosis Differentiation

Liver malignancies, particularly hepatocellular carcinoma (HCC), pose a formidable global health challenge. Conventional diagnostic techniques frequently fall short in precision, especially at advanced HCC stages. In response, we have developed a novel diagnostic strategy that integrates hyperspectral imaging with deep learning. This innovative approach captures detailed spectral data from tissue samples, pinpointing subtle cellular differences that elude traditional methods. A sophisticated deep convolutional neural network processes this data, effectively distinguishing high-grade liver cancer from cirrhosis with an accuracy of 89.45%, a sensitivity of 90.29%, and a specificity of 88.64%. For HCC differentiation specifically, it achieves an impressive accuracy of 93.73%, sensitivity of 92.53%, and specificity of 90.07%. Our results underscore the potential of this technique as a precise, rapid, and non-invasive diagnostic tool that surpasses existing clinical methods in staging liver cancer and differentiating cirrhosis.

Exact Decomposition of Joint Low Rankness and Local Smoothness Plus Sparse Matrices

It is known that the decomposition in low-rank and sparse matrices (L+S for short) can be achieved by several Robust PCA techniques. Besides the low rankness, the local smoothness (LSS) is a vitally essential prior for many real-world matrix data such as hyperspectral images and surveillance videos, which makes such matrices have low-rankness and local smoothness property at the same time. This poses an interesting question: Can we make a matrix decomposition in terms of L&LSS +S form exactly? To address this issue, we propose in this paper a new RPCA model based on three-dimensional correlated total variation regularization (3DCTV-RPCA for short) by fully exploiting and encoding the prior expression underlying such joint low-rank and local smoothness matrices. Specifically, using a modification of Golfing scheme, we prove that under some mild assumptions, the proposed 3DCTV-RPCA model can decompose both components exactly, which should be the first theoretical guarantee among all such related methods combining low rankness and local smoothness. In addition, by utilizing Fast Fourier Transform (FFT), we propose an efficient ADMM algorithm with a solid convergence guarantee for solving the resulting optimization problem. Finally, a series of experiments on both simulations and real applications are carried out to demonstrate the general validity of the proposed 3DCTV-RPCA model.

An Exploration of Recent Intelligent Image Analysis Techniques for Visual Pavement Surface Condition Assessment

Road pavement condition assessment is essential for maintenance, asset management, and budgeting for pavement infrastructure. Countries allocate a substantial annual budget to maintain and improve local, regional, and national highways. Pavement condition is assessed by measuring several pavement characteristics such as roughness, surface skid resistance, pavement strength, deflection, and visual surface distresses. Visual inspection identifies and quantifies surface distresses, and the condition is assessed using standard rating scales. This paper critically analyzes the research trends in the academic literature, professional practices and current commercial solutions for surface condition ratings by civil authorities. We observe that various surface condition rating systems exist, and each uses its own defined subset of pavement characteristics to evaluate pavement conditions. It is noted that automated visual sensing systems using intelligent algorithms can help reduce the cost and time required for assessing the condition of pavement infrastructure, especially for local and regional road networks. However, environmental factors, pavement types, and image collection devices are significant in this domain and lead to challenging variations. Commercial solutions for automatic pavement assessment with certain limitations exist. The topic is also a focus of academic research. More recently, academic research has pivoted toward deep learning, given that image data is now available in some form. However, research to automate pavement distress assessment often focuses on the regional pavement condition assessment standard that a country or state follows. We observe that the criteria a region adopts to make the evaluation depends on factors such as pavement construction type, type of road network in the area, flow and traffic, environmental conditions, and region's economic situation. We summarized a list of publicly available datasets for distress detection and pavement condition assessment. We listed approaches focusing on crack segmentation and methods concentrating on distress detection and identification using object detection and classification. We segregated the recent academic literature in terms of the camera's view and the dataset used, the year and country in which the work was published, the F1 score, and the architecture type. It is observed that the literature tends to focus more on distress identification ("presence/absence" detection) but less on distress quantification, which is essential for developing approaches for automated pavement rating.

Comparative Analysis of Automated Text Summarization Techniques: The Case of Ethiopian Languages

Nowadays, there is an abundance of information available from both online and offline sources. For a single topic, we can get more than hundreds of sources containing a wealth of information. The ability to extract or generate a summary of popular content allows users to quickly search for content and obtain preliminary data in the shortest amount of time. Manually extracting useful information from them is a difficult task. Automatic text summarization (ATS) systems are being developed to address this issue. Text summarization is the process of extracting useful information from large documents and compressing it into a summary while retaining all the relevant contents. This review paper provides a broad overview of ATS research works in various Ethiopian languages such as Amharic, Afan Oromo, and Tigrinya using different text summarization approaches. The work has identified the novel and recommended state-of-the-art techniques and methods for future researchers in the area and provides knowledge and useful support to new researchers in this field by providing a concise overview of the various feature extraction methods and classification techniques required for different types of ATS approaches applied to the Ethiopian languages. Finally, different recommendations for future researchers have been forwarded.

A New Probability Distribution for SAR Image Modeling

This article introduces exponentiated transmuted-inverted beta (ET-IB) distribution, supported by a continuous positive real line, as a synthetic aperture radar (SAR) imagery descriptor. It is an extension of the inverted beta distribution, an important texture model for SAR imagery. The considered distribution extension approach increases the flexibility of the baseline distribution, and is a new probabilistic model useful in SAR image applications. Besides introducing the new model, the maximum likelihood method is discussed for parameter estimation. Numerical experiments are performed to validate the use of the ET-IB distribution as a SAR amplitude image descriptor. Finally, three measured SAR images referring to forest, ocean, and urban regions are considered, and the performance of the proposed distribution is compared to distributions usually considered in this field. The proposed distribution outperforms the competitor models for modeling SAR images in terms of some selected goodness-of-fit measures. The results show that the ET-IB distribution is suitable as a SAR descriptor and can be used to develop image-processing tools in remote sensing applications.

Self-Supervised Nonlinear Transform-Based Tensor Nuclear Norm for Multi-Dimensional Image Recovery

Recently, transform-based tensor nuclear norm (TNN) minimization methods have received increasing attention for recovering third-order tensors in multi-dimensional imaging problems. The main idea of these methods is to perform the linear transform along the third mode of third-order tensors and then minimize the nuclear norm of frontal slices of the transformed tensor. The main aim of this paper is to propose a nonlinear multilayer neural network to learn a nonlinear transform by solely using the observed tensor in a self-supervised manner. The proposed network makes use of the low-rank representation of the transformed tensor and data-fitting between the observed tensor and the reconstructed tensor to learn the nonlinear transform. Extensive experimental results on different data and different tasks including tensor completion, background subtraction, robust tensor completion, and snapshot compressive imaging demonstrate the superior performance of the proposed method over state-of-the-art methods.

Using Artificial Intelligence for Space Challenges: A Survey

Artificial intelligence is applied to many fields and contributes to many important applications and research areas, such as intelligent data processing, natural language processing, autonomous vehicles, and robots. The adoption of artificial intelligence in several fields has been the subject of many research papers. Still, recently, the space sector is a field where artificial intelligence is receiving significant attention. This paper aims to survey the most relevant problems in the field of space applications solved by artificial intelligence techniques. We focus on applications related to mission design, space exploration, and Earth observation, and we provide a taxonomy of the current challenges. Moreover, we present and discuss current solutions proposed for each challenge to allow researchers to identify and compare the state of the art in this context.

3D Sparse SAR Image Reconstruction Based on Cauchy Penalty and Convex Optimization

Three-dimensional (3D) synthetic aperture radar (SAR) images can provide comprehensive 3D spatial information for environmental monitoring, high dimensional mapping and radar cross sectional (RCS) measurement. However, the SAR image obtained by the traditional matched filtering (MF) method has a high sidelobe and is easily disturbed by noise. In order to obtain high-quality 3D SAR images, sparse signal processing has been used in SAR imaging in recent years. However, the typical L1 regularization model is a biased estimation, which tends to underestimate the target intensity. Therefore, in this article, we present a 3D sparse SAR image reconstruction method combining the Cauchy penalty and improved alternating direction method of multipliers (ADMM). The Cauchy penalty is a non-convex penalty function, which can estimate the target intensity more accurately than L1. At the same time, the objective function maintains convexity via the convex non-convex (CNC) strategy. Compared with L1 regularization, the proposed method can reconstruct the image more accurately and improve the image quality. Finally, three indexes suitable for SAR images are used to evaluate the performance of the method under different conditions. Simulation and experimental results verify the effectiveness of the proposed method.

Hyperspectral Image Mixed Noise Removal Using a Subspace Projection Attention and Residual Channel Attention Network

Although the existing deep-learning-based hyperspectral image (HSI) denoising methods have achieved tremendous success, recovering high-quality HSIs in complex scenes that contain mixed noise is still challenging. Besides, these methods have not fully explored the local and global spatial–spectral information of HSIs. To address the above issues, a novel HSI mixed noise removal network called subspace projection attention and residual channel attention network (SPARCA-Net) is proposed. Specifically, we propose an orthogonal subspace projection attention (OSPA) module to adaptively learn to generate bases of the signal subspace and project the input into such space to remove noise. By leveraging the local and global spatial relations, OSPA is able to reconstruct the local structure of the feature maps more precisely. We further propose a residual channel attention (RCA) module to emphasize the interdependence between feature maps and exploit the global channel correlation of them, which could enhance the channel-wise adaptive learning. In addition, multiscale joint spatial–spectral input and residual learning strategies are employed to capture multiscale spatial–spectral features and reduce the degradation problem, respectively. Synthetic and real HSI data experiments demonstrated that the proposed HSI denoising network outperforms many of the advanced methods in both quantitative and qualitative assessments.

A Two-Branch Convolutional Neural Network Based on Multi-Spectral Entropy Rate Superpixel Segmentation for Hyperspectral Image Classification

Convolutional neural networks (CNNs) can extract advanced features of joint spectral–spatial information, which are useful for hyperspectral image (HSI) classification. However, the patch-based neighborhoods of samples with fixed sizes are usually used as the input of the CNNs, which cannot dig out the homogeneousness between the pixels within and outside of the patch. In addition, the spatial features are quite different in different spectral bands, which are not fully utilized by the existing methods. In this paper, a two-branch convolutional neural network based on multi-spectral entropy rate superpixel segmentation (TBN-MERS) is designed for HSI classification. Firstly, entropy rate superpixel (ERS) segmentation is performed on the image of each spectral band in an HSI, respectively. The segmented images obtained are stacked band by band, called multi-spectral entropy rate superpixel segmentation image (MERSI), and then preprocessed to serve as the input of one branch in TBN-MERS. The preprocessed HSI is used as the input of the other branch in TBN-MERS. TBN-MERS extracts features from both the HSI and the MERSI and then utilizes the fused spectral–spatial features for the classification of HSIs. TBN-MERS makes full use of the joint spectral–spatial information of HSIs at the scale of superpixels and the scale of neighborhood. Therefore, it achieves excellent performance in the classification of HSIs. Experimental results on four datasets demonstrate that the proposed TBN-MERS can effectively extract features from HSIs and significantly outperforms some state-of-the-art methods with a few training samples.

Weighted Group Sparsity-Constrained Tensor Factorization for Hyperspectral Unmixing

Recently, unmixing methods based on nonnegative tensor factorization have played an important role in the decomposition of hyperspectral mixed pixels. According to the spatial prior knowledge, there are many regularizations designed to improve the performance of unmixing algorithms, such as the total variation (TV) regularization. However, these methods mostly ignore the similar characteristics among different spectral bands. To solve this problem, this paper proposes a group sparse regularization that uses the weighted constraint of the L2,1 norm, which can not only explore the similar characteristics of the hyperspectral image in the spectral dimension, but also keep the data smooth characteristics in the spatial dimension. In summary, a non-negative tensor factorization framework based on weighted group sparsity constraint is proposed for hyperspectral images. In addition, an effective alternating direction method of multipliers (ADMM) algorithm is used to solve the algorithm proposed in this paper. Compared with the existing popular methods, experiments conducted on three real datasets fully demonstrate the effectiveness and advancement of the proposed method.

Tensor Completion via Complementary Global, Local, and Nonlocal Priors

Completing missing entries in multidimensional visual data is a typical ill-posed problem that requires appropriate exploitation of prior information of the underlying data. Commonly used priors can be roughly categorized into three classes: global tensor low-rankness, local properties, and nonlocal self-similarity (NSS); most existing works utilize one or two of them to implement completion. Naturally, there arises an interesting question: can one concurrently make use of multiple priors in a unified way, such that they can collaborate with each other to achieve better performance? This work gives a positive answer by formulating a novel tensor completion framework which can simultaneously take advantage of the global-local-nonlocal priors. In the proposed framework, the tensor train (TT) rank is adopted to characterize the global correlation; meanwhile, two Plug-and-Play (PnP) denoisers, including a convolutional neural network (CNN) denoiser and the color block-matching and 3 D filtering (CBM3D) denoiser, are incorporated to preserve local details and exploit NSS, respectively. Then, we design a proximal alternating minimization algorithm to efficiently solve this model under the PnP framework. Under mild conditions, we establish the convergence guarantee of the proposed algorithm. Extensive experiments show that these priors organically benefit from each other to achieve state-of-the-art performance both quantitatively and qualitatively.

Retrieval of Total Phosphorus Concentration in the Surface Water of Miyun Reservoir Based on Remote Sensing Data and Machine Learning Algorithms

Some essential water conservation areas in China have continuously suffered from various serious problems such as water pollution and water quality deterioration in recent decades and thus called for real-time water pollution monitoring system underwater resources management. On the basis of the remote sensing data and ground monitoring data, this study firstly constructed a more accurate retrieval model for total phosphorus (TP) concentration by comparing 12 machine learning algorithms, including support vector machine (SVM), artificial neural network (ANN), Bayesian ridge regression (BRR), lasso regression (Lasso), elastic net (EN), linear regression (LR), decision tree regressor (DTR), K neighbor regressor (KNR), random forest regressor (RFR), extra trees regressor (ETR), AdaBoost regressor (ABR) and gradient boosting regressor (GBR). Then, this study applied the constructed retrieval model to explore the spatial-temporal evolution of the Miyun Reservoir and finally assessed the water quality. The results showed that the model of TP concentration built by the ETR algorithm had the best accuracy, with the coefficient R2 reaching over 85% and the mean absolute error lower than 0.000433. The TP concentration in Miyun Reservoir was between 0.0380 and 0.1298 mg/L, and there was relatively significant spatial and temporal heterogeneity. It changed remarkably during the periods of the flood season, winter tillage, planting, and regreening, and it was lower in summer than in other seasons. Moreover, the TP in the southwest part of the reservoir was generally lower than in the northeast, as there was less human activities interference. According to the Environmental Quality Standard for the surface water environment, the water quality of Miyun Reservoir was overall safe, except only for an over-standard case occurrence in the spring and September. These conclusions can provide a significant scientific reference for water quality monitoring and management in Miyun Reservoir.

Extended Blind End-member and Abundance Estimation with Spatial Total Variation for Hyperspectral Imaging

Blind linear unmixing (BLU) methods allow the separation of multi and hyperspectral data into end-members and abundance maps in an unsupervised fashion. However, due to incident noise, the abundance maps can exhibit high presence of granularity. To address this problem, in this paper, we present a novel proposal for BLU that considers spatial coherence in the abundance estimations, through a total spatial variation component. The proposed BLU formulation is based on the blind end-member and abundance extraction perspective with total spatial variation (EBEAE-STV). In EBEAE-STV, internal abundances are added to incorporate the spatial coherence in the cost function, which is solved by a coordinates descent algorithm. The results with synthetic data show that the proposed algorithm can significantly decrease the granularity in the estimated abundances, and the estimation errors and computational times are lower compared to state of the art methodologies.Clinical relevance- The proper and robust estimation of end-members and their respective contributions (abundances) in multi-spectral and hyper-spectral images from the proposed EBEAE-STV methodology might provide useful information in several biomedical applications, such as chemometric analysis on different biological samples, tumor identification and brain tissue classification for hyper-spectral imaging, among others.

EBARec-BS: Effective Band Attention Reconstruction Network for Hyperspectral Imagery Band Selection

Hyperspectral band selection (BS) is an effective means to avoid the Hughes phenomenon and heavy computational burden in hyperspectral image processing. However, most of the existing BS methods fail to fully consider the interaction between spectral bands and cannot comprehensively consider the representativeness and redundancy of the selected band subset. To solve these problems, we propose an unsupervised effective band attention reconstruction framework for band selection (EBARec-BS) in this article. The framework utilizes the EBARec network to learn the representativeness of each band to the original band set and measures the redundancy between the bands by calculating the distance of each unselected band to the selected band subset. Subsequently, by designing an adaptive weight to balance the influence of the representativeness metric and redundancy metric on the band evaluation, a final band scoring function is obtained to select a band subset that well represents the original hyperspectral image and has low redundancy. Experiments on three well-known hyperspectral data sets indicate that compared with the existing BS methods, the proposed EBARec-BS is robust to noise bands and can effectively select the band subset with higher classification accuracy and less redundant information.

Using Canopy Measurements to Predict Soybean Seed Yield

Predicting soybean [Glycine max (L.) Merr.] seed yield is of interest for crop producers to make important agronomic and economic decisions. Evaluating the soybean canopy across a range of common agronomic practices, using canopy measurements, provides a large inference for soybean producers. The individual and synergistic relationships between fractional green canopy cover (FGCC), photosynthetically active radiation (PAR) interception, and a normalized difference vegetative index (NDVI) measurements taken throughout the growing season to predict soybean seed yield in North Dakota, USA, were investigated in 12 environments. Canopy measurements were evaluated across early and late planting dates, 407,000 and 457,000 seeds ha−1 seeding rates, 0.5 and 0.8 relative maturities, and 30.5 and 61 cm row spacings. The single best yield predictor was an NDVI measurement at R5 (beginning of seed development) with a coefficient of determination of 0.65 followed by an FGCC measurement at R5 (R2 = 0.52). Stepwise and Lasso multiple regression methods were used to select the best prediction models using the canopy measurements explaining 69% and 67% of the variation in yield, respectively. Including plant density, which can be easily measured by a producer, with an individual canopy measurement did not improve the explanation in yield. Using FGCC to estimate yield across the growing season explained a range of 49% to 56% of yield variation, and a single FGCC measurement at R5 (R2 = 0.52) being the most efficient and practical method for a soybean producer to estimate yield.

Building Outline Extraction Directly Using the U2-Net Semantic Segmentation Model from High-Resolution Aerial Images and a Comparison Study

Deep learning techniques have greatly improved the efficiency and accuracy of building extraction using remote sensing images. However, high-quality building outline extraction results that can be applied to the field of surveying and mapping remain a significant challenge. In practice, most building extraction tasks are manually executed. Therefore, an automated procedure of a building outline with a precise position is required. In this study, we directly used the U2-net semantic segmentation model to extract the building outline. The extraction results showed that the U2-net model can provide the building outline with better accuracy and a more precise position than other models based on comparisons with semantic segmentation models (Segnet, U-Net, and FCN) and edge detection models (RCF, HED, and DexiNed) applied for two datasets (Nanjing and Wuhan University (WHU)). We also modified the binary cross-entropy loss function in the U2-net model into a multiclass cross-entropy loss function to directly generate the binary map with the building outline and background. We achieved a further refined outline of the building, thus showing that with the modified U2-net model, it is not necessary to use non-maximum suppression as a post-processing step, as in the other edge detection models, to refine the edge map. Moreover, the modified model is less affected by the sample imbalance problem. Finally, we created an image-to-image program to further validate the modified U2-net semantic segmentation model for building outline extraction.

Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model

As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 - hyperspectral and multispectral data, Berlin - hyperspectral and synthetic aperture radar (SAR) data, Augsburg - hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL.

Cascaded Convolutional Neural Network-Based Hyperspectral Image Resolution Enhancement via an Auxiliary Panchromatic Image

Owing to the limits of incident energy and hardware system, hyperspectral (HS) images always suffer from low spatial resolution, compared with multispectral (MS) or panchromatic (PAN) images. Therefore, image fusion has emerged as a useful technology that is able to combine the characteristics of high spectral and spatial resolutions of HS and PAN/MS images. In this paper, a novel HS and PAN image fusion method based on convolutional neural network (CNN) is proposed. The proposed method incorporates the ideas of both hyper-sharpening and MS pan-sharpening techniques, thereby employing a two-stage cascaded CNN to reconstruct the anticipated high-resolution HS image. Technically, the proposed CNN architecture consists of two sub-networks, the detail injection sub-network and unmixing sub-network. The former aims at producing a latent high-resolution MS image, whereas the latter estimates the desired high-resolution abundance maps by exploring the spatial and spectral information of both HS and MS images. Moreover, two model-training fashions are presented in this paper for the sake of effectively training our network. Experiments on simulated and real remote sensing data demonstrate that the proposed method can improve the spatial resolution and spectral fidelity of HS image, and achieve better performance than some state-of-the-art HS pan-sharpening algorithms.

Effects of Assimilating Clear-Sky FY-3D MWHS2 Radiance on the Numerical Simulation of Tropical Storm Ampil

A new advanced microwave humidity sounder FY-3D MWHS2 radiance has been assimilated under the clear-sky conditions by implementing its data assimilation interface. The case of the tropical storm Ampil in 2018 is selected to address the effectiveness of the new-built module in the initialization and forecast of typhoons. Apart from the experiment assimilating both the Global Telecommunications System (GTS) data and the FY-3D MWHS2 radiance data, an experiment with only GTS data is also conducted for comparison. The results show that the bias correction of this humidity sounder is effective, and the analysis field after assimilating its radiance data matches well with the observation. The increment of specific humidity below the middle layers is evident after the assimilation of the radiance data. Besides, the geopotential height increment and the specific humidity increment at 500 hPa and 850 hPa, respectively, are favorable, resulting in more accurate rain belt distribution and a higher fraction skill score (FSS). In the deterministic forecast, the track error of the FY-3D MWHS2 experiment is consistently less than 90 km.

Utilization of Image, LiDAR and Gamma-Ray Information to Improve Environmental Sustainability of Cut-to-Length Wood Harvesting Operations in Peatlands: A Management Systems Perspective

Forest industry corporations use quality management systems in their wood procurement operations. Spatial quality data are used to improve the quality of wood harvesting and to achieve environmental sustainability. Some studies have proposed new management systems based on LiDAR. The main aim of this study was to investigate how efficiently planning systems can select areas for wood harvesting a priori with respect to avoiding harvesting damage caused by forest machinery. A literature review revealed the possibility of using GISs, and case studies showed the criteria required to predict the required quality levels. Terrestrial LiDAR can be utilized in authorities’ quality control systems, but it is inefficient for preplanning without terrestrial gamma-ray data collection. Airborne LiDAR and gamma-ray information about forest soils can only be used for planning larger regions at the forest level because the information includes too much uncertainty to allow it to be used for planning in small-sized areas before wood harvesting operations involving wood procurement. In addition, airborne LiDAR is not accurate enough, even at the forest level, for the planning of wood procurement systems because wood harvesting remains challenging without field measurements. Therefore, there is a need for the use of manual ground-penetrating radar for determining the peat layer thickness and the depth to the groundwater table.

Sparse Constrained Low Tensor Rank Representation Framework for Hyperspectral Unmixing

Recently, non-negative tensor factorization (NTF) as a very powerful tool has attracted the attention of researchers. It is used in the unmixing of hyperspectral images (HSI) due to its excellent expression ability without any information loss when describing data. However, most of the existing unmixing methods based on NTF fail to fully explore the unique properties of data, for example, low rank, that exists in both the spectral and spatial domains. To explore this low-rank structure, in this paper we learn the different low-rank representations of HSI in the spectral, spatial and non-local similarity modes. Firstly, HSI is divided into many patches, and these patches are clustered multiple groups according to the similarity. Each similarity group can constitute a 4-D tensor, including two spatial modes, a spectral mode and a non-local similarity mode, which has strong low-rank properties. Secondly, a low-rank regularization with logarithmic function is designed and embedded in the NTF framework, which simulates the spatial, spectral and non-local similarity modes of these 4-D tensors. In addition, the sparsity of the abundance tensor is also integrated into the unmixing framework to improve the unmixing performance through the L2,1 norm. Experiments on three real data sets illustrate the stability and effectiveness of our algorithm compared with five state-of-the-art methods.

Multidataset Independent Subspace Analysis With Application to Multimodal Fusion

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data

This paper addresses the problem of semi-supervised transfer learning with limited cross-modality data in remote sensing. A large amount of multi-modal earth observation images, such as multispectral imagery (MSI) or synthetic aperture radar (SAR) data, are openly available on a global scale, enabling parsing global urban scenes through remote sensing imagery. However, their ability in identifying materials (pixel-wise classification) remains limited, due to the noisy collection environment and poor discriminative information as well as limited number of well-annotated training images. To this end, we propose a novel cross-modal deep-learning framework, called X-ModalNet, with three well-designed modules: self-adversarial module, interactive learning module, and label propagation module, by learning to transfer more discriminative information from a small-scale hyperspectral image (HSI) into the classification task using a large-scale MSI or SAR data. Significantly, X-ModalNet generalizes well, owing to propagating labels on an updatable graph constructed by high-level features on the top of the network, yielding semi-supervised cross-modality learning. We evaluate X-ModalNet on two multi-modal remote sensing datasets (HSI-MSI and HSI-SAR) and achieve a significant improvement in comparison with several state-of-the-art methods.

Classification of Hyperspectral In Vivo Brain Tissue Based on Linear Unmixing

Hyperspectral imaging is a multidimensional optical technique with the potential of providing fast and accurate tissue classification. The main challenge is the adequate processing of the multidimensional information usually linked to long processing times and significant computational costs, which require expensive hardware. In this study, we address the problem of tissue classification for intraoperative hyperspectral images of in vivo brain tissue. For this goal, two methodologies are introduced that rely on a blind linear unmixing (BLU) scheme for practical tissue classification. Both methodologies identify the characteristic end-members related to the studied tissue classes by BLU from a training dataset and classify the pixels by a minimum distance approach. The proposed methodologies are compared with a machine learning method based on a supervised support vector machine (SVM) classifier. The methodologies based on BLU achieve speedup factors of ~459× and ~429× compared to the SVM scheme, while keeping constant and even slightly improving the classification performance.

PercepPan: Towards Unsupervised Pan-Sharpening Based on Perceptual Loss

In the literature of pan-sharpening based on neural networks, high resolution multispectral images as ground-truth labels generally are unavailable. To tackle the issue, a common method is to degrade original images into a lower resolution space for supervised training under the Wald’s protocol. In this paper, we propose an unsupervised pan-sharpening framework, referred to as “perceptual pan-sharpening”. This novel method is based on auto-encoder and perceptual loss, and it does not need the degradation step for training. For performance boosting, we also suggest a novel training paradigm, called “first supervised pre-training and then unsupervised fine-tuning”, to train the unsupervised framework. Experiments on the QuickBird dataset show that the framework with different generator architectures could get comparable results with the traditional supervised counterpart, and the novel training paradigm performs better than random initialization. When generalizing to the IKONOS dataset, the unsupervised framework could still get competitive results over the supervised ones.

NPSA: Nonorthogonal Principal Skewness Analysis

Principal skewness analysis (PSA) has been introduced for feature extraction in hyperspectral imagery. As a thirdorder generalization of principal component analysis (PCA), its solution of searching for the local maximum skewness direction is transformed into the problem of calculating the eigenpairs (the eigenvalues and the corresponding eigenvectors) of a coskewness tensor. By combining a fixed-point method with an orthogonal constraint, the new eigenpairs are prevented from converging to the same previously determined maxima. However, in general, the eigenvectors of the supersymmetric tensor are not inherently orthogonal, which implies that the results obtained by the search strategy used in PSA may unavoidably deviate from the actual eigenpairs. In this paper, we propose a new nonorthogonal search strategy to so lve this problem and the new algorithm is named nonorthogonal principal skewness analysis (NPSA). The contribution of NPSA lies in the finding that the search space of the eigenvector to be determined can be enlarged by using the orthogonal complement of the Kronecker product of the previous eigenvector with itself, instead of its orthogonal complement space. We also give a detailed theoretical proof on why we can obtain the more accurate eigenpairs through the new search strategy by comparison with PSA. In addition, after some algebraic derivations, the complexity of the presented algorithm is also greatly reduced. Experiments with both simulated data and real multi/hyperspectral imagery demonstrate its validity in feature extraction.

A Blind Multiscale Spatial Regularization Framework for Kernel-based Spectral Unmixing

Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might comprise intricate interactions between their fractional abundances and nonlinear contributions. In this paper, we consider a multiscale regularization strategy for nonlinear spectral unmixing with kernels. The proposed methodology splits the unmixing problem into two sub-problems at two different spatial scales: a coarse scale containing low-dimensional structures, and the original fine scale. The coarse spatial domain is defined using superpixels that result from a multiscale transformation. Spectral unmixing is then formulated as the solution of quadratically constrained optimization problems, which are solved efficiently by exploring their strong duality and a reformulation of their dual cost functions in the form of root-finding problems. Furthermore, we employ a theory-based statistical framework to devise a consistent strategy to estimate all required parameters, including both the regularization parameters of the algorithm and the number of superpixels of the transformation, resulting in a truly blind (from the parameters setting perspective) unmixing method. Experimental results attest the superior performance of the proposed method when comparing with other, state-of-the-art, related strategies.