Feature Learning Using Spatial-Spectral Hypergraph Discriminant Analysis for Hyperspectral Image

Enhancing land cover object classification in hyperspectral imagery through an efficient spectral-spatial feature learning approach

The classification of land cover objects in hyperspectral imagery (HSI) has significantly advanced due to the development of convolutional neural networks (CNNs). However, challenges such as limited training data and high dimensionality negatively impact classification performance. Traditional CNN-based methods predominantly utilize 2D CNNs for feature extraction, which inadequately exploit the inter-band correlations in HSIs. While 3D CNNs can capture joint spectral-spatial information, they often encounter issues related to network depth and complexity. To address these issues, we propose an innovative land cover object classification approach in HSIs that integrates segmented principal component analysis (Seg-PCA) with hybrid 3D-2D CNNs. Our approach leverages Seg-PCA for effective feature extraction and employs the minimum-redundancy maximum relevance (mRMR) criterion for feature selection. By combining the strengths of both 3D and 2D CNNs, our method efficiently extracts spectral-spatial features. These features are then processed through fully connected dense layers and a softmax layer for classification. Extensive experiments on three widely used HSI datasets demonstrate that our method consistently outperforms existing state-of-the-art techniques in classification performance. These results highlight the efficacy of our approach and its potential to significantly enhance the classification of land cover objects in hyperspectral imagery.

Secure impulsive tracking of multi-agent systems with directed hypergraph topologies against hybrid deception attacks

This research delves into the challenges of achieving secure consensus tracking within multi-agent systems characterized by directed hypergraph topologies, in the face of hybrid deception attacks. The hybrid discrete and continuous deception attacks are targeted at the controller communication channels and the hyperedges, respectively. To overcome these threats, an impulsive control mechanism based on hypergraph theory are introduced, and sufficient conditions are established, under which consensus can be maintained in a mean-square bounded sense, supported by rigorous mathematical proofs. Furthermore, the investigation quantifies the relationship between the mean-square bounded consensus of the multi-agent system and the intensity of the deception attacks, delineating a specific range for this error metric. The robustness and effectiveness of the proposed control method are verified through comprehensive simulation experiments, demonstrating its applicability in varied scenarios influenced by these sophisticated attacks. This study underscores the potential of hypergraph-based strategies in enhancing system resilience against complex hybrid attacks.

Deep Spatial-Spectral Joint-Sparse Prior Encoding Network for Hyperspectral Target Detection

Hyperspectral target detection aims to locate targets of interest in the scene, and deep learning-based detection methods have achieved the best results. However, black box network architectures are usually designed to directly learn the mapping between the original image and the discriminative features in a single data-driven manner, a choice that lacks sufficient interpretability. On the contrary, this article proposes a novel deep spatial-spectral joint-sparse prior encoding network (JSPEN), which reasonably embeds the domain knowledge of hyperspectral target detection into the neural network, and has explicit interpretability. In JSPEN, the sparse encoded prior information with spatial-spectral constraints is learned end-to-end from hyperspectral images (HSIs). Specifically, an adaptive joint spatial-spectral sparse model (AS2JSM) is developed to mine the spatial-spectral correlation of HSIs and improves the accuracy of data representation. An optimization algorithm is designed for iteratively solving AS2JSM, and JSPEN is proposed to simulate the iterative optimization process in the algorithm. Each basic module of JSPEN one-to-one corresponds to the operation in the optimization algorithm so that each intermediate result in the network has a clear explanation, which is convenient for intuitive analysis of the operation of the network. With end-to-end training, JSPEN can automatically capture the general sparse properties of HSIs and faithfully characterize the features of background and target. Experimental results verify the effectiveness and accuracy of the proposed method. Code is available at https://github.com/Jiahuiqu/JSPEN.

Hypergraph-Based Numerical Spiking Neural Membrane Systems with Novel Repartition Protocols

The classic spiking neural P (SN P) systems abstract the real biological neural network into a simple structure based on graphs, where neurons can only communicate on the plane. This study proposes the hypergraph-based numerical spiking neural membrane (HNSNM) systems with novel repartition protocols. Through the introduction of hypergraphs, the HNSNM systems can characterize the high-order relationships among neurons and extend the traditional neuron structure to high-dimensional nonlinear spaces. The HNSNM systems also abstract two biological mechanisms of synapse creation and pruning, and use plasticity rules with repartition protocols to achieve planar, hierarchical and spatial communications among neurons in hypergraph neuron structures. Through imitating register machines, the Turing universality of the HNSNM systems is proved by using them as number generating and accepting devices. A universal HNSNM system consisting of 41 neurons is constructed to compute arbitrary functions. By solving NP-complete problems using the subset sum problem as an example, the computational efficiency and effectiveness of HNSNM systems are verified.

Semi-Supervised Multiscale Dynamic Graph Convolution Network for Hyperspectral Image Classification

In recent years, convolutional neural networks (CNNs)-based methods achieve cracking performance on hyperspectral image (HSI) classification tasks, due to its hierarchical structure and strong nonlinear fitting capacity. Most of them, however, are supervised approaches that need a large number of labeled data to train them. Conventional convolution kernels are fixed shape of rectangular with fixed sizes, which are good at capturing short-range relations between pixels within HSIs but ignore the long-range context within HSIs, limiting their performance. To overcome the limitations mentioned above, we present a dynamic multiscale graph convolutional network (GCN) classifier (DMSGer). DMSGer first constructs a relatively small graph at region-level based on a superpixel segmentation algorithm and metric-learning. A dynamic pixel-level feature update strategy is then applied to the region-level adjacency matrix, which can help DMSGer learn the pixel representation dynamically. Finally, to deeply understand the complex contents within HSIs, our model is expanded into a multiscale version. On the one hand, by introducing graph learning theory, DMSGer accomplishes HSI classification tasks in a semi-supervised manner, relieving the pressure of collecting abundant labeled samples. Superpixels are generally in irregular shapes and sizes which can group only similar pixels in a neighborhood. On the other hand, based on the proposed dynamic-GCN, the pixel-level and region-level information can be captured simultaneously in one graph convolution layer such that the classification results can be improved. Also, due to the proper multiscale expansion, more helpful information can be captured from HSIs. Extensive experiments were conducted on four public HSIs, and the promising results illustrate that our DMSGer is robust in classifying HSIs. Our source codes are available at https://github.com/TangXu-Group/DMSGer.

An Interpretable Unsupervised Unrolling Network for Hyperspectral Pansharpening

Existing deep convolutional neural networks (CNNs) have recently achieved great success in pansharpening. However, most deep CNN-based pansharpening models are based on "black-box" architecture and require supervision, making these methods rely heavily on the ground-truth data and lose their interpretability for specific problems during network training. This study proposes a novel interpretable unsupervised end-to-end pansharpening network, called as IU2PNet, which explicitly encodes the well-studied pansharpening observation model into an unsupervised unrolling iterative adversarial network. Specifically, we first design a pansharpening model, whose iterative process can be computed by the half-quadratic splitting algorithm. Then, the iterative steps are unfolded into a deep interpretable iterative generative dual adversarial network (iGDANet). Generator in iGDANet is interwoven by multiple deep feature pyramid denoising modules and deep interpretable convolutional reconstruction modules. In each iteration, the generator establishes an adversarial game with the spatial and spectral discriminators to update both spectral and spatial information without ground-truth images. Extensive experiments show that, compared with the state-of-the-art methods, our proposed IU2PNet exhibits very competitive performance in terms of quantitative evaluation metrics and qualitative visual effects.

Two-Dimensional Unsupervised Feature Selection via Sparse Feature Filter

Unsupervised feature selection is a vital yet challenging topic for effective data learning. Recently, 2-D feature selection methods show good performance on image analysis by utilizing the structure information of image. Current 2-D methods usually adopt a sparse regularization to spotlight the key features. However, such scheme introduces additional hyperparameter needed for pruning, limiting the applicability of unsupervised algorithms. To overcome these challenges, we design a feature filter to estimate the weight of image features for unsupervised feature selection. Theoretical analysis shows that a sparse regularization can be derived from the feature filter by transformation, indicating that the filter plays the same role as the popular sparse regularization does. We deploy two distinct strategies in terms of feature selection, called multiple feature filters and single common feature filter. The former divides the optimization problem into multiple independent subproblems and selects features that meet the respective interests of each subproblem. The latter selects features that are in the interest of the overall optimization problem. Extensive experiments on seven benchmark datasets show that our unsupervised 2-D weight-based feature selection methods achieve superior performance over the state-of-the-art methods.

Identification of white degradable and non-degradable plastics in food field: A dynamic residual network coupled with hyperspectral technology

In the food field, with the improvement of people's health and environmental protection awareness, degradable plastics have become a trend to replace non-degradable plastics. However, their appearance is very similar, making it difficult to distinguish them. This work proposed a rapid identification method for white non-degradable and degradable plastics. Firstly, a hyperspectral imaging system was used to collect the hyperspectral images of the plastics in visible and near-infrared bands (380-1038 nm). Secondly, a residual network (ResNet) was designed according to the characteristics of hyperspectral information. Finally, a dynamic convolution module was introduced into the ResNet to establish a dynamic residual network (Dy-ResNet) to adaptively mine the data features and realize the classification of the degradable and non-degradable plastics. Dy-ResNet had better classification performance than the other classical deep learning methods. The classification accuracy of the degradable and non-degradable plastics was 99.06%. In conclusion, hyperspectral imaging technology was combined with Dy-ResNet to identify the white non-degradable and degradable plastics effectively.

Cost-Sensitive Hypergraph Learning With F-Measure Optimization

The imbalanced issue among data is common in many machine-learning applications, where samples from one or more classes are rare. To address this issue, many imbalanced machine-learning methods have been proposed. Most of these methods rely on cost-sensitive learning. However, we note that it is infeasible to determine the precise cost values even with great domain knowledge for those cost-sensitive machine-learning methods. So in this method, due to the superiority of F-measure on evaluating the performance of imbalanced data classification, we employ F-measure to calculate the cost information and propose a cost-sensitive hypergraph learning method with F-measure optimization to solve the imbalanced issue. In this method, we employ the hypergraph structure to explore the high-order relationships among the imbalanced data. Based on the constructed hypergraph structure, we optimize the cost value with F-measure and further conduct cost-sensitive hypergraph learning with the optimized cost information. The comprehensive experiments validate the effectiveness of the proposed method.

Generating Hypergraph-Based High-Order Representations of Whole-Slide Histopathological Images for Survival Prediction

Patient survival prediction based on gigapixel whole-slide histopathological images (WSIs) has become increasingly prevalent in recent years. A key challenge of this task is achieving an informative survival-specific global representation from those WSIs with highly complicated data correlation. This article proposes a multi-hypergraph based learning framework, called "HGSurvNet," to tackle this challenge. HGSurvNet achieves an effective high-order global representation of WSIs via multilateral correlation modeling in multiple spaces and a general hypergraph convolution network. It has the ability to alleviate over-fitting issues caused by the lack of training data by using a new convolution structure called hypergraph max-mask convolution. Extensive validation experiments were conducted on three widely-used carcinoma datasets: Lung Squamous Cell Carcinoma (LUSC), Glioblastoma Multiforme (GBM), and National Lung Screening Trial (NLST). Quantitative analysis demonstrated that the proposed method consistently outperforms state-of-the-art methods, coupled with the Bayesian Concordance Readjust loss. We also demonstrate the individual effectiveness of each module of the proposed framework and its application potential for pathology diagnosis and reporting empowered by its interpretability potential.

Fault diagnosis of rotor based on Semi-supervised Multi-Graph Joint Embedding

Traditional graph embedding methods only consider the pairwise relationship between fault data. But in practical applications, the relationship of high-dimensional fault data usually is multiple classes corresponding to multiple samples. Therefore, the hypergraph structure is introduced to fully portray the complex structural relationship of high-dimensional fault data. However, during the construction of the hypergraph, the hyperedge weight is usually set as the sum of the similarities between every two vertices contained within the hyperedge, and this "averaging effect" causes the relationship between data sample points with high similarity to be weakened, while the relationship between data sample points with low similarity to be strengthened. This phenomenon also leads to the hypergraph cannot accurately portray the relationship of high-dimensional data, which reduces the fault classification accuracy. To address this issue, a novel dimensionality reduction method named Semi-supervised Multi-Graph Joint Embedding (SMGJE) is proposed and applied to rotor fault diagnosis. SMGJE constructs simple graphs and hypergraphs with the same sample points and characterizes the structure of high-dimensional data in a multi-graph joint embedding. The edges of the simple graph are the direct description of the similarity between sample points so that SMGJE can overcome this "averaging effect" of the hypergraph. The effectiveness of the proposed method is verified by two different fault datasets.

E2E-LIADE: End-to-End Local Invariant Autoencoding Density Estimation Model for Anomaly Target Detection in Hyperspectral Image

Hyperspectral anomaly target detection (also known as hyperspectral anomaly detection (HAD)] is a technique aiming to identify samples with atypical spectra. Although some density estimation-based methods have been developed, they may suffer from two issues: 1) separated two-stage optimization with inconsistent objective functions makes the representation learning model fail to dig out characterization customized for HAD and 2) incapability of learning a low-dimensional representation that preserves the inherent information from the original high-dimensional spectral space. To address these problems, we propose a novel end-to-end local invariant autoencoding density estimation (E2E-LIADE) model. To satisfy the assumption on the manifold, the E2E-LIADE introduces a local invariant autoencoder (LIA) to capture the intrinsic low-dimensional manifold embedded in the original space. Augmented low-dimensional representation (ALDR) can be generated by concatenating the local invariant constrained by a graph regularizer and the reconstruction error. In particular, an end-to-end (E2E) multidistance measure, including mean-squared error (MSE) and orthogonal projection divergence (OPD), is imposed on the LIA with respect to hyperspectral data. More important, E2E-LIADE simultaneously optimizes the ALDR of the LIA and a density estimation network in an E2E manner to avoid the model being trapped in a local optimum, resulting in an energy map in which each pixel represents a negative log likelihood for the spectrum. Finally, a postprocessing procedure is conducted on the energy map to suppress the background. The experimental results demonstrate that compared to the state of the art, the proposed E2E-LIADE offers more satisfactory performance.

Composite Kernel of Mutual Learning on Mid-Level Features for Hyperspectral Image Classification

By training different models and averaging their predictions, the performance of the machine-learning algorithm can be improved. The performance optimization of multiple models is supposed to generalize further data well. This requires the knowledge transfer of generalization information between models. In this article, a multiple kernel mutual learning method based on transfer learning of combined mid-level features is proposed for hyperspectral classification. Three-layer homogenous superpixels are computed on the image formed by PCA, which is used for computing mid-level features. The three mid-level features include: 1) the sparse reconstructed feature; 2) combined mean feature; and 3) uniqueness. The sparse reconstruction feature is obtained by a joint sparse representation model under the constraint of three-scale superpixels' boundaries and regions. The combined mean features are computed with average values of spectra in multilayer superpixels, and the uniqueness is obtained by the superposed manifold ranking values of multilayer superpixels. Next, three kernels of samples in different feature spaces are computed for mutual learning by minimizing the divergence. Then, a combined kernel is constructed to optimize the sample distance measurement and applied by employing SVM training to build classifiers. Experiments are performed on real hyperspectral datasets, and the corresponding results demonstrated that the proposed method can perform significantly better than several state-of-the-art competitive algorithms based on MKL and deep learning.

Deep Manifold Embedding for Hyperspectral Image Classification

Deep learning methods have played a more important role in hyperspectral image classification. However, general deep learning methods mainly take advantage of the samplewise information to formulate the training loss while ignoring the intrinsic data structure of each class. Due to the high spectral dimension and great redundancy between different spectral channels in the hyperspectral image, these former training losses usually cannot work so well for the deep representation of the image. To tackle this problem, this work develops a novel deep manifold embedding method (DMEM) for deep learning in hyperspectral image classification. First, each class in the image is modeled as a specific nonlinear manifold, and the geodesic distance is used to measure the correlation between the samples. Then, based on the hierarchical clustering, the manifold structure of the data can be captured and each nonlinear data manifold can be divided into several subclasses. Finally, considering the distribution of each subclass and the correlation between different subclasses under data manifold, DMEM is constructed as the novel training loss to incorporate the special classwise information in the training process and obtain discriminative representation for the hyperspectral image. Experiments over four real-world hyperspectral image datasets have demonstrated the effectiveness of the proposed method when compared with general sample-based losses and showed superiority when compared with state-of-the-art methods.

Deep Feature Aggregation Framework Driven by Graph Convolutional Network for Scene Classification in Remote Sensing

Scene classification of high spatial resolution (HSR) images can provide data support for many practical applications, such as land planning and utilization, and it has been a crucial research topic in the remote sensing (RS) community. Recently, deep learning methods driven by massive data show the impressive ability of feature learning in the field of HSR scene classification, especially convolutional neural networks (CNNs). Although traditional CNNs achieve good classification results, it is difficult for them to effectively capture potential context relationships. The graphs have powerful capacity to represent the relevance of data, and graph-based deep learning methods can spontaneously learn intrinsic attributes contained in RS images. Inspired by the abovementioned facts, we develop a deep feature aggregation framework driven by graph convolutional network (DFAGCN) for the HSR scene classification. First, the off-the-shelf CNN pretrained on ImageNet is employed to obtain multilayer features. Second, a graph convolutional network-based model is introduced to effectively reveal patch-to-patch correlations of convolutional feature maps, and more refined features can be harvested. Finally, a weighted concatenation method is adopted to integrate multiple features (i.e., multilayer convolutional features and fully connected features) by introducing three weighting coefficients, and then a linear classifier is employed to predict semantic classes of query images. Experimental results performed on the UCM, AID, RSSCN7, and NWPU-RESISC45 data sets demonstrate that the proposed DFAGCN framework obtains more competitive performance than some state-of-the-art methods of scene classification in terms of OAs.

Hyperspectral Image Classification via Discriminant Gabor Ensemble Filter

For a broad range of applications, hyperspectral image (HSI) classification is a hot topic in remote sensing, and convolutional neural network (CNN)-based methods are drawing increasing attention. However, to train millions of parameters in CNN requires a large number of labeled training samples, which are difficult to collect. A conventional Gabor filter can effectively extract spatial information with different scales and orientations without training, but it may be missing some important discriminative information. In this article, we propose the Gabor ensemble filter (GEF), a new convolutional filter to extract deep features for HSI with fewer trainable parameters. GEF filters each input channel by some fixed Gabor filters and learnable filters simultaneously, then reduces the dimensions by some learnable 1×1 filters to generate the output channels. The fixed Gabor filters can extract common features with different scales and orientations, while the learnable filters can learn some complementary features that Gabor filters cannot extract. Based on GEF, we design a network architecture for HSI classification, which extracts deep features and can learn from limited training samples. In order to simultaneously learn more discriminative features and an end-to-end system, we propose to introduce the local discriminant structure for cross-entropy loss by combining the triplet hard loss. Results of experiments on three HSI datasets show that the proposed method has significantly higher classification accuracy than other state-of-the-art methods. Moreover, the proposed method is speedy for both training and testing.

Design and Implementation of Trace Inspection System Based upon Hyperspectral Imaging Technology

Trace inspection is a key technology for collecting crime scenes in the criminal investigation department. A lot of information can be obtained by restoring and analyzing the remaining traces on the scene. However, with the development of digital technology, digital trace inspection has become more and more popular. So, the main research of this article is the design and realization of the trace inspection system based on hyperspectral imaging technology. This article proposes nondestructive testing technology in hyperspectral imaging technology. Combining basic principles of spectroscopy and the image of residual traces such as car tires, shoe soles, and blood stains, it can identify the key traces. Then, based on the image denoising and least squares support vector machine method, this study improves the accuracy and restoration of the image. Therefore, this study designs a test for the trace inspection system for testing hyperspectral imaging technology. The test items include the performance of the trace inspection system, the noise reduction of the trace inspection system, and the ability of the trace inspection system to inspect blood stains. The final collected data are improved to get the trace inspection system based on hyperspectral imaging technology proposed in this study. Compared with the traditional trace inspection system, the experimental results show that the trace inspection system based on hyperspectral imaging technology can improve the accuracy by 5%–28%, compared with the traditional trace inspection system. The image restoration degree of the hyperspectral imaging technology trace inspection system can be improved by 1%–19%, compared with the traditional trace inspection system.

SpaSSA: Superpixelwise Adaptive SSA for Unsupervised Spatial-Spectral Feature Extraction in Hyperspectral Image

Singular spectral analysis (SSA) has recently been successfully applied to feature extraction in hyperspectral image (HSI), including conventional (1-D) SSA in spectral domain and 2-D SSA in spatial domain. However, there are some drawbacks, such as sensitivity to the window size, high computational complexity under a large window, and failing to extract joint spectral-spatial features. To tackle these issues, in this article, we propose superpixelwise adaptive SSA (SpaSSA), that is superpixelwise adaptive SSA for exploiting local spatial information of HSI. The extraction of local (instead of global) features, particularly in HSI, can be more effective for characterizing the objects within an image. In SpaSSA, conventional SSA and 2-D SSA are combined and adaptively applied to each superpixel derived from an oversegmented HSI. According to the size of the derived superpixels, either SSA or 2-D singular spectrum analysis (2D-SSA) is adaptively applied for feature extraction, where the embedding window in 2D-SSA is also adaptive to the size of the superpixel. Experimental results on the three datasets have shown that the proposed SpaSSA outperforms both SSA and 2D-SSA in terms of classification accuracy and computational complexity. By combining SpaSSA with the principal component analysis (SpaSSA-PCA), the accuracy of land-cover analysis can be further improved, outperforming several state-of-the-art approaches.

Class Incremental Learning With Few-Shots Based on Linear Programming for Hyperspectral Image Classification

Hyperspectral imaging (HSI) classification has drawn tremendous attention in the field of Earth observation. In the big data era, explosive growth has occurred in the amount of data obtained by advanced remote sensors. Inevitably, new data classes and refined categories appear continuously, and such data are limited in terms of the timeliness of application. These characteristics motivate us to build an HSI classification model that learns new classifying capability rapidly within a few shots while maintaining good performance on the original classes. To achieve this goal, we propose a linear programming incremental learning classifier (LPILC) that can enable existing deep learning classification models to adapt to new datasets. Specifically, the LPILC learns the new ability by taking advantage of the well-trained classification model within one shot of the new class without any original class data. The entire process requires minimal new class data, computational resources, and time, thereby making LPILC a suitable tool for some time-sensitive applications. Moreover, we utilize the proposed LPILC to implement fine-grained classification via the well-trained original coarse-grained classification model. We demonstrate the success of LPILC with extensive experiments based on three widely used hyperspectral datasets, namely, PaviaU, Indian Pines, and Salinas. The experimental results reveal that the proposed LPILC outperforms state-of-the-art methods under the same data access and computational resource. The LPILC can be integrated into any sophisticated classification model, thereby bringing new insights into incremental learning applied in HSI classification.

Convex Non-Negative Matrix Factorization With Adaptive Graph for Unsupervised Feature Selection

Unsupervised feature selection (UFS) aims to remove the redundant information and select the most representative feature subset from the original data, so it occupies a core position for high-dimensional data preprocessing. Many proposed approaches use self-expression to explore the correlation between the data samples or use pseudolabel matrix learning to learn the mapping between the data and labels. Furthermore, the existing methods have tried to add constraints to either of these two modules to reduce the redundancy, but no prior literature embeds them into a joint model to select the most representative features by the computed top ranking scores. To address the aforementioned issue, this article presents a novel UFS method via a convex non-negative matrix factorization with an adaptive graph constraint (CNAFS). Through convex matrix factorization with adaptive graph constraint, it can dig up the correlation between the data and keep the local manifold structure of the data. To our knowledge, it is the first work that integrates pseudo label matrix learning into the self-expression module and optimizes them simultaneously for the UFS solution. Besides, two different manifold regularizations are constructed for the pseudolabel matrix and the encoding matrix to keep the local geometrical structure. Eventually, extensive experiments on the benchmark datasets are conducted to prove the effectiveness of our method. The source code is available at: https://github.com/misteru/CNAFS.

Symmetric All Convolutional Neural-Network-Based Unsupervised Feature Extraction for Hyperspectral Images Classification

Recently, deep-learning-based feature extraction (FE) methods have shown great potential in hyperspectral image (HSI) processing. Unfortunately, it also brings a challenge that the training of the deep learning networks always requires large amounts of labeled samples, which is hardly available for HSI data. To address this issue, in this article, a novel unsupervised deep-learning-based FE method is proposed, which is trained in an end-to-end style. The proposed framework consists of an encoder subnetwork and a decoder subnetwork. The structure of the two subnetworks is symmetric for obtaining better downsampling and upsampling representation. Considering both spectral and spatial information, 3-D all convolution nets and deconvolution nets are used to structure the encoder subnetwork and decoder subnetwork, respectively. However, 3-D convolution and deconvolution kernels bring more parameters, which can deteriorate the quality of the obtained features. To alleviate this problem, a novel cost function with a sparse regular term is designed to obtain more robust feature representation. Experimental results on publicly available datasets indicate that the proposed method can obtain robust and effective features for subsequent classification tasks.

Hypergraph Learning: Methods and Practices

Hypergraph learning is a technique for conducting learning on a hypergraph structure. In recent years, hypergraph learning has attracted increasing attention due to its flexibility and capability in modeling complex data correlation. In this paper, we first systematically review existing literature regarding hypergraph generation, including distance-based, representation-based, attribute-based, and network-based approaches. Then, we introduce the existing learning methods on a hypergraph, including transductive hypergraph learning, inductive hypergraph learning, hypergraph structure updating, and multi-modal hypergraph learning. After that, we present a tensor-based dynamic hypergraph representation and learning framework that can effectively describe high-order correlation in a hypergraph. To study the effectiveness and efficiency of hypergraph generation and learning methods, we conduct comprehensive evaluations on several typical applications, including object and action recognition, Microblog sentiment prediction, and clustering. In addition, we contribute a hypergraph learning development toolkit called THU-HyperG.

Hyperspectral Image Classification Based on 3D Asymmetric Inception Network with Data Fusion Transfer Learning

Hyperspectral image (HSI) classification has been marked by exceptional progress in recent years. Much of this progess has come from advances in convolutional neural networks (CNNs). Different from the RGB images, HSI images are captured by various remote sensors with different spectral configurations. Moreover, each HSI dataset only contains very limited training samples and thus the model is prone to overfitting when using deep CNNs. In this paper, we first propose a 3D asymmetric inception network, AINet, to overcome the overfitting problem. With the emphasis on spectral signatures over spatial contexts of HSI data, the 3D convolution layer of AINet is replaced with two asymmetric inception units, i.e., a space inception unit and spectrum inception unit, to convey and classify the features effectively. In addition, we exploited a data-fusion transfer learning strategy to improve model initialization and classification performance. Extensive experiments show that the proposed approach outperforms all of the state-of-the-art methods via several HSI benchmarks, including Pavia University, Indian Pines and Kennedy Space Center (KSC).

Manifold-Based Multi-Deep Belief Network for Feature Extraction of Hyperspectral Image

Deep belief networks (DBNs) have been widely applied in hyperspectral imagery (HSI) processing. However, the original DBN model fails to explore the prior knowledge of training samples which limits the discriminant capability of extracted features for classification. In this paper, we proposed a new deep learning method, termed manifold-based multi-DBN (MMDBN), to obtain deep manifold features of HSI. MMDBN designed a hierarchical initialization method that initializes the network by local geometric structure hidden in data. On this basis, a multi-DBN structure is built to learn deep features in each land-cover class, and it was used as the front-end of the whole model. Then, a discrimination manifold layer is developed to improve the discriminability of extracted deep features. To discover the manifold structure contained in HSI, an intrinsic graph and a penalty graph are constructed in this layer by using label information of training samples. After that, the deep manifold features can be obtained for classification. MMDBN not only effectively extracts the deep features from each class in HSI, but also maximizes the margins between different manifolds in low-dimensional embedding space. Experimental results on Indian Pines, Salinas, and Botswana datasets reach 78.25%, 90.48%, and 97.35% indicating that MMDBN possesses better classification performance by comparing with some state-of-the-art methods.

Mixed Structure with 3D Multi-Shortcut-Link Networks for Hyperspectral Image Classification

A hyperspectral image classification method based on a mixed structure with a 3D multi-shortcut-link network (MSLN) was proposed for the features of few labeled samples, excess noise, and heterogeneous homogeneity of features in hyperspectral images. First, the spatial–spectral joint features of hyperspectral cube data were extracted through 3D convolution operation; then, the deep network was constructed and the 3D MSLN mixed structure was used to fuse shallow representational features and deep abstract features, while the hybrid activation function was utilized to ensure the integrity of nonlinear data. Finally, the global self-adaptive average pooling and L-softmax classifier were introduced to implement the terrain classification of hyperspectral images. The mixed structure proposed in this study could extract multi-channel features with a vast receptive field and reduce the continuous decay of shallow features while improving the utilization of representational features and enhancing the expressiveness of the deep network. The use of the dropout mechanism and L-softmax classifier endowed the learned features with a better generalization property and intraclass cohesion and interclass separation properties. Through experimental comparative analysis of six groups of datasets, the results showed that this method, compared with the existing deep-learning-based hyperspectral image classification methods, could satisfactorily address the issues of degeneration of the deep network and “the same object with distinct spectra, and distinct objects with the same spectrum.” It could also effectively improve the terrain classification accuracy of hyperspectral images, as evinced by the overall classification accuracies of all classes of terrain objects in the six groups of datasets: 97.698%, 98.851%, 99.54%, 97.961%, 97.698%, and 99.138%.

A Novel 2D-3D CNN with Spectral-Spatial Multi-Scale Feature Fusion for Hyperspectral Image Classification

Multifarious hyperspectral image (HSI) classification methods based on convolutional neural networks (CNN) have been gradually proposed and achieve a promising classification performance. However, hyperspectral image classification still suffers from various challenges, including abundant redundant information, insufficient spectral-spatial representation, irregular class distribution, and so forth. To address these issues, we propose a novel 2D-3D CNN with spectral-spatial multi-scale feature fusion for hyperspectral image classification, which consists of two feature extraction streams, a feature fusion module as well as a classification scheme. First, we employ two diverse backbone modules for feature representation, that is, the spectral feature and the spatial feature extraction streams. The former utilizes a hierarchical feature extraction module to capture multi-scale spectral features, while the latter extracts multi-stage spatial features by introducing a multi-level fusion structure. With these network units, the category attribute information of HSI can be fully excavated. Then, to output more complete and robust information for classification, a multi-scale spectral-spatial-semantic feature fusion module is presented based on a Decomposition-Reconstruction structure. Last of all, we innovate a classification scheme to lift the classification accuracy. Experimental results on three public datasets demonstrate that the proposed method outperforms the state-of-the-art methods.

Convolutional Neural Network Based on Bandwise-Independent Convolution and Hard Thresholding for Hyperspectral Band Selection

Band selection has been widely utilized in hyperspectral image (HSI) classification to reduce the dimensionality of HSIs. Recently, deep-learning-based band selection has become of great interest. However, existing deep-learning-based methods usually implement band selection and classification in isolation, or evaluate selected spectral bands by training the deep network repeatedly, which may lead to the loss of discriminative bands and increased computational cost. In this article, a novel convolutional neural network (CNN) based on bandwise-independent convolution and hard thresholding (BHCNN) is proposed, which combines band selection, feature extraction, and classification into an end-to-end trainable network. In BHCNN, a band selection layer is constructed by designing bandwise 1×1 convolutions, which perform for each spectral band of input HSIs independently. Then, hard thresholding is utilized to constrain the weights of convolution kernels with unselected spectral bands to zero. In this case, these weights are difficult to update. To optimize these weights, the straight-through estimator (STE) is devised by approximating the gradient. Furthermore, a novel coarse-to-fine loss calculated by full and selected spectral bands is defined to improve the interpretability of STE. In the subsequent layers of BHCNN, multiscale 3-D dilated convolutions are constructed to extract joint spatial-spectral features from HSIs with selected spectral bands. The experimental results on several HSI datasets demonstrate that the proposed method uses selected spectral bands to achieve more encouraging classification performance than current state-of-the-art band selection methods.

A 3D-2D Convolutional Neural Network and Transfer Learning for Hyperspectral Image Classification

As one of the fast evolution of remote sensing and spectral imagery techniques, hyperspectral image (HSI) classification has attracted considerable attention in various fields, including land survey, resource monitoring, and among others. Nonetheless, due to a lack of distinctiveness in the hyperspectral pixels of separate classes, there is a recurrent inseparability obstacle in the primary space. Additionally, an open challenge stems from examining efficient techniques that can speedily classify and interpret the spectral-spatial data bands within a more precise computational time. Hence, in this work, we propose a 3D-2D convolutional neural network and transfer learning model where the early layers of the model exploit 3D convolutions to modeling spectral-spatial information. On top of it are 2D convolutional layers to handle semantic abstraction mainly. Toward simplicity and a highly modularized network for image classification, we leverage the ResNeXt-50 block for our model. Furthermore, improving the separability among classes and balance of the interclass and intraclass criteria, we engaged principal component analysis (PCA) for the best orthogonal vectors for representing information from HSIs before feeding to the network. The experimental result shows that our model can efficiently improve the hyperspectral imagery classification, including an instantaneous representation of the spectral-spatial information. Our model evaluation on five publicly available hyperspectral datasets, Indian Pines (IP), Pavia University Scene (PU), Salinas Scene (SA), Botswana (BS), and Kennedy Space Center (KSC), was performed with a high classification accuracy of 99.85%, 99.98%, 100%, 99.82%, and 99.71%, respectively. Quantitative results demonstrated that it outperformed several state-of-the-arts (SOTA), deep neural network-based approaches, and standard classifiers. Thus, it has provided more insight into hyperspectral image classification.

Group-Wise Learning for Aurora Image Classification With Multiple Representations

In conventional aurora image classification methods, it is general to employ only one single feature representation to capture the morphological characteristics of aurora images, which is difficult to describe the complicated morphologies of different aurora categories. Although several studies have proposed to use multiple feature representations, the inherent correlation among these representations are usually neglected. To address this problem, we propose a group-wise learning (GWL) method for the automatic aurora image classification using multiple representations. Specifically, we first extract the multiple feature representations for aurora images, and then construct a graph in each of multiple feature spaces. To model the correlation among different representations, we partition multiple graphs into several groups via a clustering algorithm. We further propose a GWL model to automatically estimate class labels for aurora images and optimal weights for the multiple representations in a data-driven manner. Finally, we develop a label fusion approach to make a final classification decision for new testing samples. The proposed GWL method focuses on the diverse properties of multiple feature representations, by clustering the correlated representations into the same group. We evaluate our method on an aurora image data set that contains 12 682 aurora images from 19 days. The experimental results demonstrate that the proposed GWL method achieves approximately 6% improvement in terms of classification accuracy, compared to the methods using a single feature representation.

Local Manifold-Based Sparse Discriminant Learning for Feature Extraction of Hyperspectral Image

Hyperspectral image (HSI) generally contains a complex manifold structure and strong sparse correlation in its nonlinear high-dimensional data space. However, the existing manifold learning and sparse learning methods usually consider the manifold structure and sparse relationship separately rather than combining manifold and sparse properties to discover the intrinsic information in the original data. To simultaneously reveal the complex sparse relation and manifold structure of HSI, a novel feature extraction (FE) method, called local manifold-based sparse discriminant learning (LMSDL), has been proposed on the basis of manifold learning and sparse representation (SR). The LMSDL method first designs a new sparse optimization model called local manifold-based SR (LMSR) to reveal the local manifold-based sparse structure of data. Then, two geometrical sparse graphs are constructed to represent the discriminant relationship between samples and the geometrical and sparse neighbors. An objective function is constructed via geometrical sparse graphs and reconstruction points to learn a projection matrix for FE. The LMSDL effectively reveals the complex sparse relation and manifold structure in high-dimensional data, and it enhances the representation ability of extracted features for HSI classification significantly. The experimental results on the three real HSI datasets show that the proposed LMSDL algorithm possesses better performance in comparison with some state-of-the-art FE methods.

L₁ Sparsity-Regularized Attention Multiple-Instance Network for Hyperspectral Target Detection

Attention-based deep multiple-instance learning (MIL) has been applied to many machine-learning tasks with imprecise training labels. It is also appealing in hyperspectral target detection, which only requires the label of an area containing some targets, relaxing the effort of labeling the individual pixel in the scene. This article proposes an L1 sparsity-regularized attention multiple-instance neural network (L1-attention MINN) for hyperspectral target detection with imprecise labels that enforces the discrimination of false-positive instances from positively labeled bags. The sparsity constraint applied to the attention estimated for the positive training bags strictly complies with the definition of MIL and maintains better discriminative ability. The proposed algorithm has been evaluated on both simulated and real-field hyperspectral (subpixel) target detection tasks, where advanced performance has been achieved over the state-of-the-art comparisons, showing the effectiveness of the proposed method for target detection from imprecisely labeled hyperspectral data.

Scheduling-Guided Automatic Processing of Massive Hyperspectral Image Classification on Cloud Computing Architectures

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

Image-domain Material Decomposition for Spectral CT using a Generalized Dictionary Learning

The spectral computed tomography (CT) has huge advantages by providing accurate material information. Unfortunately, due to the instability or overdetermination of material decomposition model, the accuracy of material decomposition can be compromised in practice. Very recently, the dictionary learning based image-domain material decomposition (DLIMD) can obtain high accuracy for material decompositions from reconstructed spectral CT images. This method can explore the correlation of material components to some extent by training a unified dictionary from all material images. In addition, the dictionary learning based prior as a penalty is applied on material components independently, and many parameters would be carefully elaborated in practice. Because the concentration of contrast agent in clinical applications is low, it can result in data inconsistency for dictionary based representation during the iteration process. To avoid the aforementioned limitations and further improve the accuracy of materials, we first construct a generalized dictionary learning based image-domain material decomposition (GDLIMD) model. Then, the material tensor image is unfolded along the mode-1 to enhance the correlation of different materials. Finally, to avoid the data inconsistency of low iodine contrast, a normalization strategy is employed. Both physical phantom and tissue-synthetic phantom experiments demonstrate the proposed GDLIMD method outperforms the DLIMD and direct inversion (DI) methods.

Improved Transformer Net for Hyperspectral Image Classification

In recent years, deep learning has been successfully applied to hyperspectral image classification (HSI) problems, with several convolutional neural network (CNN) based models achieving an appealing classification performance. However, due to the multi-band nature and the data redundancy of the hyperspectral data, the CNN model underperforms in such a continuous data domain. Thus, in this article, we propose an end-to-end transformer model entitled SAT Net that is appropriate for HSI classification and relies on the self-attention mechanism. The proposed model uses the spectral attention mechanism and the self-attention mechanism to extract the spectral–spatial features of the HSI image, respectively. Initially, the original HSI data are remapped into multiple vectors containing a series of planar 2D patches after passing through the spectral attention module. On each vector, we perform linear transformation compression to obtain the sequence vector length. During this process, we add the position–coding vector and the learnable–embedding vector to manage capturing the continuous spectrum relationship in the HSI at a long distance. Then, we employ several multiple multi-head self-attention modules to extract the image features and complete the proposed network with a residual network structure to solve the gradient dispersion and over-fitting problems. Finally, we employ a multilayer perceptron for the HSI classification. We evaluate SAT Net on three publicly available hyperspectral datasets and challenge our classification performance against five current classification methods employing several metrics, i.e., overall and average classification accuracy and Kappa coefficient. Our trials demonstrate that SAT Net attains a competitive classification highlighting that a Self-Attention Transformer network and is appealing for HSI classification.

Learning dual-margin model for visual tracking

Existing trackers usually exploit robust features or online updating mechanisms to deal with target variations which is a key challenge in visual tracking. However, the features being robust to variations remain little spatial information, and existing online updating methods are prone to overfitting. In this paper, we propose a dual-margin model for robust and accurate visual tracking. The dual-margin model comprises an intra-object margin between different target appearances and an inter-object margin between the target and the background. The proposed method is able to not only distinguish the target from the background but also perceive the target changes, which tracks target appearance changing and facilitates accurate target state estimation. In addition, to exploit rich off-line video data and learn general rules of target appearance variations, we train the dual-margin model on a large off-line video dataset. We perform tracking under a Siamese framework using the constructed appearance set as templates. The proposed method achieves accurate and robust tracking performance on five public datasets while running in real-time. The favorable performance against the state-of-the-art methods demonstrates the effectiveness of the proposed algorithm.

Hyperspectral Imaging for Identification of an Invasive Plant Mikania micrantha Kunth

Mile-a-minute weed (Mikania micrantha Kunth) is considered as one of top 100 most dangerous invasive species in the world. A fast and accurate detection technology will be needed to identify M. micrantha. It will help to mitigate the extensive ecologic and economic damage on our ecosystems caused by this alien plant. Hyperspectral technology fulfills the above requirement. However, when working with hyperspectral images, preprocessing, dimension reduction, and classifier are fundamental to achieving reliable recognition accuracy and efficiency. The spectral data of M. micrantha were collected using hyperspectral imaging in the spectral range of 450-998 nm. A different combination of preprocessing methods, principal component analysis (for dimension reduction), and three classifiers were used to analyze the collected hyperspectral images. The results showed that a combination of Savitzky-Golay (SG) smoothing, principal component analysis (PCA), and random forest (RF) achieved an accuracy (A) of 88.71%, an average accuracy (AA) of 88.68%, and a Kappa of 0.7740 with an execution time of 9.647 ms. In contrast, the combination of SG, PCA and a support vector machine (SVM) resulted in a weaker performance in terms of A (84.68%), AA(84.66%), and Kappa (0.6934), but with less execution time (1.318 ms). According to the requirements for specific identification accuracy and time cost, SG-PCA-RF and SG-PCA-SVM might represent two promising methods for recognizing M. micrantha in the wild.

Large-Scale, Multiple Level-of-Detail Change Detection from Remote Sensing Imagery Using Deep Visual Feature Clustering

In the era of big data, where massive amounts of remotely sensed imagery can be obtained from various satellites accompanied by the rapid change in the surface of the Earth, new techniques for large-scale change detection are necessary to facilitate timely and effective human understanding of natural and human-made phenomena. In this research, we propose a chip-based change detection method that is enabled by using deep neural networks to extract visual features. These features are transformed into deep orthogonal visual features that are then clustered based on land cover characteristics. The resulting chip cluster memberships allow arbitrary level-of-detail change analysis that can also support irregular geospatial extent based agglomerations. The proposed methods naturally support cross-resolution temporal scenes without requiring normalization of the pixel resolution across scenes and without requiring pixel-level coregistration processes. This is achieved with configurable spatial locality comparisons between years, where the aperture of a unit of measure can be a single chip, a small neighborhood of chips, or a large irregular geospatial region. The performance of our proposed method has been validated using various quantitative and statistical metrics in addition to presenting the visual geo-maps and the percentage of the change. The results show that our proposed method efficiently detected the change from a large scale area.

Dimensionality Reduction of Hyperspectral Image Based on Local Constrained Manifold Structure Collaborative Preserving Embedding

Graph learning is an effective dimensionality reduction (DR) manner to analyze the intrinsic properties of high dimensional data, it has been widely used in the fields of DR for hyperspectral image (HSI) data, but they ignore the collaborative relationship between sample pairs. In this paper, a novel supervised spectral DR method called local constrained manifold structure collaborative preserving embedding (LMSCPE) was proposed for HSI classification. At first, a novel local constrained collaborative representation (CR) model is designed based on the CR theory, which can obtain more effective collaborative coefficients to characterize the relationship between samples pairs. Then, an intraclass collaborative graph and an interclass collaborative graph are constructed to enhance the intraclass compactness and the interclass separability, and a local neighborhood graph is constructed to preserve the local neighborhood structure of HSI. Finally, an optimal objective function is designed to obtain a discriminant projection matrix, and the discriminative features of various land cover types can be obtained. LMSCPE can characterize the collaborative relationship between sample pairs and explore the intrinsic geometric structure in HSI. Experiments on three benchmark HSI data sets show that the proposed LMSCPE method is superior to the state-of-the-art DR methods for HSI classification.

Error-Tolerant Deep Learning for Remote Sensing Image Scene Classification

Due to its various application potentials, the remote sensing image scene classification (RSSC) has attracted a broad range of interests. While the deep convolutional neural network (CNN) has recently achieved tremendous success in RSSC, its superior performances highly depend on a large number of accurately labeled samples which require lots of time and manpower to generate for a large-scale remote sensing image scene dataset. In contrast, it is not only relatively easy to collect coarse and noisy labels but also inevitable to introduce label noise when collecting large-scale annotated data in the remote sensing scenario. Therefore, it is of great practical importance to robustly learn a superior CNN-based classification model from the remote sensing image scene dataset containing non-negligible or even significant error labels. To this end, this article proposes a new RSSC-oriented error-tolerant deep learning (RSSC-ETDL) approach to mitigate the adverse effect of incorrect labels of the remote sensing image scene dataset. In our proposed RSSC-ETDL method, learning multiview CNNs and correcting error labels are alternatively conducted in an iterative manner. It is noted that to make the alternative scheme work effectively, we propose a novel adaptive multifeature collaborative representation classifier (AMF-CRC) that benefits from adaptively combining multiple features of CNNs to correct the labels of uncertain samples. To quantitatively evaluate the performance of error-tolerant methods in the remote sensing domain, we construct remote sensing image scene datasets with: 1) simulated noisy labels by corrupting the open datasets with varying error rates and 2) real noisy labels by deploying the greedy annotation strategies that are practically used to accelerate the process of annotating remote sensing image scene datasets. Extensive experiments on these datasets demonstrate that our proposed RSSC-ETDL approach outperforms the state-of-the-art approaches.

Robust Classification Technique for Hyperspectral Images Based on 3D-Discrete Wavelet Transform

Hyperspectral image classification is an emerging and interesting research area that has attracted several researchers to contribute to this field. Hyperspectral images have multiple narrow bands for a single image that enable the development of algorithms to extract diverse features. Three-dimensional discrete wavelet transform (3D-DWT) has the advantage of extracting the spatial and spectral information simultaneously. Decomposing an image into a set of spatial–spectral components is an important characteristic of 3D-DWT. It has motivated us to perform the proposed research work. The novelty of this work is to bring out the features of 3D-DWT applicable to hyperspectral images classification using Haar, Fejér-Korovkin and Coiflet filters. Three-dimensional-DWT is implemented with the help of three stages of 1D-DWT. The first two stages of 3D-DWT are extracting spatial resolution, and the third stage is extracting the spectral content. In this work, the 3D-DWT features are extracted and fed to the following classifiers (i) random forest (ii) K-nearest neighbor (KNN) and (iii) support vector machine (SVM). Exploiting both spectral and spatial features help the classifiers to provide a better classification accuracy. A comparison of results was performed with the same classifiers without DWT features. The experiments were performed using Salinas Scene and Indian Pines hyperspectral datasets. From the experiments, it has been observed that the SVM with 3D-DWT features performs better in terms of the performance metrics such as overall accuracy, average accuracy and kappa coefficient. It has shown significant improvement compared to the state of art techniques. The overall accuracy of 3D-DWT+SVM is 88.3%, which is 14.5% larger than that of traditional SVM (77.1%) for the Indian Pines dataset. The classification map of 3D-DWT + SVM is more closely related to the ground truth map.

Candidate region aware nested named entity recognition

Named entity recognition (NER) is crucial in various natural language processing (NLP) tasks. However, the nested entities which are common in practical corpus are often ignored in most of current NER models. To extract the nested entities, two categories of models (i.e., feature-based and neural network-based approaches) are proposed. However, the feature-based models suffer from the complicated feature engineering and often heavily rely on the external resources. Discarding the heavy feature engineering, recent neural network-based methods which treat the nested NER as a classification task are designed but still suffer from the heavy class imbalance issue and the high computational cost. To solve these problems, we propose a neural multi-task model with two modules: Binary Sequence Labeling and Candidate Region Classification to extract the nested entities. Extensive experiments are conducted on the public datasets. Comparing with recent neural network-based approaches, our proposed model achieves the better performance and obtains the higher efficiency.

Advanced Meta-Heuristics, Convolutional Neural Networks, and Feature Selectors for Efficient COVID-19 X-Ray Chest Image Classification

The chest X-ray is considered a significant clinical utility for basic examination and diagnosis. The human lung area can be affected by various infections, such as bacteria and viruses, leading to pneumonia. Efficient and reliable classification method facilities the diagnosis of such infections. Deep transfer learning has been introduced for pneumonia detection from chest X-rays in different models. However, there is still a need for further improvements in the feature extraction and advanced classification stages. This paper proposes a classification method with two stages to classify different cases from the chest X-ray images based on a proposed Advanced Squirrel Search Optimization Algorithm (ASSOA). The first stage is the feature learning and extraction processes based on a Convolutional Neural Network (CNN) model named ResNet-50 with image augmentation and dropout processes. The ASSOA algorithm is then applied to the extracted features for the feature selection process. Finally, the Multi-layer Perceptron (MLP) Neural Network's connection weights are optimized by the proposed ASSOA algorithm (using the selected features) to classify input cases. A Kaggle chest X-ray images (Pneumonia) dataset consists of 5,863 X-rays is employed in the experiments. The proposed ASSOA algorithm is compared with the basic Squirrel Search (SS) optimization algorithm, Grey Wolf Optimizer (GWO), and Genetic Algorithm (GA) for feature selection to validate its efficiency. The proposed (ASSOA + MLP) is also compared with other classifiers, based on (SS + MLP), (GWO + MLP), and (GA + MLP), in performance metrics. The proposed (ASSOA + MLP) algorithm achieved a classification mean accuracy of (99.26%). The ASSOA + MLP algorithm also achieved a classification mean accuracy of (99.7%) for a chest X-ray COVID-19 dataset tested from GitHub. The results and statistical tests demonstrate the high effectiveness of the proposed method in determining the infected cases.

Spectral and Spatial Global Context Attention for Hyperspectral Image Classification

Recently, hyperspectral image (HSI) classification has attracted increasing attention in the remote sensing field. Plenty of CNN-based methods with diverse attention mechanisms (AMs) have been proposed for HSI classification due to AMs being able to improve the quality of feature representations. However, some of the previous AMs squeeze global spatial or channel information directly by pooling operations to yield feature descriptors, which inadequately utilize global contextual information. Besides, some AMs cannot exploit the interactions among channels or positions with the aid of nonlinear transformation well. In this article, a spectral-spatial network with channel and position global context (GC) attention (SSGCA) is proposed to capture discriminative spectral and spatial features. Firstly, a spectral-spatial network is designed to extract spectral and spatial features. Secondly, two novel GC attentions are proposed to optimize the spectral and spatial features respectively for feature enhancement. The channel GC attention is used to capture channel dependencies to emphasize informative features while the position GC attention focuses on position dependencies. Both GC attentions aggregate global contextual features of positions or channels adequately, following a nonlinear transformation. Experimental results on several public HSI datasets demonstrate that the spectral-spatial network with GC attentions outperforms other related methods.

Endmember Estimation with Maximum Distance Analysis

Endmember estimation plays a key role in hyperspectral image unmixing, often requiring an estimation of the number of endmembers and extracting endmembers. However, most of the existing extraction algorithms require prior knowledge regarding the number of endmembers, being a critical process during unmixing. To bridge this, a new maximum distance analysis (MDA) method is proposed that simultaneously estimates the number and spectral signatures of endmembers without any prior information on the experimental data containing pure pixel spectral signatures and no noise, being based on the assumption that endmembers form a simplex with the greatest volume over all pixel combinations. The simplex includes the farthest pixel point from the coordinate origin in the spectral space, which implies that: (1) the farthest pixel point from any other pixel point must be an endmember, (2) the farthest pixel point from any line must be an endmember, and (3) the farthest pixel point from any plane (or affine hull) must be an endmember. Under this scenario, the farthest pixel point from the coordinate origin is the first endmember, being used to create the aforementioned point, line, plane, and affine hull. The remaining endmembers are extracted by repetitively searching for the pixel points that satisfy the above three assumptions. In addition to behaving as an endmember estimation algorithm by itself, the MDA method can co-operate with existing endmember extraction techniques without the pure pixel assumption via generalizing them into more effective schemes. The conducted experiments validate the effectiveness and efficiency of our method on synthetic and real data.

Adaptive Weighting Feature Fusion Approach Based on Generative Adversarial Network for Hyperspectral Image Classification

Recently, generative adversarial network (GAN)-based methods for hyperspectral image (HSI) classification have attracted research attention due to their ability to alleviate the challenges brought by having limited labeled samples. However, several studies have demonstrated that existing GAN-based HSI classification methods are limited in redundant spectral knowledge and cannot extract discriminative characteristics, thus affecting classification performance. In addition, GAN-based methods always suffer from the model collapse, which seriously hinders their development. In this study, we proposed a semi-supervised adaptive weighting feature fusion generative adversarial network (AWF2-GAN) to alleviate these problems. We introduced unlabeled data to address the issue of having a small number of samples. First, to build valid spectral–spatial feature engineering, the discriminator learns both the dense global spectrum and neighboring separable spatial context via well-designed extractors. Second, a lightweight adaptive feature weighting component is proposed for feature fusion; it considers four predictive fusion options, that is, adding or concatenating feature maps with similar or adaptive weights. Finally, for the mode collapse, the proposed AWF2-GAN combines supervised central loss and unsupervised mean minimization loss for optimization. Quantitative results on two HSI datasets show that our AWF2-GAN achieves superior performance over state-of-the-art GAN-based methods.

Dictionary learning based image-domain material decomposition for spectral CT

The potential huge advantage of spectral computed tomography (CT) is that it can provide accurate material identification and quantitative tissue information by material decomposition. However, material decomposition is a typical inverse problem, where the noise can be magnified. To address this issue, we develop a dictionary learning based image-domain material decomposition (DLIMD) method for spectral CT to achieve accurate material components with better image quality. Specifically, a set of image patches are extracted from the mode-1 unfolding of normalized material images decomposed by direct inversion to train a unified dictionary using the K-SVD technique. Then, the DLIMD model is established to explore the redundant similarities of the material images, where the split-Bregman is employed to optimize the model. Finally, more constraints (i.e. volume conservation and the bounds of each pixel within material maps) are integrated into the DLIMD model. Numerical phantom, physical phantom and preclinical experiments are performed to evaluate the performance of the proposed DLIMD in material decomposition accuracy, material image edge preservation and feature recovery.