Unsupervised band selection of medical hyperspectral images guided by data gravitation and weak correlation

BACKGROUND AND OBJECTIVE

Medical hyperspectral images (MHSIs) are used for a contact-free examination of patients without harmful radiation. However, high-dimensionality images contain large amounts of data that are sparsely distributed in a high-dimensional space, which leads to the "curse of dimensionality" (called Hughes' phenomenon) and increases the complexity and cost of data processing and storage. Hence, there is a need for spectral dimensionality reduction before the clinical application of MHSIs. Some dimensionality-reducing strategies have been proposed; however, they distort the data within MHSIs.

METHODS

To compress dimensionality without destroying the original data structure, we propose a method that involves data gravitation and weak correlation-based ranking (DGWCR) for removing bands of noise from MHSIs while clustering signal-containing bands. Band clustering is done by using the connection centre evolution (CCE) algorithm and selecting the most representative bands in each cluster based on the composite force. The bands within the clusters are ranked using the new entropy-containing matrix, and a global ranking of bands is obtained by applying an S-shaped strategy. The source code is available at https://www.github.com/zhangchenglong1116/DGWCR.

RESULTS

Upon feeding the reduced-dimensional images into various classifiers, the experimental results demonstrated that the small number of bands selected by the proposed DGWCR consistently achieved higher classification accuracy than the original data. Unlike other reference methods (e.g. the latest deep-learning-based strategies), DGWCR chooses the spectral bands with the least redundancy and greatest discrimination.

CONCLUSION

In this study, we present a method for efficient band selection for MHSIs that alleviates the "curse of dimensionality". Experiments were validated with three MHSIs in the human brain, and they outperformed several other band selection methods, demonstrating the clinical potential of DGWCR.

Tropical altitudinal gradient soil organic carbon and nitrogen estimation using Specim IQ portable imaging spectrometer

The largest actively cycling terrestrial carbon pool, soil, has been disturbed during latest centuries by human actions through reduction of woody land cover. Soil organic carbon (SOC) content can reliably be estimated in laboratory conditions, but more cost-efficient and mobile techniques are needed for large-scale monitoring of SOC e.g. in remote areas. We demonstrate the capability of a mobile hyperspectral camera operating in the visible-near infrared wavelength range for practical estimation of soil organic carbon (SOC) and nitrogen content, to support efficient monitoring of soil properties. The 191 soil samples were collected in Taita Taveta County, Kenya representing an altitudinal gradient comprising five typical land use types: agroforestry, cropland, forest, shrubland and sisal estate. The soil samples were imaged using a Specim IQ hyperspectral camera under controlled laboratory conditions, and their carbon and nitrogen content was determined with a combustion analyzer. We use machine learning for estimating SOC and N content based on the spectral images, studying also automatic selection of informative wavelengths and quantification of prediction uncertainty. Five alternative methods were all found to perform well with a cross-validated R² of approximately 0.8 and an RMSE of one percentage point, demonstrating feasibility of the proposed imaging setup and computational pipeline.

Selecting informative bands for partial least squares regressions improves their goodness-of-fits to estimate leaf photosynthetic parameters from hyperspectral data

The plant photosynthetic capacity determines the photosynthetic rates of the terrestrial biosphere. Timely approaches to obtain the spatiotemporal variations of the photosynthetic parameters are urgently needed to grasp the gas exchange rhythms of the terrestrial biosphere. While partial least squares regression (PLSR) is a promising way to predict the photosynthetic parameters maximum carboxylation rate (V_cmax) and maximum electron transport rate (J_max) rapidly and non-destructively from hyperspectral data, the approach, however, faces a high risk of overfitting and remains a high hurdle for applications. In this study, we propose to incorporate proper band selection techniques for PLSR analysis to refine the goodness-of-fit (GoF) in estimating V_cmax and J_max. Different band selection procedures coupled with different hyperspectral forms (reflectance, apparent absorption, as well as derivatives) were examined. Our results demonstrate that the GoFs of PLSR models could be greatly improved by combining proper band selection methods (especially the iterative stepwise elimination approach) rather than using full bands as commonly done with PLSR. The results also show that the 1st order derivative spectra had a balance between accuracy (R² = 0.80 for V_cmax, and 0.94 for J_max) and denoising (when a Gaussian noise was added to each leaf reflectance spectrum at each wavelength with a standard deviation of 1%) on retrieving photosynthetic parameters from hyperspectral data. Our results clearly illustrate the advantage of using the band selection approach for PLSR dimensionality reduction and model optimization, highlighting the superiority of using derivative spectra for V_cmax and J_max estimations, which should provide valuable insights for retrieving photosynthetic parameters from hyperspectral remotely sensed data.

Self-spectral learning with GAN based spectral-spatial target detection for hyperspectral image

To alleviate the shortcomings of target detection in only one aspect and reduce redundant information among adjacent bands, we propose a spectral-spatial target detection (SSTD) framework in deep latent space based on self-spectral learning (SSL) with a spectral generative adversarial network (GAN). The concept of SSL is introduced into hyperspectral feature extraction in an unsupervised fashion with the purpose of background suppression and target saliency. In particular, a novel structure-to-structure selection rule that takes full account of the structure, contrast, and luminance similarity is established to interpret the mapping relationship between the latent spectral feature space and the original spectral band space, to generate the optimal spectral band subset without any prior knowledge. Finally, the comprehensive result is achieved by nonlinearly combining the spatial detection on the fused latent features with the spectral detection on the selected band subset and the corresponding selected target signature. This paper paves a novel self-spectral learning way for hyperspectral target detection and identifies sensitive bands for specific targets in practice. Comparative analyses demonstrate that the proposed SSTD method presents superior detection performance compared with CSCR, ACE, CEM, hCEM, and ECEM.

Heavy metal pollution at mine sites estimated from reflectance spectroscopy following correction for skewed data

The heavy metal concentration of soil samples often exhibits a skewed distribution, especially for soil samples from mining areas with an extremely high concentration of heavy metals. In this study, to model soil contamination in mining areas using reflectance spectroscopy, the skewed distribution was corrected and heavy metal concentration estimated. In total, 46 soil samples from a mining area, along with corresponding field soil spectra, were collected. Laboratory spectra of the soil samples and the field spectra were used to estimate copper (Cu) concentration in the mining area. A logarithmic transformation was used to correct the skewed distribution, and based on the sorption of Cu on spectrally active soil constituents, the spectral bands associated with iron oxides were extracted from the visible and near-infrared (VNIR) region and used in the estimation. A genetic algorithm was adopted for band selection, and partial least squares regression was used to calibrate the estimation model. After transforming the distribution of Cu concentration, the accuracies (R²) of the estimation of Cu concentration using laboratory and field spectra separately were 0.94 and 0.96. The results indicate that Cu concentration in the mining area can be estimated using reflectance spectroscopy following correction of skewed distribution.

Hyperspectral band selection using genetic algorithm and support vector machines for early identification of charcoal rot disease in soybean stems

BACKGROUND

Charcoal rot is a fungal disease that thrives in warm dry conditions and affects the yield of soybeans and other important agronomic crops worldwide. There is a need for robust, automatic and consistent early detection and quantification of disease symptoms which are important in breeding programs for the development of improved cultivars and in crop production for the implementation of disease control measures for yield protection. Current methods of plant disease phenotyping are predominantly visual and hence are slow and prone to human error and variation. There has been increasing interest in hyperspectral imaging applications for early detection of disease symptoms. However, the high dimensionality of hyperspectral data makes it very important to have an efficient analysis pipeline in place for the identification of disease so that effective crop management decisions can be made. The focus of this work is to determine the minimal number of most effective hyperspectral wavebands that can distinguish between healthy and diseased soybean stem specimens early on in the growing season for proper management of the disease. 111 hyperspectral data cubes representing healthy and infected stems were captured at 3, 6, 9, 12, and 15 days after inoculation. We utilized inoculated and control specimens from 4 different genotypes. Each hyperspectral image was captured at 240 different wavelengths in the range of 383-1032 nm. We formulated the identification of best waveband combination from 240 wavebands as an optimization problem. We used a combination of genetic algorithm as an optimizer and support vector machines as a classifier for the identification of maximally-effective waveband combination.

RESULTS

A binary classification between healthy and infected soybean stem samples using the selected six waveband combination (475.56, 548.91, 652.14, 516.31, 720.05, 915.64 nm) obtained a classification accuracy of 97% for the infected class. Furthermore, we achieved a classification accuracy of 90.91% for test samples from 3 days after inoculation using the selected six waveband combination.

CONCLUSIONS

The results demonstrated that these carefully-chosen wavebands are more informative than RGB images alone and enable early identification of charcoal rot infection in soybean. The selected wavebands could be used in a multispectral camera for remote identification of charcoal rot infection in soybean.

Trainable spectral difference learning with spatial starting for hyperspectral image denoising

Because of the limited reflected energy and incoming illumination in an individual band, the reflected energy captured by a hyperspectral sensor might be low and there is inevitable noise that significantly decreases the performance of the subsequent analysis. Denoising is therefore of first importance in hyperspectral image (HSI) analysis and interpretation. However, most HSI denoising methods remove noise with the important spectral information being severely distorted. This paper presents an HSI denoising method using trainable spectral difference learning with spatial initialization (called HDnTSDL) aimed at preserving the spectral information. In the proposed HDnTSDL model, a key band is automatically selected and denoised. The denoised key band acts as a starting point to reconstruct the rest of the non-key bands. Meanwhile, a deep convolutional neural network (CNN) with trainable non-linearity functions is proposed to learn the spectral difference mapping. Then, the rest of the non-key bands are denoised under the guidance of the learned spectral difference with the key band as a starting point. Experiments have been conducted on five databases with both indoor and outdoor scenes. Comparative analyses validate that the proposed method: (i) presents superior performance in spatial recovery and spectral preservation, and (ii) requires less computational time than state-of-the-art methods.