Membranous nephropathy classification using microscopic hyperspectral imaging and tensor patch-based discriminative linear regression

Identification of gastric cancer types based on hyperspectral imaging technology

Gastric cancer is becoming the second biggest cause of death from cancer. Treatment and prognosis of different types of gastric cancer vary greatly. However, the routine pathological examination is limited to the tissue level and is easily affected by subjective factors. In our study, we examined gastric mucosal samples from 50 normal tissue and 90 cancer tissues. Hyperspectral imaging technology was used to obtain spectral information. A two-classification model for normal tissue and cancer tissue identification and a four-classification model for cancer type identification are constructed based on the improved deep residual network (IDRN). The accuracy of the two-classification model and four-classification model are 0.947 and 0.965. Hyperspectral imaging technology was used to extract molecular information to realize real-time diagnosis and accurate typing. The results show that hyperspectral imaging technique has good effect on diagnosis and type differentiation of gastric cancer, which is expected to be used in auxiliary diagnosis and treatment.

Nondestructive detection of Pleurotus geesteranus strain degradation based on micro-hyperspectral imaging and machine learning

In the production of edible fungi, the use of degraded strains in cultivation incurs significant economic losses. Based on micro-hyperspectral imaging and machine learning, this study proposes an early, nondestructive method for detecting different degradation degrees of Pleurotus geesteranus strains. In this study, an undegraded strain and three different degradation-level strains were used. During the mycelium growth, 600 micro-hyperspectral images were obtained. Based on the average transmittance spectra of the region of interest (ROI) in the range of 400-1000 nm and images at feature bands, feature spectra and images were extracted using the successive projections algorithm (SPA) and the deep residual network (ResNet50), respectively. Different feature input combinations were utilized to establish support vector machine (SVM) classification models. Based on the results, the spectra-input-based model performed better than the image-input-based model, and feature extraction improved the classification results for both models. The feature-fusion-based SPA+ResNet50-SVM model was the best; the accuracy rate of the test set was up to 90.8%, which was better than the accuracy rates of SPA-SVM (83.3%) and ResNet50-SVM (80.8%). This study proposes a nondestructive method to detect the degradation of Pleurotus geesteranus strains, which could further inspire new methods for the phenotypic identification of edible fungi.

Deep multi-task learning for nephropathy diagnosis on immunofluorescence images

BACKGROUND AND OBJECTIVE

As an advanced technique, immunofluorescence (IF) is one of the most widely-used medical image for nephropathy diagnosis, due to its ease of acquisition with low cost. In practice, the clinically collected IF images are commonly corrupted by blurs at different degrees, mainly because of the inaccurate focus at the acquisition stage. Although deep neural network (DNN) methods achieve the great success in nephropathy diagnosis, their performance dramatically drops over the blurred IF images. This significantly limits the potential of leveraging the advanced DNN techniques in real-world nephropathy diagnosis scenarios.

METHODS

This paper first establishes two IF databases with synthetic blurs (IFVB) and real-world blurs (Real-IF) for nephropathy diagnosis, respectively, including 1,659 patients and 6,521 IF images with various degrees of blurs. According to the analysis on these two databases, we propose a deep hierarchical multi-task learning based nephropathy diagnosis (DeepMT-ND) method to bridge the gap between the low-level vision and high-level medical tasks. Specifically, DeepMT-ND simultaneously handles the main task of automatic nephropathy diagnosis, as well as the auxiliary tasks of image quality assessment (IQA) and de-blurring.

RESULTS

Extensive experiments show the superiority of our DeepMT-ND in terms of diagnosis accuracy and generalization ability. For instance, our method performs better than nephrologists with at least 15.4% and 6.5% accuracy improvements in IFVB and Real-IF, respectively. Meanwhile, our method also achieves comparable performance in two auxiliary tasks of IQA and de-blurring on blurred IF images.

CONCLUSIONS

In this paper, we propose a new DeepMT-ND method for nephropathy diagnosis on blurred IF images. The proposed hierarchical multi-task learning framework provides the new scope to narrow the gap between the low-level vision and high-level medical tasks, and will contribute to nephropathy diagnosis in clinical scenarios. The diagnosis accuracy and generalization ability of DeepMT-ND are experimentally verified to be effective over both synthetic and real-world databases.

SpeCamX: mobile app that turns unmodified smartphones into multispectral imagers

We present the development of SpeCamX, a mobile application that enables an unmodified smartphone into a multispectral imager. Multispectral imaging provides detailed spectral information about objects or scenes, but its accessibility has been limited due to its specialized requirements for the device. SpeCamX overcomes this limitation by utilizing the RGB photographs captured by smartphones and converting them into multispectral images spanning a range of 420 to 680 nm without a need for internal modifications or external attachments. The app also includes plugin functions for extracting medical information from the resulting multispectral data cube. In a clinical study, SpeCamX was used to implement an augmented smartphone bilirubinometer, predicting blood bilirubin levels (BBL) with superior performance in accuracy, efficiency and stability compared to default smartphone cameras. This innovative technology democratizes multispectral imaging, making it accessible to a wider audience and opening new possibilities for both medical and non-medical applications.

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

Spectral Reflectance Recovery from the Quadcolor Camera Signals Using the Interpolation and Weighted Principal Component Analysis Methods

The recovery of surface spectral reflectance using the quadcolor camera was numerically studied. Assume that the RGB channels of the quadcolor camera are the same as the Nikon D5100 tricolor camera. The spectral sensitivity of the fourth signal channel was tailored using a color filter. Munsell color chips were used as reflective surfaces. When the interpolation method or the weighted principal component analysis (wPCA) method is used to reconstruct spectra, using the quadcolor camera can effectively reduce the mean spectral error of the test samples compared to using the tricolor camera. Except for computation time, the interpolation method outperforms the wPCA method in spectrum reconstruction. A long-pass optical filter can be applied to the fourth channel for reducing the mean spectral error. A short-pass optical filter can be applied to the fourth channel for reducing the mean color difference, but the mean spectral error will be larger. Due to the small color difference, the quadcolor camera using an optimized short-pass filter may be suitable as an imaging colorimeter. It was found that an empirical design rule to keep the color difference small is to reduce the error in fitting the color-matching functions using the camera spectral sensitivity functions.

Hyperspectral Imaging for Clinical Applications

Measuring morphological and biochemical features of tissue is crucial for disease diagnosis and surgical guidance, providing clinically significant information related to pathophysiology. Hyperspectral imaging (HSI) techniques obtain both spatial and spectral features of tissue without labeling molecules such as fluorescent dyes, which provides rich information for improved disease diagnosis and treatment. Recent advances in HSI systems have demonstrated its potential for clinical applications, especially in disease diagnosis and image-guided surgery. This review summarizes the basic principle of HSI and optical systems, deep-learning-based image analysis, and clinical applications of HSI to provide insight into this rapidly growing field of research. In addition, the challenges facing the clinical implementation of HSI techniques are discussed.

On the Optimization of Regression-Based Spectral Reconstruction

Spectral reconstruction (SR) algorithms attempt to recover hyperspectral information from RGB camera responses. Recently, the most common metric for evaluating the performance of SR algorithms is the Mean Relative Absolute Error (MRAE)-an ℓ1 relative error (also known as percentage error). Unsurprisingly, the leading algorithms based on Deep Neural Networks (DNN) are trained and tested using the MRAE metric. In contrast, the much simpler regression-based methods (which actually can work tolerably well) are trained to optimize a generic Root Mean Square Error (RMSE) and then tested in MRAE. Another issue with the regression methods is-because in SR the linear systems are large and ill-posed-that they are necessarily solved using regularization. However, hitherto the regularization has been applied at a spectrum level, whereas in MRAE the errors are measured per wavelength (i.e., per spectral channel) and then averaged. The two aims of this paper are, first, to reformulate the simple regressions so that they minimize a relative error metric in training-we formulate both ℓ2 and ℓ1 relative error variants where the latter is MRAE-and, second, we adopt a per-channel regularization strategy. Together, our modifications to how the regressions are formulated and solved leads to up to a 14% increment in mean performance and up to 17% in worst-case performance (measured with MRAE). Importantly, our best result narrows the gap between the regression approaches and the leading DNN model to around 8% in mean accuracy.