System design for accurately estimating the spectral reflectance of art paintings

Spectral reflectance reconstruction based on wideband multi-illuminant imaging and a modified particle swarm optimization algorithm

A method for spectral reflectance factor reconstruction based on wideband multi-illuminant imaging was proposed, using a programmable LED lighting system and modified Bare Bones Particle Swarm Optimization algorithms. From a set of 16 LEDs with different spectral power distributions, nine light sources with correlated color temperatures in the range of 1924 K - 15746 K, most of them daylight simulators, were generated. Samples from three color charts (X-Rite ColorChecker Digital SG, SCOCIE ScoColor paint chart, and SCOCIE ScoColor textile chart), were captured by a color industrial camera under the nine light sources, and used in sequence as training and/or testing colors. The spectral reconstruction models achieved under multi-illuminant imaging were trained and tested using the canonical Bare Bones Particle Swarm Optimization and its proposed modifications, along with six additional and commonly used algorithms. The impacts of different illuminants, illuminant combinations, algorithms, and training colors on reconstruction accuracy were studied comprehensively. The results indicated that training colors covering larger regions of color space give more accurate reconstructions of spectral reflectance factors, and combinations of two illuminants with a large difference of correlated color temperature achieve more than twice the accuracy of that under a single illuminant. Specifically, the average reconstruction error by the method proposed in this paper for patches from two color charts under A + D90 light sources was 0.94 and 1.08 CIEDE2000 color difference units. The results of the experiment also confirmed that some reconstruction algorithms are unsuitable for predicting spectral reflectance factors from multi-illuminant images due to the complexity of optimization problems and insufficient accuracy. The proposed reconstruction method has many advantages, such as being simple in operation, with no requirement of prior knowledge, and easy to implement in non-contact color measurement and color reproduction devices.

Sensor simulation using a spectrum tunable LED system

This study developed a method to simulate the sensor responses and verify the effectiveness on spectral reconstruction by a spectrum tunable LED system. Studies have shown that the spectral reconstruction accuracy could be improved by including multiple channels in a digital camera. However, the real sensors with designed spectral sensitivities were hard to manufacture and validate. Therefore, the presence of a quick and reliable validation mechanism was preferred when performing evaluation. In this study, two novel approaches, i.e., channel-first and illumination-first simulations, were proposed to replicate the designed sensors with the use of a monochrome camera and a spectrum-tunable LED illumination system. In the channel-first method, the spectral sensitivities of three extra sensor channels were optimized theoretically for an RGB camera and then simulated by matching the corresponding illuminants in the LED system. The illumination-first method optimized the spectral power distribution (SPD) of the lights using the LED system, and the extra channels could be determined accordingly. The results of practical experiments showed that the proposed methods were effective to simulate the responses of the extra sensor channels.

Spectral Reflectance Estimation from Camera Responses Using Local Optimal Dataset

A novel method is proposed to estimate surface-spectral reflectance from camera responses using a local optimal reflectance dataset. We adopt a multispectral imaging system that involves an RGB camera capturing multiple images under multiple light sources. A spectral reflectance database is utilized to locally determine the candidates to optimally estimate the spectral reflectance. The proposed estimation method comprises two stages: (1) selecting the local optimal reflectance dataset and (2) determining the best estimate using only the local optimal dataset. In (1), the camera responses are predicted for the respective reflectances in the database, and then the prediction errors are calculated to select the local optimal dataset. In (2), multiple methods are used; in particular, the Wiener and linear minimum mean square error estimators are used to calculate all statistics, based only on the local optimal dataset, and linear and quadratic programming methods are used to solve optimization problems with constraints. Experimental results using different mobile phone cameras show that the estimation accuracy has improved drastically. A much smaller local optimal dataset among spectral reflectance databases is enough to obtain the optimal estimates. The method has potential applications including fields of color science, image science and technology, computer vision, and graphics.

Estimation of the perceptual color gamut on displays

A new method was developed to evaluate the perceived gamut of a display. A multispectral image of a white gypsum sphere was projected onto various highly chromatic lights, producing a series of images with distinct hue perceptions at the gamut boundary of displays. These images were subsequently used in a psychophysical experiment to examine the perceived color gamut. Afterwards, the visual results were further compared with the prediction results from various uniform color spaces (UCSs) and color appearance models (CAMs). The present results demonstrate that CAM16-UCS provides the most accurate prediction across the entire color gamut, whereas the cyan-to-blue region is more poorly predicted than the other hue regions for all CAMs and UCSs investigated.

Hyperspectral image super-resolution via a multi-stage scheme without employing spatial degradation

Recently, it has become popular to obtain a high spatial resolution hyperspectral image (HR-HSI) by fusing a low spatial resolution hyperspectral image (LR-HSI) with a high spatial resolution RGB image (HR-RGB). Existing HSI super-resolution methods are designed based on a known spatial degeneration. In practice, it is difficult to obtain correct spatial degradation, which restricts the performance of existing methods. Therefore, we propose a multi-stage scheme without employing the spatial degradation model. The multi-stage scheme consists of three stages: initialization, modification, and refinement. According to the angle similarity between the HR-RGB pixel and LR-HSI spectra, we first initialize a spectrum for each HR-RGB pixel. Then, we propose a polynomial function to modify the initialized spectrum so that the RGB color values of the modified spectrum are the same as the HR-RGB. Finally, the modified HR-HSI is refined by a proposed optimization model, in which a novel, to the best of our knowledge, spectral-spatial total variation (SSTV) regularizer is investigated to keep the spectral and spatial structure of the reconstructed HR-HSI. The experimental results on two public datasets and our real-world images demonstrate our method outperforms eight state-of-the-art existing methods in terms of both reconstruction accuracy and computational efficiency.

Hyperspectral image super-resolution based on the transfer of both spectra and multi-level features

Existing hyperspectral image (HSI) super-resolution methods fusing a high-resolution RGB image (HR-RGB) and a low-resolution HSI (LR-HSI) always rely on spatial degradation and handcrafted priors, which hinders their practicality. To address these problems, we propose a novel, to the best of our knowledge, method with two transfer models: a window-based linear mixing (W-LM) model and a feature transfer model. Specifically, W-LM initializes a high-resolution HSI (HR-HSI) by transferring the spectra from the LR-HSI to the HR-RGB. By using the proposed feature transfer model, the HR-RGB multi-level features extracted by a pre-trained convolutional neural network (CNN) are then transferred to the initialized HR-HSI. The proposed method fully exploits spectra of LR-HSI and multi-level features of HR-RGB and achieves super-resolution without requiring the spatial degradation model and any handcrafted priors. The experimental results for 32 × super-resolution on two public datasets and our real image set demonstrate the proposed method outperforms eight state-of-the-art existing methods.

SRT: A Spectral Reconstruction Network for GF-1 PMS Data Based on Transformer and ResNet

The time of acquiring remote sensing data was halved after the joint operation of Gao Fen-6 (GF-6) and Gao Fen-1 (GF-1) satellites. Meanwhile, GF-6 added four bands, including the “red-edge” band that can effectively reflect the unique spectral characteristics of crops. However, GF-1 data do not contain these bands, which greatly limits their application to crop-related joint monitoring. In this paper, we propose a spectral reconstruction network (SRT) based on Transformer and ResNet to reconstruct the missing bands of GF-1. SRT is composed of three modules: (1) The transformer feature extraction module (TFEM) fully extracts the correlation features between spectra. (2) The residual dense module (RDM) reconstructs local features and avoids the vanishing gradient problem. (3) The residual global construction module (RGM) reconstructs global features and preserves texture details. Compared with competing methods, such as AWAN, HRNet, HSCNN-D, and M2HNet, the proposed method proved to have higher accuracy by a margin of the mean relative absolute error (MRAE) and root mean squared error (RMSE) of 0.022 and 0.009, respectively. It also achieved the best accuracy in supervised classification based on support vector machine (SVM) and spectral angle mapper (SAM).

An Efficient and Portable LED Multispectral Imaging System and Its Application to Human Tongue Detection

LED illumination-based multispectral imaging has a fast switching ability, high cost-effectiveness, and a simple structure. It has been used in some applications, especially color recognition. In this paper, we introduce an efficient and portable LED multispectral imaging system for human tongue detection. The spatial pixels are categorized based on cosine similarity to reduce the optimization calculation times. Further, segment linear calibration is used to improve the recovery quality. Simulation results show that this method greatly improves the reconstruction speed and that recovered images maintain a high spatial and spectral quality. This LED multispectral imaging system captures images quickly and obtains multispectral images in a timely fashion. We also built a small prototype for human tongue detection in traditional Chinese medicine. The recovered spectra were used to calculate the tongue body color and fur color. By combining these with the spatial information, the fur distribution and fur thickness were analyzed. The results of this study verified the effectiveness of this LED multispectral imaging system. Further experiments will be undertaken for the quantitative analysis of tongue features. The study was approved by the Institutional Review Board (or Ethics Committee) of Shenzhen Institute of Advanced Technology Chinese Academy of Sciences.

Improved method for spectral reflectance estimation and application to mobile phone cameras

We propose an improved method for estimating surface-spectral reflectance from the image data acquired by an RGB digital camera. We suppose a multispectral image acquisition system in the visible range, where a camera captures multiple images for the scene of an object under multiple light sources. First, the observed image data are described using the camera spectral sensitivities, the surface-spectral reflectance, the illuminant spectral power distributions, an additive noise term, and a gain parameter. Then, the optimal reflectance estimate is determined to minimize the mean-square error between the estimate and the original surface-spectral reflectance. We attempt to further improve the estimation accuracy and develop a novel linear estimator in a more general form than the Wiener estimator. Furthermore, we calibrate the imaging system using a reference standard sample. Finally, experiments are performed to validate the proposed method for estimating the surface-spectral reflectance using different mobile phone cameras.

Depth estimation using spectrally varying defocus blur

This paper proposes a method to estimate depth from a single multispectral image by using a lens property known as chromatic aberration. Chromatic aberration causes light passing through a lens to be refracted depending on the wavelength. The refraction causes the angle of rays to vary depending on their wavelength and a change in focal length, which leads to a defocus blur for different wavelengths. We propose a theory to recover a continuous depth map from the blur in a single multispectral image that includes chromatic aberration. The proposed method needs only a standard wide-aperture lens, which naturally exhibits chromatic aberration, and a multispectral camera. Moreover, we use a simple yet effective depth-of-field synthesis method to calculate the derivatives and obtain all-in-focus images necessary to approximate spectral derivatives. We verified the effectiveness of the proposed method on various real-world scenes.

Measurement and Estimation of Spectral Sensitivity Functions for Mobile Phone Cameras

Mobile phone cameras are often significantly more useful than professional digital single-lens reflex (DSLR) cameras. Knowledge of the camera spectral sensitivity function is important in many fields that make use of images. In this study, methods for measuring and estimating spectral sensitivity functions for mobile phone cameras are developed. In the direct measurement method, the spectral sensitivity at each wavelength is measured using monochromatic light. Although accurate, this method is time-consuming and expensive. The indirect estimation method is based on color samples, in which the spectral sensitivities are estimated from the input data of color samples and the corresponding output RGB values from the camera. We first present an imaging system for direct measurements. A variety of mobile phone cameras are measured using the system to create a database of spectral sensitivity functions. The features of the measured spectral sensitivity functions are then studied using principal component analysis (PCA) and the statistical features of the spectral functions extracted. We next describe a normal method to estimate the spectral sensitivity functions using color samples and point out some drawbacks of the method. A method to solve the estimation problem using the spectral features of the sensitivity functions in addition to the color samples is then proposed. The estimation is stable even when only a small number of spectral features are selected. Finally, the results of the experiments to confirm the feasibility of the proposed method are presented. We establish that our method is excellent in terms of both the data volume of color samples required and the estimation accuracy of the spectral sensitivity functions.

Direct estimation of NIR reflection spectra utilizing a snapshot-type spectrometer with photonic crystal multi-spectral filter arrays

We fabricated 16-, 25-, 36-, and 64-channel distributed passband-type multi-spectral filter arrays by utilizing a multilayer-type photonic crystal and integrated them onto a CCD to form a snapshot-type spectroscopic sensor. Reflection spectra from target objects (fruits) under broadband light illumination were estimated directly using the Wiener estimation method. A root mean square error of the reflectivity on the order of 2∼5% was obtained under optical shot noise with 6×6 pixel binning. A number of constituent filters of 36 was sufficient for this type of fruit spectral measurement. We also visualized reflection images at specified wavelengths by applying the estimation method to a multiple filter region on the sensor.

Reflectance spectra reconstruction from trichromatic camera based on kernel partial least square method

A novel spectral reflectance reconstruction method based on kernel partial least square (KPLS) regression is proposed. The proposed method integrates the partial least square algorithm and kernel function to estimate the reflectance spectra from 9-channel multispectral imaging system using commercial trichromatic camera. The performance of the proposed method is demonstrated in comparison with the existing methods using simulated and real camera responses from Munsell Matte color and IT8.7/3 dataset. The experimental results show that the proposed method is superior or at least equivalent to its counterparts and satisfactory enough for color management purpose.

NIR spectrum estimation utilizing a photonic crystal distributed passband-type multiple filter array

A near-infrared (NIR) spectral sensor consisting of a 25-channel dielectric multi-patterned filter array (MFA) and CCD is proposed and fabricated. The MFA consists of a wavy dielectric multilayer with a gradient layer profile on a silica substrate with surface grating. The incoming NIR spectrum is predicted by Wiener estimation utilizing the MFA's spectral responses and a set of training spectral data. Estimation performance is evaluated under various optical shot-noise conditions.

Accurate color imaging of pathology slides using holography and absorbance spectrum estimation of histochemical stains

Holographic microscopy presents challenges for color reproduction due to the usage of narrow-band illumination sources, which especially impacts the imaging of stained pathology slides for clinical diagnoses. Here, an accurate color holographic microscopy framework using absorbance spectrum estimation is presented. This method uses multispectral holographic images acquired and reconstructed at a small number (e.g., three to six) of wavelengths, estimates the absorbance spectrum of the sample, and projects it onto a color tristimulus. Using this method, the wavelength selection is optimized to holographically image 25 pathology slide samples with different tissue and stain combinations to significantly reduce color errors in the final reconstructed images. The results can be used as a practical guide for various imaging applications and, in particular, to correct color distortions in holographic imaging of pathology samples spanning different dyes and tissue types.

Weighted spectral reconstruction method for discrimination of bacterial species with low signal-to-noise ratio Raman measurements

Raman spectroscopy is a label-free and non-destructive spectroscopic technique that has been explored for bacterial identification. However, noise often interferes with the interesting Raman peaks because the Raman signal is inherently weak, especially for bacterial samples. Although this problem can be solved by increasing the exposure time or the power of the excitation laser, a longer acquisition time is required or the risk of sample damage is increased. In contrast, short exposure time and low laser power often lead to inadequate acquisition of Raman scattering, in which the Raman spectra with low signal-to-noise ratio (SNR) is difficult to be further analyzed. In order to quickly and accurately characterize biological samples by using low SNR Raman measurements, a weighted spectral reconstruction based method was developed and tested on Raman spectra with low SNR from 20 bacterial samples of two species. Principal component analysis followed by support vector machine was applied on the reference Raman spectra and the spectra recovered from the low SNR Raman measurements by the proposed method, the traditional spectral reconstruction method, and four other commonly used de-noising methods for the discrimination of bacterial species. The results showed that a classification accuracy of 90% was achieved based on our method, which was comparable to that of the reference Raman spectra and showed significant advantages over other spectral recovery methods. Therefore, the weighted spectral reconstruction method can preserve the most biochemical information for the bacterial species' identification while removing the noise from the low SNR Raman spectra, in which the advantages of lesser sample damage and shorter acquisition time would promote wider biomedical applications of Raman spectroscopy.

Raman spectra recovered from low SNR Raman measurements by weighted spectral reconstruction method show excellent preservation of information about bacterial identification.

Spectral diffuse reflectance and autofluorescence imaging can perform early prediction of blood vessel occlusion in skin flaps

Flap transfer has become a common technique in reconstructive surgery. However, a significant number of compromised skin flaps are not successfully salvaged because the current clinical method for flap assessment relies heavily on the clinician's experience. Vascular occlusion is the major reason for flap failure, thus the accurate and objective early prediction of blood vessel occlusion is vitally important. Our parallel point measurement study has demonstrated the great potential of joint diffuse reflectance and autofluorescence spectroscopy in the early detection and differentiation of venous and arterial occlusion in skin flaps. Unfortunately, the technique of point measurements is not suitable to examine a large skin flap when a high spatial resolution is required. In this study, we attempted to overcome this problem by performing spectral diffuse reflectance and autofluorescence imaging on a rat skin flap model. Both imaging data and reconstructed spectra were used to statistically differentiate control flaps, arterially occluded flaps and venously occluded flaps. Our preliminary results suggest that the technique of joint diffuse reflectance and autofluorescence spectroscopic imaging can achieve high classification accuracy thus could be used to detect and differentiate flaps with venous and arterial occlusion accurately at an early time point in a large skin flap. Typical reconstructed spectra of (a) diffuse reflectance and (b) autofluorescence after normalization.

Data-driven imaging of tissue inflammation using RGB-based hyperspectral reconstruction toward personal monitoring of dermatologic health

Sensitive and accurate assessment of dermatologic inflammatory hyperemia in otherwise grossly normal-appearing skin conditions is beneficial to laypeople for monitoring their own skin health on a regular basis, to patients for looking for timely clinical examination, and to primary care physicians or dermatologists for delivering effective treatments. We propose that mathematical hyperspectral reconstruction from RGB images in a simple imaging setup can provide reliable visualization of hemoglobin content in a large skin area. Without relying on a complicated, expensive, and slow hyperspectral imaging system, we demonstrate the feasibility of determining heterogeneous or multifocal areas of inflammatory hyperemia associated with experimental photocarcinogenesis in mice. We envision that RGB-based reconstructed hyperspectral imaging of subclinical inflammatory hyperemic foci could potentially be integrated with the built-in camera (RGB sensor) of a smartphone to develop a simple imaging device that could offer affordable monitoring of dermatologic health.

Self-training-based spectral image reconstruction for art paintings with multispectral imaging

A self-training-based spectral reflectance recovery method was developed to accurately reconstruct the spectral images of art paintings with multispectral imaging. By partitioning the multispectral images with the k-means clustering algorithm, the training samples are directly extracted from the art painting itself to restrain the deterioration of spectral estimation caused by the material inconsistency between the training samples and the art painting. Coordinate paper is used to locate the extracted training samples. The spectral reflectances of the extracted training samples are acquired indirectly with a spectroradiometer, and the circle Hough transform is adopted to detect the circle measuring area of the spectroradiometer. Through simulation and a practical experiment, the implementation of the proposed method is explained in detail, and it is verified to have better reflectance recovery performance than that using the commercial target and is comparable to the approach using a painted color target.

Preparation of Silver Nanoparticles in Poly(N-vinylpyrrolidone)/Ethanol Solutions

Silver nanoparticles have been synthesized from Poly(N-vinylpyrrolidone) (PVP)/ethanol solutions, and six different PVP to silver nitrate weight ratios (PVP:AgNO3) are studied in reduction of silver nitrate in ethanol with the presence of PVP as a stabilizer. The produced silver nanoparticles showed strong plasmon resonance peak centered at around 405[Formula: see text]nm in UV-Vis spectra. The particle morphologies were also examined and compared under secondary electron microscopy (SEM) and transmission electron microscopy (TEM). Energy dispersion X-ray spectroscopy (EDX) was utilized to determine the formation of silver nanoparticles. We found that the particle size and morphology were strongly dependent on the PVP:AgNO3 weight ratio. The average size of silver particles decreased from 19.25[Formula: see text]nm to 10.03[Formula: see text]nm as the weight ratio of PVP:AgNO3 increased from 1:1 to 20:1.

Image-based spectral transmission estimation using "sensitivity comparison"

Although digital cameras have been used for spectral reflectance estimation, transmission measurement has rarely been considered in studies. This study presents a method named sensitivity comparison (SC) for spectral transmission estimation. The method needs neither a priori knowledge from the samples nor statistical information of a given reflectance dataset. As with spectrophotometers, the SC method needs one shot for calibration and another shot for measurement. The method exploits the sensitivity of the camera in the absence and presence of transparent colored objects for transmission estimation. 138 Kodak Wratten Gelatin filter transmissions were used for controlling the proposed method. Using modeling of the imaging system in different levels of noise, the performance of the proposed method was compared with a training-based Matrix R method. For checking the performance of the SC method in practice, 33 manmade colored transparent films were used in a conventional three-channel camera. The method generated promising results using different error metrics.

Spectral bidirectional texture function reconstruction by fusing multiple-color and spectral images

Spectral bidirectional texture function (BTF) is essential for accurate reproduction of material appearance due to its nature of conveying both spatial and spectral information. A practical issue is that the acquisition of raw spectral BTFs is time-consuming. To resolve the limitation, this paper proposes a novel framework for efficient spectral BTF acquisition and reconstruction. The framework acquires red-green-blue (RGB) BTF images and just one spectral image. The full spectral BTFs are reconstructed by fusing the RGB and spectral images based on nonnegative matrix factorization (NMF). Experimental results indicate that the accuracy of spectral reflectance reconstruction is higher than that of existing algorithms. With the reconstructed spectral BTFs, the material appearance can be reproduced with high fidelity under various illumination conditions.

Metric for evaluation of filter efficiency in spectral cameras

Although metric functions that show the performance of a colorimetric imaging device have been investigated, a metric for performance analysis of a set of filters in wideband filter-based spectral cameras has rarely been studied. Based on a generalization of Vora's Measure of Goodness (MOG) and the spanning theorem, a single function metric that estimates the effectiveness of a filter set is introduced. The improved metric, named MMOG, varies between one, for a perfect, and zero, for the worst possible set of filters. Results showed that MMOG exhibits a trend that is more similar to the mean square of spectral reflectance reconstruction errors than does Vora's MOG index, and it is robust to noise in the imaging system. MMOG as a single metric could be exploited for further analysis of manufacturing errors.

Potential of hybrid adaptive filtering in inflammatory lesion detection from capsule endoscopy images

A new feature extraction technique for the detection of lesions created from mucosal inflammations in Crohn’s disease, based on wireless capsule endoscopy (WCE) images processing is presented here. More specifically, a novel filtering process, namely Hybrid Adaptive Filtering (HAF), was developed for efficient extraction of lesion-related structural/textural characteristics from WCE images, by employing Genetic Algorithms to the Curvelet-based representation of images. Additionally, Differential Lacunarity (DLac) analysis was applied for feature extraction from the HAF-filtered images. The resulted scheme, namely HAF-DLac, incorporates support vector machines for robust lesion recognition performance. For the training and testing of HAF-DLac, an 800-image database was used, acquired from 13 patients who undertook WCE examinations, where the abnormal cases were grouped into mild and severe, according to the severity of the depicted lesion, for a more extensive evaluation of the performance. Experimental results, along with comparison with other related efforts, have shown that the HAF-DLac approach evidently outperforms them in the field of WCE image analysis for automated lesion detection, providing higher classification results, up to 93.8% (accuracy), 95.2% (sensitivity), 92.4% (specificity) and 92.6% (precision). The promising performance of HAF-DLac paves the way for a complete computer-aided diagnosis system that could support physicians’ clinical practice.

Updated version of an interim connection space LabPQR for spectral color reproduction: LabLab

In this paper, we propose a new interim connection space (ICS) called LabLab, which is an updated version of LabPQR, to overcome the drawback that the last three dimensions of LabPQR have no definite colorimetric meanings. We extended and improved the method by which the first three dimensions of LabPQR are deduced to obtain an ICS consisting of two sets of CIELAB values under different illuminants, and the reconstructed spectra from LabLab were obtained by minimizing colorimetric errors by means of the computational formula of the CIE-XYZ tristimulus values combined with least-squares best fit. The improvement obtained from the proposed method was tested to compress and reconstruct the reflectance spectra of the 1950 Natural Color System color chips and more than 50,000 ISO SOCS color patches as well as six multispectral images acquired by multispectral image acquisition systems using 1600 glossy Munsell color chips as training samples. The performance was evaluated by the mean values of color differences between the original and reconstructed spectra under the CIE 1931 standard colorimetric observer and the CIE standard illuminants D50, D55, D65, D75, F2, F7, F11, and A as well as five multichip white LED light sources. The mean and maximum values of the root mean square errors between the original and reconstructed spectra were also calculated. The experimental results show that the proposed three LabLab interim connection spaces significantly outperform principal component analysis, LabPQR, XYZLMS, Fairman-Brill, and LabRGB in colorimetric reconstruction accuracy at the cost of slight reduction of spectral reconstruction accuracy and illuminant independence of color differences of the suggested LabLab interim connection spaces outperform other interim connection spaces. In addition, the presented LabLab interim connection spaces could be quite compatible with the extensively used colorimetric management system since each dimension has definite colorimetric meanings and is perceptually uniform.

Improved method for skin reflectance reconstruction from camera images

A improved spectral reflectance reconstruction method is developed to transform camera RGB to spectral reflectance for skin images. Rather than using conventional direct or two-step processes, we transform camera RGB to skin reflectance directly using a principal component analysis (PCA) approach. The novelty in our direct method (RGB to spectra) is the use of a skin-specific colour characterisation chart with spectra closer to human skin spectra, and a new database of skin reflectances to derive the PCA bases. The experimental results using the facial images of 17 subjects demonstrate that our new direct method gives a significantly better performance than conventional, two-step methods and direct methods with traditional characterization charts. This new spectral reconstruction algorithm is sufficiently precise to reconstruct spectral properites relating to chromophores and its performance is within the acceptable range for maxillofacial soft tissue prostheses (error < 3 ΔE*_ab units).

Numerical Demultiplexing of Color Image Sensor Measurements via Non-linear Random Forest Modeling

The simultaneous capture of imaging data at multiple wavelengths across the electromagnetic spectrum is highly challenging, requiring complex and costly multispectral image devices. In this study, we investigate the feasibility of simultaneous multispectral imaging using conventional image sensors with color filter arrays via a novel comprehensive framework for numerical demultiplexing of the color image sensor measurements. A numerical forward model characterizing the formation of sensor measurements from light spectra hitting the sensor is constructed based on a comprehensive spectral characterization of the sensor. A numerical demultiplexer is then learned via non-linear random forest modeling based on the forward model. Given the learned numerical demultiplexer, one can then demultiplex simultaneously-acquired measurements made by the color image sensor into reflectance intensities at discrete selectable wavelengths, resulting in a higher resolution reflectance spectrum. Experimental results demonstrate the feasibility of such a method for the purpose of simultaneous multispectral imaging.

Accurate Measurements of Spectral Reflectance in Picasso's Guernica Painting

The use of non-invasive spectral measurements to control the conservation status is a part of the preventive conservation of artworks which nowadays is becoming increasingly interesting. This paper describes how to use a spectral measuring device and an illumination system specifically designed for such a task in a very large dimension artwork painting (7.8 m wide × 3.5 m high). The system, controlled by a Cartesian robot, allows spectral measurements in a spectral range of 400-780 nm. The measured data array has a total of 2201 circular regions with 5.5 mm spot diameter placed on a square grid. Colorimetric calculations performed from these spectral measurements may be used to characterize color shifts related to reflectance changes in specific areas of the paint. A color shifting from the expected gray has been shown.

Multispectral Image Out-of-Focus Deblurring Using Interchannel Correlation

Out-of-focus blur occurs frequently in multispectral imaging systems when the camera is well focused at a specific (reference) imaging channel. As the effective focal lengths of the lens are wavelength dependent, the blurriness levels of the images at individual channels are different. This paper proposes a multispectral image deblurring framework to restore out-of-focus spectral images based on the characteristic of interchannel correlation (ICC). The ICC is investigated based on the fact that a high-dimensional color spectrum can be linearly approximated using rather a few number of intrinsic spectra. In the method, the spectral images are classified into an out-of-focus set and a well-focused set via blurriness computation. For each out-of-focus image, a guiding image is derived from the well-focused spectral images and is used as the image prior in the deblurring framework. The out-of-focus blur is modeled as a Gaussian point spread function, which is further employed as the blur kernel prior. The regularization parameters in the image deblurring framework are determined using generalized cross validation, and thus the proposed method does not need any parameter tuning. The experimental results validate that the method performs well on multispectral image deblurring and outperforms the state of the arts.

Optimization of advanced Wiener estimation methods for Raman reconstruction from narrow-band measurements in the presence of fluorescence background

Raman spectroscopy has shown great potential in biomedical applications. However, intrinsically weak Raman signals cause slow data acquisition especially in Raman imaging. This problem can be overcome by narrow-band Raman imaging followed by spectral reconstruction. Our previous study has shown that Raman spectra free of fluorescence background can be reconstructed from narrow-band Raman measurements using traditional Wiener estimation. However, fluorescence-free Raman spectra are only available from those sophisticated Raman setups capable of fluorescence suppression. The reconstruction of Raman spectra with fluorescence background from narrow-band measurements is much more challenging due to the significant variation in fluorescence background. In this study, two advanced Wiener estimation methods, i.e. modified Wiener estimation and sequential weighted Wiener estimation, were optimized to achieve this goal. Both spontaneous Raman spectra and surface enhanced Raman spectra were evaluated. Compared with traditional Wiener estimation, two advanced methods showed significant improvement in the reconstruction of spontaneous Raman spectra. However, traditional Wiener estimation can work as effectively as the advanced methods for SERS spectra but much faster. The wise selection of these methods would enable accurate Raman reconstruction in a simple Raman setup without the function of fluorescence suppression for fast Raman imaging.

Differentiating Biological Colours with Few and Many Sensors: Spectral Reconstruction with RGB and Hyperspectral Cameras

BACKGROUND

The ability to discriminate between two similar or progressively dissimilar colours is important for many animals as it allows for accurately interpreting visual signals produced by key target stimuli or distractor information. Spectrophotometry objectively measures the spectral characteristics of these signals, but is often limited to point samples that could underestimate spectral variability within a single sample. Algorithms for RGB images and digital imaging devices with many more than three channels, hyperspectral cameras, have been recently developed to produce image spectrophotometers to recover reflectance spectra at individual pixel locations. We compare a linearised RGB and a hyperspectral camera in terms of their individual capacities to discriminate between colour targets of varying perceptual similarity for a human observer.

MAIN FINDINGS

(1) The colour discrimination power of the RGB device is dependent on colour similarity between the samples whilst the hyperspectral device enables the reconstruction of a unique spectrum for each sampled pixel location independently from their chromatic appearance. (2) Uncertainty associated with spectral reconstruction from RGB responses results from the joint effect of metamerism and spectral variability within a single sample.

CONCLUSION

(1) RGB devices give a valuable insight into the limitations of colour discrimination with a low number of photoreceptors, as the principles involved in the interpretation of photoreceptor signals in trichromatic animals also apply to RGB camera responses. (2) The hyperspectral camera architecture provides means to explore other important aspects of colour vision like the perception of certain types of camouflage and colour constancy where multiple, narrow-band sensors increase resolution.

Sequential weighted Wiener estimation for extraction of key tissue parameters in color imaging: a phantom study

Key tissue parameters, e.g., total hemoglobin concentration and tissue oxygenation, are important biomarkers in clinical diagnosis for various diseases. Although point measurement techniques based on diffuse reflectance spectroscopy can accurately recover these tissue parameters, they are not suitable for the examination of a large tissue region due to slow data acquisition. The previous imaging studies have shown that hemoglobin concentration and oxygenation can be estimated from color measurements with the assumption of known scattering properties, which is impractical in clinical applications. To overcome this limitation and speed-up image processing, we propose a method of sequential weighted Wiener estimation (WE) to quickly extract key tissue parameters, including total hemoglobin concentration (CtHb), hemoglobin oxygenation (StO2), scatterer density (α), and scattering power (β), from wide-band color measurements. This method takes advantage of the fact that each parameter is sensitive to the color measurements in a different way and attempts to maximize the contribution of those color measurements likely to generate correct results in WE. The method was evaluated on skin phantoms with varying CtHb, StO2, and scattering properties. The results demonstrate excellent agreement between the estimated tissue parameters and the corresponding reference values. Compared with traditional WE, the sequential weighted WE shows significant improvement in the estimation accuracy. This method could be used to monitor tissue parameters in an imaging setup in real time.

Multispectral filter arrays: recent advances and practical implementation

Thanks to some technical progress in interferencefilter design based on different technologies, we can finally successfully implement the concept of multispectral filter array-based sensors. This article provides the relevant state-of-the-art for multispectral imaging systems and presents the characteristics of the elements of our multispectral sensor as a case study. The spectral characteristics are based on two different spatial arrangements that distribute eight different bandpass filters in the visible and near-infrared area of the spectrum. We demonstrate that the system is viable and evaluate its performance through sensor spectral simulation.

Single-shot digital holography using a spectral estimation technique

We demonstrate a technique capable of obtaining spectral information and a three-dimensional (3D) profile of an object with a single-shot exposure. This technique is based on digital holography and the spectral estimation technique. In the demonstration of this technique, we simultaneously use three laser lines operating at 473, 532, and 633 nm to record the multiple complex amplitudes of the object corresponding to the wavelengths and obtain reconstructed monochrome images of each wavelength. A spectral estimation technique is applied to estimate the spectral reflectance of the object from the reconstructed monochrome images. We experimentally succeed in estimating the spectral reflectance of a lemon by using the technique.

Parallel phase-shifting digital holography using spectral estimation technique

We propose a parallel phase-shifting digital holography using a spectral estimation technique, which enables the instantaneous acquisition of spectral information and three-dimensional (3D) information of a moving object. In this technique, an interference fringe image that contains six holograms with two phase shifts for three laser lines, such as red, green, and blue, is recorded by a space-division multiplexing method with single-shot exposure. The 3D monochrome images of these three laser lines are numerically reconstructed by a computer and used to estimate the spectral reflectance distribution of object using a spectral estimation technique. Preliminary experiments demonstrate the validity of the proposed technique.

Hybrid-resolution spectral video system using low-resolution spectral sensor

This paper presents a prototype of a spectral video system based on hybrid resolution spectral imaging. The system consists of a commercial three-channel color camera and a low-resolution spectral sensor which captures a 68-pixel spectral image by a single snap-shot. By combining the measurement data from both devices, the system produces high-resolution spectral image data frame by frame. The accuracy of the spectral data measured by the system is evaluated at some selected regions in the image. As a result, it is confirmed that spectra can be measured with less or around 10% of normalized root mean squared error. In addition, the capture of spectral videos in 3 frame-per-second and the real-time color reproduction in the same frame rate from the spectral video are demonstrated.

Recovery of Raman spectra with low signal-to-noise ratio using Wiener estimation

Raman spectroscopy is a powerful non-destructive technique for qualitatively and quantitatively characterizing materials. However, noise often obscures interesting Raman peaks due to the inherently weak Raman signal, especially in biological samples. In this study, we develop a method based on spectral reconstruction to recover Raman spectra with low signal-to-noise ratio (SNR). The synthesis of narrow-band measurements from low-SNR Raman spectra eliminates the effect of noise by integrating the Raman signal along the wavenumber dimension, which is followed by spectral reconstruction based on Wiener estimation to recover the Raman spectrum with high spectral resolution. Non-negative principal components based filters are used in the synthesis to ensure that most variance contained in the original Raman measurements are retained. A total of 25 agar phantoms and 20 bacteria samples were measured and data were used to validate our method. Four commonly used de-noising methods in Raman spectroscopy, i.e. Savitzky-Golay (SG) algorithm, finite impulse response (FIR) filtration, wavelet transform and factor analysis, were also evaluated on the same set of data in addition to the proposed method for comparison. The proposed method showed the superior accuracy in the recovery of Raman spectra from measurements with extremely low SNR, compared with the four commonly used de-noising methods.

Channel selection for multispectral color imaging using binary differential evolution

In multispectral color imaging, there is a demand to select a reduced number of optimal imaging channels to simultaneously speed up the image acquisition process and keep reflectance reconstruction accuracy. In this paper, the channel selection problem is cast as the binary optimization problem, and is consequently solved using a novel binary differential evolution (DE) algorithm. In the proposed algorithm, we define the mutation operation using a differential table of swapping pairs, and deduce the trial solutions using neighboring self-crossover. In this manner, the binary DE algorithm can well adapt to the channel selection problem. The proposed algorithm is evaluated on the multispectral color imaging system on both synthetic and real data sets. It is verified that high color accuracy is achievable by only using a reduced number of channels using the proposed method. In addition, as binary DE is a global optimization algorithm in nature, it performs better than the traditional sequential channel selection algorithm.

Adaptive global training set selection for spectral estimation of printed inks using reflectance modeling

The performance of learning-based spectral estimation is greatly influenced by the set of training samples selected to create the reconstruction model. Training sample selection schemes can be categorized into global and local approaches. Most of the previously proposed global training schemes aim to reduce the number of training samples, or a selection of representative samples, to maintain the generality of the training dataset. This work relates to printed ink reflectance estimation for quality assessment in in-line print inspection. We propose what we believe is a novel global training scheme that models a large population of realistic printable ink reflectances. Based on this dataset, we used a recursive top-down algorithm to reject clusters of training samples that do not enhance the performance of a linear least-square regression (pseudoinverse-based estimation) process. A set of experiments with real camera response data of a 12-channel multispectral camera system illustrate the advantages of this selection scheme over some other state-of-the-art algorithms. For our data, our method of global training sample selection outperforms other methods in terms of estimation quality and, more importantly, can quickly handle large datasets. Furthermore, we show that reflectance modeling is a reasonable, convenient tool to generate large training sets for print inspection applications.

Link functions and Matérn kernel in the estimation of reflectance spectra from RGB responses

We evaluate three link functions (square root, logit, and copula) and Matérn kernel in the kernel-based estimation of reflectance spectra of the Munsell Matte collection in the 400-700 nm region. We estimate reflectance spectra from RGB camera responses in case of real and simulated responses and show that a combination of link function and a kernel regression model with a Matérn kernel decreases spectral errors when compared to a Gaussian mixture model or kernel regression with the Gaussian kernel. Matérn kernel produces performance similar to the thin plate spline model, but does not require a parametric polynomial part in the model.

A real-time spectral mapper as an emerging diagnostic technology in biomedical sciences

Real time spectral imaging and mapping at video rates can have tremendous impact not only on diagnostic sciences but also on fundamental physiological problems. We report the first real-time spectral mapper based on the combination of snap-shot spectral imaging and spectral estimation algorithms. Performance evaluation revealed that six band imaging combined with the Wiener algorithm provided high estimation accuracy, with error levels lying within the experimental noise. High accuracy is accompanied with much faster, by 3 orders of magnitude, spectral mapping, as compared with scanning spectral systems. This new technology is intended to enable spectral mapping at nearly video rates in all kinds of dynamic bio-optical effects as well as in applications where the target-probe relative position is randomly and fast changing.

Oximetry of retinal capillaries by multicomponent analysis

Retinal oximetry of capillaries was performed for early detection of retinal vascular abnormalities, which are caused predominantly by complications of systemic circulatory diseases. As the conventional method for determining absorbance is not applicable to capillaries, multicomponent analysis was used to estimate the absorbance spectra of the retinal blood vessels. In this analysis, the capillary spectrum was classified as intermediate between those of the retinal arteries and veins, enabling relative estimation of oxygen saturation in the capillaries. This method could be useful for early recognition of disturbances in the peripheral circulation. Furthermore, a spectroscopic ophthalmoscope system based on the proposed method was developed to examine the human retina. A clinical trial of this system demonstrated that oximetry of the retinal capillaries may be an improvement over the present diagnosis for patients of malignant hypertension.

Autofocus for multispectral camera using focus symmetry

A multispectral camera acquires spectral color images with high fidelity by splitting the light spectrum into more than three bands. Because of the shift of focal length with wavelength, the focus of each channel should be mechanically adjusted in order to obtain sharp images. Because progressive adjustment is quite time consuming, the clear focus must be determined by using a limited number of images. This paper exploits the symmetry of focus measure distribution and proposes a simple yet efficient autofocus method. The focus measures are computed using first-order image derivatives, and the focus curve is obtained by spline interpolation. The optimal focus position, which maximizes the symmetry of the focus measure distribution, is then computed according to distance metrics. The effectiveness of the proposed method is validated in the multispectral camera system, and it is also applicable to relevant imaging systems.

Hybrid-resolution multispectral imaging using color filter array

Hybrid-resolution multispectral imaging is a framework to acquire multispectral images through a reconstruction procedure using two types of measurement data with different spatial and spectral resolutions. In this paper, we propose a new method for such a framework on the basis of a full-resolution RGB image and the data obtained from an image sensor with a multispectral filter array (MSFA). In the proposed method, a small region of each image band is reconstructed as a linear combination of RGB images, where the weighting coefficients are determined using MSFA data. The effectiveness of the proposed approach is shown by simulations using spectral images of natural scenes.

Modified Wiener estimation of diffuse reflectance spectra from RGB values by the synthesis of new colors for tissue measurements

We present a new method for the accurate estimation of diffuse reflectance spectra from RGB values based on Wiener estimation. In the proposed method, a system matrix obtained from the original RGB values is combined with a set of synthetic optical filters to generate another three values corresponding to new colors. A modified Wiener matrix can then be created with the original RGB values and the new color values, which will yield a more accurate estimation because of the new color information that has been incorporated. This method was tested on in vivo color measurements from 200 skin sites in 10 volunteers. The results show that the proposed method is able to improve the estimation accuracy significantly compared with the traditional Wiener estimation method. The fast speed of this method may enable the estimation of diffuse reflectance spectra at multiple tissue locations from color images in real time, which provides a cost-effective alternative to spectral imaging with the additional advantage of high spectral resolution.

Chromoendoscopy in small bowel capsule endoscopy: Blue mode or Fuji Intelligent Colour Enhancement?

INTRODUCTION

Virtual chromoendoscopy is used to enhance surface patterns and colour differences. One type of virtual chromoendoscopy is the Fuji Intelligent Colour Enhancement (FICE). Although widely applied in conventional endoscopy, data on FICE application in capsule endoscopy are limited. Furthermore, the validity of Blue filter (feature of RAPID(®) software) has not been examined. AIM/S: We aimed to qualitatively evaluate the use of FICE and Blue filter enhancement, in images of lesions obtained during small bowel capsule endoscopy, comparing them with similar, conventional (white light) images.

METHODS

A total of 167 images (6 different lesion categories) obtained from 200 capsule endoscopy examinations. Two gastroenterologists examined the images with white light, FICE and Blue filter in regards to the visibility of blood vessels, the contrast of the mucosal surface, and the demarcation of lesion borders. The agreed scores were: improved, similar, worse. Inter-observer agreement was calculated.

RESULTS

For all lesion categories, Blue filter provided image improvement (compared to white light) in 83%, (inter-observer agreement: 0.786). With FICE 1, improvement was observed in 34%, worse image in 55.9%, (inter-observer agreement: 0.646). With FICE 2, improvement was observed in 8.6%, worse in 77.5%, (inter-observer agreement: 0.617). With FICE 3, improvement was seen in 7.7%, worse in 79.9% (inter-observer agreement: 0.669).

CONCLUSION

Comparing with FICE, Blue filter offers better image enhancement in capsule endoscopy.

Improvement of color reproduction in color digital holography by using spectral estimation technique

We propose a color digital holography by using spectral estimation technique to improve the color reproduction of objects. In conventional color digital holography, there is insufficient spectral information in holograms, and the color of the reconstructed images depend on only reflectances at three discrete wavelengths used in the recording of holograms. Therefore the color-composite image of the three reconstructed images is not accurate in color reproduction. However, in our proposed method, the spectral estimation technique was applied, which has been reported in multispectral imaging. According to the spectral estimation technique, the continuous spectrum of object can be estimated and the color reproduction is improved. The effectiveness of the proposed method was confirmed by a numerical simulation and an experiment, and, in the results, the average color differences are decreased from 35.81 to 7.88 and from 43.60 to 25.28, respectively.