Improved method for spectral reflectance estimation and application to mobile phone cameras

A Method for Estimating Fluorescence Emission Spectra from the Image Data of Plant Grain and Leaves Without a Spectrometer

This study proposes a method for estimating the spectral images of fluorescence spectral distributions emitted from plant grains and leaves without using a spectrometer. We construct two types of multiband imaging systems with six channels, using ordinary off-the-shelf cameras and a UV light. A mobile phone camera is used to detect the fluorescence emission in the blue wavelength region of rice grains. For plant leaves, a small monochrome camera is used with additional optical filters to detect chlorophyll fluorescence in the red-to-far-red wavelength region. A ridge regression approach is used to obtain a reliable estimate of the spectral distribution of the fluorescence emission at each pixel point from the acquired image data. The spectral distributions can be estimated by optimally selecting the ridge parameter without statistically analyzing the fluorescence spectra. An algorithm for optimal parameter selection is developed using a cross-validation technique. In experiments using real rice grains and green leaves, the estimated fluorescence emission spectral distributions by the proposed method are compared to the direct measurements obtained with a spectroradiometer and the estimates obtained using the minimum norm estimation method. The estimated images of fluorescence emissions are presented for rice grains and green leaves. The reliability of the proposed estimation method is demonstrated.

Spectral Reflectance Estimation from Camera Response Using Local Optimal Dataset and Neural Networks

In this study, a novel method is proposed to estimate surface-spectral reflectance from camera responses that combine model-based and training-based approaches. An imaging system is modeled using the spectral sensitivity functions of an RGB camera, spectral power distributions of multiple light sources, unknown surface-spectral reflectance, additive noise, and a gain parameter. The estimation procedure comprises two main stages: (1) selecting the local optimal reflectance dataset from a reflectance database and (2) determining the best estimate by applying a neural network to the local optimal dataset only. In stage (1), the camera responses are predicted for the respective reflectances in the database, and the optimal candidates are selected in the order of lowest prediction error. In stage (2), most reflectance training data are obtained by a convex linear combination of local optimal data using weighting coefficients based on random numbers. A feed-forward neural network with one hidden layer is used to map the observation space onto the spectral reflectance space. In addition, the reflectance estimation is repeated by generating multiple sets of random numbers, and the median of a set of estimated reflectances is determined as the final estimate of the reflectance. Experimental results show that the estimation accuracies exceed those of other methods.

Controlling the target pattern of projected LED arrays for smart lighting

High-resolution, pixelated LED arrays allow flexible illumination. By addressing certain areas of the LED matrix and projecting the emitted light, selective illumination can be achieved. When combined with computer vision, smart, autonomous lighting systems are within reach. However, limitations of the used projection optics, in combination with the fact that the LED array and camera can be at a different position, severely complicates the problem of calculating which LED pixels to address in order to achieve a desired target pattern. This work proposes a least-squares deconvolution-based calculation method to solve this problem. The method relies on an initial calibration step that characterizes the complete point-spread-function of the LED array for the considered illumination configuration. This allows using the system for various settings. The method is experimentally validated for an off-axis illumination configuration that demonstrates the accuracy and flexibility of the approach. Because the proposed algorithm is fast and guarantees a global optimum, it opens new avenues towards accurate, smart and adaptive illumination.

Spectral Reflectance Estimation from Camera Responses Using Local Optimal Dataset

A novel method is proposed to estimate surface-spectral reflectance from camera responses using a local optimal reflectance dataset. We adopt a multispectral imaging system that involves an RGB camera capturing multiple images under multiple light sources. A spectral reflectance database is utilized to locally determine the candidates to optimally estimate the spectral reflectance. The proposed estimation method comprises two stages: (1) selecting the local optimal reflectance dataset and (2) determining the best estimate using only the local optimal dataset. In (1), the camera responses are predicted for the respective reflectances in the database, and then the prediction errors are calculated to select the local optimal dataset. In (2), multiple methods are used; in particular, the Wiener and linear minimum mean square error estimators are used to calculate all statistics, based only on the local optimal dataset, and linear and quadratic programming methods are used to solve optimization problems with constraints. Experimental results using different mobile phone cameras show that the estimation accuracy has improved drastically. A much smaller local optimal dataset among spectral reflectance databases is enough to obtain the optimal estimates. The method has potential applications including fields of color science, image science and technology, computer vision, and graphics.

Point-by-point visual enhancement with spatially and spectrally tunable laser illumination

Vision is responsible for most of the information that humans perceive of the surrounding world. Many studies attempt to enhance the visualization of the entire scene by optimizing and tuning the overall illumination spectrum. However, by using a spatially uniform illumination spectrum for the entire scene, only certain global color shifts with respect to a reference illumination spectrum can be realized, resulting in moderate visual enhancement. In this paper, a new visual enhancement method is presented that relies on a spatially variable illumination spectrum. Such an approach can target much more dedicated visual enhancements by optimizing the incident illumination spectrum to the surface reflectance at each position. First, a geometric calibration of the projector-camera system is carried out for determining the spatial mapping from the projected pixel grid to the imaged pixel grid. Secondly, the scene is segmented for implementing the visual enhancement approach. And finally, one of three visual enhancement scenarios is applied by projecting the required color image onto the considered segmented scene. The experimental results show that the visual salience of the scene or region of interest can be efficiently enhanced when our proposed method is applied to achieve colorfulness enhancement, hue tuning, and background lightness reduction.

Irradiance Independent Spectrum Reconstruction from Camera Signals Using the Interpolation Method

The spectrum of light captured by a camera can be reconstructed using the interpolation method. The reconstructed spectrum is a linear combination of the reference spectra, where the weighting coefficients are calculated from the signals of the pixel and the reference samples by interpolation. This method is known as the look-up table (LUT) method. It is irradiance-dependent due to the dependence of the reconstructed spectrum shape on the sample irradiance. Since the irradiance can vary in field applications, an irradiance-independent LUT (II-LUT) method is required to recover spectral reflectance. This paper proposes an II-LUT method to interpolate the spectrum in the normalized signal space. Munsell color chips irradiated with D65 were used as samples. Example cameras are a tricolor camera and a quadcolor camera. Results show that the proposed method can achieve the irradiance independent spectrum reconstruction and computation time saving at the expense of the recovered spectral reflectance error. Considering that the irradiance variation will introduce additional errors, the actual mean error using the II-LUT method might be smaller than that of the ID-LUT method. It is also shown that the proposed method outperformed the weighted principal component analysis method in both accuracy and computation speed.

An improved spectral estimation method based on color perception features of mobile phone camera

We use the mobile phone camera as a new spectral imaging device to obtain raw responses of samples for spectral estimation and propose an improved sequential adaptive weighted spectral estimation method. First, we verify the linearity of the raw response of the cell phone camera and investigate its feasibility for spectral estimation experiments. Then, we propose a sequential adaptive spectral estimation method based on the CIE1976 L*a*b* (CIELAB) uniform color space color perception feature. The first stage of the method is to weight the training samples and perform the first spectral reflectance estimation by considering the Lab color space color perception features differences between samples, and the second stage is to adaptively select the locally optimal training samples and weight them by the first estimated root mean square error (RMSE), and perform the second spectral reconstruction. The novelty of the method is to weight the samples by using the sample in CIELAB uniform color space perception features to more accurately characterize the color difference. By comparing with several existing methods, the results show that the method has the best performance in both spectral error and chromaticity error. Finally, we apply this weighting strategy based on the CIELAB color space color perception feature to the existing method, and the spectral estimation performance is greatly improved compared with that before the application, which proves the effectiveness of this weighting method.

Spectral Reflectance Recovery from the Quadcolor Camera Signals Using the Interpolation and Weighted Principal Component Analysis Methods

The recovery of surface spectral reflectance using the quadcolor camera was numerically studied. Assume that the RGB channels of the quadcolor camera are the same as the Nikon D5100 tricolor camera. The spectral sensitivity of the fourth signal channel was tailored using a color filter. Munsell color chips were used as reflective surfaces. When the interpolation method or the weighted principal component analysis (wPCA) method is used to reconstruct spectra, using the quadcolor camera can effectively reduce the mean spectral error of the test samples compared to using the tricolor camera. Except for computation time, the interpolation method outperforms the wPCA method in spectrum reconstruction. A long-pass optical filter can be applied to the fourth channel for reducing the mean spectral error. A short-pass optical filter can be applied to the fourth channel for reducing the mean color difference, but the mean spectral error will be larger. Due to the small color difference, the quadcolor camera using an optimized short-pass filter may be suitable as an imaging colorimeter. It was found that an empirical design rule to keep the color difference small is to reduce the error in fitting the color-matching functions using the camera spectral sensitivity functions.