Mathematical definition and analysis of the retinex algorithm

Double-function enhancement algorithm for low-illumination images based on retinex theory

In order to solve the problems of noise amplification and excessive enhancement caused by low contrast and uneven illumination in the process of low-illumination image enhancement, a high-quality image enhancement algorithm is proposed in this paper. First, the total-variation model is used to obtain the smoothed V- and S-channel images, and the adaptive gamma transform is used to enhance the details of the smoothed V-channel image. Then, on this basis, the improved multi-scale retinex algorithms based on the logarithmic function and on the hyperbolic tangent function, respectively, are used to obtain different V-channel enhanced images, and the two images are fused according to the local intensity amplitude of the images. Finally, the three-dimensional gamma function is used to correct the fused image, and then adjust the image saturation. A lightness-order-error (LOE) approach is used to measure the naturalness of the enhanced image. The experimental results show that compared with other classical algorithms, the LOE value of the proposed algorithm can be reduced by 79.95% at most. Compared with other state-of-the-art algorithms, the LOE value can be reduced by 53.43% at most. Compared with some algorithms based on deep learning, the LOE value can be reduced by 52.13% at most. The algorithm proposed in this paper can effectively reduce image noise, retain image details, avoid excessive image enhancement, and obtain a better visual effect while ensuring the enhancement effect.

Mathematical insights into the original Retinex algorithm for image enhancement

The Retinex theory, originally developed by Land and McCann as a computation model of the human color sensation, has become, with time, a pillar of digital image enhancement. In this area, the Retinex algorithm is widely used to improve the quality of any input image by increasing the visibility of its content and details, enhancing its colorfulness, and weakening, or even removing, some undesired effects of the illumination. The algorithm was originally described by its creators in terms of a sequence of image processing operations and was not fully formalized mathematically. Later, works focusing on aspects of the original formulation and adopting some of its principles tried to frame the algorithm within a mathematical formalism: this yielded every time a partial rendering of the model and resulted in several interesting model variants. The purpose of the present work is to fill a gap in the Retinex-related literature by providing a complete mathematical formalization of the original Retinex algorithm. The overarching goals of this work are to provide mathematical insights into the Retinex theory, promote awareness of the use of the model within image enhancement, and enable better appreciation of differences and similarities with later models based on Retinex principles. For this purpose, we compare our model with others proposed in the literature, paying particular attention to the work published in 2005 by Provenzi and others.

Low-Light Image Enhancement Based on Constraint Low-Rank Approximation Retinex Model

Images captured in a low-light environment are strongly influenced by noise and low contrast, which is detrimental to tasks such as image recognition and object detection. Retinex-based approaches have been continuously explored for low-light enhancement. Nevertheless, Retinex decomposition is a highly ill-posed problem. The estimation of the decomposed components should be combined with proper constraints. Meanwhile, the noise mixed in the low-light image causes unpleasant visual effects. To address these problems, we propose a Constraint Low-Rank Approximation Retinex model (CLAR). In this model, two exponential relative total variation constraints were imposed to ensure that the illumination is piece-wise smooth and that the reflectance component is piece-wise continuous. In addition, the low-rank prior was introduced to suppress the noise in the reflectance component. With a tailored separated alternating direction method of multipliers (ADMM) algorithm, the illumination and reflectance components were updated accurately. Experimental results on several public datasets verify the effectiveness of the proposed model subjectively and objectively.

LGN-CNN: A biologically inspired CNN architecture

In this paper we introduce a biologically inspired Convolutional Neural Network (CNN) architecture called LGN-CNN that has a first convolutional layer composed of a single filter that mimics the role of the Lateral Geniculate Nucleus (LGN). The first layer of the neural network shows a rotational symmetric pattern justified by the structure of the net itself that turns up to be an approximation of a Laplacian of Gaussian (LoG). The latter function is in turn a good approximation of the receptive field profiles (RFPs) of the cells in the LGN. The analogy with the visual system is established, emerging directly from the architecture of the neural network. A proof of rotation invariance of the first layer is given on a fixed LGN-CNN architecture and the computational results are shown. Thus, contrast invariance capability of the LGN-CNN is investigated and a comparison between the Retinex effects of the first layer of LGN-CNN and the Retinex effects of a LoG is provided on different images. A statistical study is done on the filters of the second convolutional layer with respect to biological data. In conclusion, the model we have introduced approximates well the RFPs of both LGN and V1 attaining similar behavior as regards long range connections of LGN cells that show Retinex effects.

Deep Dichromatic Model Estimation Under AC Light Sources

The dichromatic reflection model has been popularly exploited for computer vison tasks, such as color constancy and highlight removal. However, dichromatic model estimation is an severely ill-posed problem. Thus, several assumptions have been commonly made to estimate the dichromatic model, such as white-light (highlight removal) and the existence of highlight regions (color constancy). In this paper, we propose a spatio-temporal deep network to estimate the dichromatic parameters under AC light sources. The minute illumination variations can be captured with high-speed camera. The proposed network is composed of two sub-network branches. From high-speed video frames, each branch generates chromaticity and coefficient matrices, which correspond to the dichromatic image model. These two separate branches are jointly learned by spatio-temporal regularization. As far as we know, this is the first work that aims to estimate all dichromatic parameters in computer vision. To validate the model estimation accuracy, it is applied to color constancy and highlight removal. Both experimental results show that the dichromatic model can be estimated accurately via the proposed deep network.

Non-uniform illumination correction based on multi-scale Retinex in digital image correlation

Digital image correlation (DIC) is an effective optical measurement method. It aims to obtain the displacement field and strain field of the measured object by correlating two digital speckle images before and after deformation. In the actual acquisition of speckle images, due to the large volume of the measured object, the light source cannot cover all areas evenly or has some random change. These issues may easily lead to a non-uniform distribution of light intensity speckle images and reduce the quality of speckle images, which affects the accuracy of DIC measurement to a certain extent. To solve this problem, a non-uniform illumination correction algorithm based on multi-scale Retinex is introduced. First, to analyze the influence of non-uniform illumination on DIC measurement accuracy, the displacement comparison experiment of the numerical simulation speckle images with different non-uniform illumination is conducted. Then, a non-uniform illumination correction algorithm based on multi-scale Retinex is applied to reduce or eliminate the effects of non-uniform illumination by the simulation experiment. Finally, the quantitative measurement of rigid body rotation and uniaxial tensile experiment in plane is studied to verify the feasibility of the correction method for the speckle images. The experimental results show that the measurement accuracy of DIC is improved significantly with the aid of non-uniform illumination variation correction.

An Image Contrast Measure Based on Retinex Principles

The image contrast is a feature capturing the variation of the image signal across the space. Such a feature is very useful to describe the local image structure at different scales and thus it is relevant to many computer vision applications, like image/texture retrieval and object recognition. In this work, we present MiRCo, a novel measure of image contrast derived from the Retinex theory. MiRCo is robust against in-plane rotations and light changes at multiple scales. Thanks to these properties, MiRCo enables an accurate and robust description of the local image structure. Here we describe and discuss the mathematical insights of MiRCo also in comparison with other popular contrast measures.

Knowledge-Driven Deep Unrolling for Robust Image Layer Separation

Single-image layer separation targets to decompose the observed image into two independent components in terms of different application demands. It is known that many vision and multimedia applications can be (re)formulated as a separation problem. Due to the fundamentally ill-posed natural of these separations, existing methods are inclined to investigate model priors on the separated components elaborately. Nevertheless, it is knotty to optimize the cost function with complicated model regularizations. Effectiveness is greatly conceded by the settled iteration mechanism, and the adaption cannot be guaranteed due to the poor data fitting. What is more, for a universal framework, the most taxing point is that one type of visual cue cannot be shared with different tasks. To partly overcome the weaknesses mentioned earlier, we delve into a generic optimization unrolling technique to incorporate deep architectures into iterations for adaptive image layer separation. First, we propose a general energy model with implicit priors, which is based on maximum a posterior, and employ the extensively accepted alternating direction method of multiplier to determine our elementary iteration mechanism. By unrolling with one general residual architecture prior and one task-specific prior, we attain a straightforward, flexible, and data-dependent image separation framework successfully. We apply our method to four different tasks, including single-image-rain streak removal, high-dynamic-range tone mapping, low-light image enhancement, and single-image reflection removal. Extensive experiments demonstrate that the proposed method is applicable to multiple tasks and outperforms the state of the arts by a large margin qualitatively and quantitatively.

Generalized equation for real-world image enhancement by Milano Retinex family

Milano Retinexes are spatial color algorithms grounded on the Retinex theory and widely applied to enhance the visual content of real-world color images. In this framework, they process the color channels of the input image independently and re-scale channel by channel the intensity of each pixel p by the so-called local reference white, i.e., a strictly positive value, computed by reworking a set of features sampled around p. The neighborhood of p to be sampled, its sampling, the features to be processed, as well as the mathematical model for the computation of the local reference white vary from algorithm to algorithm, determining different levels of enhancement. Based on the analysis of a group of Milano Retinexes, this work proves that the Milano Retinex local reference whites can be expressed by a generalized equation whose parameters model specific aspects of the Milano Retinex spatial color processing. In particular, tuning these parameters leads to different Milano Retinex implementations. This study contributes to a better understanding of the similarities and differences among the members of the Milano Retinex family, and provides new taxonomic schemes of them based on their own mathematical properties.

STAR: A Structure and Texture Aware Retinex Model

Retinex theory is developed mainly to decompose an image into the illumination and reflectance components by analyzing local image derivatives. In this theory, larger derivatives are attributed to the changes in reflectance, while smaller derivatives are emerged in the smooth illumination. In this paper, we utilize exponentiated local derivatives (with an exponent γ) of an observed image to generate its structure map and texture map. The structure map is produced by been amplified with γ > 1, while the texture map is generated by been shrank with γ < 1. To this end, we design exponential filters for the local derivatives, and present their capability on extracting accurate structure and texture maps, influenced by the choices of exponents γ. The extracted structure and texture maps are employed to regularize the illumination and reflectance components in Retinex decomposition. A novel Structure and Texture Aware Retinex (STAR) model is further proposed for illumination and reflectance decomposition of a single image. We solve the STAR model by an alternating optimization algorithm. Each sub-problem is transformed into a vectorized least squares regression, with closed-form solutions. Comprehensive experiments on commonly tested datasets demonstrate that, the proposed STAR model produce better quantitative and qualitative performance than previous competing methods, on illumination and reflectance decomposition, low-light image enhancement, and color correction. The code is publicly available at https://github.com/csjunxu/STAR.

SuPeR: Milano Retinex implementation exploiting a regular image grid

A spatial color algorithm grounded on the Retinex theory is known as a Milano Retinex. This type of algorithm performs image enhancement by processing spatial and color cues in the neighborhood of each image pixel. Because this local, pixel-wise analysis is time consuming, optimization techniques are needed to expand the use of Milano Retinexes to applications that require fast or even real-time image processing. In this work, we propose SuPeR, an efficient optimization of the Milano Retinex local spatial color processing that exploits superpixels, which are as the regular, rectangular blocks of a grid that partitions the image support. Image enhancement is obtained by reworking channel-wise the intensity of each pixel based on the maximum color intensities of the blocks and on its distance from the blocks. The experiments, carried out on real-world image datasets, show good performance compared to other Milano Retinexes.

A neural field model for color perception unifying assimilation and contrast

We address the question of color-space interactions in the brain, by proposing a neural field model of color perception with spatial context for the visual area V1 of the cortex. Our framework reconciles two opposing perceptual phenomena, known as simultaneous contrast and chromatic assimilation. They have been previously shown to act synergistically, so that at some point in an image, the color seems perceptually more similar to that of adjacent neighbors, while being more dissimilar from that of remote ones. Thus, their combined effects are enhanced in the presence of a spatial pattern, and can be measured as larger shifts in color matching experiments. Our model supposes a hypercolumnar structure coding for colors in V1, and relies on the notion of color opponency introduced by Hering. The connectivity kernel of the neural field exploits the balance between attraction and repulsion in color and physical spaces, so as to reproduce the sign reversal in the influence of neighboring points. The color sensation at a point, defined from a steady state of the neural activities, is then extracted as a nonlinear percept conveyed by an assembly of neurons. It connects the cortical and perceptual levels, because we describe the search for a color match in asymmetric matching experiments as a mathematical projection on color sensations. We validate our color neural field alongside this color matching framework, by performing a multi-parameter regression to data produced by psychophysicists and ourselves. All the results show that we are able to explain the nonlinear behavior of shifts observed along one or two dimensions in color space, which cannot be done using a simple linear model.

The color perception produced by an image heavily depends on the spatial distribution of its colors. From this “color in context” phenomenon, extensively studied in psychophysics for decades, has arisen the question in neuroscience of how color and space interact in the brain. Visual signals are indeed processed in such a way that neighboring pixels make the perception at some point different from its real color, inducing a color shift. In this work, we propose to emulate perception in context by modeling the activity of color sensitive neurons with a neural field. Our framework unifies two antagonistic effects, assimilation and contrast, which have been suggested to occur simultaneously but at different scales. We use the notion of color opponency inspired by the work of Hering, so as to express these effects as a combination of attraction and repulsion in physical and color spaces. We introduce the concept of “color sensation”, and show how to rigorously link the neural field model to perceptual shifts, by considering color matching as a mathematical projection on color sensations. The results show that our model is able to reproduce some nontrivial behaviors of the color shifts observed in experiments.

STAR: A Segmentation-Based Approximation of Point-Based Sampling Milano Retinex for Color Image Enhancement

Milano Retinex is a family of spatial color algorithms inspired by Retinex and mainly devoted to the image enhancement. In the so-called point-based sampling Milano Retinex algorithms, this task is accomplished by processing the color of each image pixel based on a set of colors sampled in its surround. This paper presents STAR, a segmentation based approximation of the point-based sampling Milano Retinex approaches: it replaces the pixel-wise image sampling by a novel, computationally efficient procedure that detects once for all the color and spatial information relevant to image enhancement from clusters of pixels output by a segmentation. The experiments reported here show that STAR performs similarly to previous point-based sampling Milano Retinex approaches and that STAR enhancement improves the accuracy of the well-known algorithm scale-invariant feature transform on the description and matching of photographs captured under difficult light conditions.

Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model

Low-light image enhancement methods based on classic Retinex model attempt to manipulate the estimated illumination and to project it back to the corresponding reflectance. However, the model does not consider the noise, which inevitably exists in images captured in low-light conditions. In this paper, we propose the robust Retinex model, which additionally considers a noise map compared with the conventional Retinex model, to improve the performance of enhancing low-light images accompanied by intensive noise. Based on the robust Retinex model, we present an optimization function that includes novel regularization terms for the illumination and reflectance. Specifically, we use norm to constrain the piece-wise smoothness of the illumination, adopt a fidelity term for gradients of the reflectance to reveal the structure details in low-light images, and make the first attempt to estimate a noise map out of the robust Retinex model. To effectively solve the optimization problem, we provide an augmented Lagrange multiplier based alternating direction minimization algorithm without logarithmic transformation. Experimental results demonstrate the effectiveness of the proposed method in low-light image enhancement. In addition, the proposed method can be generalized to handle a series of similar problems, such as the image enhancement for underwater or remote sensing and in hazy or dusty conditions.

GRASS: A Gradient-Based Random Sampling Scheme for Milano Retinex

Retinex is an early and famous theory attempting to estimate the human color sensation derived from an observed scene. When applied to a digital image, the original implementation of retinex estimates the color sensation by modifying the pixels channel intensities with respect to a local reference white, selected from a set of random paths. The spatial search of the local reference white influences the final estimation. The recent algorithm energy-driven termite retinex (ETR), as well as its predecessor termite retinex, has introduced a new path-based image aware sampling scheme, where the paths depend on local visual properties of the input image. Precisely, the ETR paths transit over pixels with high gradient magnitude that have been proved to be important for the formation of color sensation. Such a sampling method enables the visit of image portions effectively relevant to the estimation of the color sensation, while it reduces the analysis of pixels with less essential and/or redundant data, i.e., the flat image regions. While the ETR sampling scheme is very efficacious in detecting image pixels salient for the color sensation, its computational complexity can be a limit. In this paper, we present a novel Gradient-based RAndom Sampling Scheme that inherits from ETR the image aware sampling principles, but has a lower computational complexity, while similar performance. Moreover, the new sampling scheme can be interpreted both as a path-based scanning and a 2D sampling.

GREAT: a gradient-based color-sampling scheme for Retinex

Modeling the local color spatial distribution is a crucial step for the algorithms of the Milano Retinex family. Here we present GREAT, a novel, noise-free Milano Retinex implementation based on an image-aware spatial color sampling. For each channel of a color input image, GREAT computes a 2D set of edges whose magnitude exceeds a pre-defined threshold. Then GREAT re-scales the channel intensity of each image pixel, called target, by the average of the intensities of the selected edges weighted by a function of their positions, gradient magnitudes, and intensities relative to the target. In this way, GREAT enhances the input image, adjusting its brightness, contrast and dynamic range. The use of the edges as pixels relevant to color filtering is justified by the importance that edges play in human color sensation. The name GREAT comes from the expression "Gradient RElevAnce for ReTinex," which refers to the threshold-based definition of a gradient relevance map for edge selection and thus for image color filtering.

A population-based approach to point-sampling spatial color algorithms

Inspired by the behavior of the human visual system, spatial color algorithms perform image enhancement by correcting the pixel channel lightness based on the spatial distribution of the intensities in the surrounding area. The two visual contrast enhancement algorithms RSR and STRESS belong to this family of models: they rescale the input based on local reference values, which are determined by exploring the image by means of random point samples, called sprays. Due to the use of sampling, they may yield a noisy output. In this paper, we introduce a probabilistic formulation of the two models: our algorithms (RSR-P and STRESS-P) rely implicitly on the whole population of possible sprays. For processing larger images, we also provide two approximated algorithms that exploit a suitable target-dependent space quantization. Those spray population-based formulations outperform RSR and STRESS in terms of the processing time required for the production of noiseless outputs. We argue that this population-based approach, which can be extended to other members of the family, complements the sampling-based approach, in that it offers not only a better control in the design of approximated algorithms, but also additional insight into individual models and their relationships. We illustrate the latter point by providing a model of halo artifact formation.

Realistic Image Rendition Using a Variable Exponent Functional Model for Retinex

The goal of realistic image rendition is to recover the acquired image under imperfect illuminant conditions, where non-uniform illumination may degrade image quality with high contrast and low SNR. In this paper, the assumption regarding illumination is modified and a variable exponent functional model for Retinex is proposed to remove non-uniform illumination and reduce halo artifacts. The theoretical derivation is provided and experimental results are presented to illustrate the effectiveness of the proposed model.

Energy-driven path search for Termite Retinex

The human color sensation depends on the local and global spatial arrangements of the colors in the scene. Emulating this dependence requires the exploration of the image in search of a white reference. The algorithm Termite Retinex explores the image by a set of paths resembling traces of a swarm of termites. Starting from this approach, we develop a novel spatial exploration scheme where the termite paths are local minimums of an energy function, which depend on the image visual content. The energy is designed to favor the visitation of regions containing information relevant to the color sensation while minimizing the coverage of less essential regions. This exploration method contributes to the investigation of the spatial properties of the color sensation and, to the best of our knowledge, is the first model relying on mathematical global conditions for the Retinex paths. The experiments show that the estimation of the color sensation obtained by means of the proposed spatial sampling is a valid alternative to the one based on Termite Retinex.

A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation

In this paper, a new probabilistic method for image enhancement is presented based on a simultaneous estimation of illumination and reflectance in the linear domain. We show that the linear domain model can better represent prior information for better estimation of reflectance and illumination than the logarithmic domain. A maximum a posteriori (MAP) formulation is employed with priors of both illumination and reflectance. To estimate illumination and reflectance effectively, an alternating direction method of multipliers is adopted to solve the MAP problem. The experimental results show the satisfactory performance of the proposed method to obtain reflectance and illumination with visually pleasing enhanced results and a promising convergence rate. Compared with other testing methods, the proposed method yields comparable or better results on both subjective and objective assessments.

Smart light random memory sprays Retinex: a fast Retinex implementation for high-quality brightness adjustment and color correction

Removing the influence of illumination on image colors and adjusting the brightness across the scene are important image enhancement problems. This is achieved by applying adequate color constancy and brightness adjustment methods. One of the earliest models to deal with both of these problems was the Retinex theory. Some of the Retinex implementations tend to give high-quality results by performing local operations, but they are computationally relatively slow. One of the recent Retinex implementations is light random sprays Retinex (LRSR). In this paper, a new method is proposed for brightness adjustment and color correction that overcomes the main disadvantages of LRSR. There are three main contributions of this paper. First, a concept of memory sprays is proposed to reduce the number of LRSR's per-pixel operations to a constant regardless of the parameter values, thereby enabling a fast Retinex-based local image enhancement. Second, an effective remapping of image intensities is proposed that results in significantly higher quality. Third, the problem of LRSR's halo effect is significantly reduced by using an alternative illumination processing method. The proposed method enables a fast Retinex-based image enhancement by processing Retinex paths in a constant number of steps regardless of the path size. Due to the halo effect removal and remapping of the resulting intensities, the method outperforms many of the well-known image enhancement methods in terms of resulting image quality. The results are presented and discussed. It is shown that the proposed method outperforms most of the tested methods in terms of image brightness adjustment, color correction, and computational speed.

Color Constancy Using Double-Opponency

The double-opponent (DO) color-sensitive cells in the primary visual cortex (V1) of the human visual system (HVS) have long been recognized as the physiological basis of color constancy. In this work we propose a new color constancy model by imitating the functional properties of the HVS from the single-opponent (SO) cells in the retina to the DO cells in V1 and the possible neurons in the higher visual cortexes. The idea behind the proposed double-opponency based color constancy (DOCC) model originates from the substantial observation that the color distribution of the responses of DO cells to the color-biased images coincides well with the vector denoting the light source color. Then the illuminant color is easily estimated by pooling the responses of DO cells in separate channels in LMS space with the pooling mechanism of sum or max. Extensive evaluations on three commonly used datasets, including the test with the dataset dependent optimal parameters, as well as the intra- and inter-dataset cross validation, show that our physiologically inspired DOCC model can produce quite competitive results in comparison to the state-of-the-art approaches, but with a relative simple implementation and without requiring fine-tuning of the method for each different dataset.

Tuning the locality of filtering with a spatially weighted implementation of random spray Retinex

The human color sensation depends on the spatial distribution of the colors in the viewed scene. This principle is at the basis of the random spray Retinex (RSR) algorithm. In this work, we modify RSR by integrating its approach with a method to weight and tune the locality of spatial image information. This modification allows for spatial control of the local effect of RSR on image color filtering. We study the performances of this spatially weighted version of RSR on a public image dataset by analyzing and comparing several image features of the output image and its local properties.

QBRIX: a quantile-based approach to retinex

In this paper, we introduce a novel probabilistic version of retinex. It is based on a probabilistic formalization of the random spray retinex sampling and contributes to the investigation of the spatial properties of the model. Various versions available of the retinex algorithm are characterized by different procedures for exploring the image content (so as to obtain, for each pixel, a reference white value), then used to rescale the pixel lightness. Here we propose an alternative procedure, which computes the reference white value from the percentile values of the pixel population. We formalize two versions of the algorithm: one with global and one with local behavior, characterized by different computational costs.

Variational Bayesian method for Retinex

In this paper, we propose a variational Bayesian method for Retinex to simulate and interpret how the human visual system perceives color. To construct a hierarchical Bayesian model, we use the Gibbs distributions as prior distributions for the reflectance and the illumination, and the gamma distributions for the model parameters. By assuming that the reflection function is piecewise continuous and illumination function is spatially smooth, we define the energy functions in the Gibbs distributions as a total variation function and a smooth function for the reflectance and the illumination, respectively. We then apply the variational Bayes approximation to obtain the approximation of the posterior distribution of unknowns so that the unknown images and hyperparameters are estimated simultaneously. Experimental results demonstrate the efficiency of the proposed method for providing competitive performance without additional information about the unknown parameters, and when prior information is added the proposed method outperforms the non-Bayesian-based Retinex methods we compared.

Noise reduction based on partial-reference, dual-tree complex wavelet transform shrinkage

This paper presents a novel way to reduce noise introduced or exacerbated by image enhancement methods, in particular algorithms based on the random spray sampling technique, but not only. According to the nature of sprays, output images of spray-based methods tend to exhibit noise with unknown statistical distribution. To avoid inappropriate assumptions on the statistical characteristics of noise, a different one is made. In fact, the non-enhanced image is considered to be either free of noise or affected by non-perceivable levels of noise. Taking advantage of the higher sensitivity of the human visual system to changes in brightness, the analysis can be limited to the luma channel of both the non-enhanced and enhanced image. Also, given the importance of directional content in human vision, the analysis is performed through the dual-tree complex wavelet transform (DTWCT). Unlike the discrete wavelet transform, the DTWCT allows for distinction of data directionality in the transform space. For each level of the transform, the standard deviation of the non-enhanced image coefficients is computed across the six orientations of the DTWCT, then it is normalized. The result is a map of the directional structures present in the non-enhanced image. Said map is then used to shrink the coefficients of the enhanced image. The shrunk coefficients and the coefficients from the non-enhanced image are then mixed according to data directionality. Finally, a noise-reduced version of the enhanced image is computed via the inverse transforms. A thorough numerical analysis of the results has been performed in order to confirm the validity of the proposed approach.

An Analysis of Visual Adaptation and Contrast Perception for Tone Mapping

Tone Mapping is the problem of compressing the range of a High-Dynamic Range image so that it can be displayed in a Low-Dynamic Range screen, without losing or introducing novel details: The final image should produce in the observer a sensation as close as possible to the perception produced by the real-world scene. We propose a tone mapping operator with two stages. The first stage is a global method that implements visual adaptation, based on experiments on human perception, in particular we point out the importance of cone saturation. The second stage performs local contrast enhancement, based on a variational model inspired by color vision phenomenology. We evaluate this method with a metric validated by psychophysical experiments and, in terms of this metric, our method compares very well with the state of the art.

Constrained pseudo-Brownian motion and its application to image enhancement

Brownian motion is a random process that finds application in many fields, and its relation to certain color perception phenomena has recently been observed. On this ground, Marini and Rizzi developed a retinex algorithm based on Brownian motion paths. However, while their approach has several advantages and delivers interesting results, it has a high computational complexity. We propose an efficient algorithm that generates pseudo-Brownian paths with a very important constraint: we can guarantee a lower bound to the number of visits to each pixel, as well as its average. Despite these constraints, we show that the paths generated have certain statistical similarities to random walk and Brownian motion. Finally, we present a retinex implementation that exploits the paths generated with our algorithm, and we compare some images it generates with those obtained with the McCann99 and Frankle and McCann's algorithms (two multiscale retinex implementations that have a low computational complexity). We find that our approach causes fewer artifacts and tends to require a smaller number of pixel comparisons to achieve similar results, thus compensating for the slightly higher computational complexity.

High dynamic range image display with halo and clipping prevention

The dynamic range of an image is defined as the ratio between the highest and the lowest luminance level. In a high dynamic range (HDR) image, this value exceeds the capabilities of conventional display devices; as a consequence, dedicated visualization techniques are required. In particular, it is possible to process an HDR image in order to reduce its dynamic range without producing a significant change in the visual sensation experienced by the observer. In this paper, we propose a dynamic range reduction algorithm that produces high-quality results with a low computational cost and a limited number of parameters. The algorithm belongs to the category of methods based upon the Retinex theory of vision and was specifically designed in order to prevent the formation of common artifacts, such as halos around the sharp edges and clipping of the highlights, that often affect methods of this kind. After a detailed analysis of the state of the art, we shall describe the method and compare the results and performance with those of two techniques recently proposed in the literature and one commercial software.

A PDE formalization of Retinex theory

In 1964 Edwin H. Land formulated the Retinex theory, the first attempt to simulate and explain how the human visual system perceives color. His theory and an extension, the "reset Retinex" were further formalized by Land and McCann. Several Retinex algorithms have been developed ever since. These color constancy algorithms modify the RGB values at each pixel to give an estimate of the color sensation without a priori information on the illumination. Unfortunately, the Retinex Land-McCann original algorithm is both complex and not fully specified. Indeed, this algorithm computes at each pixel an average of a very large set of paths on the image. For this reason, Retinex has received several interpretations and implementations which, among other aims, attempt to tune down its excessive complexity. In this paper, it is proved that if the paths are assumed to be symmetric random walks, the Retinex solutions satisfy a discrete screened Poisson equation. This formalization yields an exact and fast implementation using only two FFTs. Several experiments on color images illustrate the effectiveness of the Retinex original theory.

Color constancy

A quarter of a century ago, the first systematic behavioral experiments were performed to clarify the nature of color constancy-the effect whereby the perceived color of a surface remains constant despite changes in the spectrum of the illumination. At about the same time, new models of color constancy appeared, along with physiological data on cortical mechanisms and photographic colorimetric measurements of natural scenes. Since then, as this review shows, there have been many advances. The theoretical requirements for constancy have been better delineated and the range of experimental techniques has been greatly expanded; novel invariant properties of images and a variety of neural mechanisms have been identified; and increasing recognition has been given to the relevance of natural surfaces and scenes as laboratory stimuli. Even so, there remain many theoretical and experimental challenges, not least to develop an account of color constancy that goes beyond deterministic and relatively simple laboratory stimuli and instead deals with the intrinsically variable nature of surfaces and illuminations present in the natural world.

Implementing the Retinex algorithm with Wilson-Cowan equations

It is shown that a form of the Wilson-Cowan equations representing large-scale activity in interacting neural populations can implement a form of the Retinex algorithm for color vision. It has also been shown recently that a color enhancement algorithm closely related to Retinex can be derived from a variational principle. It follows that a variational principle exists for equations of Wilson-Cowan type. Thus the Wilson-Cowan equations are the Euler-Lagrange solution of the minimization of an energy functional. This result suggests many interesting neural applications.

A perceptually inspired variational framework for color enhancement

Basic phenomenology of human color vision has been widely taken as an inspiration to devise explicit color correction algorithms. The behavior of these models in terms of significative image features (such as, e.g., contrast and dispersion) can be difficult to characterize. To cope with this, we propose to use a variational formulation of color contrast enhancement that is inspired by the basic phenomenology of color perception. In particular, we devise a set of basic requirements to be fulfilled by an energy to be considered as 'perceptually inspired', showing that there is an explicit class of functionals satisfying all of them. We single out three explicit functionals that we consider of basic interest, showing similarities and differences with existing models. The minima of such functionals is computed using a gradient descent approach. We also present a general methodology to reduce the computational cost of the algorithms under analysis from O(N2) to O(N logN), being N the number of pixels of the input image.

A spatially variant white-patch and gray-world method for color image enhancement driven by local contrast

Starting from the revolutionary Retinex by Land and McCann, several further perceptually inspired color correction models have been developed with different aims, e.g. reproduction of color sensation, robust features recognition, enhancement of color images. Such models have a differential, spatially-variant and non-linear nature and they can coarsely be distinguished between white-patch (WP) and gray-world (GW) algorithms. In this paper we show that the combination of a pure WP algorithm (Random Spray Retinex (RSR) )and an essentially GW one (Automatic Color Equalization (ACE)) leads to a more robust and better performing model (RACE). The choice of RSR and ACE follows from the recent identification of a unified spatially-variant approach for both algorithms. Mathematically, the originally distinct non-linear and differential mechanisms of RSR and ACE have been fused using the spray technique and local average operations. The investigation of RACE allowed us to put in evidence a common drawback of differential models: corruption of uniform image areas. To overcome this intrinsic defect, we devised a local and global contrast-based and image-driven regulation mechanism that has a general applicability to perceptually inspired color correction algorithms. Tests, comparisons and discussions are presented.

Perceptual color correction through variational techniques

In this paper, we present a discussion about perceptual-based color correction of digital images in the framework of variational techniques. We propose a novel image functional whose minimization produces a perceptually inspired color enhanced version of the original. The variational formulation permits a more flexible local control of contrast adjustment and attachment to data. We show that a numerical implementation of the gradient descent technique applied to this energy functional coincides with the equation of automatic color enhancement (ACE), a particular perceptual-based model of color enhancement. Moreover, we prove that a numerical approximation of the Euler-Lagrange equation reduces the computational complexity of ACE from theta(N2) to theta(N log N), where N is the total number of pixels in the image.

Random spray Retinex: a new Retinex implementation to investigate the local properties of the model

In order to investigate the local filtering behavior of the Retinex model, we propose a new implementation in which paths are replaced by 2-D pixel sprays, hence the name "random spray Retinex." A peculiar feature of this implementation is the way its parameters can be controlled to perform spatial investigation. The parameters' tuning is accomplished by an unsupervised method based on quantitative measures. This procedure has been validated via user panel tests. Furthermore, the spray approach has faster performances than the path-wise one. Tests and results are presented and discussed.