Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model

ACformer: A unified transformer for arbitrary-frame image exposure correction

Both the single-image exposure correction (SEC) methods and multi-image exposure fusion (MEF) methods aim to obtain a well-exposed image, but from different number of input image(s). Despite their promising performance on the specific SEC or MEF task, few of these methods explores the inherent correlation behind the same goal of the SEC or MEF tasks. In this paper, we propose to unify the SEC and MEF tasks into a unified task of "Arbitrary-Frame Exposure Correction" (AF-EC) with arbitrary number of input frames. To tackle our AF-EC task, we develop an Arbitrary-Frame Exposure Correction Transformer (ACformer) as an integrated model to tackle the AF-EC task, and achieves mutually boosted performance on both the SEC and MEF tasks. Our ACformer consists mainly of the proposed Parallel Feature Fusion and Correction (PFFC) module. It simultaneously exploits feature-level exposure correction by Spatial Self-Attention and Channel Self-Attention blocks from each input image, as well as Temporal Self-Attention blocks to fuse the features from arbitrary numbers of input frames for feature-level exposure fusion. Experiments on the two commonly used datasets demonstrates that our ACformer outperforms the comparison methods designed specifically for the SEC or MEF tasks, in terms of both objective metrics and subjective visual quality. The code and pretrained models will be publicly released.

Low-Light Image and Video Enhancement for More Robust Computer Vision Tasks: A Review

Computer vision aims to enable machines to understand the visual world. Computer vision encompasses numerous tasks, namely action recognition, object detection and image classification. Much research has been focused on solving these tasks, but one that remains relatively uncharted is light enhancement (LE). Low-light enhancement (LLE) is crucial as computer vision tasks fail in the absence of sufficient lighting, having to rely on the addition of peripherals such as sensors. This review paper will shed light on this (focusing on video enhancement) subfield of computer vision, along with the other forementioned computer vision tasks. The review analyzes both traditional and deep learning-based enhancers and provides a comparative analysis on recent models in the field. The review also analyzes how popular computer vision tasks are improved and made more robust when coupled with light enhancement algorithms. Results show that deep learners outperform traditional enhancers, with supervised learners obtaining the best results followed by zero-shot learners, while computer vision tasks are improved with light enhancement coupling. The review concludes by highlighting major findings such as that although supervised learners obtain the best results, due to a lack of real-world data and robustness to new data, a shift to zero-shot learners is required.

Transferring enhanced material knowledge via image quality enhancement and feature distillation for pavement condition identification

In the context of rapid advancements in autonomous driving technology, ensuring passengers' safety and comfort has become a priority. Obstacle or road detection systems, especially accurate pavement condition identification in unfavorable weather or time circumstances, play a crucial role in the safe operation and comfortable riding experience of autonomous vehicles. To this end, we propose a novel framework based on image quality enhancement and feature distillation (IQEFD) for detecting diverse pavement conditions during the day and night to achieve state classification. The IQEFD model first leverages ConvNeXt as its backbone to extract high-quality basic features. Then, a bidirectional fusion module embedded with a hybrid attention mechanism (HAM) is devised to effectively extract multi-scale refined features, thereby mitigating information loss during continuous upsampling and downsampling. Subsequently, the refined features are fused with the enhanced features extracted through the image enhancement network Zero-DCE to generate the fused attention features. Lastly, the enhanced features serve as the guidance online for the fused attention features through feature distillation, transferring enhanced material knowledge and achieving alignment between feature representations. Extensive experimental results on two publicly available datasets validate that IQEFD can accurately classify a variety of pavement conditions, including dry, wet, and snowy conditions, especially showing satisfactory and robust performance in noisy nighttime images. In detail, the IQEFD model achieves the accuracies of 98.04% and 98.68% on the YouTube-w-ALI and YouTube-w/o-ALI datasets, respectively, outperforming the state-of-the-art baselines. It is worth noting that IQEFD has a certain generalization ability on a classical material image dataset named MattrSet, with an average accuracy of 75.86%. This study provides a novel insight into pavement condition identification. The source code of IQEFD will be made available at https://github.com/rainzyx/IQEFD .

Interpretable Optimization-Inspired Unfolding Network for Low-Light Image Enhancement

Retinex model-based methods have shown to be effective in layer-wise manipulation with well-designed priors for low-light image enhancement (LLIE). However, the hand-crafted priors and conventional optimization algorithm adopted to solve the layer decomposition problem result in the lack of adaptivity and efficiency. To this end, this paper proposes a Retinex-based deep unfolding network (URetinex-Net++), which unfolds an optimization problem into a learnable network to decompose a low-light image into reflectance and illumination layers. By formulating the decomposition problem as an implicit priors regularized model, three learning-based modules are carefully designed, responsible for data-dependent initialization, high-efficient unfolding optimization, and fairly-flexible component adjustment, respectively. Particularly, the proposed unfolding optimization module, introducing two networks to adaptively fit implicit priors in the data-driven manner, can realize noise suppression and details preservation for decomposed components. URetinex-Net++ is a further augmented version of URetinex-Net, which introduces a cross-stage fusion block to alleviate the color defect in URetinex-Net. Therefore, boosted performance on LLIE can be obtained in both visual quality and quantitative metrics, where only a few parameters are introduced and little time is cost. Extensive experiments on real-world low-light images qualitatively and quantitatively demonstrate the effectiveness and superiority of the proposed URetinex-Net++ over state-of-the-art methods.

Zero-Shot Sand-Dust Image Restoration

Natural sand-dust weather is complicated, and synthetic sand-dust datasets cannot accurately reflect the properties of real sand-dust images. Sand-dust image enhancement and restoration methods that are based on enhancement, on priors, or on data-driven may not perform well in some scenes. Therefore, it is important to develop a robust sand-dust image restoration method to improve the information processing ability of computer vision. In this paper, we propose a new zero-shot learning method based on an atmospheric scattering physics model to restore sand-dust images. The technique has two advantages: First, as it is unsupervised, the model can be trained without any prior knowledge or image pairs. Second, the method obtains transmission and atmospheric light by learning and inferring from a single real sand-dust image. Extensive experiments are performed and evaluated both qualitatively and quantitatively. The results show that the proposed method works better than the state-of-the-art algorithms for enhancing and restoring sand-dust images.

A fast sand-dust video quality improvement method based on color correction and illumination compensation

Sand-dust weather seriously reduces the effectiveness of computer vision equipment acquisition. To solve this problem, a fast sand-dust video quality improvement method based on color correction and illumination compensation is proposed in this paper. The mapping function strategy designed in the paper has two methods for dealing with sand-dust video frames. The first method has two steps: one is to correct the color cast of sand-dust video frames using a color correction and stretching algorithm, and the other is to use an illumination compensation algorithm to supplement and enhance the missing light to make the frame clearer. The second method uses the mapping functions of each color channel to improve the quality of the sand-dust video frames to be processed to reduce the amount of calculation. The first frame of the video is processed using the first method. Then, the processing method of each frame after the first frame of the video is determined according to its interframe detection value with the buffer frame. The first method is used to improve the quality of frames whose interframe detection values are less than the threshold value, and the second method is used to improve the quality of frames whose interframe detection values are not less than the threshold value until all frames are processed to obtain the sand-dust video with quality improvement. The experimental results are compared with existing relevant methods through qualitative and quantitative comprehensive experiments on sand-dust videos and images. It is proven that our improved frame method has the best visual effect in improving the quality of sand-dust images, and the quantitative evaluation indicators are the best. The mapping function strategy can improve the processing efficiency of videos in the experimental data by an average of 2.08 times compared with the total time of framewise processing.

Residual Quotient Learning for Zero-Reference Low-Light Image Enhancement

Recently, neural networks have become the dominant approach to low-light image enhancement (LLIE), with at least one-third of them adopting a Retinex-related architecture. However, through in-depth analysis, we contend that this most widely accepted LLIE structure is suboptimal, particularly when addressing the non-uniform illumination commonly observed in natural images. In this paper, we present a novel variant learning framework, termed residual quotient learning, to substantially alleviate this issue. Instead of following the existing Retinex-related decomposition-enhancement-reconstruction process, our basic idea is to explicitly reformulate the light enhancement task as adaptively predicting the latent quotient with reference to the original low-light input using a residual learning fashion. By leveraging the proposed residual quotient learning, we develop a lightweight yet effective network called ResQ-Net. This network features enhanced non-uniform illumination modeling capabilities, making it more suitable for real-world LLIE tasks. Moreover, due to its well-designed structure and reference-free loss function, ResQ-Net is flexible in training as it allows for zero-reference optimization, which further enhances the generalization and adaptability of our entire framework. Extensive experiments on various benchmark datasets demonstrate the merits and effectiveness of the proposed residual quotient learning, and our trained ResQ-Net outperforms state-of-the-art methods both qualitatively and quantitatively. Furthermore, a practical application in dark face detection is explored, and the preliminary results confirm the potential and feasibility of our method in real-world scenarios.

Illumination-aware divide-and-conquer network for improperly-exposed image enhancement

Improperly-exposed images often have unsatisfactory visual characteristics like inadequate illumination, low contrast, and the loss of small structures and details. The mapping relationship from an improperly-exposed condition to a well-exposed one may vary significantly due to the presence of multiple exposure conditions. Consequently, the enhancement methods that do not pay specific attention to this issue tend to yield inconsistent results when applied to the same scene under different exposure conditions. In order to obtain consistent enhancement results for various exposures while restoring rich details, we propose an illumination-aware divide-and-conquer network (IDNet). Specifically, to address the challenge of directly learning a sophisticated nonlinear mapping from an improperly-exposed condition to a well-exposed one, we utilize the discrete wavelet transform (DWT) to decompose the image into the low-frequency (LF) component, which primarily captures brightness and contrast, and the high-frequency (HF) components that depict fine-scale structures. To mitigate the inconsistency in correction across various exposures, we extract a conditional feature from the input that represents illumination-related global information. This feature is then utilized to modulate the dynamic convolution weights, enabling precise correction of the LF component. Furthermore, as the co-located positions of LF and HF components are highly correlated, we create a mask to distill useful knowledge from the corrected LF component, and integrate it into the HF component to support the restoration of fine-scale details. Extensive experimental results demonstrate that the proposed IDNet is superior to several state-of-the-art enhancement methods on two datasets with multiple exposures.

FPGA-based low-light image enhancement using Retinex algorithm and coarse-grained reconfigurable architecture

Advancements in digital imaging and video processing are often challenged by low-light environments, leading to degraded visual quality. This affects critical sectors such as medical imaging, aerospace, and underwater exploration, where uneven lighting can compromise safety and clarity. To enhance image quality in low-light conditions using a computationally efficient system. This paper introduces an FPGA-based system utilizing the Retinex algorithm for low-light image enhancement, implemented on a Coarse-Grained Reconfigurable Architecture (CGRA). The system is designed using Verilog HDL on a Xilinx FPGA, prioritizing hardware optimization to achieve high-quality outputs with minimal latency. The system achieves a processing rate of 60 frames per second (fps) for images with a resolution of 720 × 576. Quantitative evaluations show a Peak Signal-to-Noise Ratio (PSNR) improvement to 43.18 dB, a Structural Similarity Index (SSIM) of 0.92, and a Mean Squared Error (MSE) reduction, demonstrating significant enhancements in image quality. The design also achieves a low power consumption of 0.186 W and efficient resource utilization, with only 2.2% of Slice LUTs and Slice Registers used. The FPGA-based system demonstrates significant improvements in image quality with high computational efficiency, proving beneficial for critical applications in various sectors.

Retinex Image Enhancement Based on Sequential Decomposition With a Plug-and-Play Framework

The Retinex model is one of the most representative and effective methods for low-light image enhancement. However, the Retinex model does not explicitly tackle the noise problem and shows unsatisfactory enhancing results. In recent years, due to the excellent performance, deep learning models have been widely used in low-light image enhancement. However, these methods have two limitations. First, the desirable performance can only be achieved by deep learning when a large number of labeled data are available. However, it is not easy to curate massive low-/normal-light paired data. Second, deep learning is notoriously a black-box model. It is difficult to explain their inner working mechanism and understand their behaviors. In this article, using a sequential Retinex decomposition strategy, we design a plug-and-play framework based on the Retinex theory for simultaneous image enhancement and noise removal. Meanwhile, we develop a convolutional neural network-based (CNN-based) denoiser into our proposed plug-and-play framework to generate a reflectance component. The final image is enhanced by integrating the illumination and reflectance with gamma correction. The proposed plug-and-play framework can facilitate both post hoc and ad hoc interpretability. Extensive experiments on different datasets demonstrate that our framework outcompetes the state-of-the-art methods in both image enhancement and denoising.

Improved YOLOv5 Network for High-Precision Three-Dimensional Positioning and Attitude Measurement of Container Spreaders in Automated Quayside Cranes

For automated quayside container cranes, accurate measurement of the three-dimensional positioning and attitude of the container spreader is crucial for the safe and efficient transfer of containers. This paper proposes a high-precision measurement method for the spreader's three-dimensional position and rotational angles based on a single vertically mounted fixed-focus visual camera. Firstly, an image preprocessing method is proposed for complex port environments. The improved YOLOv5 network, enhanced with an attention mechanism, increases the detection accuracy of the spreader's keypoints and the container lock holes. Combined with image morphological processing methods, the three-dimensional position and rotational angle changes of the spreader are measured. Compared to traditional detection methods, the single-camera-based method for three-dimensional positioning and attitude measurement of the spreader employed in this paper achieves higher detection accuracy for spreader keypoints and lock holes in experiments and improves the operational speed of single operations in actual tests, making it a feasible measurement approach.

Low-illumination and noisy bridge crack image restoration by deep CNN denoiser and normalized flow module

When applying deep learning and image processing techniques for bridge crack detection, the obtained images in real-world scenarios have severe image degradation problem. This study focuses on restoring low-illumination bridge crack images corrupted by noise to improve the accuracy of subsequent crack detection and semantic segmentation. The proposed algorithm consists of a deep CNN denoiser and a normalized flow-based brightness enhancement module. By taking the noise spectrum as an input, the deep CNN denoiser restores image at a broad range of noise levels. The normalized flow module, employs a conditional encoder and a reversible network to map the distribution of normally exposed images to a Gaussian distribution, effectively improving the image brightness. Extensive experiments have demonstrated the approach can usefully recover low-illumination images corrupted by noise compared to the state-of-the-art methods. Furthermore, the algorithm presented in this study can also be applied to other image quality restoration with high generalization and robust abilities. And the semantic segmentation accuracy of the restored image is significantly improved.

Fast, Zero-Reference Low-Light Image Enhancement with Camera Response Model

Low-light images are prevalent in intelligent monitoring and many other applications, with low brightness hindering further processing. Although low-light image enhancement can reduce the influence of such problems, current methods often involve a complex network structure or many iterations, which are not conducive to their efficiency. This paper proposes a Zero-Reference Camera Response Network using a camera response model to achieve efficient enhancement for arbitrary low-light images. A double-layer parameter-generating network with a streamlined structure is established to extract the exposure ratio K from the radiation map, which is obtained by inverting the input through a camera response function. Then, K is used as the parameter of a brightness transformation function for one transformation on the low-light image to realize enhancement. In addition, a contrast-preserving brightness loss and an edge-preserving smoothness loss are designed without the requirement for references from the dataset. Both can further retain some key information in the inputs to improve precision. The enhancement is simplified and can reach more than twice the speed of similar methods. Extensive experiments on several LLIE datasets and the DARK FACE face detection dataset fully demonstrate our method's advantages, both subjectively and objectively.

Efficient detection of driver fatigue state based on all-weather illumination scenarios

Among the causes of the annually traffic accidents, driving fatigue is the main culprit. In consequence, it is of great practical significance to carry out the research of driving fatigue detection and early warning system. However, there are still two problems in the latest methods of driving fatigue detection: one is that a single information cannot precisely reflect the actual state of the driver in different fatigue phases, another one is the detection effect is not very well or even difficult to detect under abnormal illumination. In this paper, the multi-task cascaded convolutional networks (MTCNN) and infrared-based remote photo-plethysmography (rPPG) theory are used to extract the driver's facial and physiological information, and the multi-modal specific fatigue information is deeply excavated, and the multi-modal feature fusion model is constructed to comprehensively analyze the driver's fatigue variation tendency. Aiming at the matter of low detection accuracy under abnormal illumination, the multi-modal features extracted from visible light images and infrared images are fused by multi-loss reconstruction (MLR) module, and the driving fatigue detection module is established which is based on Bi-LSTM model by utilizing fatigue timing. The experiments were validated under all-weather illumination scenarios and were carried out on the datasets NTHU-DDD, UTA-RLDDD and FAHD. The results show that the multi-modal driving fatigue detection model has better performance than the single-modal model, and the accuracy is improved by 8.1%. In the abnormal illumination such as strong and weak light, the accuracy of the method can reach 91.7% at the highest and 83.6% at the lowest. Meanwhile, in the normal illumination, it can reach 93.2%.

Contrast Enhancement Method Using Region-Based Dynamic Clipping Technique for LWIR-Based Thermal Camera of Night Vision Systems

In the autonomous driving industry, there is a growing trend to employ long-wave infrared (LWIR)-based uncooled thermal-imaging cameras, capable of robustly collecting data even in extreme environments. Consequently, both industry and academia are actively researching contrast-enhancement techniques to improve the quality of LWIR-based thermal-imaging cameras. However, most research results only showcase experimental outcomes using mass-produced products that already incorporate contrast-enhancement techniques. Put differently, there is a lack of experimental data on contrast enhancement post-non-uniformity (NUC) and temperature compensation (TC) processes, which generate the images seen in the final products. To bridge this gap, we propose a histogram equalization (HE)-based contrast enhancement method that incorporates a region-based clipping technique. Furthermore, we present experimental results on the images obtained after applying NUC and TC processes. We simultaneously conducted visual and qualitative performance evaluations on images acquired after NUC and TC processes. In the visual evaluation, it was confirmed that the proposed method improves image clarity and contrast ratio compared to conventional HE-based methods, even in challenging driving scenarios such as tunnels. In the qualitative evaluation, the proposed method demonstrated upper-middle-class rankings in both image quality and processing speed metrics. Therefore, our proposed method proves to be effective for the essential contrast enhancement process in LWIR-based uncooled thermal-imaging cameras intended for autonomous driving platforms.

Research on Unsupervised Low-Light Railway Fastener Image Enhancement Method Based on Contrastive Learning GAN

The railway fastener, as a crucial component of railway tracks, directly influences the safety and stability of a railway system. However, in practical operation, fasteners are often in low-light conditions, such as at nighttime or within tunnels, posing significant challenges to defect detection equipment and limiting its effectiveness in real-world scenarios. To address this issue, this study proposes an unsupervised low-light image enhancement algorithm, CES-GAN, which achieves the model's generalization and adaptability under different environmental conditions. The CES-GAN network architecture adopts a U-Net model with five layers of downsampling and upsampling structures as the generator, incorporating both global and local discriminators to help the generator to preserve image details and textures during the reconstruction process, thus enhancing the realism and intricacy of the enhanced images. The combination of the feature-consistency loss, contrastive learning loss, and illumination loss functions in the generator structure, along with the discriminator loss function in the discriminator structure, collectively promotes the clarity, realism, and illumination consistency of the images, thereby improving the quality and usability of low-light images. Through the CES-GAN algorithm, this study provides reliable visual support for railway construction sites and ensures the stable operation and accurate operation of fastener identification equipment in complex environments.

Content-illumination coupling guided low-light image enhancement network

Current low-light enhancement algorithms fail to suppress noise when enhancing brightness, and may introduces structural distortion and color distortion caused by halos or artifacts. This paper proposes a content-illumination coupling guided low-light image enhancement network (CICGNet), it develops a truss topology based on Retinex as backbone to decompose low-light image component in an end-to-end way. The preservation of content features and the enhancement of illumination features are carried out along with depth and width direction of the truss topology. Each submodule uses the same resolution input and output to avoid the introduction of noise. Illumination component prevents misestimation of global and local illumination by using pre- and post-activation features at different depth levels, this way could avoid possible halos and artifacts. The network progressively enhances the illumination component and maintains the content component stage-by-stage. The proposed algorithm demonstrates better performance compared with advanced attention-based low-light enhancement algorithms and state-of-the-art image restoration algorithms. We also perform extensive ablation studies and demonstrate the impact of low-light enhancement algorithm on the downstream task of computer vision. Code is available at: https://github.com/Ruini94/CICGNet .

Restoration of UAV-Based Backlit Images for Geological Mapping of a High-Steep Slope

Unmanned aerial vehicle (UAV)-based geological mapping is significant for understanding the geological structure in the high-steep slopes, but the images obtained in these areas are inevitably influenced by the backlit effect because of the undulating terrain and the viewpoint change of the camera mounted on the UAV. To handle this concern, a novel backlit image restoration method is proposed that takes the real-world application into account and addresses the color distortion issue existing in backlit images captured in high-steep slope scenes. Specifically, there are two main steps in the proposed method, which consist of the backlit removal and the color and detail enhancement. The backlit removal first eliminates the backlit effect using the Retinex strategy, and then the color and detail enhancement step improves the image color and sharpness. The author designs extensive comparison experiments from multiple angles and applies the proposed method to different engineering applications. The experimental results show that the proposed method has potential compared to other main-stream methods both in qualitative visual effects and universal quantitative evaluation metrics. The backlit images processed by the proposed method are significantly improved by the process of feature key point matching, which is very conducive to the fine construction of 3D geological models of the high-steep slope.

MTIE-Net: Multi-technology fusion of low-light image enhancement network

Images obtained in low-light scenes are often accompanied by problems such as low visibility, blurred details, and color distortion, enhancing them can effectively improve the visual effect and provide favorable conditions for advanced visual tasks. In this study, we propose a Multi-Technology Fusion of Low-light Image Enhancement Network (MTIE-Net) that modularizes the enhancement task. MTIE-Net consists of a residual dense decomposition network (RDD-Net) based on Retinex theory, an encoder-decoder denoising network (EDD-Net), and a parallel mixed attention-based self-calibrated illumination enhancement network (PCE-Net). The low-light image is first decomposed by RDD-Net into a lighting map and reflectance map; EDD-Net is used to process noise in the reflectance map; Finally, the lighting map is fused with the denoised reflectance map as an input to PCE-Net, using the Fourier transform for illumination enhancement and detail recovery in the frequency domain. Numerous experimental results show that MTIE-Net outperforms the comparison methods in terms of image visual quality enhancement improvement, denoising, and detail recovery. The application in nighttime face detection also fully demonstrates its promise as a pre-processing means in practical applications.

DRLIE: Flexible Low-Light Image Enhancement via Disentangled Representations

Low-light image enhancement (LIME) aims to convert images with unsatisfied lighting into desired ones. Different from existing methods that manipulate illumination in uncontrollable manners, we propose a flexible framework to take user-specified guide images as references to improve the practicability. To achieve the goal, this article models an image as the combination of two components, that is, content and exposure attribute, from an information decoupling perspective. Specifically, we first adopt a content encoder and an attribute encoder to disentangle the two components. Then, we combine the scene content information of the low-light image with the exposure attribute of the guide image to reconstruct the enhanced image through a generator. Extensive experiments on public datasets demonstrate the superiority of our approach over state-of-the-art alternatives. Particularly, the proposed method allows users to enhance images according to their preferences, by providing specific guide images. Our source code and the pretrained model are available at https://github.com/Linfeng-Tang/DRLIE.

GMRE-iUnet: Isomorphic Unet fusion model for PET and CT lung tumor images

Lung tumor PET and CT image fusion is a key technology in clinical diagnosis. However, the existing fusion methods are difficult to obtain fused images with high contrast, prominent morphological features, and accurate spatial localization. In this paper, an isomorphic Unet fusion model (GMRE-iUnet) for lung tumor PET and CT images is proposed to address the above problems. The main idea of this network is as following: Firstly, this paper constructs an isomorphic Unet fusion network, which contains two independent multiscale dual encoders Unet, it can capture the features of the lesion region, spatial localization, and enrich the morphological information. Secondly, a Hybrid CNN-Transformer feature extraction module (HCTrans) is constructed to effectively integrate local lesion features and global contextual information. In addition, the residual axial attention feature compensation module (RAAFC) is embedded into the Unet to capture fine-grained information as compensation features, which makes the model focus on local connections in neighboring pixels. Thirdly, a hybrid attentional feature fusion module (HAFF) is designed for multiscale feature information fusion, it aggregates edge information and detail representations using local entropy and Gaussian filtering. Finally, the experiment results on the multimodal lung tumor medical image dataset show that the model in this paper can achieve excellent fusion performance compared with other eight fusion models. In CT mediastinal window images and PET images comparison experiment, AG, EI, QAB/F, SF, SD, and IE indexes are improved by 16.19%, 26%, 3.81%, 1.65%, 3.91% and 8.01%, respectively. GMRE-iUnet can highlight the information and morphological features of the lesion areas and provide practical help for the aided diagnosis of lung tumors.

UIEOGP: an underwater image enhancement method based on optical geometric properties

Due to the inconsistent absorption and scattering effects of different wavelengths of light, underwater images often suffer from color casts, blurred details, and low visibility. To address this image degradation problem, we propose a robust and efficient underwater image enhancement method named UIEOGP. It can be divided into the following three steps. First, according to the light attenuation effect presented by Lambert Beer's law, combined with the variance change after attenuation, we estimate the depth of field in the underwater image. Then, we propose a local-based color correction algorithm to address the color cast issue in underwater images, employing the statistical distribution law. Finally, drawing inspiration from the law of light propagation, we propose detail enhancement algorithms, each based on the geometric properties of circles and ellipses, respectively. The enhanced images produced by our method feature vibrant colors, improved contrast, and sharper detail. Extensive experiments show that our method outperforms current state-of-the-art methods. In further experiments, we found that our method is beneficial for downstream tasks of underwater image processing, such as the detection of keypoints and edges in underwater images.

An H-GrabCut Image Segmentation Algorithm for Indoor Pedestrian Background Removal

In the context of predicting pedestrian trajectories for indoor mobile robots, it is crucial to accurately measure the distance between indoor pedestrians and robots. This study aims to address this requirement by extracting pedestrians as regions of interest and mitigating issues related to inaccurate depth camera distance measurements and illumination conditions. To tackle these challenges, we focus on an improved version of the H-GrabCut image segmentation algorithm, which involves four steps for segmenting indoor pedestrians. Firstly, we leverage the YOLO-V5 object recognition algorithm to construct detection nodes. Next, we propose an enhanced BIL-MSRCR algorithm to enhance the edge details of pedestrians. Finally, we optimize the clustering features of the GrabCut algorithm by incorporating two-dimensional entropy, UV component distance, and LBP texture feature values. The experimental results demonstrate that our algorithm achieves a segmentation accuracy of 97.13% in both the INRIA dataset and real-world tests, outperforming alternative methods in terms of sensitivity, missegmentation rate, and intersection-over-union metrics. These experiments confirm the feasibility and practicality of our approach. The aforementioned findings will be utilized in the preliminary processing of indoor mobile robot pedestrian trajectory prediction and enable path planning based on the predicted results.

A Survey of Deep Learning-Based Low-Light Image Enhancement

Images captured under poor lighting conditions often suffer from low brightness, low contrast, color distortion, and noise. The function of low-light image enhancement is to improve the visual effect of such images for subsequent processing. Recently, deep learning has been used more and more widely in image processing with the development of artificial intelligence technology, and we provide a comprehensive review of the field of low-light image enhancement in terms of network structure, training data, and evaluation metrics. In this paper, we systematically introduce low-light image enhancement based on deep learning in four aspects. First, we introduce the related methods of low-light image enhancement based on deep learning. We then describe the low-light image quality evaluation methods, organize the low-light image dataset, and finally compare and analyze the advantages and disadvantages of the related methods and give an outlook on the future development direction.

Low-light image enhancement based on Retinex-Net with color restoration

Low-light images often suffer from a variety of degradation problems such as loss of detail, color distortions, and prominent noise. In this paper, the Retinex-Net model and loss function with color restoration are proposed to reduce color distortion in low-light image enhancement. The model trains the decom-net and color recovery-net to achieve decomposition of low-light images and color restoration of reflected images, respectively. First, a convolutional neural network and the designed loss functions are used in the decom-net to decompose the low-light image pair into an optimal reflection image and illumination image as the input of the network, and the reflection image after normal light decomposition is taken as the label. Then, an end-to-end color recovery network with a simplified model and time complexity is learned and combined with the color recovery loss function to obtain the correction reflection map with higher perception quality, and gamma correction is applied to the decomposed illumination image. Finally, the corrected reflection image and the illumination image are synthesized to get the enhanced image. The experimental results show that the proposed network model has lower brightness-order-error (LOE) and natural image quality evaluator (NIQE) values, and the average LOE and NIQE values of the low-light dataset images can be reduced to 942 and 6.42, respectively, which significantly improves image quality compared with other low-light enhancement methods. Generally, our proposed method can effectively improve image illuminance and restore color information in the end-to-end learning process of low-light images.

Low-light image enhancement via adaptive frequency decomposition network

Images captured in low light conditions suffer from low visibility, blurred details and strong noise, resulting in unpleasant visual appearance and poor performance of high level visual tasks. To address these problems, existing approaches have attempted to enhance the visibility of low-light images using convolutional neural networks (CNN). However, due to the insufficient consideration of the characteristics of the information of different frequency layers in the image, most of them yield blurry details and amplified noise. In this work, to fully extract and utilize these information, we proposed a novel Adaptive Frequency Decomposition Network (AFDNet) for low-light image enhancement. An Adaptive Frequency Decomposition (AFD) module is designed to adaptively extract low and high frequency information of different granularities. Specifically, the low-frequency information is employed for contrast enhancement and noise suppression in low-scale space and high-frequency information is for detail restoration in high-scale space. Meanwhile, a new frequency loss function are proposed to guarantee AFDNet's recovery capability for different frequency information. Extensive experiments on various publicly available datasets show that AFDNet outperforms the existing state-of-the-art methods both quantitatively and visually. In addition, our results showed that the performance of the face detection can be effectively improved by using AFDNet as pre-processing.

A Low-Illumination Enhancement Method Based on Structural Layer and Detail Layer

Low-illumination image enhancement technology is a topic of interest in the field of image processing. However, while improving image brightness, it is difficult to effectively maintain the texture and details of the image, and the quality of the image cannot be guaranteed. In order to solve this problem, this paper proposed a low-illumination enhancement method based on structural and detail layers. Firstly, we designed an SRetinex-Net model. The network is mainly divided into two parts: a decomposition module and an enhancement module. Second, the decomposition module mainly adopts the SU-Net structure, which is an unsupervised network that decomposes the input image into a structural layer image and detail layer image. Afterward, the enhancement module mainly adopts the SDE-Net structure, which is divided into two branches: the SDE-S branch and the SDE-D branch. The SDE-S branch mainly enhances and adjusts the brightness of the structural layer image through Ehnet and Adnet to prevent insufficient or overexposed brightness enhancement in the image. The SDE-D branch is mainly denoised and enhanced with textural details through a denoising module. This network structure can greatly reduce computational costs. Moreover, we also improved the total variation optimization model as a mixed loss function and added structural metrics and textural metrics as variables on the basis of the original loss function, which can well separate the structure edge and texture edge. Numerous experiments have shown that our structure has a more significant impact on the brightness and detail preservation of image restoration.

Deep polyp image enhancement using region of interest with paired supervision

Endoscopic medical imaging in complex curved intestinal structures are prone to uneven illumination, low contrast and lack of texture information. These problems may lead to diagnostic challenges. This paper described the first supervised deep learning based image fusion framework to enable the polyp region highlight through a global image enhancement and a local region of interest (ROI) with paired supervision. Firstly, we conducted a dual attention based network in global image enhancement. The Detail Attention Maps was used to preserve more image details and the Luminance Attention Maps was used to adjust the global illumination of the image. Secondly, we adopted the advanced polyp segmentation network ACSNet to obtain the accurate mask image of lesion region in local ROI acquisition. Finally, a new image fusion strategy was proposed to realize the local enhancement effect of polyp image. Experimental results show that our method can highlight the local details of the lesion area better and reach the optimal comprehensive performance with comparing with 16 traditional and state-of-the-art enhancement algorithms. And 8 doctors and 12 medical students were asked to evaluate our method for assisting clinical diagnosis and treatment effectively. Furthermore, the first paired image dataset LHI was constructed, which will be made available as an open source to research communities.

Dental image enhancement network for early diagnosis of oral dental disease

Intelligent robotics and expert system applications in dentistry suffer from identification and detection problems due to the non-uniform brightness and low contrast in the captured images. Moreover, during the diagnostic process, exposure of sensitive facial parts to ionizing radiations (e.g., X-Rays) has several disadvantages and provides a limited angle for the view of vision. Capturing high-quality medical images with advanced digital devices is challenging, and processing these images distorts the contrast and visual quality. It curtails the performance of potential intelligent and expert systems and disincentives the early diagnosis of oral and dental diseases. The traditional enhancement methods are designed for specific conditions, and network-based methods rely on large-scale datasets with limited adaptability towards varying conditions. This paper proposed a novel and adaptive dental image enhancement strategy based on a small dataset and proposed a paired branch Denticle-Edification network (Ded-Net). The input dental images are decomposed into reflection and illumination in a multilayer Denticle network (De-Net). The subsequent enhancement operations are performed to remove the hidden degradation of reflection and illumination. The adaptive illumination consistency is maintained through the Edification network (Ed-Net). The network is regularized following the decomposition congruity of the input data and provides user-specific freedom of adaptability towards desired contrast levels. The experimental results demonstrate that the proposed method improves visibility and contrast and preserves the edges and boundaries of the low-contrast input images. It proves that the proposed method is suitable for intelligent and expert system applications for future dental imaging.

Low-Light Image Enhancement with Contrast Increase and Illumination Smooth

In image enhancement, maintaining the texture and attenuating noise are worth discussing. To address these problems, we propose a low-light image enhancement method with contrast increase and illumination smooth. First, we calculate the maximum map and the minimum map of RGB channels, and then we set maximum map as the initial value for illumination and introduce minimum map to smooth illumination. Second, we use the histogram-equalized version of the input image to construct the weight for the illumination map. Third, we propose an optimization problem to obtain the smooth illumination and refined reflectance. Experimental results show that our method can achieve better performance compared to the state-of-the-art methods.

Enhancing retinal images in low-light conditions using semidecoupled decomposition

Eye diseases that are common and many diseases that result in visual ailments, such as diabetes and vascular disease, can be diagnosed through retinal imaging. The enhancement of retinal images often helps in diagnosing diseases related to retinal organ failure. However, today's image enhancement methods may lead to artificial boundaries, sudden color gradation, and the loss of image details. Therefore, to prevent these side effects, a new method of retinal image enhancement is proposed. In this work, we propose a new method for enhancing the overall contrast of colored retinal images. That is, we propose low-light image enhancement using a new retinex method based on a powerful semidecoupled retinex method. In particular, illumination layer I gradually approximates the S input image according to the file. This leads to a complete Gaussian transformation model, while the R-layer reflectance is estimated jointly by S and intermediary by I to suppress image noise simultaneously during R estimation on the publicly available Messidor database. From our assessment measurements (PSNR and SSIM), we show that this proposed method is effective in comparison with the relevant and recently proposed retinal imaging methods; moreover, the color, which is determined by the data, does not change the image structure. Finally, a technique is presented to improve the pronounced color of a retinal image, which is useful for ophthalmologists to screen for retinal disease more effectively. Moreover, this technique can be used in the development of robotics for imaging tests to search for clinical markers.

Research on steel rail surface defects detection based on improved YOLOv4 network

Introduction

The surface images of steel rails are extremely difficult to detect and recognize due to the presence of interference such as light changes and texture background clutter during the acquisition process.

Methods

To improve the accuracy of railway defects detection, a deep learning algorithm is proposed to detect the rail defects. Aiming at the problems of inconspicuous rail defects edges, small size and background texture interference, the rail region extraction, improved Retinex image enhancement, background modeling difference, and threshold segmentation are performed sequentially to obtain the segmentation map of defects. For the classification of defects, Res2Net and CBAM attention mechanism are introduced to improve the receptive field and small target position weights. The bottom-up path enhancement structure is removed from the PANet structure to reduce the parameter redundancy and enhance the feature extraction of small targets.

Results

The results show the average accuracy of rail defects detection reaches 92.68%, the recall rate reaches 92.33%, and the average detection time reaches an average of 0.068 s per image, which can meet the real-time of rail defects detection.

Discussion

Comparing the improved method with the mainstream target detection algorithms such as Faster RCNN, SSD, YOLOv3 and other algorithms, the improved YOLOv4 has excellent comprehensive performance for rail defects detection, the improved YOLOv4 model obviously better than several others in P _r , R _c , and F1 value, and can be well-applied to rail defect detection projects.

A Retinex-based variational model for noise suppression and nonuniform illumination correction in corneal confocal microscopy images

Objective.Corneal confocal microscopy (CCM) image analysis is a non-invasivein vivoclinical technique that can quantify corneal nerve fiber damage. However, the acquired CCM images are often accompanied by speckle noise and nonuniform illumination, which seriously affects the analysis and diagnosis of the diseases.Approach.In this paper, first we propose a variational Retinex model for the inhomogeneity correction and noise removal of CCM images. In this model, the Beppo Levi space is introduced to constrain the smoothness of the illumination layer for the first time, and the fractional order differential is adopted as the regularization term to constrain reflectance layer. Then, a denoising regularization term is also constructed with Block Matching 3D (BM3D) to suppress noise. Finally, by adjusting the uneven illumination layer, we obtain the final results. Second, an image quality evaluation metric is proposed to evaluate the illumination uniformity of images objectively.Main results.To demonstrate the effectiveness of our method, the proposed method is tested on 628 low-quality CCM images from the CORN-2 dataset. Extensive experiments show the proposed method outperforms the other four related methods in terms of noise removal and uneven illumination suppression.SignificanceThis demonstrates that the proposed method may be helpful for the diagnostics and analysis of eye diseases.

4D LUT: Learnable Context-Aware 4D Lookup Table for Image Enhancement

Image enhancement aims at improving the aesthetic visual quality of photos by retouching the color and tone, and is an essential technology for professional digital photography. Recent years deep learning-based image enhancement algorithms have achieved promising performance and attracted increasing popularity. However, typical efforts attempt to construct a uniform enhancer for all pixels' color transformation. It ignores the pixel differences between different content (e.g., sky, ocean, etc.) that are significant for photographs, causing unsatisfactory results. In this paper, we propose a novel learnable context-aware 4-dimensional lookup table (4D LUT), which achieves content-dependent enhancement of different contents in each image via adaptively learning of photo context. In particular, we first introduce a lightweight context encoder and a parameter encoder to learn a context map for the pixel-level category and a group of image-adaptive coefficients, respectively. Then, the context-aware 4D LUT is generated by integrating multiple basis 4D LUTs via the coefficients. Finally, the enhanced image can be obtained by feeding the source image and context map into fused context-aware 4D LUT via quadrilinear interpolation. Compared with traditional 3D LUT, i.e., RGB mapping to RGB, which is usually used in camera imaging pipeline systems or tools, 4D LUT, i.e., RGBC(RGB+Context) mapping to RGB, enables finer control of color transformations for pixels with different content in each image, even though they have the same RGB values. Experimental results demonstrate that our method outperforms other state-of-the-art methods in widely-used benchmarks.

LTF-NSI: a novel local transfer function based on neighborhood similarity index for medical image enhancement

Medical image is an essential tool used in quantitative and qualitative evaluation of different diseases. Medical imaging methods such as fluorescein angiography (FA), optical coherence tomography angiography (OCTA), computed tomography (CT), optical coherence tomography (OCT), and X-ray are used for diagnosis. These imaging modalities suffer from low contrast, which leads to deterioration in the image quality. Consequently, this causes limitation in the usage of medical images in clinical routine and hindered its potential by depriving clinicians from assessing useful information that are needed in disease monitoring, treatment, progression, and decision-making. To overcome this limitation, we propose a novel local transfer function for medical image enhancement algorithm using the pixel neighborhood constraint. The proposed algorithm uses block-wise intensity distribution to generate the regional similarity index. The regional similarity index transformed each centered pixel in the block, to generate a new similarity image. An intuitive optimization algorithm is utilized to optimize the proposed algorithm parameters. Experimentation results show that the proposed LTF-NSI performs better than the state-of-the-art methods and improves the interpretability and perception of the medical images, which can provide clinicians and computer vision program with good quantitative and qualitative information.

Low light color balancing and denoising by machine learning based approximation for underwater images

It is difficult for underwater archaeologists to recover the fine details of a captured image on the seabed when the image quality worsens due to the presence of more noisy artefacts, a mismatched device colour map, and a blurry image. To resolve this problem, we present a machine learning-based image restoration model (ML-IRM) for improving the visual quality of underwater images that have been deteriorated. Using this model, a home-made bowl set-up is created in which a different liquid concentration is used to replicate seabed water variation, and an object is dipped, or a video is played behind the bowl to recognise the object texture captured image in high-resolution for training the image restoration model is proposed. Gaussian and bidirectional pre-processing filters are used to both the high and low frequency components of the training image, respectively. To improve the clarity of the high-frequency channel background, soft-thresholding decreases the presence of distracting artefacts. On the other hand, the ML-IRM model can effectively keep the object textures on a low frequency channel. Experiment findings show that the proposed ML-IRM model improves the quality of seabed images, eliminates colour mismatches, and allows for more detailed information extraction. Blue shadow, green shadow, hazy, and low light test samples are randomly selected from all five datasets including U45 [1], EUVP [2], DUIE [3], UIEB [4], UM-ImageNet [5], and the proposed model. Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are computed for each condition separately. We list the values of PSNR (at 16.99 dB, 15.96 dB, 18.09 dB, 15.67 dB, 9.39 dB, 17.98 dB, 19.32 dB, 14.27 dB, 12.07 dB, and 25.47 dB) and SSIM (at 0.52, 0.57, 0.33, 0.47, 0.44, and 0.23, respectively. Similarly, it demonstrates that the proposed ML-IRM achieves a satisfactory result in terms of colour correction and contrast adjustment when applied to the problem of improving underwater images captured in low light. To do so, high-resolution images were captured in two low-light conditions (after 6 p.m. and again at 6 a.m.) for the training image datasets, and the results of their observations were compared to those of other existing state-of-the-art-methods.

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination. Recent advances in this area are dominated by deep learning-based solutions, where many learning strategies, network structures, loss functions, training data, etc. have been employed. In this paper, we provide a comprehensive survey to cover various aspects ranging from algorithm taxonomy to unsolved open issues. To examine the generalization of existing methods, we propose a low-light image and video dataset, in which the images and videos are taken by different mobile phones' cameras under diverse illumination conditions. Besides, for the first time, we provide a unified online platform that covers many popular LLIE methods, of which the results can be produced through a user-friendly web interface. In addition to qualitative and quantitative evaluation of existing methods on publicly available and our proposed datasets, we also validate their performance in face detection in the dark. This survey together with the proposed dataset and online platform could serve as a reference source for future study and promote the development of this research field. The proposed platform and dataset as well as the collected methods, datasets, and evaluation metrics are publicly available and will be regularly updated. Project page: https://www.mmlab-ntu.com/project/lliv_survey/index.html.

Robust contrast enhancement method using a retinex model with adaptive brightness for detection applications

Low light image enhancement with adaptive brightness, color and contrast preservation in degraded visual conditions (e.g., extreme dark background, lowlight, back-light, mist. etc.) is becoming more challenging for machine cognition applications than anticipated. A realistic image enhancement framework should preserve brightness and contrast in robust scenarios. The extant direct enhancement methods amplify objectionable structure and texture artifacts, whereas network-based enhancement approaches are based on paired or large-scale training datasets, raising fundamental concerns about their real-world applicability. This paper presents a new framework to get deep into darkness in degraded visual conditions following the fundamental of retinex-based image decomposition. We separate the reflection and illumination components to perform independent weighted enhancement operations on each component to preserve the visual details with a balance of brightness and contrast. A comprehensive weighting strategy is proposed to constrain image decomposition while disrupting the irregularities of high frequency reflection and illumination to improve the contrast. At the same time, we propose to guide the illumination component with a high-frequency component for structure and texture preservation in degraded visual conditions. Unlike existing approaches, the proposed method works regardless of the training data type (i.e., low light, normal light, or normal and low light pairs). A deep into darkness network (D2D-Net) is proposed to maintain the visual balance of smoothness without compromising the image quality. We conduct extensive experiments to demonstrate the superiority of the proposed enhancement. We test the performance of our method for object detection tasks in extremely dark scenarios. Experimental results demonstrate that our method maintains the balance of visual smoothness, making it more viable for future interactive visual applications.