1000 fps computational ghost imaging using LED-based structured illumination

Single-Pixel Imaging Based on Deep Learning Enhanced Singular Value Decomposition

We propose and demonstrate a single-pixel imaging method based on deep learning network enhanced singular value decomposition. The theoretical framework and the experimental implementation are elaborated and compared with the conventional methods based on Hadamard patterns or deep convolutional autoencoder network. Simulation and experimental results show that the proposed approach is capable of reconstructing images with better quality especially under a low sampling ratio down to 3.12%, or with fewer measurements or shorter acquisition time if the image quality is given. We further demonstrate that it has better anti-noise performance by introducing noises in the SPI systems, and we show that it has better generalizability by applying the systems to targets outside the training dataset. We expect that the developed method will find potential applications based on single-pixel imaging beyond the visible regime.

Efficient single-pixel imaging based on a compact fiber laser array and untrained neural network

This paper presents an efficient scheme for single-pixel imaging (SPI) utilizing a phase-controlled fiber laser array and an untrained deep neural network. The fiber lasers are arranged in a compact hexagonal structure and coherently combined to generate illuminating light fields. Through the utilization of high-speed electro-optic modulators in each individual fiber laser module, the randomly modulated fiber laser array enables rapid speckle projection onto the object of interest. Furthermore, the untrained deep neural network is incorporated into the image reconstructing process to enhance the quality of the reconstructed images. Through simulations and experiments, we validate the feasibility of the proposed method and successfully achieve high-quality SPI utilizing the coherent fiber laser array at a sampling ratio of 1.6%. Given its potential for high emitting power and rapid modulation, the SPI scheme based on the fiber laser array holds promise for broad applications in remote sensing and other applicable fields.

Single-pixel imaging based on metasurface fuzzy coding

Single-pixel imaging, renowned for its high sensitivity, robustness against interference, and superior resolution, has become increasingly prominent in the field of optical research. Over recent years, a diverse array of light modulation devices and methodologies has been devised to accomplish megahertz modulations rates. This work presents a single-pixel imaging scheme based on the fuzzy coding of metasurfaces. This unique encoding technique manipulates the quality of the mask pattern by adjusting the pixel count within the metasurface units. Notably, we expand the metasurface units to effectively mitigate the position sensitivity during movement or rotations, thus easing the challenge for the detector in collecting the correct light intensity during sub-mask transitions. A detailed analysis is drawn of the reconstruction quality of fuzzy masks. Simultaneously, we provide simulations of single-pixel imaging under the condition where the fuzzy-coded metasurface is moving. This work provides a new, to the best of our knowledge, mask generation mode for high-speed spatial light modulation.

Robust compressed ghost imaging against environmental influence factors

Ghost imaging based on sparse sampling is sensitive to the environmental influence factors frequently encountered in practice, such as instrumental drift and ambient light change, which could cause degradation of image quality. In this manuscript, we report a robust compressed sensing technique which could effectively reduce the influence of measurement errors on image quality. For demonstration purposes, we implement the proposed technique to ghost imaging, namely differential compressed sensing ghost imaging (DCSGI). By applying differential measurements n times, the first n Taylor expansion polynomials of the error could be eliminated in n-order DCSGI. It has been verified theoretically and experimentally that DCSGI works well with typical errors which exists in the realities of ghost imaging applications, while the conventional approach can hardly. In addition, the proposed technique may also replace conventional compressed sensing in other applications for anti-interference high-quality reconstruction.

Ghost imaging based on Fermat spiral laser array designed for remote sensing

We propose a Fermat spiral laser array as illumination source in ghost imaging. Due to the aperiodic structure, the Fermat spiral laser array generates illuminating light field without spatial periodicity on the normalized second-order intensity correlation function. A single-pixel detector is used to receive the signal light from object for image reconstruction. The effects of laser array parameters on the quality of ghost imaging are analyzed comprehensively. Through experimental demonstration, the Fermat spiral laser array successfully achieves ghost imaging with high quality by combining with the compressive sensing reconstruction algorithm. This method is expected to be applied in remote sensing by combining with phased and collimated fiber laser array equipped with the high emitting power and high-speed modulation frequency.

Advances in Ghost Imaging of Moving Targets: A Review

Ghost imaging is a novel imaging technique that utilizes the intensity correlation property of an optical field to retrieve information of the scene being measured. Due to the advantages of simple structure, high detection efficiency, etc., ghost imaging exhibits broad application prospects in the fields of space remote sensing, optical encryption transmission, medical imaging, and so on. At present, ghost imaging is gradually developing toward practicality, in which ghost imaging of moving targets is becoming a much-needed breakthrough link. At this stage, we can improve the imaging speed and improve the imaging quality to seek a more optimized ghost imaging scheme for moving targets. Based on the principle of moving target ghost imaging, this review summarizes and compares the existing methods for ghost imaging of moving targets. It also discusses the research direction and the technical challenges at the current stage to provide references for further promotion of the instantiation of ghost imaging applications.

Double filter iterative ghost imaging for high quality edge and image acquisition

Improving imaging quality and reducing time consumption are the key problems that need to be solved in the practical application of ghost imaging. Hence, we demonstrate a double filter iterative ghost imaging method, which adopts the joint iteration of projected Landweber iterative regularization and double filtering based on block matching three dimensional filtering and guided filtering to achieve high-quality image reconstruction under low measurement and low iteration times. This method combines the advantages of ill-posed problem solution of projected Landweber iterative regularization with double filtering joint iterative de-noising and edge preservation. The numerical simulation results show that our method outperforms the comparison method by 4 to 6 dB in terms of peak signal-to-noise ratio for complex binary target 'rice' and grayscale target 'aircraft' after 1500 measurements. The comparison results of experiments and numerical simulations using similar aircraft targets show that this method is superior to the comparison method, especially in terms of richer and more accurate edge detection results. This method can simultaneously obtain high quality reconstructed image and edge feature information under low measurement and iteration times, which is of great value for the practical application fields of imaging and edge detection at the same time, such as intelligent driving, remote sensing and other fields.

High-resolution self-corrected single-pixel imaging through dynamic and complex scattering media

Imaging with single-pixel detectors becomes attractive in many applications where pixelated detectors are not available or cannot work. Based on a correlation between the probing patterns and the realizations, optical imaging with single-pixel detector offers an indirect way to recover a sample. It is well recognized that single-pixel optical imaging through dynamic and complex scattering media is challenging, and dynamic scaling factors lead to serious mismatches between the probing patterns and the realizations. In this paper, we report self-corrected imaging to realize high-resolution object reconstruction through dynamic and complex scattering media using a parallel detection with dual single-pixel detectors. The proposed method can supervise and self-correct dynamic scaling factors, and can implement high-resolution object reconstruction through dynamic and complex scattering media where conventional methods could not work. Spatial resolution of 44.19 µm is achieved which approaches diffraction limit (40.0 µm) in the designed optical setup. The achievable spatial resolution is dependent on pixel size of spatial light modulator. It is experimentally validated that the proposed method shows unprecedented robustness against complex scattering. The proposed self-corrected imaging provides a solution for ghost recovery, enabling high-resolution object reconstruction in complex scattering environments.

Dual-mode adaptive-SVD ghost imaging

In this paper, we present a dual-mode adaptive singular value decomposition ghost imaging (A-SVD GI), which can be easily switched between the modes of imaging and edge detection. It can adaptively localize the foreground pixels via a threshold selection method. Then only the foreground region is illuminated by the singular value decomposition (SVD) - based patterns, consequently retrieving high-quality images with fewer sampling ratios. By changing the selecting range of foreground pixels, the A-SVD GI can be switched to the mode of edge detection to directly reveal the edge of objects, without needing the original image. We investigate the performance of these two modes through both numerical simulations and experiments. We also develop a single-round scheme to halve measurement numbers in experiments, instead of separately illuminating positive and negative patterns in traditional methods. The binarized SVD patterns, generated by the spatial dithering method, are modulated by a digital micromirror device (DMD) to speed up the data acquisition. This dual-mode A-SVD GI can be applied in various applications, such as remote sensing or target recognition, and could be further extended for multi-modality functional imaging/detection.

Ghost images with controllable visibility and spatial resolution

We designed a kind of speckle field with controllable visibility and speckle grain size through a modified Gerchberg-Saxton algorithm based on Fresnel diffraction. Ghost images with independently controllable visibility and spatial resolution were demonstrated based on the designed speckle fields, which could be of much higher visibility and spatial resolution than those with pseudothermal light. In addition, speckle fields capable of reconstructing ghost images simultaneously on multiple different planes were customized. These results could be of potential applications on optical encryption and optical tomography.

Low sampling high quality image reconstruction and segmentation based on array network ghost imaging

High-quality imaging under low sampling time is an important step in the practical application of computational ghost imaging (CGI). At present, the combination of CGI and deep learning has achieved ideal results. However, as far as we know, most researchers focus on one single pixel CGI based on deep learning, and the combination of array detection CGI and deep learning with higher imaging performance has not been mentioned. In this work, we propose a novel multi-task CGI detection method based on deep learning and array detector, which can directly extract target features from one-dimensional bucket detection signals at low sampling times, especially output high-quality reconstruction and image-free segmentation results at the same time. And this method can realize fast light field modulation of modulation devices such as digital micromirror device to improve the imaging efficiency by binarizing the trained floating-point spatial light field and fine-tuning the network. Meanwhile, the problem of partial information loss in the reconstructed image due to the detection unit gap in the array detector has also been solved. Simulation and experimental results show that our method can simultaneously obtain high-quality reconstructed and segmented images at sampling rate of 0.78 %. Even when the signal-to-noise ratio of the bucket signal is 15 dB, the details of the output image are still clear. This method helps to improve the applicability of CGI and can be applied to resource-constrained multi-task detection scenarios such as real-time detection, semantic segmentation, and object recognition.

Mid-infrared single-pixel imaging at the single-photon level

Single-pixel cameras have recently emerged as promising alternatives to multi-pixel sensors due to reduced costs and superior durability, which are particularly attractive for mid-infrared (MIR) imaging pertinent to applications including industry inspection and biomedical diagnosis. To date, MIR single-pixel photon-sparse imaging has yet been realized, which urgently calls for high-sensitivity optical detectors and high-fidelity spatial modulators. Here, we demonstrate a MIR single-photon computational imaging with a single-element silicon detector. The underlying methodology relies on nonlinear structured detection, where encoded time-varying pump patterns are optically imprinted onto a MIR object image through sum-frequency generation. Simultaneously, the MIR radiation is spectrally translated into the visible region, thus permitting infrared single-photon upconversion detection. Then, the use of advanced algorithms of compressed sensing and deep learning allows us to reconstruct MIR images under sub-Nyquist sampling and photon-starving illumination. The presented paradigm of single-pixel upconversion imaging is featured with single-pixel simplicity, single-photon sensitivity, and room-temperature operation, which would establish a new path for sensitive imaging at longer infrared wavelengths or terahertz frequencies, where high-sensitivity photon counters and high-fidelity spatial modulators are typically hard to access.

Single-pixel imaging of a translational object

Image-free tracking methods based on single-pixel detectors (SPDs) can track a moving object at a very high frame rate, but they rarely can achieve simultaneous imaging of such an object. In this study, we propose a method for simultaneously obtaining the relative displacements and images of a translational object. Four binary Fourier patterns and two differential Hadamard patterns are used to modulate one frame of the object and then modulated light signals are obtained by SPD. The relative displacements and image of the moving object can be gradually obtained along with the detection. The proposed method does not require any prior knowledge of the object and its motion. The method has been verified by simulations and experiments, achieving a frame rate of 3332 Hz to acquire relative displacements of a translational object at a spatial resolution of 128 × 128 pixels using a 20000-Hz digital micro-mirror device. This proposed method can broaden the application of image-free tracking methods and obtain spatial information about moving objects.

Compressed ultrahigh-speed single-pixel imaging by swept aggregate patterns

Single-pixel imaging (SPI) has emerged as a powerful technique that uses coded wide-field illumination with sampling by a single-point detector. Most SPI systems are limited by the refresh rates of digital micromirror devices (DMDs) and time-consuming iterations in compressed-sensing (CS)-based reconstruction. Recent efforts in overcoming the speed limit in SPI, such as the use of fast-moving mechanical masks, suffer from low reconfigurability and/or reduced accuracy. To address these challenges, we develop SPI accelerated via swept aggregate patterns (SPI-ASAP) that combines a DMD with laser scanning hardware to achieve pattern projection rates of up to 14.1 MHz and tunable frame sizes of up to 101×103 pixels. Meanwhile, leveraging the structural properties of S-cyclic matrices, a lightweight CS reconstruction algorithm, fully compatible with parallel computing, is developed for real-time video streaming at 100 frames per second (fps). SPI-ASAP allows reconfigurable imaging in both transmission and reflection modes, dynamic imaging under strong ambient light, and offline ultrahigh-speed imaging at speeds of up to 12,000 fps.

Anti-noise computational imaging using unsupervised deep learning

Computational imaging enables spatial information retrieval of objects with the use of single-pixel detectors. By combining measurements and computational methods, it is possible to reconstruct images in a variety of situations that are challenging or impossible with traditional multi-pixel cameras. However, these systems typically suffer from significant loss of imaging quality due to various noises when the measurement conditions are single-photon detecting, undersampling and complicated. Here, we provide an unsupervised deep learning (UnDL) based anti-noise approach to deal with this problem. The proposed method does not require any clean experimental data to pre-train, so it effectively alleviates the difficulty of model training (especially for the biomedical imaging scene which is difficult to obtain training ground truth inherently). Our results show that an UnDL based imaging approach outperforms conventional single-pixel computational imaging methods considerably in reconstructing the target image against noise. Moreover, the well-trained model is generalized to image a real biological sample and can accurately image 64 × 64 resolution objects with a high speed of 20 fps at 5% sampling ratio. This method can be used in various solvers for general computational imaging and is expected to effectively suppress noises for high-quality biomedical imaging in generalizable complicated environments.

Single-pixel imaging of a randomly moving object

Single-pixel imaging enjoys advantages of low budget, broad spectrum, and high imaging speed. However, existing methods cannot clearly reconstruct the object that is fast rotating or randomly moving. In this work, we put forward an effective method to image a randomly moving object based on geometric moment analysis. To the best of our knowledge, this is the first work that reconstructs the shape and motion state of the target without prior knowledge of the speed or position. By using the cake-cutting order Hadamard illumination patterns and low-order geometric moment patterns, we obtain a high-quality video stream of the target which moves at high and varying translational and rotational speeds. The efficient method as verified by simulation and experimental results has great potential for practical applications such as Brownian motion microscopy and remote sensing.

Real-time single-pixel imaging using a system on a chip field-programmable gate array

Unlike conventional imaging, the single-pixel imaging technique uses a single-element detector, which enables high sensitivity, broad wavelength, and noise robustness imaging. However, it has several challenges, particularly requiring extensive computations for image reconstruction with high image quality. Therefore, high-performance computers are required for real-time reconstruction with higher image quality. In this study, we developed a compact dedicated computer for single-pixel imaging using a system on a chip field-programmable gate array (FPGA), which enables real-time reconstruction at 40 frames per second with an image size of 128 × 128 pixels. An FPGA circuit was implemented with the proposed reconstruction algorithm to obtain higher image quality by introducing encoding mask pattern optimization. The dedicated computer can accelerate the reconstruction 10 times faster than a recent CPU. Because it is very compact compared with typical computers, it can expand the application of single-pixel imaging to the Internet of Things and outdoor applications.

Deblurring Ghost Imaging Reconstruction Based on Underwater Dataset Generated by Few-Shot Learning

Underwater ghost imaging based on deep learning can effectively reduce the influence of forward scattering and back scattering of water. With the help of data-driven methods, high-quality results can be reconstructed. However, the training of the underwater ghost imaging requires enormous paired underwater datasets, which are difficult to obtain directly. Although the Cycle-GAN method solves the problem to some extent, the blurring degree of the fuzzy class of the paired underwater datasets generated by Cycle-GAN is relatively unitary. To solve this problem, a few-shot underwater image generative network method is proposed. Utilizing the proposed few-shot learning image generative method, the generated paired underwater datasets are better than those obtained by the Cycle-GAN method, especially under the condition of few real underwater datasets. In addition, to reconstruct high-quality results, an underwater deblurring ghost imaging method is proposed. The reconstruction method consists of two parts: reconstruction and deblurring. The experimental and simulation results show that the proposed reconstruction method has better performance in deblurring at a low sampling rate, compared with existing underwater ghost imaging methods based on deep learning. The proposed reconstruction method can effectively increase the clarity degree of the underwater reconstruction target at a low sampling rate and promotes the further applications of underwater ghost imaging.

Multiple-Image Reconstruction of a Fast Periodic Moving/State-Changed Object Based on Compressive Ghost Imaging

We propose a multiple-image reconstruction scheme of a fast periodic moving/state-changed object with a slow bucket detector based on compressive ghost imaging, named MIPO-CSGI. To obtain N frames of an object with fast periodic moving/state-changed, N random speckle patterns are generated in each cycle of the object, which are then used to illuminate the object one by one. The total energy reflected from the object is recorded by a slow bucket detector at each cycle time T. Each group with N random speckle patterns is programmed as one row of a random matrix, and each row of the matrix element corresponds to one measurement of the slow bucket detector. Finally, the compressive sensing algorithm is applied to the constructed matrix and bucket detector signals, resulting in the direct acquisition of multiple images of the object. The feasibility of our method has been demonstrated in both numerical simulations and experiments. Hence, even with a slow bucket detector, MIPO-CSGI can image a fast periodic moving/state-changed object effectively.

Motion Deblurring for Single-Pixel Spatial Frequency Domain Imaging

The single-pixel imaging technique is applied to spatial frequency domain imaging (SFDI) to bring significant performance advantages in band extension and sensitivity enhancement. However, the large number of samplings required can cause severe quality degradations in the measured image when imaging a moving target. This work presents a novel method of motion deblurring for single-pixel SFDI. In this method, the Fourier coefficients of the reflected image are measured by the Fourier single-pixel imaging technique. On this basis, a motion-degradation-model-based compensation, which is derived by the phase-shift and frequency-shift properties of Fourier transform, is adopted to eliminate the effects of target displacements on the measurements. The target displacements required in the method are obtained using a fast motion estimation approach. A series of numerical and experimental validations show that the proposed method can effectively deblur the moving targets and accordingly improves the accuracy of the extracted optical properties, rendering it a potentially powerful way of broadening the clinical application of single-pixel SFDI.

Gigahertz-rate random speckle projection for high-speed single-pixel image classification

Imaging techniques based on single-pixel detection, such as ghost imaging, can reconstruct or recognize a target scene from multiple measurements using a sequence of random mask patterns. However, the processing speed is limited by the low rate of the pattern generation. In this study, we propose an ultrafast method for random speckle pattern generation, which has the potential to overcome the limited processing speed. The proposed approach is based on multimode fiber speckles induced by fast optical phase modulation. We experimentally demonstrate dynamic speckle projection with phase modulation at 10 GHz rates, which is five to six orders of magnitude higher than conventional modulation approaches using spatial light modulators. Moreover, we combine the proposed generation approach with a wavelength-division multiplexing technique and apply it for image classification. As a proof-of-concept demonstration, we show that 28×28-pixel images of digits acquired at GHz rates can be accurately classified using a simple neural network. The proposed approach opens a novel pathway for an all-optical image processor.

Single pixel imaging at high pixel resolutions

The usually reported pixel resolution of single pixel imaging (SPI) varies between 32 × 32 and 256 × 256 pixels falling far below imaging standards with classical methods. Low resolution results from the trade-off between the acceptable compression ratio, the limited DMD modulation frequency, and reasonable reconstruction time, and has not improved significantly during the decade of intensive research on SPI. In this paper we show that image measurement at the full resolution of the DMD, which lasts only a fraction of a second, is possible for sparse images or in a situation when the field of view is limited but is a priori unknown. We propose the sampling and reconstruction strategies that enable us to reconstruct sparse images at the resolution of 1024 × 768 within the time of 0.3s. Non-sparse images are reconstructed with less details. The compression ratio is on the order of 0.4% which corresponds to an acquisition frequency of 7Hz. Sampling is differential, binary, and non-adaptive, and includes information on multiple partitioning of the image which later allows us to determine the actual field of view. Reconstruction is based on the differential Fourier domain regularized inversion (D-FDRI). The proposed SPI framework is an alternative to both adaptive SPI, which is challenging to implement in real time, and to classical compressive sensing image recovery methods, which are very slow at high resolutions.

Secondary Complementary Balancing Compressive Imaging with a Free-Space Balanced Amplified Photodetector

Single-pixel imaging (SPI) has attracted widespread attention because it generally uses a non-pixelated photodetector and a digital micromirror device (DMD) to acquire the object image. Since the modulated patterns seen from two reflection directions of the DMD are naturally complementary, one can apply complementary balanced measurements to greatly improve the measurement signal-to-noise ratio and reconstruction quality. However, the balance between two reflection arms significantly determines the quality of differential measurements. In this work, we propose and demonstrate a simple secondary complementary balancing mechanism to minimize the impact of the imbalance on the imaging system. In our SPI setup, we used a silicon free-space balanced amplified photodetector with 5 mm active diameter which could directly output the difference between two optical input signals in two reflection arms. Both simulation and experimental results have demonstrated that the use of secondary complementary balancing can result in a better cancellation of direct current components of measurements, and can acquire an image quality slightly better than that of single-arm single-pixel complementary measurement scheme (which is free from the trouble of optical imbalance) and over 20 times better than that of double-arm dual-pixel complementary measurement scheme under optical imbalance conditions.

Multi-Object Positioning and Imaging Based on Single-Pixel Imaging Using Binary Patterns

Single-pixel imaging (SPI) is a new type of imaging technology that uses a non-scanning single-pixel detector to image objects and has important application prospects and value in many fields. Most of the modulators currently used in SPI systems are digital micromirror device (DMD) modulators, which use a higher frequency for binary modulation than other alternatives. When modulating grayscale information, the modulation frequency is significantly reduced. This paper conducts research on multiple discrete objects in a scene and proposes using binary patterns to locate and image these objects. Compared with the existing methods of using gray patterns to locate and image multiple objects, the method proposed in this paper is more suitable for DMD-type SPI systems and has wider applicability and greater prospects. The principle of the proposed method is introduced, and the effectiveness of the method is experimentally verified. The experimental results show that, compared to traditional SPI methods, the number of patterns required by the proposed method is reduced by more than 85%.

A Single-Pixel Imaging Scheme with Obstacle Detection

Single-pixel imaging (SPI) utilizes a second-order correlation of structured illumination light field and a single-pixel detector to form images. As the single-pixel detector provides no spatial resolution, a structured illumination light field generated by devices such as a spatial light modulator substitutes the role of array camera to retrieve pixel-wise spatial information. Due to its unique imaging modality, SPI has certain advantages. Meanwhile, its counterintuitive configuration and reciprocity relation to traditional array cameras have been studied to understand its fundamental principle. According to previous studies, the non-spatial detection property makes it possible for SPI to resist scattering in the detection part. In this work, we study the influence of an obstacle aperture in the detection part of SPI. We notice that such an obstacle aperture can restrict the field-of-view (FOV) of SPI, which can be diminished by a scattering process. We investigate these properties with experiment results and analysis under geometry optics. We believe that our study will be helpful in understanding the counterintuitive configuration of SPI and its reciprocity to traditional imaging.

Ghost image restoring using random speckles created by a liquid crystal cell

A liquid crystal cell is used to produce correlated light beams with speckle structures for implementation of pseudo-thermal ghost imaging. The liquid crystal cell makes it possible to provide random spatial intensity distributions, which are characterized by a low coefficient of mutual cross correlations. Ghost imaging of an object representing an amplitude mask is demonstrated. The quality of the reconstructed images was estimated by the method of structural similarity.

Ghost panorama using a convex mirror

Computational ghost imaging or single-pixel imaging enables the image formation of an unknown scene using a lens-free photodetector. In this Letter, we present a computational panoramic ghost imaging system that can achieve a full-color panorama using a single-pixel photodetector, where a convex mirror performs the optical transformation of the engineered Hadamard-based circular illumination pattern from unidirectionally to omnidirectionally. To our best knowledge, it is the first time to propose the concept of ghost panoramas and realize preliminary experimentations. It is foreseeable that ghost panoramas will have more advantages in imaging and detection in many extreme conditions (e.g., scattering/turbulence and unconventional spectra), as well as broad application prospects.

Single-pixel imaging: An overview of different methods to be used for 3D space reconstruction in harsh environments

Different imaging solutions have been proposed over the last few decades, aimed at three-dimensional (3D) space reconstruction and obstacle detection, either based on stereo-vision principles using active pixel sensors operating in the visible part of the spectra or based on active Near Infra-Red (NIR) illumination applying the time-of-flight principle, to mention just a few. If extremely low quantum efficiencies for NIR active illumination yielded by silicon-based detector solutions are considered together with the huge photon noise levels produced by the background illumination accompanied by Rayleigh scattering effects taking place in outdoor applications, the operating limitations of these systems under harsh weather conditions, especially if relatively low-power active illumination is used, are evident. If longer wavelengths for active illumination are applied to overcome these issues, indium gallium arsenide (InGaAs)-based photodetectors become the technology of choice, and for low-cost solutions, using a single InGaAs photodetector or an InGaAs line-sensor becomes a promising choice. In this case, the principles of Single-Pixel Imaging (SPI) and compressive sensing acquire a paramount importance. Thus, in this paper, we review and compare the different SPI developments reported. We cover a variety of SPI system architectures, modulation methods, pattern generation and reconstruction algorithms, embedded system approaches, and 2D/3D image reconstruction methods. In addition, we introduce a Near Infra-Red Single-Pixel Imaging (NIR-SPI) sensor aimed at detecting static and dynamic objects under outdoor conditions for unmanned aerial vehicle applications.

Single-pixel camera based on a spinning mask

Single-pixel imaging (SPI) has been intensively studied in recent years for its capacity to obtain 2D images using a non-pixelated detector. However, the traditional modulation modality using an iteratively refreshed spatial light modulator has significantly restricted its imaging speed, which is a primary barrier to its widespread application. In this work, we propose and demonstrate a new, to the best of our knowledge, SPI scheme using a spinning mask for modulation. An annular binary mask is designed and spun to perform fast spatial modulation, neglecting the iterative modulation modality that limits SPI speed. A multi-spectral SPI system at 100 frames per second is demonstrated, covering a wide range of spectra, from ultraviolet to short-wave infrared light. We believe that this elegant and low-cost scheme will enable SPI to pave its way for practical application.

Differential real-time single-pixel imaging with Fourier domain regularization: applications to VIS-IR imaging and polarization imaging

The speed and quality of single-pixel imaging (SPI) are fundamentally limited by image modulation frequency and by the levels of optical noise and compression noise. In an approach to come close to these limits, we introduce a SPI technique, which is inherently differential, and comprises a novel way of measuring the zeroth spatial frequency of images and makes use of varied thresholding of sampling patterns. With the proposed sampling, the entropy of the detection signal is increased in comparison to standard SPI protocols. Image reconstruction is obtained with a single matrix-vector product so the cost of the reconstruction method scales proportionally with the number of measured samples. A differential operator is included in the reconstruction and following the method is based on finding the generalized inversion of the modified measurement matrix with regularization in the Fourier domain. We demonstrate 256 × 256 SPI at up to 17 Hz at visible and near-infrared wavelength ranges using 2 polarization or spectral channels. A low bit-resolution data acquisition device with alternating-current-coupling can be used in the measurement indicating that the proposed method combines improved noise robustness with a differential removal of the direct current component of the signal.

Retina-like Imaging and Its Applications: A Brief Review

The properties of the human eye retina, including space-variant resolution and gaze characters, provide many advantages for numerous applications that simultaneously require a large field of view, high resolution, and real-time performance. Therefore, retina-like mechanisms and sensors have received considerable attention in recent years. This paper provides a review of state-of-the-art retina-like imaging techniques and applications. First, we introduce the principle and implementing methods, including software and hardware, and describe the comparisons between them. Then, we present typical applications combined with retina-like imaging, including three-dimensional acquisition and reconstruction, target tracking, deep learning, and ghost imaging. Finally, the challenges and outlook are discussed to further study for practical use. The results are beneficial for better understanding retina-like imaging.

Single pixel imaging at megahertz switching rates via cyclic Hadamard masks

Optical imaging is commonly performed with either a camera and wide-field illumination or with a single detector and a scanning collimated beam; unfortunately, these options do not exist at all wavelengths. Single-pixel imaging offers an alternative that can be performed with a single detector and wide-field illumination, potentially enabling imaging applications in which the detection and illumination technologies are immature. However, single-pixel imaging currently suffers from low imaging rates owing to its reliance on configurable spatial light modulators, generally limited to 22 kHz rates. We develop an approach for rapid single-pixel imaging which relies on cyclic patterns coded onto a spinning mask and demonstrate it for in vivo imaging of C. elegans worms. Spatial modulation rates of up to 2.4 MHz, imaging rates of up to 72 fps, and image-reconstruction times of down to 1.5 ms are reported, enabling real-time visualization of dynamic objects.

Bionic Birdlike Imaging Using a Multi-Hyperuniform LED Array

Digital cameras obtain color information of the scene using a chromatic filter, usually a Bayer filter, overlaid on a pixelated detector. However, the periodic arrangement of both the filter array and the detector array introduces frequency aliasing in sampling and color misregistration during demosaicking process which causes degradation of image quality. Inspired by the biological structure of the avian retinas, we developed a chromatic LED array which has a geometric arrangement of multi-hyperuniformity, which exhibits an irregularity on small-length scales but a quasi-uniformity on large scales, to suppress frequency aliasing and color misregistration in full color image retrieval. Experiments were performed with a single-pixel imaging system using the multi-hyperuniform chromatic LED array to provide structured illumination, and 208 fps frame rate was achieved at 32 × 32 pixel resolution. By comparing the experimental results with the images captured with a conventional digital camera, it has been demonstrated that the proposed imaging system forms images with less chromatic moiré patterns and color misregistration artifacts. The concept proposed verified here could provide insights for the design and the manufacturing of future bionic imaging sensors.

Single-pixel imaging of dynamic objects using multi-frame motion estimation

Single-pixel imaging (SPI) enables the visualization of objects with a single detector by using a sequence of spatially modulated illumination patterns. For natural images, the number of illumination patterns may be smaller than the number of pixels when compressed-sensing algorithms are used. Nonetheless, the sequential nature of the SPI measurement requires that the object remains static until the signals from all the required patterns have been collected. In this paper, we present a new approach to SPI that enables imaging scenarios in which the imaged object, or parts thereof, moves within the imaging plane during data acquisition. Our algorithms estimate the motion direction from inter-frame cross-correlations and incorporate it in the reconstruction model. Moreover, when the illumination pattern is cyclic, the motion may be estimated directly from the raw data, further increasing the numerical efficiency of the algorithm. A demonstration of our approach is presented for both numerically simulated and measured data.

Full-color computational ghost imaging using a chromatic LED array and image interpolation

Computational ghost imaging has been an interesting topic for the imaging research community. However, low resolution and quality of image have been a major problem inhibiting the application of computational ghost imaging technique. In this work, we develop a chromatic 64 × 64 LED array which provides high-speed structured illumination up to 2.5 MHz for computational ghost imaging. Importantly, rather than using regular Cartesian arrangement which is commonly used in a digital camera’s detection array, the LED chips on chromatic LED array we propose are arranged in a special way we refer to as basket-weave sampling. The experimental results demonstrate that our proposed arrangement outperforms Cartesian arrangement for storing high-frequency information of colored pictures, with averaged root mean squared error (RMSE) reduced by 4.6%. Meanwhile, considering the physical structure of the LED array, we propose a targeted interpolation algorithm for resulting images obtained from the experiment, and results show that our algorithm has lower averaged RMSE by 2% when compared to bilinear algorithm and by 6.4% when compared to bicubic algorithm.

Deep Compressed Imaging via Optimized-Pattern Scanning

The need for high-speed imaging in applications such as biomedicine, surveillance and consumer electronics has called for new developments of imaging systems. While the industrial effort continuously pushes the advance of silicon focal plane array image sensors, imaging through a single-pixel detector has gained significant interests thanks to the development of computational algorithms. Here, we present a new imaging modality, Deep Compressed Imaging via Optimized-Pattern Scanning (DeCIOPS), which can significantly increase the acquisition speed for a single-detector-based imaging system. We project and scan an illumination pattern across the object and collect the sampling signal with a single-pixel detector. We develop an innovative end-to-end optimized auto-encoder, using a deep neural network and compressed sensing algorithm, to optimize the illumination pattern, which allows us to reconstruct faithfully the image from a small number of samples, and with a high frame rate. Compared with the conventional switching-mask based single-pixel camera and point scanning imaging systems, our method achieves a much higher imaging speed, while retaining a similar imaging quality. We experimentally validated this imaging modality in the settings of both continuous-wave (CW) illumination and pulsed light illumination and showed high-quality image reconstructions with a high compressed sampling rate. This new compressed sensing modality could be widely applied in different imaging systems, enabling new applications which require high imaging speed.

Denoising ghost imaging under a small sampling rate via deep learning for tracking and imaging moving objects

Ghost imaging (GI) usually requires a large number of samplings, which limit the performance especially when dealing with moving objects. We investigated a deep learning method for GI, and the results show that it can enhance the quality of images with the sampling rate even down to 3.7%. With a convolutional denoising auto-encoder network trained with numerical data, blurry images from few samplings can be denoised. Then those outputs are used to reconstruct both the trajectory and clear image of the moving object via cross-correlation based GI, with the number of required samplings reduced by two-thirds.

Improving single pixel imaging performance in high noise condition by under-sampling

Single-pixel imaging could be a superior solution for imaging applications where the detector array is very expensive or not even available. Sampling order, sampling ratio, noise and type of transforms affect the quality of the reconstructed image. Here, we compare the performance of single pixel imaging (SPI) with Hadamard transform (HT) and discrete cosine transform (DCT) in the presence of noise. The trade-off between adding image information and adding noise in each coefficient measurement results in an optimum number of measurements for reconstruction image quality. In addition, DCT shows higher image quality with fewer measurements than HT does. We then demonstrate our SPI with optimum sampling strategy for a large set of images and lab experiments and finally put forward a quality control technique, which is corroborated by the practical experiments. Our results suggest a practical approach for SPI to improve the speed and achieve the highest possible image quality.

Computational ghost imaging with spatiotemporal encoding pseudo-random binary patterns

Computational ghost imaging (CGI) can reconstruct the pixelated image of a target without lenses and image sensors. In almost all spatial CGI systems using various patterns reported in the past, people often only focus on the distribution of patterns in the spatial dimension but ignore the possibility of encoding in the time dimension or even the space-time dimension. Although the random illumination pattern in CGI always brings some inevitable background noise to the recovered image, it has considerable advantages in optical encryption, authentication, and watermarking technologies. In this paper, we focus on stimulating the potential of random lighting patterns in the space-time dimension for embedding large amounts of information. Inspired by binary CGI and second-order correlation operations, we design two novel generation schemes of pseudo-random patterns for information embedding that are suitable for different scenarios. Specifically, we embed a total of 10,000 ghost images (64 × 64 pixels) of the designed Hadamard-matrix-based data container patterns in the framework of CGI, and these ghost images can be quantitatively decoded to two 8-bit standard grayscale images, with a total data volume of 1, 280, 000 bits. Our scheme has good noise resistance and a low symbol error rate. One can design the number of lighting patterns and the information capacity of the design patterns according to the trade-off between accuracy and efficiency. Our scheme, therefore, paves the way for CGI using random lighting patterns to embed large amounts of information and provides new insights into CGI-based encryption, authentication, and watermarking technologies.

Image recovery of ghost imaging with sparse spatial frequencies

When the spatial frequencies of the object are insufficiently sampled, the reconstruction of ghost imaging will suffer from repetitive visual artifacts, which cannot be effectively tackled by existing ghost imaging reconstruction techniques. In this Letter, extensions of the CLEAN algorithm applied in ghost imaging are explored to eliminate those artifacts. Combined with the point spread function estimation using the second-order coherence measurement in ghost imaging, our modified CLEAN algorithm is demonstrated to have a fast and noteworthy improvement against the spatial-frequency insufficiency, even for the extreme sparse sampling cases. A brief explanation of the algorithm and performance analysis are given.

Single-pixel imaging 12 years on: a review

Modern cameras typically use an array of millions of detector pixels to capture images. By contrast, single-pixel cameras use a sequence of mask patterns to filter the scene along with the corresponding measurements of the transmitted intensity which is recorded using a single-pixel detector. This review considers the development of single-pixel cameras from the seminal work of Duarte et al. up to the present state of the art. We cover the variety of hardware configurations, design of mask patterns and the associated reconstruction algorithms, many of which relate to the field of compressed sensing and, more recently, machine learning. Overall, single-pixel cameras lend themselves to imaging at non-visible wavelengths and with precise timing or depth resolution. We discuss the suitability of single-pixel cameras for different application areas, including infrared imaging and 3D situation awareness for autonomous vehicles.

Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection

Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.

Deep learning based projector defocus compensation in single-pixel imaging

Fourier single-pixel imaging (FSI) uses a digital projector to illuminate the target with Fourier basis patterns, and captures the back-scattered light with a photodetector to reconstruct a high-quality target image. Like other single-pixel imaging (SPI) schemes, FSI requires the projector to be focused on the target for best performance. In case the projector lens is defocused, the projected patterns are blurred and their interaction with the target produces a low-quality image. To address this problem, we propose a fast, adaptive, and highly-scalable deep learning (DL) approach for projector defocus compensation in FSI. Specifically, we employ a deep convolutional neural network (DCNN), which learns to offset the effects of projector defocusing through training on a large image set reconstructed with varying defocus parameters. The model is further trained on experimental data to make it robust against system bias. Experimental results demonstrate the efficacy of our method in reconstructing high-quality images at high projector defocusing. Comparative results indicate the superiority of our method over conventional FSI and existing projector defocus rectification method. The proposed work can also be extended to other SPI methods influenced by projector defocusing, and open avenues for applying DL to correct optical anomalies in SPI.

DeepGhost: real-time computational ghost imaging via deep learning

The potential of random pattern based computational ghost imaging (CGI) for real-time applications has been offset by its long image reconstruction time and inefficient reconstruction of complex diverse scenes. To overcome these problems, we propose a fast image reconstruction framework for CGI, called "DeepGhost", using deep convolutional autoencoder network to achieve real-time imaging at very low sampling rates (10-20%). By transferring prior-knowledge from STL-10 dataset to physical-data driven network, the proposed framework can reconstruct complex unseen targets with high accuracy. The experimental results show that the proposed method outperforms existing deep learning and state-of-the-art compressed sensing methods used for ghost imaging under similar conditions. The proposed method employs deep architecture with fast computation, and tackles the shortcomings of existing schemes i.e., inappropriate architecture, training on limited data under controlled settings, and employing shallow network for fast computation.

Ghost polarimetry with unpolarized pseudo-thermal light

We present an experimental implementation of the ghost polarimetry concept in unpolarized light, which allows obtaining complete information on the spatial distribution of polarization properties of objects with linear dichroism. It is theoretically shown that it is possible to restore the spatial distribution of the azimuth and a value of anisotropy of such objects. The developed technique allows us to free up the object arm from all additional optical elements, including polarizers. The experimental results of measuring the dichroism parameters of a test four-sectional sample are presented, which demonstrate the efficiency of the method and confirm the correctness of the developed theoretical model.

Adaptive ghost imaging

Traditional ghost imaging applies correlated algorithms to reconstruct the image of an object. However, it fundamentally requires some spatial distributions of the correlated light beam, e.g. random illumination, which hardly exists in reality. Here, different from the localized analysis used in the traditional ghost imaging, a spatial and temporal global analysis of the whole measurements is proposed. Therefore, we demonstrate a new ghost imaging modality, called adaptive ghost imaging (AGI), that utilizes the difference of successive frames as the correlation pattern to generate the image. As a result, AGI can work with any varying illuminations including, but not limited to, random illumination. We believe that AGI will make the ghost imaging easier, more applicable and closer to reality.

Long Exposure Short Pulse Synchronous Phase Lock Method for Capturing High Dynamic Surface Shape

In infrared weak target detection systems, high-frequency vibrating mirrors (VMs) are a core component. The dynamic surface shape of the VM has a direct impact on imaging quality and the optical modulation effect, so its measurement is necessary but also very difficult. Measurement of the dynamic surface shape of VMs requires a transiently acquired image series, but traditional methods cannot perform this task, as, when the VM is vibrating at a frequency of 3033 Hz, using high-speed cameras to acquire the images would result in frame rates exceeding 1.34 MFPS, which is currently technically impossible. In this paper, we propose the long exposure short pulse synchronous phase lock (LSPL) method, which can capture the dynamic surface shape using a camera working at 10 FPS. In addition, our proposed approach uses a single laser pulse and can achieve the dynamic surface shape measurement on a single frame image.

Image-free classification of fast-moving objects using "learned" structured illumination and single-pixel detection

Object classification generally relies on image acquisition and subsequent analysis. Real-time classification of fast-moving objects is a challenging task. Here we propose an approach for real-time classification of fast-moving objects without image acquisition. The key to the approach is to use structured illumination and single-pixel detection to acquire the object features directly. A convolutional neural network (CNN) is trained to learn the object features. The "learned" object features are then used as structured patterns for structured illumination. Object classification can be achieved by picking up the resulting light signals by a single-pixel detector and feeding the single-pixel measurements to the trained CNN. In our experiments, we show that accurate and real-time classification of fast-moving objects can be achieved. Potential applications of the proposed approach include rapid classification of flowing cells, assembly-line inspection, and aircraft classification in defense applications. Benefiting from the use of a single-pixel detector, the approach might be applicable for hidden moving object classification.

Gallium nitride micro-light-emitting diode structured light sources for multi-modal optical wireless communications systems

Gallium nitride-based light-emitting diodes (LEDs) have revolutionized the lighting industry with their efficient generation of blue and green light. While broad-area (square millimetre) devices have become the dominant LED lighting technology, fabricating LEDs into micro-scale pixels (micro-LEDs) yields further advantages for optical wireless communications (OWC), and for the development of smart-lighting applications such as tracking and imaging. The smaller active areas of micro-LEDs result in high current density operation, providing high modulation bandwidths and increased optical power density. Fabricating micro-LEDs in array formats allows device layouts to be tailored for target applications and provides additional degrees of freedom for OWC systems. Temporal and spatial control is crucial to use the full potential of these micro-scale sources, and is achieved by bonding arrays to pitch-matched complementary metal-oxide-semiconductor control electronics. These compact, integrated chips operate as digital-to-light converters, providing optical signals from digital inputs. Applying the devices as projection systems allows structured light patterns to be used for tracking and self-location, while simultaneously providing space-division multiple access communication links. The high-speed nature of micro-LED array devices, combined with spatial and temporal control, allows many modes of operation for OWC providing complex functionality with chip-scale devices. This article is part of the theme issue 'Optical wireless communication'.

Imaging high-speed moving targets with a single-pixel detector

Single-pixel imaging (SPI) has recently been intensively studied as an alternative to the traditional focal plane array (FPA) technology. However, limited by the refresh rate of spatial light modulators (SLM) and inherent reconstruction mechanism, SPI is inappropriate for high-speed moving targets. To break through this limitation, we propose a novel SPI scheme for high-speed moving targets. In our scenario, the spatial encoding for the target is done by the movement of the target relative to a static pseudo-random illumination pattern. In this process, a series of single-pixel signals are generated that corresponds to the overlap between the target and certain parts of the illumination structure. This correspondence can be utilized for image reconstruction in the same way as normal SPI. In addition, compressive sensing and deep learning algorithms are used for reconstruction, respectively. Reasonable reconstructions can be obtained with a sampling ratio of only 6%. Experimental verification together with theoretical analysis has shown that our scheme is able to image high-speed moving targets that could be alternatively achieved by a fast FPA camera. Our scheme keeps the inherent advantages of SPI and meanwhile extend its application to moving targets. It is believed that this technology will have wide application in many situations.