Temporal phase unwrapping: application to surface profiling of discontinuous objects

Real-time 3D shape measurement of dynamic scenes using fringe projection profilometry: lightweight NAS-optimized dual frequency deep learning approach

Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.

Phase-shifted imaging on multi-directional induction thermography

A novel multi-directional eddy current thermography (ECT) system is presented generating sets of directional phase images that have been fused with a processing pipeline allowing for an improved probability of detection (POD). Inhomogeneous electromagnetic Joule heating derived from the diversion of induced eddy currents provoked by cracks, altering its path around as well as under its bottom, is the principal phenomenon enabling its usage as a non-destructive-evaluation (NDE) technique. Most induction thermography systems employ inductors derived from old designs, optimized for localized heating with a fixed magnetic field direction. This provokes a directional detection blind-spot for surfaces with random crack orientations. In this paper we have observed that the pattern associated with the thermal response distribution can be geometrically correlated to the relative orientation of the magnetic field regarding the crack, conforming to a rotating feature that has not been described before. Extracting the apparent motion as an optical flow, with a phase-shifting interpolation of the intermediate orientations, generates a signal that enables a robust segmentation of a wide variety of defects in ferritic and austenitic alloys. Its performance has been evaluated with two 'Hit/Miss' POD studies TIG welds Inconel 718 and Haynes 282 alloys. Results show an increased detectability regarding the manual labelling of the defects in the same directional set, employing the same input.

Unifying temporal phase unwrapping framework using deep learning

Temporal phase unwrapping (TPU) is significant for recovering an unambiguous phase of discontinuous surfaces or spatially isolated objects in fringe projection profilometry. Generally, temporal phase unwrapping algorithms can be classified into three groups: the multi-frequency (hierarchical) approach, the multi-wavelength (heterodyne) approach, and the number-theoretic approach. For all of them, extra fringe patterns of different spatial frequencies are required for retrieving the absolute phase. Due to the influence of image noise, people have to use many auxiliary patterns for high-accuracy phase unwrapping. Consequently, image noise limits the efficiency and the measurement speed greatly. Further, these three groups of TPU algorithms have their own theories and are usually applied in different ways. In this work, for the first time to our knowledge, we show that a generalized framework using deep learning can be developed to perform the TPU task for different groups of TPU algorithms. Experimental results show that benefiting from the assistance of deep learning the proposed framework can mitigate the impact of noise effectively and enhance the phase unwrapping reliability significantly without increasing the number of auxiliary patterns for different TPU approaches. We believe that the proposed method demonstrates great potential for developing powerful and reliable phase retrieval techniques.

Temporal phase unwrapping based on unequal phase-shifting code

In fringe projection profilometry (FPP) based on temporal phase unwrapping (TPU), reducing the number of projecting patterns has become one of the most important works in recent years. To remove the 2π ambiguity independently, this paper proposes a TPU method based on unequal phase-shifting code. Wrapped phase is still calculated from N-step conventional phase-shifting patterns with equal phase-shifting amount to guarantee the measuring accuracy. Particularly, a series of different phase-shifting amounts relative to the first phase-shifting pattern are set as codewords, and encoded to different periods to generate one coded pattern. When decoding, Fringe order with a large number can be determined from the conventional and coded wrapped phases. In addition, we develop a self-correction method to eliminate the deviation between the edge of fringe order and the 2π discontinuity. Thus, the proposed method can achieve TPU but need to only project one additional coded pattern (e. g. 3+1), which can significantly benefit dynamic 3D shape reconstruction. The theoretical and experimental analysis verify that the proposed method performs high robustness on the reflectivity of the isolated object while ensuring the measuring speed.

Simultaneous phase and amplitude modulation for dual-sensitivity profilometry of discontinuous objects

Fringe projection profilometry (FPP) is a well-known technique for digitizing solids. In FPP, straight fringes are projected over a digitizing solid, and a digital camera grabs the projected fringes. The sensitivity of FPP depends on the spatial frequency of the projected fringes. The projected fringes as seen by the camera are phase modulated by the surface of the digitizing object; the demodulated phase is usually wrapped. If the digitizing object has discontinuities larger than the fringe period, the phase jumps are lost. To preserve large phase discontinuities, one must use very low spatial frequency (low-sensitivity) fringes. The drawback of low-sensitivity FPP is that the demodulated phase has low signal-to-noise ratio (SNR). Much higher SNR is obtained by projecting shorter wavelength, at the cost of obtaining wrapped phase. A way out of this problem is to use dual-wavelength FPP (DW-FPP). In DW-FPP, two sets of projected fringes are used, one with long wavelength and another with shorter wavelength. Due to harmonics and gamma distortion, in DW-FPP, one usually needs four phase-shifted fringes for each sensitivity. Here we are proposing to combine the two sensitivities simultaneously, one coded in phase (PM) and the other coded in amplitude (AM), in order to obtain phase and amplitude modulated (DW-PAM) fringes. The low-sensitivity phase is coded as AM of the DW-PAM fringes. The main advantage of DW-PAM fringes is that one reduces the number of phase-shifted fringes by half: instead of using eight phase-shifted fringes (four for low and four for high sensitivities), one would need only four DW-PAM fringes. Of course, if one wants to increase the harmonic rejection of the recovered phase, one may use a higher order phase-shifting algorithm (PSA).

Multi-scale band-limited illumination profilometry for robust three-dimensional surface imaging at video rate

Dynamic three-dimensional (3D) surface imaging by phase-shifting fringe projection profilometry has been widely implemented in diverse applications. However, existing techniques fall short in simultaneously providing the robustness in solving spatially isolated 3D objects, the tolerance of large variation in surface reflectance, and the flexibility of tunable working distances with meter-square-level fields of view (FOVs) at video rate. In this work, we overcome these limitations by developing multi-scale band-limited illumination profilometry (MS-BLIP). Supported by the synergy of dual-level intensity projection, multi-frequency fringe projection, and an iterative method for distortion compensation, MS-BLIP can accurately discern spatially separated 3D objects with highly varying reflectance. MS-BLIP is demonstrated by dynamic 3D imaging of a translating engineered box and a rotating vase. With an FOV of up to 1.7 m × 1.1 m and a working distance of up to 2.8 m, MS-BLIP is applied to capturing full human-body movements at video rate.

Deep learning in optical metrology: a review

With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional "physics-based" approach, deep-learning-enabled optical metrology is a kind of "data-driven" approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.

Calibration method of multi-projector display system with extra-large FOV and quantitative registration accuracy analysis

The calibration of multi-projector display with extra-large field of view (FOV) and quantitative registration analysis for realizing perfect visual splicing is crucial and difficult. In this paper, we present a novel calibration method to realize the seamless splicing for a multi-projector display system with extra-large FOV. The display consists of 24 projectors, covering the range of 360 degrees in the longitude direction and 210 degrees in the latitude direction. A wide-angle camera fixed on a rotating optical system is used to scan the entire display scene and establish point-to-point correspondence between projector pixels and spatial points using the longitude and latitude information. Local longitude table and latitude table are established on the target of the wide-angle camera. A deterministic method is proposed to locate the North Pole of the display. The local tables corresponding to different camera views can be unified based on the image of the North Pole to form global longitude and latitude tables of arbitrary free-form surface. The mapping between the projector pixels and the camera pixels is established by inverse projection technique, and then each pixel of each projector can be appointed a pair of unique longitude and latitude values. A quantitative registration accuracy analysis method is proposed for multi-projector display system, in which, three-frequency temporal unwrapping method based on coded longitude and latitude values is applied to calculate the registration accuracy. Experiments prove that the registration error of the multi-projector system is less than 0.4 pixels.

Single-shot circular fringe projection for the profiling of objects having surface discontinuities

Fringe projection profilometry (FPP) is a widely used non-contact optical method for 3D profiling of objects. The commonly used linear fringe pattern in FPP has periodic intensity variations along the lateral direction. As a result, the linear fringe pattern used in FPP cannot uniquely represent the lateral shift induced by the objects having surface discontinuities. Thus, unambiguous surface profiling of objects, especially with surface discontinuities, using a single linear fringe image having a single fringe frequency, is unfeasible. This paper proposes using a radially symmetric circular fringe pattern as the structured light pattern for accurate unambiguous surface profiling of sudden height-discontinuous objects. To the best of our knowledge, this is the only method that can reconstruct discontinuous height profiles with the help of a single fringe image having a single frequency. The performance of the proposed algorithm is evaluated on several synthetic and real objects having smooth variations and discontinuities. Compared to the well-known fringe projection methods, the results depict that for a tolerable range of error, the proposed method can be applied for the reconstruction of objects with 4 times higher dynamic range and even at much lower fringe frequencies.

Unsupervised deep learning for 3D reconstruction with dual-frequency fringe projection profilometry

The fringe projection profilometry (FPP) technique has been widely applied in three-dimensional (3D) reconstruction in industry for its high speed and high accuracy. Recently, deep learning has been successfully applied in FPP to achieve high-accuracy and robust 3D reconstructions in an efficient way. However, the network training needs to generate and label numerous ground truth 3D data, which can be time-consuming and labor-intensive. In this paper, we propose to design an unsupervised convolutional neural network (CNN) model based on dual-frequency fringe images to fix the problem. The fringe reprojection model is created to transform the output height map to the corresponding fringe image to realize the unsupervised training of the CNN. Our network takes two fringe images with different frequencies and outputs the corresponding height map. Unlike most of the previous works, our proposed network avoids numerous data annotations and can be trained without ground truth 3D data for unsupervised learning. Experimental results verify that our proposed unsupervised model (1) can get competitive-accuracy reconstruction results compared with previous supervised methods, (2) has excellent anti-noise and generalization performance and (3) saves time for dataset generation and labeling (3.2 hours, one-sixth of the supervised method) and computer space for dataset storage (1.27 GB, one-tenth of the supervised method).

Phase-Shifting Projected Fringe Profilometry Using Binary-Encoded Patterns

A phase unwrapping method for phase-shifting projected fringe profilometry is presented. It did not require additional projections to identify the fringe orders. The pattern used for the phase extraction could be used for phase unwrapping directly. By spatially encoding the fringe patterns that were used to perform the phase-shifting technique with binary contrasts, fringe orders could be discerned. For spatially isolated objects or surfaces with large depth discontinuities, unwrapping could be identified without ambiguity. Even though the surface color or reflectivity varied periodically with position, it distinguished the fringe order very well.

Phase Demodulation Method for Fringe Projection Measurement Based on Improved Variable-Frequency Coded Patterns

The phase-to-height imaging model, as a three-dimensional (3D) measurement technology, has been commonly applied in fringe projection to assist surface profile measurement, where the efficient and accurate calculation of phase plays a critical role in precise imaging. To deal with multiple extra coded patterns and 2π jump error caused to the existing absolute phase demodulation methods, a novel method of phase demodulation is proposed based on dual variable-frequency (VF) coded patterns. In this paper, the frequency of coded fringe is defined as the number of coded fringes within a single sinusoidal fringe period. First, the effective wrapped phase (EWP) as calculated using the four-step phase shifting method was split into the wrapped phase region with complete period and the wrapped phase region without complete period. Second, the fringe orders in wrapped phase region with complete period were decoded according to the frequency of the VF coded fringes and the continuous characteristic of the fringe order. Notably, the sampling frequency of fast Fourier transform (FFT) was determined by the length of the decoding interval and can be adjusted automatically with the variation in height of the object. Third, the fringe orders in wrapped phase region without complete period were decoded depending on the consistency of fringe orders in the connected region of wrapped phase. Last, phase demodulation was performed. The experimental results were obtained to confirm the effectiveness of the proposed method in the phase demodulation of both discontinuous objects and highly abrupt objects.

245 MHz bandwidth organic light-emitting diodes used in a gigabit optical wireless data link

Organic optoelectronic devices combine high-performance, simple fabrication and distinctive form factors. They are widely integrated in smart devices and wearables as flexible, high pixel density organic light emitting diode (OLED) displays, and may be scaled to large area by roll-to-roll printing for lightweight solar power systems. Exceptionally thin and flexible organic devices may enable future integrated bioelectronics and security features. However, as a result of their low charge mobility, these are generally thought to be slow devices with microsecond response times, thereby limiting their full scope of potential applications. By investigating the factors limiting their bandwidth and overcoming them, we demonstrate here exceptionally fast OLEDs with bandwidths in the hundreds of MHz range. This opens up a wide range of potential applications in spectroscopy, communications, sensing and optical ranging. As an illustration of this, we have demonstrated visible light communication using OLEDs with data rates exceeding 1 gigabit per second.

Improved unwrapped phase retrieval method of a fringe projection profilometry system based on fewer phase-coding patterns

In this paper, based on two additional phase-coding patterns, an improved phase demodulation method is proposed. First, six equally spaced coding phases in the interval [$ - \pi $-π, $\pi $π] are embedded in different periods of the coded fringes following a certain sequence. Subsequently, since a group of phase orders can be uniquely determined by the four adjacent coding phases, the phase-order map of the object can be generated. To ensure the accuracy of decoding results, the interference coding numbers should be corrected in advance. In the meantime, the connected regions exhibiting the same orders are classified and then labeled for simplifying the decoding process. The simulation results verify the feasibility of the proposed method. By two groups of 3D imaging experiments, the applicability of this method to multiple objects and discontinuous objects is confirmed.

Absolute Depth Measurement Using Multiphase Normalized Cross-Correlation for Precise Optical Profilometry

In a multifrequency phase-shifting (MFPS) algorithm, the temporal phase unwrapping algorithm can extend the unambiguous phase range by transforming the measurement range from a short fringe pitch into an extended synthetic pitch of two different frequencies. However, this undesirably amplifies the uncertainty of measurement, with each single-frequency phase map retaining its measurement uncertainty, which is carried over to the final unwrapped phase maps in fringe-order calculations. This article analyzes possible causes and proposes a new absolute depth measurement algorithm to minimize the propagation of measurement uncertainty. Developed from normalized cross-correlation (NCC), the proposed algorithm can minimize wrong fringe-order calculations in the MFPS algorithm. The experimental results demonstrated that the proposed measurement method could effectively calibrate the wrong fringe order. Moreover, some extremely low signal-to-noise ratio (SNR) regions of a captured image could be correctly reconstructed (for surface profiles). The present findings confirmed measurement precision at one standard deviation below 5.4 µm, with an absolute distance measurement of 16 mm. The measurement accuracy of the absolute depth could be significantly improved from an unacceptable level of measured errors down to 0.5% of the overall measuring range. Additionally, the proposed algorithm was capable of extracting the absolute phase map in other optical measurement applications, such as distance measurements using interferometry.

High-speed three-dimensional shape measurement based on shifting Gray-code light

The measuring technique combining a phase-shifting algorithm and Gray-code light has been widely used in three-dimensional (3D) shape measurement for static scenes because of its high robustness and anti-noise ability. However, in the high-speed measurement, phase unwrapping errors occur easily on the boundaries of adjacent Gray-code words because of the defocus of the projector, the motion of the objects and the non-uniform reflectivity of the surface. To overcome this challenge, a high-speed 3D shape measurement method based on shifting Gray-code light has been proposed in this paper. Firstly, the average intensity of three captured phase-shifting fringe images are used as a pixel-wise threshold to binarize the Gray codes and to eliminate most phase unwrapping errors caused by the non-uniform reflectivity, ambient light variations, and the defocus of projector. Then, the shifting Gray-code (SGC) coding strategy is proposed to avoid the remaining errors of phase unwrapping on the edge of the code words. In this strategy, no additional patterns are projected, and two sets of decoding words with staggered boundaries are built in the temporal sequences for one wrapped phase. Finally, the simple, efficient, and robust phase unwrapping can be achieved in the high-speed dynamic measurement. This proposed method has been applied to reconstruct 3D shape of randomly collapsing objects in a large depth range, and the experimental results demonstrate that it can reliably obtain high-quality shape and texture information at 310 frames per second.

Multi-Wavelength Digital-Phase-Shifting Moiré Based on Moiré Wavelength

Multi-wavelength digital-phase-shifting moiré was demonstrated using multiple moiré wavelengths determined by system calibration over the full working depth. The method uses the extended noisy phase map as a reference to unwrap the phase map with a shorter wavelength, and thus achieve a less noisy and more accurate continuous phase map. The moiré wavelength calibration determines a moiré-wavelength to height relationship that permits pixelwise refinement of the moiré wavelength and height during 3D reconstruction. Only a single pattern has to be projected and, thus, a single image captured to compute each phase map with a different wavelength to perform digital-phase-shifting moiré temporal phase unwrapping. Only two captured images are required for two-wavelength phase unwrapping and three captured images for three-wavelength phase unwrapping. The method has been demonstrated in the 3D surface-shape measurement of an object with surface discontinuities and spatially isolated objects.

Pixelwise Phase Unwrapping Based on Ordered Periods Phase Shift

The existing phase-shift methods are effective in achieving high-speed, high-precision, high-resolution, real-time shape measurement of moving objects; however, a phase-unwrapping method that can handle the motion of target objects in a real environment and is robust against global illumination as well is yet to be established. Accordingly, a robust and highly accurate method for determining the absolute phase, using a minimum of three steps, is proposed in this study. In this proposed method, an order structure that rearranges the projection pattern for each period of the sine wave is introduced, so that solving the phase unwrapping problem comes down to calculating the pattern order. Using simulation experiments, it has been confirmed that the proposed method can be used in high-speed, high-precision, high-resolution, three-dimensional shape measurements even in situations with high-speed moving objects and presence of global illumination. In this study, an experimental measurement system was configured with a high-speed camera and projector, and real-time measurements were performed with a processing time of 1.05 ms and a throughput of 500 fps.

GOBO projection for 3D measurements at highest frame rates: a performance analysis

Aperiodic sinusoidal patterns that are cast by a GOBO (GOes Before Optics) projector are a powerful tool for optically measuring the surface topography of moving or deforming objects with very high speed and accuracy. We optimised the first experimental setup that we were able to measure inflating car airbags at frame rates of more than 50 kHz while achieving a 3D point standard deviation of ~500 µm. Here, we theoretically investigate the method of GOBO projection of aperiodic sinusoidal fringes. In a simulation-based performance analysis, we examine the parameters that influence the accuracy of the measurement result and identify an optimal pattern design that yields the highest measurement accuracy. We compare the results with those that were obtained via GOBO projection of phase-shifted sinusoidal fringes. Finally, we experimentally verify the theoretical findings. We show that the proposed technique has several advantages over conventional fringe projection techniques, as the easy-to-build and cost-effective GOBO projector can provide a high radiant flux, allows high frame rates, and can be used over a wide spectral range.

Reference-plane-based fast pixel-by-pixel absolute phase retrieval for height measurement

Absolute phase retrieval is essential for height measurement in digital fringe projection. However, projections of additional structured patterns that are normally required for phase unwrapping increase the measurement complexity. In this paper, we propose two reference-plane-based pixel-by-pixel absolute phase retrieval techniques with as few projections as possible, suitable for different object depth ranges. The wrapped phase on the object is absolutely unwrapped by referring just to the absolute phase map on the reference plane. Single-frequency absolute phase retrieval with one-reference-plane-based calibration is first proposed for objects within a height limit that equals a calibrated system constant. To extend the measurement depth range, dual-frequency absolute phase retrieval with two parallel reference planes is further proposed. The additional low frequency is used to choose the unwrapping reference from the two reference plane phases for unwrapping the high-frequency phase. Moreover, the proposed techniques are capable of high-frequency absolute phase unwrapping for objects with step-height surface discontinuities. Experiments have been conducted to demonstrate the efficiency of the proposed two techniques.

Optimal wavelength selection strategy in temporal phase unwrapping with projection distance minimization

Micro Fourier transform profilometry (μFTP) is a recently developed computational framework for high-speed dynamic 3D shape measurement of transient scenes based on fringe projection. It has been demonstrated that by using high-frame-rate fringe projection hardware, μFTP can achieve accurate, denser, unambiguous, and motion-artifact-free 3D reconstruction at a speed up to 10,000 Hz. μFTP utilizes a temporal phase unwrapping algorithm, so-called projection distance minimization (PDM), in which multiple wavelengths are used to solve the phase ambiguity optimally in the maximum-likelihood sense. However, it has been found that the choice of the wavelengths is essential to the unambiguous measurement range as well as the unwrapping reliability in the presence of noise. In this work, the relations between the wavelength combination and the noise resistance ability of PDM are analyzed and investigated in detail by analytical, emulational, and experimental means. This leads to a qualitative conclusion that the noise resistance ability of PDM is fundamentally determined by the value of each item in wavelength ratio: a smaller value of each item in wavelength ratio means better noise resistance ability in phase unwrapping. Our result provides a guideline for optimal wavelengths selection in order to improve the noise resistance ability of a practical fringe projection system. Simulations and experiments based on a microscopic fringe projection system are demonstrated to validate the correctness of our conclusion.

Single-shot 3D motion picture camera with a dense point cloud

We discuss physical and information theoretical limits of optical 3D metrology. Based on these principal considerations we introduce a novel single-shot 3D movie camera that almost reaches these limits. The camera is designed for the 3D acquisition of macroscopic live scenes. Like a hologram, each movie-frame encompasses the full 3D information about the object surface and the observation perspective can be varied while watching the 3D movie. The camera combines single-shot ability with a point cloud density close to the theoretical limit. No space-bandwidth is wasted by pattern codification. With 1-megapixel sensors, the 3D camera delivers nearly 300,000 independent 3D points within each frame. The 3D data display a lateral resolution and a depth precision only limited by physics. The approach is based on multi-line triangulation. The requisite low-cost technology is simple. Only two properly positioned synchronized cameras solve the profound ambiguity problem omnipresent in 3D metrology.

Phase-shifting profilometry combined with Gray-code patterns projection: unwrapping error removal by an adaptive median filter

Phase-shifting profilometry combined with Gray-code patterns projection has been widely used for 3D measurement. In this technique, a phase-shifting algorithm is used to calculate the wrapped phase, and a set of Gray-code binary patterns is used to determine the unwrapped phase. In the real measurement, the captured Gray-code patterns are no longer binary, resulting in phase unwrapping errors at a large number of erroneous pixels. Although this problem has been attended and well resolved by a few methods, it remains challenging when a measured object has step-heights and the captured patterns contain invalid pixels. To effectively remove unwrapping errors and simultaneously preserve step-heights, in this paper, an effective method using an adaptive median filter is proposed. Both simulations and experiments can demonstrate its effectiveness.

Multi-subzone algorithm for absolute phase retrieval in digital fringe projection profilometry

Codewords are important in encoded absolute phase retrieval techniques such as two-frequency, gray-code, and phase-coding. Each sinusoidal fringe is marked by a unique codeword so that an absolute fringe order can be determined by decoding the codeword. However, due to the limited number of unique codewords, sinusoidal fringe patterns do not contain high-frequency fringes without the use of additional patterns. A multi-subzone coding and decoding algorithm is thus proposed to overcome this limitation. Three multi-subzone coding methods based on two-frequency, gray-code, and phase-coding techniques are presented. The coding creates multiple subzones of unique codewords and the decoding enables it to use non-unique codewords to identify absolute fringe order. Specifically, the range of fringe order is estimated by the use of a wrapped phase map and the absolute fringe order is identified by a codeword. Experimental studies demonstrate the advantages of the proposed algorithm over existing coding methods. The proposed algorithm is suitable to measure objects with large step-height surface discontinuities.

Spatiotemporal three-dimensional phase unwrapping in digital speckle pattern interferometry

We propose a hybrid spatiotemporal three-dimensional phase unwrapping algorithm for use in digital speckle pattern interferometry (DSPI). The feature of the proposed algorithm is the integration of one-dimensional temporal and two-dimensional spatial phase unwrapping algorithms. By demodulating the phase on a single reference point or multiple reference points using temporal phase unwrapping and on each separated phase map region using spatial phase unwrapping, the DSPI with the spatiotemporal three-dimensional phase unwrapping algorithm can realize the measurement of dynamic absolute displacements and the determination of abrupt phase changes which are usually caused by object discontinuities. We demonstrate that the presented algorithm can overcome the drawbacks of the traditional spatial and temporal phase unwrapping algorithms.

Fast temporal phase unwrapping method for the fringe reflection technique based on the orthogonal grid fringes

In traditional temporal phase unwrapping (TPU) algorithms, wrapped phases with different spatial frequencies are obtained from several groups of phase shift fringes to calculate the unwrapped phase. Therefore, the necessary quantity of captured fringes is very large, especially for the fringe reflection technique (FRT), since a pair of phases should be unwrapped to get the slopes of two perpendicular directions. In this paper, we propose a fast TPU algorithm based on the orthogonal grid fringes by which only one image is needed to extract the two integer phases for each frequency instead of two groups of phase shift fringes, and then they can be added into the wrapped phases separately to complete the unwrapping. There are ridge errors in the direct unwrapped phases, but they are significantly suppressed by our pseudo-phase-shift strategy without any extra captured fringes. The proposed method is robust and effective where the fringe amount used for unwrapping is only 1/4 of the previous similar algorithm and 1/6-1/8 of the traditional TPU methods. The detailed comparison of measurement time is also given, which demonstrate that the FRT measurement can be accelerated in most cases by our method. The algorithm is validated by the experiments, which still works well for the severely defocusing fringes or complex specimen.

Temporal phase-unwrapping of static surfaces with 2-sensitivity fringe-patterns

Here we describe a 2-step temporal phase unwrapping formula that uses 2-sensitivity demodulated phases for measuring static surfaces. The first phase demodulation has at most 1-wavelength sensitivity and the second one is G-times (G>>1.0) more sensitive. Measuring static surfaces with 2-sensitivity fringe patterns is well known and recent published methods combine 2-sensitivities measurements mostly by triangulation. Two important applications for our 2-step unwrapping algorithm is profilometry and synthetic aperture radar (SAR) interferometry. In these two applications the object or surface being analyzed is static and highly discontinuous; so temporal unwrapping is the best strategy to follow. Phase-demodulation in profilometry and SAR interferometry is very similar because both share similar mathematical models.

Theory and algorithms of an efficient fringe analysis technology for automatic measurement applications

Some advances in fringe analysis technology for phase computing are presented. A full scheme for phase evaluation, applicable to automatic applications, is proposed. The proposal consists of: a fringe-pattern normalization method, Fourier fringe-normalized analysis, generalized phase-shifting processing for inhomogeneous nonlinear phase shifts and spatiotemporal visibility, and a phase-unwrapping method by a rounding-least-squares approach. The theoretical principles of each algorithm are given. Numerical examples and an experimental evaluation are presented.

Valid point detection in fringe projection profilometry

Fringe projection profilometry has become one of the most popular 3D information acquisition techniques being developed over the past three decades. However, the general and practical issues on valid point detection, including object segmentation, error correction and noisy point removal, have not been studied thoroughly. Furthermore, existing valid point detection techniques require multiple case-dependent thresholds which increase processing inconvenience. In this paper, we proposed a new valid point detection framework, which includes the k-means clustering for automatic background segmentation, unwrapping error correction based on theoretical analysis, and noisy point detection in both temporal and spatial directions with automatic threshold setting. Experimental results are given to validate the proposed framework.

Three-dimensional trace measurements for fast-moving objects using binary-encoded fringe projection techniques

A fringe projection technique to trace the shape of a fast-moving object is proposed. A binary-encoded fringe pattern is illuminated by a strobe lamp and then projected onto the moving object at a sequence of time. Phases of the projected fringes obtained from the sequent measurements are extracted by the Fourier transform method. Unwrapping is then performed with reference to the binary-encoded fringe pattern. Even though the inspected object is colorful, fringe orders can be identified. A stream of profiles is therefore retrieved from the sequent unwrapped phases. This makes it possible to analyze physical properties of the dynamic objects. Advantages of the binary-encoded fringe pattern for phase unwrapping also include (1) reliable performance for colorful objects, spatially isolated objects, and surfaces with large depth discontinuities; (2) unwrapped errors only confined in a local area; and (3) low computation cost.

Recovery of absolute height from wrapped phase maps for fringe projection profilometry

A novel multi-frequency fringe projection profilometry is presented in this paper. Fringe patterns with multiple frequencies are projected onto an object by a digital micro-mirror device projector. The approach involves an improved Fourier transform profilometry method with an additional π phase shifting stage and hence the acquisition of two source images. A peak searching algorithm is then employed to obtain the real height profile of the object together with a mathematical proof of this algorithm. In our method, the height of each point on the object is measured independently and the phase unwrapping procedure is avoided, enabling the measurement of objects with large depth discontinuities, where the phase unwrapping is difficult. The measurement result is given to validate the method in the paper. Our technique has great potential in industrial applications where the measurement of objects with complex shape and large discontinuities is needed.

Absolute phase map recovery of two fringe patterns with flexible selection of fringe wavelengths

A novel approach is proposed to unwrap the phase maps of two fringe patterns in fringe pattern projection-based profilometry. In contrast to existing techniques, where spatial frequencies (i.e., the number of fringes on a pattern) of the two fringe patterns must be integers and coprime, the proposed method is applicable for any two fringe patterns with different fringe wavelengths (i.e., the number of pixels in a fringe) and thus provides more flexibility in the use of fringe patterns. Moreover, compared to the existing techniques, the proposed method is simpler in its implementation and has better antierror capability. Theoretical analysis and experiment results are presented to confirm the effectiveness of the proposed method.

Binary pattern deflectometry

Deflectometry is widely used to accurately calculate the slopes of any specular reflective surface, ranging from car bodies to nanometer-level mirrors. This paper presents a new deflectometry technique using binary patterns of increasing frequency to retrieve the surface slopes. Binary Pattern Deflectometry allows almost instant, simple, and accurate slope retrieval, which is required for applications using mobile devices. The paper details the theory of this deflectometry method and the challenges of its implementation. Furthermore, the binary pattern method can also be combined with a classic phase-shifting method to eliminate the need of a complex unwrapping algorithm and retrieve the absolute phase, especially in cases like segmented optics, where spatial algorithms have difficulties. Finally, whether it is used as a stand-alone or combined with phase-shifting, the binary patterns can, within seconds, calculate the slopes of any specular reflective surface.

Common-path spectral interferometry with temporal carrier for highly sensitive surface plasmon resonance sensing

Incorporating the temporal carrier technique with common-path spectral interferometry, we have successfully demonstrated an advanced surface plasmon resonance (SPR) biosensing system which achieves refractive index resolution (RIR) up to 2 × 10(-8) refractive index unit (RIU) over a wide dynamic range of 3 × 10(-2) RIU. While this is accomplished by optimizing the SPR differential phase sensing conditions with just a layer of gold, we managed to address the spectral phase discontinuity with a novel spectral-temporal phase measurement scheme. As the new optical setup supersedes its Michelson counterpart in term of simplicity, we believe that it is a significant contribution for practical SPR sensing applications.

Simultaneous multiple view high resolution surface geometry acquisition using structured light and mirrors

Knowledge of the surface geometry of an imaging subject is important in many applications. This information can be obtained via a number of different techniques, including time of flight imaging, photogrammetry, and fringe projection profilometry. Existing systems may have restrictions on instrument geometry, require expensive optics, or require moving parts in order to image the full surface of the subject. An inexpensive generalised fringe projection profilometry system is proposed that can account for arbitrarily placed components and use mirrors to expand the field of view. It simultaneously acquires multiple views of an imaging subject, producing a cloud of points that lie on its surface, which can then be processed to form a three dimensional model. A prototype of this system was integrated into an existing Diffuse Optical Tomography and Bioluminescence Tomography small animal imaging system and used to image objects including a mouse-shaped plastic phantom, a mouse cadaver, and a coin. A surface mesh generated from surface capture data of the mouse-shaped plastic phantom was compared with ideal surface points provided by the phantom manufacturer, and 50% of points were found to lie within 0.1mm of the surface mesh, 82% of points were found to lie within 0.2mm of the surface mesh, and 96% of points were found to lie within 0.4mm of the surface mesh.

Flexible error-reduction method for shape measurement by temporal phase unwrapping: phase averaging method

Temporal phase unwrapping is an important method for shape measurement in structured light projection. Its measurement errors mainly come from both the camera noise and nonlinearity. Analysis found that least-squares fitting cannot completely eliminate nonlinear errors, though it can significantly reduce the random errors. To further reduce the measurement errors of current temporal phase unwrapping algorithms, in this paper, we proposed a phase averaging method (PAM) in which an additional fringe sequence at the highest fringe density is employed in the process of data processing and the phase offset of each set of the four frames is carefully chosen according to the period of the phase nonlinear errors, based on fast classical temporal phase unwrapping algorithms. This method can decrease both the random errors and the systematic errors with statistical averaging. In addition, the length of the additional fringe sequence can be changed flexibly according to the precision of the measurement. Theoretical analysis and simulation experiment results showed the validity of the proposed method.

Frequency selection in absolute phase maps recovery with two frequency projection fringes

In a recent published work we proposed a technique to recover the absolute phase maps of two fringe patterns with different spatial frequencies. It is demonstrated that a number of selected frequency pairs can be used for the proposed approach, but the published work did not provide a guideline for frequency selection. In addition, the performance of the proposed technique in terms of its anti-noise capability is not addressed. In this paper, the rules for selecting the two frequencies are presented based on theoretical analysis of the proposed technique. Also, when the two frequencies are given, the anti-noise capability of technique is formulated and evaluated. These theoretical conclusions are verified by experimental results.

Integration of robust filters and phase unwrapping algorithms for image reconstruction of objects containing height discontinuities

For 3D objects with height discontinuities, the image reconstruction performance of interferometric systems is adversely affected by the presence of noise in the wrapped phase map. Various schemes have been proposed for detecting residual noise, speckle noise and noise at the lateral surfaces of the discontinuities. However, in most schemes, some noisy pixels are missed and noise detection errors occur. Accordingly, this paper proposes two robust filters (designated as Filters A and B, respectively) for improving the performance of the phase unwrapping process for objects with height discontinuities. Filter A comprises a noise and phase jump detection scheme and an adaptive median filter, while Filter B replaces the detected noise with the median phase value of an N × N mask centered on the noisy pixel. Filter A enables most of the noise and detection errors in the wrapped phase map to be removed. Filter B then detects and corrects any remaining noise or detection errors during the phase unwrapping process. Three reconstruction paths are proposed, Path I, Path II and Path III. Path I combines the path-dependent MACY algorithm with Filters A and B, while Paths II and III combine the path-independent cellular automata (CA) algorithm with Filters A and B. In Path II, the CA algorithm operates on the whole wrapped phase map, while in Path III, the CA algorithm operates on multiple sub-maps of the wrapped phase map. The simulation and experimental results confirm that the three reconstruction paths provide a robust and precise reconstruction performance given appropriate values of the parameters used in the detection scheme and filters, respectively. However, the CA algorithm used in Paths II and III is relatively inefficient in identifying the most suitable unwrapping paths. Thus, of the three paths, Path I yields the lowest runtime.

Flexible structured-light-based three-dimensional profile reconstruction method considering lens projection-imaging distortion

Structured-light profilometry is a powerful tool to reconstruct the three-dimensional (3D) profile of an object. Accurate profile acquisition is often hindered by not only the nonlinear response (i.e., gamma effect) of electronic devices but also the projection-imaging distortion of lens used in the system. In this paper, a flexible 3D profile reconstruction method based on a nonlinear iterative optimization is proposed to correct the errors caused by the lens distortion. It can be easily extended to measurements for which a more complex projection-imaging distortion model is required. Experimental work shows that the root-mean-square (RMS) error is reduced by eight times and highly accurate results with errors of less than 1‰ can be achieved by the proposed method.

Three-dimensional shape measurement of aspheric mirrors with fringe reflection photogrammetry

Three-dimensional (3D) shape measurement of an aspheric mirror with fringe reflection photogrammetry involves three steps: correspondence matching, triangulation, and bundle adjustment. Correspondence matching is realized by absolute phase tracking and triangulation is computed by the intersection of reflection and incidence rays. The main contribution in this paper is constraint bundle adjustment for carefully dealing with lens distortion in the process of ray intersection, as compared to the well-known grating reflection photogrammetry. Additionally, a free frame is proposed to alleviate troublesome system geometrical calibration, and constraint bundle adjustment is operated in the free frame to refine the 3D shape. Simulation and experiment demonstrate that constraint bundle adjustment can improve absolute measurement accuracy of aspheric mirrors.

Period coded phase shifting strategy for real-time 3-D structured light illumination

Phase shifting structured light illumination for range sensing involves projecting a set of grating patterns where accuracy is determined, in part, by the number of stripes. However, high pattern frequencies introduce ambiguities during phase unwrapping. This paper proposes a process for embedding a period cue into the projected pattern set without reducing the signal-to-noise ratio. As a result, each period of the high frequency signal can be identified. The proposed method can unwrap high frequency phase and achieve high measurement precision without increasing the pattern number. Therefore, the proposed method can significantly benefit real-time applications. The method is verified by theoretical and experimental analysis using prototype system built to achieve 120 fps at 640 × 480 resolution.

A novel encoded-phase technique for phase measuring profilometry

Three-dimensional (3-D) shape measurement using a novel encoded-phase grating is proposed. The projected sinusoidal fringe patterns are designed with wrapped and encoded phase instead of monotonic and unwrapped phase. Phase values of the projected fringes on the surface are evaluated by phase-shift technique. The absolute phase is then restored with reference to the encoded information, which is extracted from the differential of the wrapped phase. To solve the phase errors at some phase-jump areas, Hilbert transform is employed. By embedding the encoded information in the wrapped phase, there is no extra pattern that needs to be projected. The experimental results identify its feasibility and show the possibility to measure the spatially isolated objects. It will be promising to analyze dynamic objects.

Recovering the absolute phase maps of two fringe patterns with selected frequencies

Phase unwrapping is an important and challenging issue in fringe pattern profilometry. In this Letter we propose an approach to recover absolute phase maps of two fringe patterns with selected frequencies. Compared to existing temporal multiple frequency algorithms, the two frequencies in our proposed algorithm can be high enough and thus enable efficient and accurate recovery of absolute phase maps. Experiment results are presented to confirm the effectiveness of the proposed technique.

High-speed measurement of three-dimensional surface profiles up to 10 μm using two-wavelength phase-shifting interferometry utilizing an injection locking technique

High-speed two-wavelength phase-shifting interferometry is presented. The technique is aimed at high-speed in-line inspection of spacers in liquid crystal display panels or wafer bumps where the measuring range is well determined and high-speed measurements are essential. With our test setup, the measuring range is extended to 10 μm by using two injection locked frequency scanning lasers that offer fast and equidistant phase shifting of interference fringes. A technique to determine the unwrapped phase map in a frequency scanning phase-shifting interferometry without the ordinary phase-unwrapping process is proposed.

Robust detection scheme on noise and phase jump for phase maps of objects with height discontinuities--theory and experiment

This paper proposes a robust noise and phase jump detection scheme for noisy phase maps containing height discontinuities. The detection scheme has two primary functions, namely to detect the positions of noise and to locate the positions of the phase jumps. Generally speaking, the removal of noise from a wrapped phase map causes a smearing of the phase jumps and therefore leads to a loss of definition in the unwrapped phase map. However, in the proposed scheme, the boundaries of the phase jump regions are preserved during the noise detection process. The validity of the proposed approach is demonstrated using the simulated and experimental wrapped phase maps of a 3D object containing height discontinuities, respectively. It is shown that the noise and phase jump detection scheme enables the precise and efficient detection of three different types of noise, namely speckle noise, residual noise, and noise at the lateral surfaces of the height discontinuities. Therefore, the proposed scheme represents an ideal solution for the pre-processing of noisy wrapped phase maps prior to their treatment using a filtering algorithm and phase unwrapping algorithm.

Optical deflection tomography with the phase-shifting schlieren

We present a new optical deflection tomography method that takes advantage of the phase-shifting schlieren. The reconstruction algorithm is based on filtered backprojection. The instrument is well adapted for three-dimensional imaging of spatially sparse objects exhibiting large refractive index variations. It achieves a 35 μm resolution with a 3 mm depth of field. Its performance is illustrated with a bundle of fibers immersed in a matching index solution.

High-speed and dense three-dimensional surface acquisition using defocused binary patterns for spatially isolated objects

The three-dimensional (3-D) shape measurement using defocused Ronchi grating is advantageous for the high contrast of fringe. This paper presents a method for measuring spatially isolated objects using defocused binary patterns. Two Ronchi grating with horizontal position difference of one-third of a period and an encoded pattern are adopted. The phase distribution of fringe pattern is obtained by Fourier analysis method. The measurement depth and range is enlarged because the third harmonic component and background illumination is eliminated with proposed method. The fringe order is identified by the encoded pattern. Three gray levels are used and the pattern is converted to binary image with error diffusion algorithm. The tolerance of encoded pattern is large. It is suited for defocused optical system. We also present a measurement system with a modified DLP projector and a high-speed camera. The 3-D surface acquisition speed of 60 frames per second (fps), with resolution of 640 × 480 points and that of 120 fps, with resolution of 320 × 240 points are archived. If the control logic of DMD was modified and a camera with higher speed was employed, the measurement speed would reach thousands fps. This makes it possible to analyze dynamic objects.

Analysis and identification of phase error in phase measuring profilometry

Both the analysis of phase errors which occur at the abrupt discontinuities in phase measuring profilometry (PMP) and the identification method are presented in this paper. The sampling effect of CCD will cause a dilution of accuracy in PMP, especially at abrupt discontinuities on the object surface. The existing methods cannot efficiently identify the abrupt discontinuities. We analyze the relationship between the phase, the height and the equivalent wavelength. By viewing the phase as the argument of a vector we find out that CCD sampling introduces errors into the measurement and the phase is nonlinear to the equivalent wavelength at the abrupt discontinuities. Therefore temporal phase unwrapping (TPU) is introduced into the measurement to identify the abrupt discontinuities. Computer simulations and practical experiment validate the feasibility of this method.