Roadmap on Recent Progress in FINCH Technology

Roadmap on computational methods in optical imaging and holography [invited]

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

Extending the Depth of Focus of an Infrared Microscope Using a Binary Axicon Fabricated on Barium Fluoride

Axial resolution is one of the most important characteristics of a microscope. In all microscopes, a high axial resolution is desired in order to discriminate information efficiently along the longitudinal direction. However, when studying thick samples that do not contain laterally overlapping information, a low axial resolution is desirable, as information from multiple planes can be recorded simultaneously from a single camera shot instead of plane-by-plane mechanical refocusing. In this study, we increased the focal depth of an infrared microscope non-invasively by introducing a binary axicon fabricated on a barium fluoride substrate close to the sample. Preliminary results of imaging the thick and sparse silk fibers showed an improved focal depth with a slight decrease in lateral resolution and an increase in background noise.

Incoherent digital holography with two polarization-sensitive phase-only spatial light modulators and reduced number of exposures

I propose methods for reducing the number of exposures in incoherent digital holography with two polarization-sensitive phase-only spatial light modulators (IDH with TPP-SLMs). In IDH with TPP-SLMs, no polarization filters are required, and not only three-dimensional (3D), but polarization information is also obtained. However, seven exposures are required to conduct filter-free polarimetric incoherent holography. In this article, the optical designs and modified phase-shifting interferometry to reduce the number of recordings are described. IDH with TPP-SLMs has the potential for filter-free single-shot multidimensional incoherent holographic imaging.

Optimizing self-interference digital holography for single-molecule localization

Self-interference digital holography (SIDH) can image incoherently emitting objects over large axial ranges from three two-dimensional images. By combining SIDH with single-molecule localization microscopy (SMLM), incoherently emitting objects can be localized with nanometer precision over a wide axial range without mechanical refocusing. However, background light substantially degrades the performance of SIDH due to the relatively large size of the hologram. To optimize the performance of SIDH, we performed simulations to study the optimal hologram radius (Rh) for different levels of background photons. The results show that by reducing the size of the hologram, we can achieve a localization precision of better than 60 nm laterally and 80 nm axially over a 10 µm axial range under the conditions of low signal level (6000 photons) with 10 photons/pixel of background noise. We then performed experiments to demonstrate our optimized SIDH system. The results show that point sources emitting as few as 2120 photons can be successfully detected. We further demonstrated that we can successfully reconstruct point-like sources emitting 4200 photons over a 10 µm axial range by light-sheet SIDH.

Polarization-filterless polarization-sensitive polarization-multiplexed phase-shifting incoherent digital holography (P4IDH)

I propose a holography technique that detects both three-dimensional (3D) and polarization information on incoherent light without any polarization filters. Two polarization-sensitive phase-only spatial light modulators are used to generate self-interference holograms with different polarization directions. 3D images for different polarization directions are retrieved from the recorded polarization-multiplexed holograms, exploiting the developed phase-shifting interferometry and numerical refocusing. Moreover, light-use efficiency is improved in general incoherent holography because there is no polarization filter. Its validity is experimentally demonstrated.

Live Cell Light Sheet Imaging with Low- and High-Spatial-Coherence Detection Approaches Reveals Spatiotemporal Aspects of Neuronal Signaling

Light sheet microscopy in live cells requires minimal excitation intensity and resolves three-dimensional (3D) information rapidly. Lattice light sheet microscopy (LLSM) works similarly but uses a lattice configuration of Bessel beams to generate a flatter, diffraction-limited z-axis sheet suitable for investigating subcellular compartments, with better tissue penetration. We developed a LLSM method for investigating cellular properties of tissue in situ. Neural structures provide an important target. Neurons are complex 3D structures, and signaling between cells and subcellular structures requires high resolution imaging. We developed an LLSM configuration based on the Janelia Research Campus design or in situ recording that allows simultaneous electrophysiological recording. We give examples of using LLSM to assess synaptic function in situ. In presynapses, evoked Ca²⁺ entry causes vesicle fusion and neurotransmitter release. We demonstrate the use of LLSM to measure stimulus-evoked localized presynaptic Ca²⁺ entry and track synaptic vesicle recycling. We also demonstrate the resolution of postsynaptic Ca²⁺ signaling in single synapses. A challenge in 3D imaging is the need to move the emission objective to maintain focus. We have developed an incoherent holographic lattice light-sheet (IHLLS) technique to replace the LLS tube lens with a dual diffractive lens to obtain 3D images of spatially incoherent light diffracted from an object as incoherent holograms. The 3D structure is reproduced within the scanned volume without moving the emission objective. This eliminates mechanical artifacts and improves temporal resolution. We focus on LLS and IHLLS applications and data obtained in neuroscience and emphasize increases in temporal and spatial resolution using these approaches.

Deep learning-based incoherent holographic camera enabling acquisition of real-world holograms for holographic streaming system

While recent research has shown that holographic displays can represent photorealistic 3D holograms in real time, the difficulty in acquiring high-quality real-world holograms has limited the realization of holographic streaming systems. Incoherent holographic cameras, which record holograms under daylight conditions, are suitable candidates for real-world acquisition, as they prevent the safety issues associated with the use of lasers; however, these cameras are hindered by severe noise due to the optical imperfections of such systems. In this work, we develop a deep learning-based incoherent holographic camera system that can deliver visually enhanced holograms in real time. A neural network filters the noise in the captured holograms, maintaining a complex-valued hologram format throughout the whole process. Enabled by the computational efficiency of the proposed filtering strategy, we demonstrate a holographic streaming system integrating a holographic camera and holographic display, with the aim of developing the ultimate holographic ecosystem of the future.

Enhanced design of multiplexed coded masks for Fresnel incoherent correlation holography

Fresnel incoherent correlation holography (FINCH) is a well-established incoherent digital holography technique. In FINCH, light from an object point splits into two, differently modulated using two diffractive lenses with different focal distances and interfered to form a self-interference hologram. The hologram numerically back propagates to reconstruct the image of the object at different depths. FINCH, in the inline configuration, requires at least three camera shots with different phase shifts between the two interfering beams followed by superposition to obtain a complex hologram that can be used to reconstruct an object's image without the twin image and bias terms. In general, FINCH is implemented using an active device, such as a spatial light modulator, to display the diffractive lenses. The first version of FINCH used a phase mask generated by random multiplexing of two diffractive lenses, which resulted in high reconstruction noise. Therefore, a polarization multiplexing method was later developed to suppress the reconstruction noise at the expense of some power loss. In this study, a novel computational algorithm based on the Gerchberg-Saxton algorithm (GSA) called transport of amplitude into phase (TAP-GSA) was developed for FINCH to design multiplexed phase masks with high light throughput and low reconstruction noise. The simulation and optical experiments demonstrate a power efficiency improvement of ~ 150 and ~ 200% in the new method in comparison to random multiplexing and polarization multiplexing, respectively. The SNR of the proposed method is better than that of random multiplexing in all tested cases but lower than that of the polarization multiplexing method.

3D single shot lensless incoherent optical imaging using coded phase aperture system with point response of scattered airy beams

Interferenceless coded aperture correlation holography (I-COACH) techniques have revolutionized the field of incoherent imaging, offering multidimensional imaging capabilities with a high temporal resolution in a simple optical configuration and at a low cost. The I-COACH method uses phase modulators (PMs) between the object and the image sensor, which encode the 3D location information of a point into a unique spatial intensity distribution. The system usually requires a one-time calibration procedure in which the point spread functions (PSFs) at different depths and/or wavelengths are recorded. When an object is recorded under identical conditions as the PSF, the multidimensional image of the object is reconstructed by processing the object intensity with the PSFs. In the previous versions of I-COACH, the PM mapped every object point to a scattered intensity distribution or random dot array pattern. The scattered intensity distribution results in a low SNR compared to a direct imaging system due to optical power dilution. Due to the limited focal depth, the dot pattern reduces the imaging resolution beyond the depth of focus if further multiplexing of phase masks is not performed. In this study, I-COACH has been realized using a PM that maps every object point into a sparse random array of Airy beams. Airy beams during propagation exhibit a relatively high focal depth with sharp intensity maxima that shift laterally following a curved path in 3D space. Therefore, sparse, randomly distributed diverse Airy beams exhibit random shifts with respect to one another during propagation, generating unique intensity distributions at different distances while retaining optical power concentrations in small areas on the detector. The phase-only mask displayed on the modulator was designed by random phase multiplexing of Airy beam generators. The simulation and experimental results obtained for the proposed method are significantly better in SNR than in the previous versions of I-COACH.

Incoherent color holography lattice light-sheet for subcellular imaging of dynamic structures

The purpose of the article is to explore the need and advantages of using the incoherent color holography lattice light-sheet (ICHLLS) to provide multiwavelength quantitative monitoring of 3D cellular dynamics in live tissue to further understand complex functions of cells and cellular compartments. We have explored the use of incoherent color holography lattice light-sheet to investigate colocalization of fluorescent markers in live cells in intact tissue. Neuronal structures provide an attractive target for incoherent color holography lattice light-sheet. The cells show a complex architecture in 3D space in which signaling both between cells and within subcellular structures requires colocalization of proteins and lipids to function. During activity and over long periods it is important in understanding these signaling functions in Parkinson's, Alzheimer's and motoneuron diseases within live cells in intact tissue. As a proof of concept this article recalls the key aspects in lattice light-sheet imaging and provides a description of the incoherent detection system configuration to actively control dual diffractive lenses phase-shifting at multiple excitation wavelengths sequentially, and per each z-galvo scanning level, with extended field-of-view. The incoherent color holography lattice light-sheet system will allow simultaneous recording of multidimensional object waves that contain intensity in 3D space, phase, and wavelength information. We measure colocalization of fluorescence indicators introduced into live cells in intact neural tissue.

Grating-based in-line geometric-phase-shifting incoherent digital holographic system toward 3D videography

Incoherent digital holography (IDH) with a sequential phase-shifting method enables high-definition 3D imaging under incoherent lights. However, sequential recording of multiple holograms renders IDH impractical for 3D videography. In this study, we propose grating-based in-line geometric-phase-shifting IDH. Our method divides orthogonal circularly polarized lights into four copies with a fabricated phase grating and subsequently creates self-interference holograms with geometric phases introduced by a segmented linear polarizer. This enables single-shot recording of holograms without the need for a specially designed image sensor, such as a polarization-sensitive sensor. Moreover, the achievable spatial resolution is higher than that of off-axis methods. As a proof-of-principle experiment, we demonstrated snapshot and video recording of 3D reflective objects using our IDH method. The results confirmed the feasibility of the proposed method.

Nonlinear Reconstruction of Images from Patterns Generated by Deterministic or Random Optical Masks-Concepts and Review of Research

Indirect-imaging methods involve at least two steps, namely optical recording and computational reconstruction. The optical-recording process uses an optical modulator that transforms the light from the object into a typical intensity distribution. This distribution is numerically processed to reconstruct the object's image corresponding to different spatial and spectral dimensions. There have been numerous optical-modulation functions and reconstruction methods developed in the past few years for different applications. In most cases, a compatible pair of the optical-modulation function and reconstruction method gives optimal performance. A new reconstruction method, termed nonlinear reconstruction (NLR), was developed in 2017 to reconstruct the object image in the case of optical-scattering modulators. Over the years, it has been revealed that the NLR can reconstruct an object's image modulated by an axicons, bifocal lenses and even exotic spiral diffractive elements, which generate deterministic optical fields. Apparently, NLR seems to be a universal reconstruction method for indirect imaging. In this review, the performance of NLR isinvestigated for many deterministic and stochastic optical fields. Simulation and experimental results for different cases are presented and discussed.

Multidimension-multiplexed full-phase-encoding holography

I propose a multidimension-multiplexed imaging method with which multiple physical quantities of light are simultaneously obtained as interference fringe images. The varieties of light are distinguished by exploiting the proposed phase-encoding technique. Neither measurements of point spread functions in advance, nor iterative calculations to derive multidimensional information, nor a laser light source is required. Multidimensional imaging of an object and simultaneous three-dimensional image recording of self-luminous light and light transmitted from an object are experimentally demonstrated. A palm-sized interferometer based on the proposed holography is developed for the experiments to show its portability and physical-filter-free multidimensional imaging ability without an antivibration structure.

Wide-viewing holographic stereogram based on self-interference incoherent digital holography

We propose a holographic stereogram synthesis method which uses holograms that are optically captured by self-interference incoherent digital holography (SIDH) based on a geometric phase lens. SIDH is a promising solution for hologram acquisition under low-coherence lighting condition. A mechanical scanning system is constructed to acquire multiple perspective holograms. Numerical simulations and experimental analyses conducted using high-resolution diffractive optical element demonstrate that the proposed method can produce a wide-viewing hologram which can realize realistic 3D scenarios with depth cues such as accommodation and motion parallax. The future objectives include the implementation of a multiple-camera system for holographic videos.

Three-Dimensional Incoherent Imaging Using Spiral Rotating Point Spread Functions Created by Double-Helix Beams [Invited]

In recent years, there has been a significant transformation in the field of incoherent imaging with new possibilities of compressing three-dimensional (3D) information into a two-dimensional intensity distribution without two-beam interference (TBI). Most incoherent 3D imagers without TBI are based on scattering by a random phase mask exhibiting sharp autocorrelation and low cross-correlation along the depth axis. Consequently, during reconstruction, high lateral and axial resolutions are obtained. Scattering based-Imaging requires a wasteful photon budget and is therefore precluded in many power-sensitive applications. This study develops a proof-of-concept 3D incoherent imaging method using a rotating point spread function termed 3D Incoherent Imaging with Spiral Beams (3DI²SB). The rotation speed of the point spread function (PSF) with displacement and the orbital angular momentum has been theoretically analyzed. The imaging characteristics of 3DI²SB were compared with a direct imaging system using a diffractive lens, and the proposed system exhibited a higher focal depth than the direct imaging system. Different computational reconstruction methods such as the Lucy-Richardson algorithm (LRA), non-linear reconstruction (NLR), and the Lucy-Richardson-Rosen algorithm (LRRA) were compared. While LRRA performed better than both LRA and NLR for an ideal case, NLR performed better than both under real experimental conditions. Both single plane imaging, as well as synthetic 3D imaging, were demonstrated. We believe that the proposed approach might cause a paradigm shift in the current state-of-the-art incoherent imaging, fluorescence microscopy, and astronomical imaging.

Automatic Detection of Age-Related Macular Degeneration Based on Deep Learning and Local Outlier Factor Algorithm

Age-related macular degeneration (AMD) is a retinal disorder affecting the elderly, and society’s aging population means that the disease is becoming increasingly prevalent. The vision in patients with early AMD is usually unaffected or nearly normal but central vision may be weakened or even lost if timely treatment is not performed. Therefore, early diagnosis is particularly important to prevent the further exacerbation of AMD. This paper proposed a novel automatic detection method of AMD from optical coherence tomography (OCT) images based on deep learning and a local outlier factor (LOF) algorithm. A ResNet-50 model with L2-constrained softmax loss was retrained to extract features from OCT images and the LOF algorithm was used as the classifier. The proposed method was trained on the UCSD dataset and tested on both the UCSD dataset and Duke dataset, with an accuracy of 99.87% and 97.56%, respectively. Even though the model was only trained on the UCSD dataset, it obtained good detection accuracy when tested on another dataset. Comparison with other methods also indicates the efficiency of the proposed method in detecting AMD.

Recent progress in digital holography with dynamic diffractive phase apertures [Invited]

Digital holography with diffractive phase apertures is a hologram recording technique in which at least one of the interfering waves is modulated by a phase mask. In this review, we survey several main milestones on digital holography with dynamic diffractive phase apertures. We begin with Fresnel incoherent correlation holography (FINCH), a hologram recorder with an aperture of a diffractive lens. FINCH has been used for many applications such as 3D imaging, fluorescence microscopy, superresolution, image processing, and imaging with sectioning ability. FINCH has played an important role by inspiring other digital holography systems based on diffractive phase aperture, such as Fourier incoherent single-channel holography and coded aperture correlation holography, which also are described in this review.

Adaptive Digital Hologram Binarization Method Based on Local Thresholding, Block Division and Error Diffusion

High-speed optical reconstruction of 3D-scenes can be achieved using digital holography with binary digital micromirror devices (DMD) or a ferroelectric spatial light modulator (fSLM). There are many algorithms for binarizing digital holograms. The most common are methods based on global and local thresholding and error diffusion techniques. In addition, hologram binarization is used in optical encryption, data compression, beam shaping, 3D-displays, nanofabrication, materials characterization, etc. This paper proposes an adaptive binarization method based on a combination of local threshold processing, hologram division into blocks, and error diffusion procedure (the LDE method). The method is applied for binarization of optically recorded and computer-generated digital holograms of flat objects and three-dimensional scenes. The quality of reconstructed images was compared with different methods of error diffusion and thresholding. Image reconstruction quality was up to 22% higher by various metrics than that one for standard binarization methods. The optical hologram reconstruction using DMD confirms the results of the numerical simulations.

Voxel characteristic estimation of integral imaging display system using self-interference incoherent digital holography

Three-dimensional (3D) images reconstructed by integral imaging display are captured as a complex hologram using self-interference incoherent digital holography (SIDH) and analyzed for the volumetric image characteristics. The integrated images can present 3D perception through not only binocular disparity but also volumetric property, which is represented in forming a volume picture element, called 'voxel', and an important criterion to distinguish the integral imaging from the multiview 3D display. Since SIDH can record the complex hologram under incoherent lighting conditions, the SIDH camera system has the advantage to measure the voxel formed with the incoherent light fields. In this paper, we propose a technique to estimate and analyze the voxel characteristics of the integral imaging system such as the depth location and resolution. The captured holograms of the integrated images are numerically reconstructed by depth for the voxel analysis. The depth location of the integrated image can be calculated and obtained using the autofocus algorithms and the focus metrics values, which also show the modalities of depth resolution. The estimation method of this paper can be applied to the accurate and quantitative analysis of the volumetric characteristics of light field 3D displays.