Influence of the overlap parameter on the convergence of the ptychographical iterative engine

Recovery of spatial frequencies in coherent diffraction imaging in the presence of a central obscuration

Coherent diffraction imaging (CDI) and its scanning version, ptychography, are lensless imaging approaches used to iteratively retrieve a sample's complex scattering amplitude from its measured diffraction patterns. These imaging methods are most useful in extreme ultraviolet (EUV) and X-ray regions of the electromagnetic spectrum, where efficient imaging optics are difficult to manufacture. CDI relies on high signal-to-noise ratio diffraction data to recover the phase, but increasing the flux can cause saturation effects on the detector. A conventional solution to this problem is to place a beam stop in front of the detector. The pixel masking method is a common solution to the problem of missing frequencies due to a beam stop. This paper describes the information redundancy in the recorded data set and expands on how the reconstruction algorithm can exploit this redundancy to estimate the missing frequencies. Thereafter, we modify the size of the beam stop in experimental and simulation data to assess the impact of the missing frequencies, investigate the extent to which the lost portion of the diffraction spectrum can be recovered, and quantify the effect of the beam stop on the image quality. The experimental findings and simulations conducted for EUV imaging demonstrate that when using a beam stop, the numerical aperture of the condenser is a crucial factor in the recovery of lost frequencies. Our thorough investigation of the reconstructed images provides information on the overall quality of reconstruction and highlights the vulnerable frequencies if the beam stop size is larger than the extent of the illumination NA. The outcome of this study can be applied to other sources of frequency loss, and it will contribute to the improvement of experiments and reconstruction algorithms in CDI.

Unsupervised physics-informed deep learning-based reconstruction for time-resolved imaging by multiplexed ptychography

We explore numerically an unsupervised, physics-informed, deep learning-based reconstruction technique for time-resolved imaging by multiplexed ptychography. In our method, the untrained deep learning model replaces the iterative algorithm's update step, yielding superior reconstructions of multiple dynamic object frames compared to conventional methodologies. More precisely, we demonstrate improvements in image quality and resolution, while reducing sensitivity to the number of recorded frames, the mutual orthogonality of different probe modes, overlap between neighboring probe beams and the cutoff frequency of the ptychographic microscope - properties that are generally of paramount importance for ptychographic reconstruction algorithms.

Efficient boundary-guided scanning for high-resolution X-ray ptychography

In the realm of X-ray ptychography experiments, a considerable amount of ptychography scans are typically performed within a field of view encompassing the target sample. While it is crucial to obtain overlapping scans in small increments over the region of interest for achieving high-resolution sample reconstruction, a significant number of these scans often redundantly measure the empty background within the wide field of view. To address this inefficiency, an innovative algorithm is proposed that introduces automatic guidance for data acquisition. The algorithm first directs the scan point to actively search for the object of interest within the field of view. Subsequently, it intelligently scans along the perimeter of the sample, strategically acquiring measurements exclusively within the boundary of the region of interest. By employing this approach, a reduction in the number of measurements required to obtain high-resolution reconstruction images is demonstrated, as compared with conventional raster scanning methods. Furthermore, the automatic guidance provided by the algorithm offers the added advantage of saving valuable time during the reconstruction process. Through practical implementation on real experiments, these findings showcase the efficacy of the proposed algorithm in enhancing the efficiency and accuracy of X-ray ptychography experiments. This novel approach holds immense potential for advancing sample analysis and imaging techniques in various scientific disciplines.

An efficient ptychography reconstruction strategy through fine-tuning of large pre-trained deep learning model

With pre-trained large models and their associated fine-tuning paradigms being constantly applied in deep learning, the performance of large models achieves a dramatic boost, mostly owing to the improvements on both data quantity and quality. Next-generation synchrotron light sources offer ultra-bright and highly coherent X-rays, which are becoming one of the largest data sources for scientific experiments. As one of the most data-intensive scanning-based imaging methodologies, ptychography produces an immense amount of data, making the adoption of large deep learning models possible. Here, we introduce and refine the architecture of a neural network model to improve the reconstruction performance, through fine-tuning large pre-trained model using a variety of datasets. The pre-trained model exhibits remarkable generalization capability, while the fine-tuning strategy enhances the reconstruction quality. We anticipate this work will contribute to the advancement of deep learning methods in ptychography, as well as in broader coherent diffraction imaging methodologies in future.

Maximum-likelihood estimation in ptychography in the presence of Poisson-Gaussian noise statistics

Optical measurements often exhibit mixed Poisson-Gaussian noise statistics, which hampers the image quality, particularly under low signal-to-noise ratio (SNR) conditions. Computational imaging falls short in such situations when solely Poissonian noise statistics are assumed. In response to this challenge, we define a loss function that explicitly incorporates this mixed noise nature. By using a maximum-likelihood estimation, we devise a practical method to account for a camera readout noise in gradient-based ptychography optimization. Our results, based on both experimental and numerical data, demonstrate that this approach outperforms the conventional one, enabling enhanced image reconstruction quality under challenging noise conditions through a straightforward methodological adjustment.

High-resolution and high-sensitivity X-ray ptychographic coherent diffraction imaging using the CITIUS detector

Ptychographic coherent diffraction imaging (PCDI) is a synchrotron X-ray microscopy technique that provides high spatial resolution and a wide field of view. To improve the performance of PCDI, the performance of the synchrotron radiation source and imaging detector should be improved. In this study, ptychographic diffraction pattern measurements using the CITIUS high-speed X-ray image detector and the corresponding image reconstruction are reported. X-rays with an energy of 6.5 keV were focused by total reflection focusing mirrors, and a flux of ∼2.6 × 1010 photons s-1 was obtained at the sample plane. Diffraction intensity data were collected at up to ∼250 Mcounts s-1 pixel-1 without saturation of the detector. Measurements of tantalum test charts and silica particles and the reconstruction of phase images were performed. A resolution of ∼10 nm and a phase sensitivity of ∼0.01 rad were obtained. The CITIUS detector can be applied to the PCDI observation of various samples using low-emittance synchrotron radiation sources and to the stability evaluation of light sources.

Unsupervised adaptive coded illumination Fourier ptychographic microscopy based on a physical neural network

Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational burden. In this paper, we propose a novel physical neural network that generates an adaptive illumination mode by incorporating temporally-encoded illumination modes as a distinct layer, aiming to improve the acquisition and calculation efficiency. Both simulations and experiments have been conducted to validate the feasibility and effectiveness of the proposed method. It is worth mentioning that, unlike previous works that obtain the intensity of a multiplexed illumination by post-combination of each sequentially illuminated and obtained low-resolution images, our experimental data is captured directly by turning on multiple LEDs with a coded illumination pattern. Our method has exhibited state-of-the-art performance in terms of both detail fidelity and imaging velocity when assessed through a multitude of evaluative aspects.

A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions

Two-dimensional observation of biological samples at hundreds of nanometers resolution or even below is of high interest for many sensitive medical applications. Recent advances have been obtained over the last ten years with computational imaging. Among them, Fourier Ptychographic Microscopy is of particular interest because of its important super-resolution factor. In complement to traditional intensity images, phase images are also produced. A large set of N raw images (with typically N = 225) is, however, required because of the reconstruction process that is involved. In this paper, we address the problem of FPM image reconstruction using a few raw images only (here, N = 37) as is highly desirable to increase microscope throughput. In contrast to previous approaches, we develop an algorithmic approach based on a physics-informed optimization deep neural network and statistical reconstruction learning. We demonstrate its efficiency with the help of simulations. The forward microscope image formation model is explicitly introduced in the deep neural network model to optimize its weights starting from an initialization that is based on statistical learning. The simulation results that are presented demonstrate the conceptual benefits of the approach. We show that high-quality images are effectively reconstructed without any appreciable resolution degradation. The learning step is also shown to be mandatory.

Broadband ptychography using curved wavefront illumination

We examine the interplay between spectral bandwidth and illumination curvature in ptychography. By tailoring the divergence of the illumination, broader spectral bandwidths can be tolerated without requiring algorithmic modifications to the forward model. In particular, a strong wavefront curvature transitions a far-field diffraction geometry to an effectively near-field one, which is less affected by temporal coherence effects. The relaxed temporal coherence requirements allow for leveraging wider spectral bandwidths and larger illumination spots. Our findings open up new avenues towards utilizing pink and broadband beams for increased flux and throughput at both synchrotron facilities and lab-scale beamlines.

Characterisation of engineered defects in extreme ultraviolet mirror substrates using lab-scale extreme ultraviolet reflection ptychography

Ptychography is a lensless imaging technique that is aberration-free and capable of imaging both the amplitude and the phase of radiation reflected or transmitted from an object using iterative algorithms. Working with extreme ultraviolet (EUV) light, ptychography can provide better resolution than conventional optical microscopy and deeper penetration than scanning electron microscope. As a compact lab-scale EUV light sources, high harmonic generation meets the high coherence requirement of ptychography and gives more flexibilities in both budget and experimental time compared to synchrotrons. The ability to measure phase makes reflection-mode ptychography a good choice for characterising both the surface topography and the internal structural changes in EUV multilayer mirrors. This paper describes the use of reflection-mode ptychography with a lab-scale high harmonic generation based EUV light source to perform quantitative measurement of the amplitude and phase reflection from EUV multilayer mirrors with engineered substrate defects. Using EUV light at 29.6nm from a tabletop high harmonic generation light source, a lateral resolution down to ∼88nm and a phase resolution of 0.08rad (equivalent to topographic height variation of 0.27nm) are achieved. The effect of surface distortion and roughness on EUV reflectivity is compared to topographic properties of the mirror defects measured using both atomic force microscopy and scanning transmission electron microscopy. Modelling of reflection properties from multilayer mirrors is used to predict the potential of a combination of on-resonance, actinic ptychographic imaging at 13.5nm and atomic force microscopy for characterising the changes in multilayered structures.

Fourier ptychographic microscopy image enhancement with bi-modal deep learning

Digital pathology based on a whole slide imaging system is about to permit a major breakthrough in automated diagnosis for rapid and highly sensitive disease detection. High-resolution FPM (Fourier ptychographic microscopy) slide scanners delivering rich information on biological samples are becoming available. They allow new effective data exploitation for efficient automated diagnosis. However, when the sample thickness becomes comparable to or greater than the microscope depth of field, we report an observation of undesirable contrast change of sub-cellular compartments in phase images around the optimal focal plane, reducing their usability. In this article, a bi-modal U-Net artificial neural network (i.e., a two channels U-Net fed with intensity and phase images) is trained to reinforce specifically targeted sub-cellular compartments contrast for both intensity and phase images. The procedure used to construct a reference database is detailed. It is obtained by exploiting the FPM reconstruction algorithm to explore images around the optimal focal plane with virtual Z-stacking calculations and selecting those with adequate contrast and focus. By construction and once trained, the U-Net is able to simultaneously reinforce targeted cell compartment visibility and compensate for any focus imprecision. It is efficient over a large field of view at high resolution. The interest of the approach is illustrated considering the use-case of Plasmodium falciparum detection in blood smear where improvement in the detection sensitivity is demonstrated without degradation of the specificity. Post-reconstruction FPM image processing with such U-Net and its training procedure is general and applicable to demanding biological screening applications.

Cryogenic electron ptychographic single particle analysis with wide bandwidth information transfer

Advances in cryogenic transmission electron microscopy have revolutionised the determination of many macromolecular structures at atomic or near-atomic resolution. This method is based on conventional defocused phase contrast imaging. However, it has limitations of weaker contrast for small biological molecules embedded in vitreous ice, in comparison with cryo-ptychography, which shows increased contrast. Here we report a single-particle analysis based on the use of ptychographic reconstruction data, demonstrating that three dimensional reconstructions with a wide information transfer bandwidth can be recovered by Fourier domain synthesis. Our work suggests future applications in otherwise challenging single particle analyses, including small macromolecules and heterogeneous or flexible particles. In addition structure determination in situ within cells without the requirement for protein purification and expression may be possible.

Direct high-resolution X-ray imaging exploiting pseudorandomness

Owing to its unique penetrating power and high-resolution capability, X-ray imaging has been an irreplaceable tool since its discovery. Despite the significance, the resolution of X-ray imaging has largely been limited by the technical difficulties on X-ray lens making. Various lensless imaging methods have been proposed, but are yet relying on multiple measurements or additional constraints on measurements or samples. Here we present coherent speckle-correlation imaging (CSI) using a designed X-ray diffuser. CSI has no prerequisites for samples or measurements. Instead, from a single shot measurement, the complex sample field is retrieved based on the pseudorandomness of the speckle intensity pattern, ensured through a diffuser. We achieve a spatial resolution of 13.9 nm at 5.46 keV, beating the feature size of the diffuser used (300 nm). The high-resolution imaging capability is theoretically explained based on fundamental and practical limits. We expect the CSI to be a versatile tool for navigating the unexplored world of nanometer.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

4D-STEM Ptychography for Electron-Beam-Sensitive Materials

Recent advances in high-speed pixelated electron detectors have substantially facilitated the implementation of four-dimensional scanning transmission electron microscopy (4D-STEM). A critical application of 4D-STEM is electron ptychography, which reveals the atomic structure of a specimen by reconstructing its transmission function from redundant convergent-beam electron diffraction patterns. Although 4D-STEM ptychography offers many advantages over conventional imaging modes, this emerging technique has not been fully applied to materials highly sensitive to electron beams. In this Outlook, we introduce the fundamentals of 4D-STEM ptychography, focusing on data collection and processing methods, and present the current applications of 4D-STEM ptychography in various materials. Next, we discuss the potential advantages of imaging electron-beam-sensitive materials using 4D-STEM ptychography and explore its feasibility by performing simulations and experiments on a zeolite material. The preliminary results demonstrate that, at the low electron dose required to preserve the zeolite structure, 4D-STEM ptychography can reliably provide higher resolution and greater tolerance to the specimen thickness and probe defocus as compared to existing imaging techniques. In the final section, we discuss the challenges and possible strategies to further reduce the electron dose for 4D-STEM ptychography. If successful, it will be a game-changer for imaging extremely sensitive materials, such as metal-organic frameworks, hybrid halide perovskites, and supramolecular crystals.

Resolution enhancement with highly curved illumination in ptychography

By deducing a formula to compute a sample from recorded diffraction intensity directly and analytically, the relationship among the highest reachable resolution of the ptychographic iterative engine (PIE), its illumination angle, and its collection angle was discussed analytically. Curved illumination was then proposed to realize the resolution enhancement for PIE, and a corresponding computing algorithm was proposed to avoid an undersampling effect without increasing the size of the computing matrix, thus realizing speedy high-resolution PIE imaging with a simple experimental setup. While theoretical analysis was carried out, the feasibility of this proposed method was verified both numerically and experimentally.

Fast scanning in x-ray microscopy: the effects of offset in the central stop position

Scanning of lightweight circular diffractive optics, separate from central stops and apertures, is emerging as an approach to exploit advances in synchrotron x-ray sources. We consider the effects in a scanning microscope of offsets between the optic and its central stop and find that scan ranges of up to about half the diameter of the optic are possible with only about a 10% increase in the focal spot width. For large scanning ranges, we present criteria for the working distance between the last aperture and the specimen to be imaged.

Phase retrieval based on a total-variation-regularized Poisson model for X-ray ptychographic imaging of low-contrast objects

Hard X-ray ptychography has become an indispensable tool for observing the microscopic structure of a thick specimen. It measures diffraction patterns by scanning an X-ray beam and visualizes the complex-valued refractive index of the specimen by a computational reconstruction called phase retrieval. The quality of imaging is dependent on the used phase-retrieval algorithm, especially when the intensity of the diffraction patterns in the high-spatial-frequency range is low and/or when the spatial overlap of the illumination area is small. In this paper, a phase-retrieval algorithm, AMPAM, based on the Poisson model and total variation (TV) is proposed. It applies alternating minimization using primal-dual splitting and gradient-descent algorithms to compute the result without matrix inversion. The imaging capability of the proposed algorithm from low-dose and/or sparsely scanned data was investigated by numerical simulations. The proposed algorithm was compared with ADPr, which is the state-of-the-art algorithm based on the TV-regularized Poisson model. The results indicated that AMPAM can provide good-quality images with a computational cost 7–11 times less than ADPr. In addition, ink toner and macroporous silica particles were imaged at SPring-8 BL24XU to confirm the applicability of the algorithm to actual measurements.

Morphological variations to a ptychographic algorithm

Ptychography is a technique widely used in microscopy for achieving high-resolution imaging. This method relies on computational processing of images gathered from diffraction patterns produced by several partial illuminations of a sample. We numerically studied the effect of using different shapes for illuminating the aforementioned sample: convex shapes, such as circles and regular polygons, and unconnected shapes that resemble a QR code. Our results suggest that the use of unconnected shapes seems to outperform convex shapes in terms of convergence and, in some cases, accuracy.

High-resolution ptychographic imaging enabled by high-speed multi-pass scanning

As a coherent diffraction imaging technique, ptychography provides high-spatial resolution beyond Rayleigh's criterion of the focusing optics, but it is also sensitively affected by the decoherence coming from the spatial and temporal variations in the experiment. Here we show that high-speed ptychographic data acquisition with short exposure can effectively reduce the impact from experimental variations. To reach a cumulative dose required for a given resolution, we further demonstrate that a continuous multi-pass scan via high-speed ptychography can achieve high-resolution imaging. This low-dose scan strategy is shown to be more dose-efficient, and has potential for radiation-sensitive sample studies and time-resolved imaging.

Towards kilohertz synchrotron coherent diffractive imaging

This work shows how spatiotemporal redundancy can overcome the twin-image stagnation mode in coherent diffractive imaging, and explores the relationship between detector frame rate and signal-to-noise ratio in the application of imaging nanoscale dynamic behaviour at kHz frame rates.

X-ray coherent diffractive imaging (CDI) techniques have been applied with widespread impact to study nanoscale material properties. New fast framing detectors may reveal dynamics that occur at millisecond timescales. This work demonstrates by simulation that kilohertz synchrotron CDI is possible, by making use of redundant information from static parts of the image field. Reconstruction ambiguities are strongly suppressed by applying a spatio­temporal constraint, obviating the need for slower methods of introducing diversity such as ptychography. The relationship between image fidelity and time resolution is investigated and shows that dynamics an order of magnitude faster can be reconstructed, compared with conventional CDI.

Wavelength-multiplexed single-shot ptychography

We present the first experimental demonstration of wavelength-multiplexing in single-shot ptychography. Specifically, we experimentally reconstruct the complex transmission profile of a wavelength-independent and wavelength-dependent object simultaneously for 532 nm and 633 nm probing wavelengths. In addition, we discuss the advantages of a more general approach to detector segmentation in single-shot ptychography. A minimization to correct for uncertainties in a priori information that is required for single-shot geometries is presented along with a novel probe constraint. Furthermore, this technique is complementary to dual-wavelength interferometry without the need for an external reference. This work is enabling to imaging technologies and applications such as broadband single-shot ptychography, time-resolved imaging by multiplexed ptychography, real-time inspection for laser micro-machining, and plasma imaging.

Analysis, Simulations, and Experiments for Far-Field Fourier Ptychography Imaging Using Active Coherent Synthetic-Aperture

Fourier ptychography (FP) is a powerful phase retrieval method that can be used to reconstruct missing high-frequency details and high-space-bandwidth products in microscopy. In this study, we further advanced the application of FP in microscopic imaging to the field of macroscopic far-field imaging, incorporating camera scanning for spatial resolution improvement. First, on the basis of the Fraunhofer diffraction mechanism and the transmission imaging model, we found the analysis of the associated theoretical fundamentals via simulations and experiments to be crucially relevant to the far-field of FP imaging. Second, we built an experimental device with long-distance imaging and experimentally demonstrated the relationship between the spectrum overlap ratio and the reconstructed high-resolution image. The simulation and experimental results showed that an overlap ratio higher than 50% had a good reconstruction effect. Third, camera scanning was used to obtain low-resolution intensity images in this study, for which the scanning range was wide and spherical wave illumination was satisfied, and therefore different positions corresponded to different aberrations of low-resolution intensity images, and even different positions of the same image had aberration differences, leading to inconsistencies in the aberrations of different images. Therefore, in the reconstruction process, we further overcame the effect of the inconsistency of aberrations of different images using the partition reconstruction method, which involves cutting the image into smaller parts for reconstruction. Finally, with the proposed partition reconstruction algorithm, we were able to resolve 40 μm line width of GBA1 resolution object and obtain a spatial resolution gain of 4× with a working distance of 2 m.

Hard X-ray ptychography at Taiwan Photon Source at 11-20 nm spatial resolution

X-ray ptychography, a technique based on scanning and processing of coherent diffraction patterns, is a non-destructive imaging technique with a high spatial resolution far beyond the focused beam size. Earlier demonstrations of hard X-ray ptychography at Taiwan Photon Source (TPS) using an in-house program successfully recorded the ptychographic diffraction patterns from a gold-made Siemens star as a test sample and retrieved the finest inner features of 25 nm. Ptychography was performed at two beamlines with different focusing optics: a pair of Kirkpatrick-Baez mirrors and a pair of nested Montel mirrors, for which the beam sizes on the focal planes were 3 µm and 200 nm and the photon energies were from 5.1 keV to 9 keV. The retrieved spatial resolutions are 20 nm to 11 nm determined by the 10-90% line-cut method and half-bit threshold of Fourier shell correlation. This article describes the experimental conditions and compensation methods, including position correction, mixture state-of-probe, and probe extension methods, of the aforementioned experiments. The discussions will highlight the criteria of ptychographic experiments at TPS as well as the opportunity to characterize beamlines by measuring factors such as the drift or instability of beams or stages and the coherence of beams. Besides, probe functions, the full complex fields illuminated on samples, can be recovered simultaneously using ptychography. Theoretically, the wavefield at any arbitrary position can be estimated from one recovered probe function undergoing wave-propagating. The verification of probe-propagating has been carried out by comparing the probe functions obtained by ptychography and undergoing wave-propagating located at 0, 500 and 1000 µm relative to the focal plane.

Parameter retrieval of small particles in dark-field Fourier ptychography and a rectangle in real-space ptychography

We present a parameter retrieval method which incorporates prior knowledge about the object into ptychography. The proposed method is applied to two applications: (1) parameter retrieval of small particles from Fourier ptychographic dark field measurements; (2) parameter retrieval of a rectangular structure with real-space ptychography. The influence of Poisson noise is discussed in the second part of the paper. The Cramér Rao Lower Bound in both applications is computed and Monte Carlo analysis is used to verify the calculated lower bound. With the computation results we report the lower bound for various noise levels and analyze the correlation of particles in application 1. For application 2 the correlation of parameters of the rectangular structure is discussed.

Accelerating ptychographic reconstructions using spectral initializations

Ptychography is a promising phase retrieval technique for label-free quantitative phase imaging. Recent advances in phase retrieval algorithms witnessed the development of spectral methods to accelerate gradient descent algorithms. Using spectral initializations on experimental data, for the first time, we report three times faster ptychographic reconstructions than with a standard gradient descent algorithm and improved resilience to noise. Coming at no additional computational cost compared to gradient-descent-based algorithms, spectral methods have the potential to be implemented in large-scale iterative ptychographic algorithms.

Upscaling X-ray nanoimaging to macroscopic specimens

Upscaling X-ray nanoimaging to macroscopic specimens has the potential for providing insights across multiple length scales, but its feasibility has long been an open question. By combining the imaging requirements and existing proof-of-principle examples in large-specimen preparation, data acquisition and reconstruction algorithms, the authors provide imaging time estimates for howX-ray nanoimaging can be scaled to macroscopic specimens. To arrive at this estimate, a phase contrast imaging model that includes plural scattering effects is used to calculate the required exposure and corresponding radiation dose. The coherent X-ray flux anticipated from upcoming diffraction-limited light sources is then considered. This imaging time estimation is in particular applied to the case of the connectomes of whole mouse brains. To image the connectome of the whole mouse brain, electron microscopy connectomics might require years, whereas optimized X-ray microscopy connectomics could reduce this to one week. Furthermore, this analysis points to challenges that need to be overcome (such as increased X-ray detector frame rate) and opportunities that advances in artificial-intelligence-based 'smart' scanning might provide. While the technical advances required are daunting, it is shown that X-ray microscopy is indeed potentially applicable to nanoimaging of millimetre- or even centimetre-size specimens.

Optimizing illumination for precise multi-parameter estimations in coherent diffractive imaging

Coherent diffractive imaging (CDI) is widely used to characterize structured samples from measurements of diffracting intensity patterns. We introduce a numerical framework to quantify the precision that can be achieved when estimating any given set of parameters characterizing the sample from measured data. The approach, based on the calculation of the Fisher information matrix, provides a clear benchmark to assess the performance of CDI methods. Moreover, by optimizing the Fisher information metric using deep learning optimization libraries, we demonstrate how to identify the optimal illumination scheme that minimizes the estimation error under specified experimental constraints. This work paves the way for an efficient characterization of structured samples at the sub-wavelength scale.

Fast digital lossy compression for X-ray ptychographic data

Increases in X-ray brightness from synchrotron light sources lead to a requirement for higher frame rates from hybrid pixel array detectors (HPADs), while also favoring charge integration over photon counting. However, transfer of the full uncompressed data will begin to constrain detector design, as well as limit the achievable continuous frame rate. Here a data compression scheme that is easy to implement in a HPAD's application-specific integrated circuit (ASIC) is described, and how different degrees of compression affect image quality in ptychography, a commonly employed coherent imaging method, is examined. Using adaptive encoding quantization, it is shown in simulations that one can digitize signals up to 16383 photons per pixel (corresponding to 14 bits of information) using only 8 or 9 bits for data transfer, with negligible effect on the reconstructed image.

Broadband X-ray ptychography using multi-wavelength algorithm

Ptychography is a rapidly developing scanning microscopy which is able to view the internal structures of samples at a high resolution beyond the illumination size. The achieved spatial resolution is theoretically dose-limited. A broadband source can provide much higher flux compared with a monochromatic source; however, it conflicts with the necessary coherence requirements of this coherent diffraction imaging technique. In this paper, a multi-wavelength reconstruction algorithm has been developed to deal with the broad bandwidth in ptychography. Compared with the latest development of mixed-state reconstruction approach, this multi-wavelength approach is more accurate in the physical model, and also considers the spot size variation as a function of energy due to the chromatic focusing optics. Therefore, this method has been proved in both simulation and experiment to significantly improve the reconstruction when the source bandwidth, illumination size and scan step size increase. It is worth mentioning that the accurate and detailed information of the energy spectrum for the incident beam is not required in advance for the proposed method. Further, we combine multi-wavelength and mixed-state approaches to jointly solve temporal and spatial partial coherence in ptychography so that it can handle various disadvantageous experimental effects. The significant relaxation in coherence requirements by our approaches allows the use of high-flux broadband X-ray sources for high-efficient and high-resolution ptychographic imaging.

Reference-free quantitative microscopic imaging of coherent arbitrary vectorial light beams

Precise spatial characterization of vectorial beams is crucial for many advanced optical experiments, but challenging when wavefront and polarization features are involved together. Here we propose a reference-free method aimed at extracting the map of the complex-amplitude components of any coherent beam at an optical-microscopy resolution. Our method exploits recent advances in ptychographic imaging approaches. We emphasize its versatility by reconstructing successfully various experimental vectorial beams including polarization and phase vortices, the exit field of a multicore fiber and a speckle pattern.

Hard X-ray ptychography for optics characterization using a partially coherent synchrotron source

Ptychography is a scanning coherent diffraction imaging technique which provides high resolution imaging and complete spatial information of the complex electric field probe and sample transmission function. Its ability to accurately determine the illumination probe has led to its use at modern synchrotrons and free-electron lasers as a wavefront-sensing technique for optics alignment, monitoring and correction. Recent developments in the ptychography reconstruction process now incorporate a modal decomposition of the illuminating probe and relax the restriction of using sources with high spatial coherence. In this article a practical implementation of hard X-ray ptychography from a partially coherent X-ray source with a large number of modes is demonstrated experimentally. A strongly diffracting Siemens star test sample is imaged using the focused beam produced by either a Fresnel zone plate or beryllium compound refractive lens. The recovered probe from each optic is back propagated in order to plot the beam caustic and determine the precise focal size and position. The power distribution of the reconstructed probe modes also allows the quantification of the beams coherence and is compared with the values predicted by a Gaussian-Schell model and the optics exit intensity.

The XFM beamline at the Australian Synchrotron

The X-ray fluorescence microscopy (XFM) beamline is an in-vacuum undulator-based X-ray fluorescence (XRF) microprobe beamline at the 3 GeV Australian Synchrotron. The beamline delivers hard X-rays in the 4-27 keV energy range, permitting K emission to Cd and L and M emission for all other heavier elements. With a practical low-energy detection cut-off of approximately 1.5 keV, low-Z detection is constrained to Si, with Al detectable under favourable circumstances. The beamline has two scanning stations: a Kirkpatrick-Baez mirror microprobe, which produces a focal spot of 2 µm × 2 µm FWHM, and a large-area scanning `milliprobe', which has the beam size defined by slits. Energy-dispersive detector systems include the Maia 384, Vortex-EM and Vortex-ME3 for XRF measurement, and the EIGER2 X 1 Mpixel array detector for scanning X-ray diffraction microscopy measurements. The beamline uses event-mode data acquisition that eliminates detector system time overheads, and motion control overheads are significantly reduced through the application of an efficient raster scanning algorithm. The minimal overheads, in conjunction with short dwell times per pixel, have allowed XFM to establish techniques such as full spectroscopic XANES fluorescence imaging, XRF tomography, fly scanning ptychography and high-definition XRF imaging over large areas. XFM provides diverse analysis capabilities in the fields of medicine, biology, geology, materials science and cultural heritage. This paper discusses the beamline status, scientific showcases and future upgrades.

Practical implementation of high-resolution electron ptychography and comparison with off-axis electron holography

Ptychography is a coherent diffractive imaging technique that can determine how an electron wave is transmitted through an object by probing it in many small overlapping regions and processing the diffraction data obtained at each point. The resulting electron transmission model describes both phase and amplitude changes to the electron wave. Ptychography has been adopted in transmission electron microscopy in recent years following advances in high-speed direct electron detectors and computer algorithms which now make the technique suitable for practical applications. Its ability to retrieve quantitative phase information at high spatial resolution makes it a plausible alternative or complement to electron holography. Furthermore, unlike off-axis electron holography, it can provide phase information without an electron bi-prism assembly or the requirement of a minimally structured region adjacent to the region of interest in the object. However, it does require a well-calibrated scanning transmission electron microscope and a well-managed workflow to manage the calibration, data acquisition and reconstruction process to yield a practical technique. Here we detail this workflow and highlight how this is greatly assisted by acquisition management software. Through experimental data and modelling we also explore the similarities and differences between high-resolution ptychography and electron holography. Both techniques show a dependence of the recovered phase on the crystalline orientation of the material which is attributable to dynamical scattering. However, the exact nature of the variation differs reflecting fundamental expectations, but nonetheless equally useful information is obtained from electron holography and the ptychographically determined object transmission function.

Reconstruction method of a ptychographic dataset with unknown positions

Wavefield drift or wobbling occurs quite often in coherent scanning systems such as satellite laser communication, laser pointing of high-power lasers, or microscopy. The uncertainty of wavefront positions might result in blurred images or large measurement errors. Here we propose an iterative approach that can retrieve both the drift positions and complex-valued distribution of the wavefield from a ptychographic diffraction intensity dataset. We demonstrate the feasibility and effectiveness of the method in numerical simulation and an optical experiment. The method requires little a priori knowledge and thus would open up new opportunities in many fields.

Continuous-wave terahertz reflective ptychography by oblique illumination

Massive usage scenarios prompt the prosperity of terahertz (THz) reflective imaging methods. In this Letter, we apply ptychography to continuous-wave THz reflective imaging. Our scheme has a compact lensless layout and uses a full-field oblique-illumination recording mode. Diffraction patterns are corrected through tilted plane correction. This method can be used to retrieve the complex-valued object function and to suppress the negative effect of non-uniform illumination. The feasibility is investigated using two metal samples.

Mixed-state electron ptychography enables sub-angstrom resolution imaging with picometer precision at low dose

Both high resolution and high precision are required to quantitatively determine the atomic structure of complex nanostructured materials. However, for conventional imaging methods in scanning transmission electron microscopy (STEM), atomic resolution with picometer precision cannot usually be achieved for weakly-scattering samples or radiation-sensitive materials, such as 2D materials. Here, we demonstrate low-dose, sub-angstrom resolution imaging with picometer precision using mixed-state electron ptychography. We show that correctly accounting for the partial coherence of the electron beam is a prerequisite for high-quality structural reconstructions due to the intrinsic partial coherence of the electron beam. The mixed-state reconstruction gains importance especially when simultaneously pursuing high resolution, high precision and large field-of-view imaging. Compared with conventional atomic-resolution STEM imaging techniques, the mixed-state ptychographic approach simultaneously provides a four-times-faster acquisition, with double the information limit at the same dose, or up to a fifty-fold reduction in dose at the same resolution.

With conventional scanning transmission electron microscopy, some sensitive materials are difficult to image with atomic resolution. The authors present a method of mixed-state electron ptychography that enables picometer precision with fast acquisition and low dosage.

Near, far, wherever you are: simulations on the dose efficiency of holographic and ptychographic coherent imaging

Different studies in X-ray microscopy have arrived at conflicting conclusions about the dose efficiency of imaging modes involving the recording of intensity distributions in the near (Fresnel regime) or far (Fraunhofer regime) field downstream of a specimen. A numerical study is presented on the dose efficiency of near-field holography, near-field ptychography and far-field ptychography, where ptychography involves multiple overlapping finite-sized illumination positions. Unlike what has been reported for coherent diffraction imaging, which involves recording a single far-field diffraction pattern, it is found that all three methods offer similar image quality when using the same fluence on the specimen, with far-field ptychography offering slightly better spatial resolution and a lower mean error. These results support the concept that (if the experiment and image reconstruction are done properly) the sample can be near or far; wherever you are, photon fluence on the specimen sets one limit to spatial resolution.

The achievable resolution for X-ray imaging of cells and other soft biological material

X-ray imaging of soft materials is often difficult because of the low contrast of the components. This particularly applies to frozen hydrated biological cells where the feature of interest can have a similar density to the surroundings. As a consequence, a high dose is often required to achieve the desired resolution. However, the maximum dose that a specimen can tolerate is limited by radiation damage. Results from 3D coherent diffraction imaging (CDI) of frozen hydrated specimens have given resolutions of ∼80 nm compared with the expected resolution of 10 nm predicted from theoretical considerations for identifying a protein embedded in water. Possible explanations for this include the inapplicability of the dose-fractionation theorem, the difficulty of phase determination, an overall object-size dependence on the required fluence and dose, a low contrast within the biological cell, insufficient exposure, and a variety of practical difficulties such as scattering from surrounding material. A recent article [Villaneuva-Perez et al. (2018), Optica, 5, 450-457] concluded that imaging by Compton scattering gave a large dose advantage compared with CDI because of the object-size dependence for CDI. An object-size dependence would severely limit the applicability of CDI and perhaps related coherence-based methods for structural studies. This article specifically includes the overall object size in the analysis of the fluence and dose requirements for coherent imaging in order to investigate whether there is a dependence on object size. The applicability of the dose-fractionation theorem is also discussed. The analysis is extended to absorption-based imaging and imaging by incoherent scattering (Compton) and fluorescence. This article includes analysis of the dose required for imaging specific low-contrast cellular organelles as well as for protein against water. This article concludes that for both absorption-based and coherent diffraction imaging, the dose-fractionation theorem applies and the required dose is independent of the overall size of the object. For incoherent-imaging methods such as Compton scattering, the required dose depends on the X-ray path length through the specimen. For all three types of imaging, the dependence of fluence and dose on a resolution d goes as 1/d 4 when imaging uniform-density voxels. The independence of CDI on object size means that there is no advantage for Compton scattering over coherent-based imaging methods. The most optimistic estimate of achievable resolution is 3 nm for imaging protein molecules in water/ice using lensless imaging methods in the water window. However, the attainable resolution depends on a variety of assumptions including the model for radiation damage as a function of resolution, the efficiency of any phase-retrieval process, the actual contrast of the feature of interest within the cell and the definition of resolution itself. There is insufficient observational information available regarding the most appropriate model for radiation damage in frozen hydrated biological material. It is advocated that, in order to compare theory with experiment, standard methods of reporting results covering parameters such as the feature examined (e.g. which cellular organelle), resolution, contrast, depth of the material (for 2D), estimate of noise and dose should be adopted.

Diffraction tomography with a deep image prior

We present a tomographic imaging technique, termed Deep Prior Diffraction Tomography (DP-DT), to reconstruct the 3D refractive index (RI) of thick biological samples at high resolution from a sequence of low-resolution images collected under angularly varying illumination. DP-DT processes the multi-angle data using a phase retrieval algorithm that is extended by a deep image prior (DIP), which reparameterizes the 3D sample reconstruction with an untrained, deep generative 3D convolutional neural network (CNN). We show that DP-DT effectively addresses the missing cone problem, which otherwise degrades the resolution and quality of standard 3D reconstruction algorithms. As DP-DT does not require pre-captured data or pre-training, it is not biased towards any particular dataset. Hence, it is a general technique that can be applied to a wide variety of 3D samples, including scenarios in which large datasets for supervised training would be infeasible or expensive. We applied DP-DT to obtain 3D RI maps of bead phantoms and complex biological specimens, both in simulation and experiment, and show that DP-DT produces higher-quality results than standard regularization techniques. We further demonstrate the generality of DP-DT, using two different scattering models, the first Born and multi-slice models. Our results point to the potential benefits of DP-DT for other 3D imaging modalities, including X-ray computed tomography, magnetic resonance imaging, and electron microscopy.

zPIE: an autofocusing algorithm for ptychography

An autofocusing algorithm for ptychography is proposed. The method optimizes a sharpness metric that would be observed in a differential interference microscope and is valid for both amplitude and phase modulating specimens. We experimentally demonstrate that the algorithm, based on the extended ptychographic iterative engine (ePIE), calibrates the sample-detector distance with an accuracy within the depth of field of the ptychographic microscope. We show that the method can be used to determine slice separation in multislice ptychography, provided there are isolated regions on each slice of the specimen that do not axially overlap.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Multi-layered full-field phase imaging using continuous-wave terahertz ptychography

Due to the unique properties of terahertz (THz) waves, THz phase imaging has been widely investigated to retrieve the absorption and phase modulation of dielectric two-dimensional thin samples, as well as multiple stacked samples. In this Letter, we apply the three-dimensional ptychographic iterative engine algorithm for continuous-wave THz full-field multi-layered phase imaging. The complex-valued transmission function of two-layered polypropylene thin plates and the corresponding probe function are reconstructed, respectively, which are immune to crosstalk of different layers. The phenomenon of the field-of-view enlargement at the second object layer is observed. This lensless compact imaging method can be potentially used for THz three-dimensional imaging.

Super-resolution near-field ptychography

Compared to far-field ptychography, near-field ptychography can reduce the requirement on the detector dynamic range, while it is able to cover a larger field of view with a fewer number of sample scans. However, its spatial resolution is limited by the detector pixel size. Here, we utilize a pixel-super-resolved approach to overcome this limitation. The method has been applied to four types of experiment configurations using planar and divergent illuminations together with two different cameras with highly contrast specifications. The proposed method works effectively for up-sampling up to 6 times. Meanwhile, it can achieve ∼5.9-fold and ∼3.1-fold resolution improvement over the 6.5-μm and 2.4-μm detector pixel size. We also demonstrate the precisely quantitative phase imaging capability of the method by using a phase resolution target. The presented method is believed to have great potential in X-ray tomography and on-chip flow cytometry.

Multimodal X-ray imaging of nanocontainer-treated macrophages and calcium distribution in the perilacunar bone matrix

Studies of biological systems typically require the application of several complementary methods able to yield statistically-relevant results at a unique level of sensitivity. Combined X-ray fluorescence and ptychography offer excellent elemental and structural imaging contrasts at the nanoscale. They enable a robust correlation of elemental distributions with respect to the cellular morphology. Here we extend the applicability of the two modalities to higher X-ray excitation energies, permitting iron mapping. Using a long-range scanning setup, we applied the method to two vital biomedical cases. We quantified the iron distributions in a population of macrophages treated with Mycobacterium-tuberculosis-targeting iron-oxide nanocontainers. Our work allowed to visualize the internalization of the nanocontainer agglomerates in the cytosol. From the iron areal mass maps, we obtained a distribution of antibiotic load per agglomerate and an average areal concentration of nanocontainers in the agglomerates. In the second application we mapped the calcium content in a human bone matrix in close proximity to osteocyte lacunae (perilacunar matrix). A concurrently acquired ptychographic image was used to remove the mass-thickness effect from the raw calcium map. The resulting ptychography-enhanced calcium distribution allowed then to observe a locally lower degree of mineralization of the perilacunar matrix.

Generation and characterization of focused helical x-ray beams

The phenomenon of orbital angular momentum (OAM) affects a variety of important applications in visible optics, including optical tweezers, free-space communication, and 3D localization for fluorescence imaging. The lack of suitable wavefront shaping optics such as spatial light modulators has inhibited the ability to impart OAM on x-ray and electron radiation in a controlled way. Here, we report the experimental observation of helical soft x-ray beams generated by holographically designed diffractive optical elements. We demonstrate that these beams rotate as a function of propagation distance and measure their vorticity and coherent mode structure using ptychography. Our results establish an approach for controlling and shaping of complex focused beams for short wavelength scanning microscopy and OAM-driven applications.

Ptychography with multiple wavelength illumination

For performing phase retrieval in the extreme ultraviolet (EUV) regime more efficiently, developing polychromatic ptychography is desirable. As an alternative to the existing ptychographic information multiplexing (PIM) method, we present an another scheme where all monochromatic exit waves are expressed in terms of the amplitude of the transmission function and the thickness function of the object. Our proposed algorithm is a gradient based method and its validity is studied numerically. In addition, the sampling issue which appears in the polychromatic ptychography scheme and its influence to the reconstruction quality are discussed.

THz coherent lensless imaging

Imaging with THz radiation has proved an important tool for both fundamental science and industrial use. Here we review a class of THz imaging implementations, named coherent lensless imaging, that reconstruct the coherent response of arbitrary samples with a minimized experimental setup based only on a coherent source and a camera. After discussing the appropriate sources and detectors to perform them, we detail the fundamental principles and implementations of THz digital holography and phase retrieval. These techniques owe a lot to imaging with different wavelengths, yet innovative concepts are also being developed in the THz range and are ready to be applied in other spectral ranges. This makes our review useful for both the THz and imaging communities, and we hope it will foster their interaction.

Imaging Correlography Using Ptychography

Imaging correlography, an effective method for long-distance imaging, recovers an object using only the knowledge of the Fourier modulus, without needing phase information. It is not sensitive to atmospheric turbulence or optical imperfections. However, the unreliability of traditional phase retrieval algorithms in imaging correlography has hindered their development. In this work, we join imaging correlography and ptychography together to overcome such obstacles. Instead of detecting the whole object, the object is measured part-by-part with a probe moving in a ptychographic way. A flexible optimization framework is proposed to reconstruct the object rapidly and reliably within a few iterations. In addition, novel image space denoising regularization is plugged into the loss function to reduce the effects of input noise and improve the perceptual quality of the recovered image. Experiments demonstrate that four-fold resolution gains are achievable for the proposed imaging method. We can obtain satisfactory results for both visual and quantitative metrics with one-sixth of the measurements in the conventional imaging correlography. Therefore, the proposed imaging technique is more suitable for long-range practical applications.