1
|
Schoonhoven R, Skorikov A, Palenstijn WJ, Pelt DM, Hendriksen AA, Batenburg KJ. How auto-differentiation can improve CT workflows: classical algorithms in a modern framework. OPTICS EXPRESS 2024; 32:9019-9041. [PMID: 38571146 DOI: 10.1364/oe.502920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 01/09/2024] [Indexed: 04/05/2024]
Abstract
Many of the recent successes of deep learning-based approaches have been enabled by a framework of flexible, composable computational blocks with their parameters adjusted through an automatic differentiation mechanism to implement various data processing tasks. In this work, we explore how the same philosophy can be applied to existing "classical" (i.e., non-learning) algorithms, focusing on computed tomography (CT) as application field. We apply four key design principles of this approach for CT workflow design: end-to-end optimization, explicit quality criteria, declarative algorithm construction by building the forward model, and use of existing classical algorithms as computational blocks. Through four case studies, we demonstrate that auto-differentiation is remarkably effective beyond the boundaries of neural-network training, extending to CT workflows containing varied combinations of classical and machine learning algorithms.
Collapse
|
2
|
Diederichs B, Herdegen Z, Strauch A, Filbir F, Müller-Caspary K. Exact inversion of partially coherent dynamical electron scattering for picometric structure retrieval. Nat Commun 2024; 15:101. [PMID: 38168078 PMCID: PMC10762228 DOI: 10.1038/s41467-023-44268-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 11/30/2023] [Indexed: 01/05/2024] Open
Abstract
The greatly nonlinear diffraction of high-energy electron probes focused to subatomic diameters frustrates the direct inversion of ptychographic data sets to decipher the atomic structure. Several iterative algorithms have been proposed to yield atomically-resolved phase distributions within slices of a 3D specimen, corresponding to the scattering centers of the electron wave. By pixelwise phase retrieval, current approaches do not only involve orders of magnitude more free parameters than necessary, but also neglect essential details of scattering physics such as the atomistic nature of the specimen and thermal effects. Here, we introduce a parametrized, fully differentiable scheme employing neural network concepts which allows the inversion of ptychographic data by means of entirely physical quantities. Omnipresent thermal diffuse scattering in thick specimens is treated accurately using frozen phonons, and atom types, positions and partial coherence are accounted for in the inverse model as relativistic scattering theory demands. Our approach exploits 4D experimental data collected in an aberration-corrected momentum-resolved scanning transmission electron microscopy setup. Atom positions in a 20 nm thick PbZr0.2Ti0.8O3 ferroelectric are measured with picometer precision, including the discrimination of different atom types and positions in mixed columns.
Collapse
Affiliation(s)
- Benedikt Diederichs
- Department of Chemistry and Centre for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany
- Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Neuherberg, Germany
| | - Ziria Herdegen
- Department of Chemistry and Centre for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany
| | - Achim Strauch
- Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons, Forschungszentrum Jülich, Jülich, Germany
| | - Frank Filbir
- Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Neuherberg, Germany
- Department of Mathematics, TUM School of Computation, Information and Technology, Technische Universität München, Garching, Germany
| | - Knut Müller-Caspary
- Department of Chemistry and Centre for NanoScience, Ludwig-Maximilians-Universität München, Munich, Germany.
- Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons, Forschungszentrum Jülich, Jülich, Germany.
| |
Collapse
|
3
|
Terzoudis-Lumsden EWC, Petersen TC, Brown HG, Pelz PM, Ophus C, Findlay SD. Resolution of Virtual Depth Sectioning from Four-Dimensional Scanning Transmission Electron Microscopy. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2023; 29:1409-1421. [PMID: 37488824 DOI: 10.1093/micmic/ozad068] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 03/15/2023] [Accepted: 05/25/2023] [Indexed: 07/26/2023]
Abstract
One approach to three-dimensional structure determination using the wealth of scattering data in four-dimensional (4D) scanning transmission electron microscopy (STEM) is the parallax method proposed by Ophus et al. (2019. Advanced phase reconstruction methods enabled by 4D scanning transmission electron microscopy, Microsc Microanal25, 10-11), which determines the scattering matrix and uses it to synthesize a virtual depth-sectioning reconstruction of the sample structure. Drawing on an equivalence with a hypothetical confocal imaging mode, we derive contrast transfer and point spread functions for this parallax method applied to weakly scattering objects, showing them identical to earlier depth-sectioning STEM modes when only bright field signal is used, but that improved depth resolution is possible if dark field signal can be used. Through a simulation-based study of doped Si, we show that this depth resolution is preserved for thicker samples, explore the impact of shot noise on the parallax reconstructions, discuss challenges to making use of dark field signal, and identify cases where the interpretation of the parallax reconstruction breaks down.
Collapse
Affiliation(s)
| | - T C Petersen
- School of Physics and Astronomy, Monash University, Melbourne, VIC 3800, Australia
- Monash Centre for Electron Microscopy, Monash University, Melbourne, VIC 3800, Australia
| | - H G Brown
- Ian Holmes Imaging Center, Bio21 Molecular Science and Biotechnology Institute, University of Melbourne, Melbourne, VIC 3052, Australia
| | - P M Pelz
- Institute of Micro- and Nanostructure Research and Center for Nanoanalysis and Electron Microscopy, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Bavaria 91058, Germany
| | - C Ophus
- National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA
| | - S D Findlay
- School of Physics and Astronomy, Monash University, Melbourne, VIC 3800, Australia
| |
Collapse
|
4
|
Sadri A, Findlay SD. Determining the Projected Crystal Structure from Four-dimensional Scanning Transmission Electron Microscopy via the Scattering Matrix. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2023; 29:967-982. [PMID: 37749695 DOI: 10.1093/micmic/ozad018] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Revised: 01/15/2023] [Accepted: 02/05/2023] [Indexed: 09/27/2023]
Abstract
We present a gradient-descent-based approach to determining the projected electrostatic potential from four-dimensional scanning transmission electron microscopy measurements of a periodic, crystalline material even when dynamical scattering occurs. The method solves for the scattering matrix as an intermediate step, but overcomes the so-called truncation problem that limited previous scattering-matrix-based projected structure determination methods. Gradient descent is made efficient by using analytic expressions for the gradients. Through simulated case studies, we show that iteratively improving the scattering matrix determination can significantly improve the accuracy of the projected structure determination.
Collapse
Affiliation(s)
- Alireza Sadri
- School of Physics and Astronomy, Monash University, Clayton, Victoria 3800, Australia
| | | |
Collapse
|
5
|
Myint P, Chu M, Tripathi A, Wojcik MJ, Zhou J, Cherukara MJ, Narayanan S, Wang J, Jiang Z. Multislice forward modeling of coherent surface scattering imaging on surface and interfacial structures. OPTICS EXPRESS 2023; 31:11261-11273. [PMID: 37155766 DOI: 10.1364/oe.481401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
To study nanostructures on substrates, surface-sensitive reflection-geometry scattering techniques such as grazing incident small angle X-ray scattering are commonly used to yield an averaged statistical structural information of the surface sample. Grazing incidence geometry can probe the absolute three-dimensional structural morphology of the sample if a highly coherent beam is used. Coherent surface scattering imaging (CSSI) is a powerful yet non-invasive technique similar to coherent X-ray diffractive imaging (CDI) but performed at small angles and grazing-incidence reflection geometry. A challenge with CSSI is that conventional CDI reconstruction techniques cannot be directly applied to CSSI because the Fourier-transform-based forward models cannot reproduce the dynamical scattering phenomenon near the critical angle of total external reflection of the substrate-supported samples. To overcome this challenge, we have developed a multislice forward model which can successfully simulate the dynamical or multi-beam scattering generated from surface structures and the underlying substrate. The forward model is also demonstrated to be able to reconstruct an elongated 3D pattern from a single shot scattering image in the CSSI geometry through fast-performing CUDA-assisted PyTorch optimization with automatic differentiation.
Collapse
|
6
|
Guzzi F, Gianoncelli A, Billè F, Carrato S, Kourousias G. Automatic Differentiation for Inverse Problems in X-ray Imaging and Microscopy. Life (Basel) 2023; 13:life13030629. [PMID: 36983785 PMCID: PMC10051220 DOI: 10.3390/life13030629] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Revised: 02/17/2023] [Accepted: 02/20/2023] [Indexed: 03/06/2023] Open
Abstract
Computational techniques allow breaking the limits of traditional imaging methods, such as time restrictions, resolution, and optics flaws. While simple computational methods can be enough for highly controlled microscope setups or just for previews, an increased level of complexity is instead required for advanced setups, acquisition modalities or where uncertainty is high; the need for complex computational methods clashes with rapid design and execution. In all these cases, Automatic Differentiation, one of the subtopics of Artificial Intelligence, may offer a functional solution, but only if a GPU implementation is available. In this paper, we show how a framework built to solve just one optimisation problem can be employed for many different X-ray imaging inverse problems.
Collapse
Affiliation(s)
- Francesco Guzzi
- Elettra—Sincrotrone Trieste, Strada Statale 14—km 163,500 in AREA Science Park, Basovizza, 34149 Trieste, Italy
- Correspondence:
| | - Alessandra Gianoncelli
- Elettra—Sincrotrone Trieste, Strada Statale 14—km 163,500 in AREA Science Park, Basovizza, 34149 Trieste, Italy
| | - Fulvio Billè
- Elettra—Sincrotrone Trieste, Strada Statale 14—km 163,500 in AREA Science Park, Basovizza, 34149 Trieste, Italy
| | - Sergio Carrato
- Department of Engineering and Architecture (DIA), University of Trieste, 34127 Trieste, Italy
| | - George Kourousias
- Elettra—Sincrotrone Trieste, Strada Statale 14—km 163,500 in AREA Science Park, Basovizza, 34149 Trieste, Italy
| |
Collapse
|
7
|
Wang T, Jiang S, Song P, Wang R, Yang L, Zhang T, Zheng G. Optical ptychography for biomedical imaging: recent progress and future directions [Invited]. BIOMEDICAL OPTICS EXPRESS 2023; 14:489-532. [PMID: 36874495 PMCID: PMC9979669 DOI: 10.1364/boe.480685] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Revised: 12/10/2022] [Accepted: 12/10/2022] [Indexed: 05/25/2023]
Abstract
Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.
Collapse
Affiliation(s)
- Tianbo Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
- These authors contributed equally to this work
| | - Shaowei Jiang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
- These authors contributed equally to this work
| | - Pengming Song
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
- These authors contributed equally to this work
| | - Ruihai Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
| | - Liming Yang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
| | - Terrance Zhang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
| | - Guoan Zheng
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT 06269, USA
| |
Collapse
|
8
|
Kharitonov K, Mehrjoo M, Ruiz-Lopez M, Keitel B, Kreis S, Gang SG, Pan R, Marras A, Correa J, Wunderer CB, Plönjes E. Single-shot ptychography at a soft X-ray free-electron laser. Sci Rep 2022; 12:14430. [PMID: 36002577 PMCID: PMC9402553 DOI: 10.1038/s41598-022-18605-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 08/16/2022] [Indexed: 11/05/2022] Open
Abstract
In this work, single-shot ptychography was adapted to the XUV range and, as a proof of concept, performed at the free-electron laser FLASH at DESY to obtain a high-resolution reconstruction of a test sample. Ptychography is a coherent diffraction imaging technique capable of imaging extended samples with diffraction-limited resolution. However, its scanning nature makes ptychography time-consuming and also prevents its application for mapping of dynamical processes. Single-shot ptychography can be realized by collecting the diffraction patterns of multiple overlapping beams in one shot and, in recent years, several concepts based on two con-focal lenses were employed in the visible regime. Unfortunately, this approach cannot be extended straightforwardly to X-ray wavelengths due to the use of refractive optics. Here, a novel single-shot ptychography setup utilizes a combination of X-ray focusing optics with a two-dimensional beam-splitting diffraction grating. It facilitates single-shot imaging of extended samples at X-ray wavelengths.
Collapse
Affiliation(s)
- Konstantin Kharitonov
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Masoud Mehrjoo
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Mabel Ruiz-Lopez
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Barbara Keitel
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Svea Kreis
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Seung-gi Gang
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Rui Pan
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Alessandro Marras
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany ,grid.7683.a0000 0004 0492 0453Center for Free-Electron Laser Science CFEL, Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Jonathan Correa
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany ,grid.7683.a0000 0004 0492 0453Center for Free-Electron Laser Science CFEL, Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Cornelia B. Wunderer
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany ,grid.7683.a0000 0004 0492 0453Center for Free-Electron Laser Science CFEL, Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | - Elke Plönjes
- grid.7683.a0000 0004 0492 0453Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| |
Collapse
|
9
|
Gureyev TE, Paganin DM, Brown HG, Quiney HM, Allen LJ. A Method for High-Resolution Three-Dimensional Reconstruction with Ewald Sphere Curvature Correction from Transmission Electron Images. MICROSCOPY AND MICROANALYSIS : THE OFFICIAL JOURNAL OF MICROSCOPY SOCIETY OF AMERICA, MICROBEAM ANALYSIS SOCIETY, MICROSCOPICAL SOCIETY OF CANADA 2022; 28:1-17. [PMID: 35485646 DOI: 10.1017/s1431927622000630] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
A method for three-dimensional reconstruction of objects from defocused images collected at multiple illumination directions in high-resolution transmission electron microscopy is presented. The method effectively corrects for the Ewald sphere curvature by taking into account the in-particle propagation of the electron beam. Numerical simulations demonstrate that the proposed method is capable of accurately reconstructing biological molecules or nanoparticles from high-resolution defocused images under conditions achievable in single-particle electron cryo-microscopy or electron tomography with realistic radiation doses, non-trivial aberrations, multiple scattering, and other experimentally relevant factors. The physics of the method is based on the well-known Diffraction Tomography formalism, but with the phase-retrieval step modified to include a conjugation of the phase (i.e., multiplication of the phase by a negative constant). At each illumination direction, numerically backpropagating the beam with the conjugated phase produces maximum contrast at the location of individual atoms in the molecule or nanoparticle. The resultant algorithm, Conjugated Holographic Reconstruction, can potentially be incorporated into established software tools for single-particle analysis, such as, for example, RELION or FREALIGN, in place of the conventional contrast transfer function correction procedure, in order to account for the Ewald sphere curvature and improve the spatial resolution of the three-dimensional reconstruction.
Collapse
Affiliation(s)
- Timur E Gureyev
- ARC Centre in Advanced Molecular Imaging, School of Physics, The University of Melbourne, Parkville, VIC3010, Australia
- School of Physics and Astronomy, Monash University, Clayton, VIC3800, Australia
| | - David M Paganin
- School of Physics and Astronomy, Monash University, Clayton, VIC3800, Australia
| | - Hamish G Brown
- ARC Centre in Advanced Molecular Imaging, School of Physics, The University of Melbourne, Parkville, VIC3010, Australia
| | - Harry M Quiney
- ARC Centre in Advanced Molecular Imaging, School of Physics, The University of Melbourne, Parkville, VIC3010, Australia
| | - Leslie J Allen
- ARC Centre in Advanced Molecular Imaging, School of Physics, The University of Melbourne, Parkville, VIC3010, Australia
| |
Collapse
|
10
|
A Parameter Refinement Method for Ptychography Based on Deep Learning Concepts. CONDENSED MATTER 2021. [DOI: 10.3390/condmat6040036] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
X-ray ptychography is an advanced computational microscopy technique, which is delivering exceptionally detailed quantitative imaging of biological and nanotechnology specimens, which can be used for high-precision X-ray measurements. However, coarse parametrisation in propagation distance, position errors and partial coherence frequently threaten the experimental viability. In this work, we formally introduce these actors, solving the whole reconstruction as an optimisation problem. A modern deep learning framework was used to autonomously correct the setup incoherences, thus improving the quality of a ptychography reconstruction. Automatic procedures are indeed crucial to reduce the time for a reliable analysis, which has a significant impact on all the fields that use this kind of microscopy. We implemented our algorithm in our software framework, SciComPty, releasing it as open-source. We tested our system on both synthetic datasets, as well as on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
Collapse
|
11
|
Kandel S, Maddali S, Nashed YSG, Hruszkewycz SO, Jacobsen C, Allain M. Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm. OPTICS EXPRESS 2021; 29:23019-23055. [PMID: 34614577 PMCID: PMC8327924 DOI: 10.1364/oe.422768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 06/04/2021] [Accepted: 06/04/2021] [Indexed: 06/13/2023]
Abstract
The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained "blind" ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.
Collapse
Affiliation(s)
- Saugat Kandel
- Applied Physics, Northwestern University, Evanston, Illinois 60208, USA
| | - S. Maddali
- Materials Science Division, Argonne National Laboratory, Lemont, IL 60439, USA
| | | | | | - Chris Jacobsen
- Advanced Photon Source, Argonne National Laboratory, Lemont, Illinois 60439, USA
- Department of Physics & Astronomy, Northwestern University, Evanston, Illinois 60208, USA
- Chemistry of Life Processes Institute, Northwestern University, Evanston, Illinois 60208, USA
| | - Marc Allain
- Aix Marseille Univ, CNRS, Centrale Marseille, Institut Fresnel, Marseille, France
| |
Collapse
|
12
|
Du M, Huang X, Jacobsen C. Using a modified double deep image prior for crosstalk mitigation in multislice ptychography. JOURNAL OF SYNCHROTRON RADIATION 2021; 28:1137-1145. [PMID: 34212877 PMCID: PMC8284408 DOI: 10.1107/s1600577521003507] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Accepted: 04/01/2021] [Indexed: 06/13/2023]
Abstract
Multislice ptychography is a high-resolution microscopy technique used to image multiple separate axial planes using a single illumination direction. However, multislice ptychography reconstructions are often degraded by crosstalk, where some features on one plane erroneously contribute to the reconstructed image of another plane. Here, the use of a modified `double deep image prior' (DDIP) architecture is demonstrated in mitigating crosstalk artifacts in multislice ptychography. Utilizing the tendency of generative neural networks to produce natural images, a modified DDIP method yielded good results on experimental data. For one of the datasets, it is shown that using DDIP could remove the need of using additional experimental data, such as from X-ray fluorescence, to suppress the crosstalk. This method may help X-ray multislice ptychography work for more general experimental scenarios.
Collapse
Affiliation(s)
- Ming Du
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
| | - Xiaojing Huang
- National Synchrotron Light Source II, Brookhaven National Laboratory, Upton, NY 11973, USA
| | - Chris Jacobsen
- Advanced Photon Source, Argonne National Laboratory, Argonne, IL 60439, USA
- Department of Physics and Astronomy, Northwestern University, Evanston, IL 60208, USA
- Chemistry of Life Processes Institute, Northwestern University, Evanston, IL 60208, USA
| |
Collapse
|
13
|
Affiliation(s)
- Lisandro Dalcin
- King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia
| | | |
Collapse
|
14
|
Ratner D, Christie F, Cryan JP, Edelen A, Lutman A, Zhang X. Recovering the phase and amplitude of X-ray FEL pulses using neural networks and differentiable models. OPTICS EXPRESS 2021; 29:20336-20352. [PMID: 34266125 DOI: 10.1364/oe.432488] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 05/28/2021] [Indexed: 06/13/2023]
Abstract
Dynamics experiments are an important use-case for X-ray free-electron lasers (XFELs), but time-domain measurements of the X-ray pulses themselves remain a challenge. Shot-by-shot X-ray diagnostics could enable a new class of simpler and potentially higher-resolution pump-probe experiments. Here, we report training neural networks to combine low-resolution measurements in both the time and frequency domains to recover X-ray pulses at high-resolution. Critically, we also recover the phase, opening the door to coherent-control experiments with XFELs. The model-based generative neural-network architecture can be trained directly on unlabeled experimental data and is fast enough for real-time analysis on the new generation of MHz XFELs.
Collapse
|