Tomographic reconstruction with a generative adversarial network

Prior frequency guided diffusion model for limited angle (LA)-CBCT reconstruction

Objective.Cone-beam computed tomography (CBCT) is widely used in image-guided radiotherapy. Reconstructing CBCTs from limited-angle acquisitions (LA-CBCT) is highly desired for improved imaging efficiency, dose reduction, and better mechanical clearance. LA-CBCT reconstruction, however, suffers from severe under-sampling artifacts, making it a highly ill-posed inverse problem. Diffusion models can generate data/images by reversing a data-noising process through learned data distributions; and can be incorporated as a denoiser/regularizer in LA-CBCT reconstruction. In this study, we developed a diffusion model-based framework, prior frequency-guided diffusion model (PFGDM), for robust and structure-preserving LA-CBCT reconstruction.Approach.PFGDM uses a conditioned diffusion model as a regularizer for LA-CBCT reconstruction, and the condition is based on high-frequency information extracted from patient-specific prior CT scans which provides a strong anatomical prior for LA-CBCT reconstruction. Specifically, we developed two variants of PFGDM (PFGDM-A and PFGDM-B) with different conditioning schemes. PFGDM-A applies the high-frequency CT information condition until a pre-optimized iteration step, and drops it afterwards to enable both similar and differing CT/CBCT anatomies to be reconstructed. PFGDM-B, on the other hand, continuously applies the prior CT information condition in every reconstruction step, while with a decaying mechanism, to gradually phase out the reconstruction guidance from the prior CT scans. The two variants of PFGDM were tested and compared with current available LA-CBCT reconstruction solutions, via metrics including peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).Main results.PFGDM outperformed all traditional and diffusion model-based methods. The mean(s.d.) PSNR/SSIM were 27.97(3.10)/0.949(0.027), 26.63(2.79)/0.937(0.029), and 23.81(2.25)/0.896(0.036) for PFGDM-A, and 28.20(1.28)/0.954(0.011), 26.68(1.04)/0.941(0.014), and 23.72(1.19)/0.894(0.034) for PFGDM-B, based on 120°, 90°, and 30° orthogonal-view scan angles respectively. In contrast, the PSNR/SSIM was 19.61(2.47)/0.807(0.048) for 30° for DiffusionMBIR, a diffusion-based method without prior CT conditioning.Significance. PFGDM reconstructs high-quality LA-CBCTs under very-limited gantry angles, allowing faster and more flexible CBCT scans with dose reductions.

Recent developments in X-ray diffraction/scattering computed tomography for materials science

X-ray diffraction/scattering computed tomography (XDS-CT) methods are a non-destructive class of chemical imaging techniques that have the capacity to provide reconstructions of sample cross-sections with spatially resolved chemical information. While X-ray diffraction CT (XRD-CT) is the most well-established method, recent advances in instrumentation and data reconstruction have seen greater use of related techniques like small angle X-ray scattering CT and pair distribution function CT. Additionally, the adoption of machine learning techniques for tomographic reconstruction and data analysis are fundamentally disrupting how XDS-CT data is processed. The following narrative review highlights recent developments and applications of XDS-CT with a focus on studies in the last five years. This article is part of the theme issue 'Exploring the length scales, timescales and chemistry of challenging materials (Part 2)'.

Deep learning enables nanoscale X-ray 3D imaging with limited data

Deep neural network can greatly improve tomography reconstruction with limited data. A recent effort of combining ptycho-tomography model with the 3D U-net demonstrated a significant reduction in both the number of projections and computation time, and showed its potential for integrated circuit imaging that requires high-resolution and fast measurement speed.

Quantum machine learning for chemistry and physics

Machine learning (ML) has emerged as a formidable force for identifying hidden but pertinent patterns within a given data set with the objective of subsequent generation of automated predictive behavior. In recent years, it is safe to conclude that ML and its close cousin, deep learning (DL), have ushered in unprecedented developments in all areas of physical sciences, especially chemistry. Not only classical variants of ML, even those trainable on near-term quantum hardwares have been developed with promising outcomes. Such algorithms have revolutionized materials design and performance of photovoltaics, electronic structure calculations of ground and excited states of correlated matter, computation of force-fields and potential energy surfaces informing chemical reaction dynamics, reactivity inspired rational strategies of drug designing and even classification of phases of matter with accurate identification of emergent criticality. In this review we shall explicate a subset of such topics and delineate the contributions made by both classical and quantum computing enhanced machine learning algorithms over the past few years. We shall not only present a brief overview of the well-known techniques but also highlight their learning strategies using statistical physical insight. The objective of the review is not only to foster exposition of the aforesaid techniques but also to empower and promote cross-pollination among future research in all areas of chemistry which can benefit from ML and in turn can potentially accelerate the growth of such algorithms.

Classification of diffraction patterns using a convolutional neural network in single-particle-imaging experiments performed at X-ray free-electron lasers

Single particle imaging (SPI) at X-ray free-electron lasers is particularly well suited to determining the 3D structure of particles at room temperature. For a successful reconstruction, diffraction patterns originating from a single hit must be isolated from a large number of acquired patterns. It is proposed that this task could be formulated as an image-classification problem and solved using convolutional neural network (CNN) architectures. Two CNN configurations are developed: one that maximizes the F1 score and one that emphasizes high recall. The CNNs are also combined with expectation-maximization (EM) selection as well as size filtering. It is observed that the CNN selections have lower contrast in power spectral density functions relative to the EM selection used in previous work. However, the reconstruction of the CNN-based selections gives similar results. Introducing CNNs into SPI experiments allows the reconstruction pipeline to be streamlined, enables researchers to classify patterns on the fly, and, as a consequence, enables them to tightly control the duration of their experiments. Incorporating non-standard artificial-intelligence-based solutions into an existing SPI analysis workflow may be beneficial for the future development of SPI experiments.

Adversarial Resolution Enhancement for Electrical Capacitance Tomography Image Reconstruction

High-quality image reconstruction is essential for many electrical capacitance tomography (CT) applications. Raw capacitance measurements are used in the literature to generate low-resolution images. However, such low-resolution images are not sufficient for proper functionality of most systems. In this paper, we propose a novel adversarial resolution enhancement (ARE-ECT) model to reconstruct high-resolution images of inner distributions based on low-quality initial images, which are generated from the capacitance measurements. The proposed model uses a UNet as the generator of a conditional generative adversarial network (CGAN). The generator’s input is set to the low-resolution image rather than the typical random input signal. Additionally, the CGAN is conditioned by the input low-resolution image itself. For evaluation purposes, a massive ECT dataset of 320 K synthetic image–measurement pairs was created. This dataset is used for training, validating, and testing the proposed model. New flow patterns, which are not exposed to the model during the training phase, are used to evaluate the feasibility and generalization ability of the ARE-ECT model. The superiority of ARE-ECT, in the efficient generation of more accurate ECT images than traditional and other deep learning-based image reconstruction algorithms, is proved by the evaluation results. The ARE-ECT model achieved an average image correlation coefficient of more than 98.8% and an average relative image error about 0.1%.

Unsupervised Deep Learning Methods for Biological Image Reconstruction and Enhancement: An overview from a signal processing perspective

Recently, deep learning approaches have become the main research frontier for biological image reconstruction and enhancement problems thanks to their high performance, along with their ultra-fast inference times. However, due to the difficulty of obtaining matched reference data for supervised learning, there has been increasing interest in unsupervised learning approaches that do not need paired reference data. In particular, self-supervised learning and generative models have been successfully used for various biological imaging applications. In this paper, we overview these approaches from a coherent perspective in the context of classical inverse problems, and discuss their applications to biological imaging, including electron, fluorescence and deconvolution microscopy, optical diffraction tomography and functional neuroimaging.

Foam-like phantoms for comparing tomography algorithms

Tomographic algorithms are often compared by evaluating them on certain benchmark datasets. For fair comparison, these datasets should ideally (i) be challenging to reconstruct, (ii) be representative of typical tomographic experiments, (iii) be flexible to allow for different acquisition modes, and (iv) include enough samples to allow for comparison of data-driven algorithms. Current approaches often satisfy only some of these requirements, but not all. For example, real-world datasets are typically challenging and representative of a category of experimental examples, but are restricted to the acquisition mode that was used in the experiment and are often limited in the number of samples. Mathematical phantoms are often flexible and can sometimes produce enough samples for data-driven approaches, but can be relatively easy to reconstruct and are often not representative of typical scanned objects. In this paper, we present a family of foam-like mathematical phantoms that aims to satisfy all four requirements simultaneously. The phantoms consist of foam-like structures with more than 100000 features, making them challenging to reconstruct and representative of common tomography samples. Because the phantoms are computer-generated, varying acquisition modes and experimental conditions can be simulated. An effectively unlimited number of random variations of the phantoms can be generated, making them suitable for data-driven approaches. We give a formal mathematical definition of the foam-like phantoms, and explain how they can be generated and used in virtual tomographic experiments in a computationally efficient way. In addition, several 4D extensions of the 3D phantoms are given, enabling comparisons of algorithms for dynamic tomography. Finally, example phantoms and tomographic datasets are given, showing that the phantoms can be effectively used to make fair and informative comparisons between tomography algorithms.

Quantum State Tomography with Conditional Generative Adversarial Networks

Quantum state tomography (QST) is a challenging task in intermediate-scale quantum devices. Here, we apply conditional generative adversarial networks (CGANs) to QST. In the CGAN framework, two dueling neural networks, a generator and a discriminator, learn multimodal models from data. We augment a CGAN with custom neural-network layers that enable conversion of output from any standard neural network into a physical density matrix. To reconstruct the density matrix, the generator and discriminator networks train each other on data using standard gradient-based methods. We demonstrate that our QST-CGAN reconstructs optical quantum states with high fidelity, using orders of magnitude fewer iterative steps, and less data, than both accelerated projected-gradient-based and iterative maximum-likelihood estimation. We also show that the QST-CGAN can reconstruct a quantum state in a single evaluation of the generator network if it has been pretrained on similar quantum states.

Artificial Intelligence Applied to Battery Research: Hype or Reality?

This is a critical review of artificial intelligence/machine learning (AI/ML) methods applied to battery research. It aims at providing a comprehensive, authoritative, and critical, yet easily understandable, review of general interest to the battery community. It addresses the concepts, approaches, tools, outcomes, and challenges of using AI/ML as an accelerator for the design and optimization of the next generation of batteries—a current hot topic. It intends to create both accessibility of these tools to the chemistry and electrochemical energy sciences communities and completeness in terms of the different battery R&D aspects covered.

3D single-shot ptychography with highly tilted illuminations

A method based on highly tilted illumination and non-paraxial iterative computation is proposed to improve the image quality of single-shot 3D ptychography. A thick sample is illuminated with a cluster of laser beams that are separated by large enough angles to record each diffraction pattern distinctly in a single exposure. 3D structure of the thick sample is accurately reconstructed from recorded diffraction patterns using a modified multi-slice algorithm to process non-paraxial illumination. Sufficient number of recorded diffraction patterns with noticeably low crosstalk enhances the fidelity of reconstruction significantly over single-shot 3D ptychography methods that are based on paraxial illumination. Experimental observations guided by the results of numerical simulations show the feasibility of the proposed method.

Limited-angle computed tomography with deep image and physics priors

Computed tomography is a well-established x-ray imaging technique to reconstruct the three-dimensional structure of objects. It has been used extensively in a variety of fields, from diagnostic imaging to materials and biological sciences. One major challenge in some applications, such as in electron or x-ray tomography systems, is that the projections cannot be gathered over all the angles due to the sample holder setup or shape of the sample. This results in an ill-posed problem called the limited angle reconstruction problem. Typical image reconstruction in this setup leads to distortion and artifacts, thereby hindering a quantitative evaluation of the results. To address this challenge, we use a generative model to effectively constrain the solution of a physics-based approach. Our approach is self-training that can iteratively learn the nonlinear mapping from partial projections to the scanned object. Because our approach combines the data likelihood and image prior terms into a single deep network, it is computationally tractable and improves performance through an end-to-end training. We also complement our approach with total-variation regularization to handle high-frequency noise in reconstructions and implement a solver based on alternating direction method of multipliers. We present numerical results for various degrees of missing angle range and noise levels, which demonstrate the effectiveness of the proposed approach.

Classification of diffraction patterns in single particle imaging experiments performed at x-ray free-electron lasers using a convolutional neural network

Single particle imaging (SPI) is a promising method of native structure determination, which has undergone fast progress with the development of x-ray free-electron lasers. Large amounts of data are collected during SPI experiments, driving the need for automated data analysis. The necessary data analysis pipeline has a number of steps including binary object classification (single versus non-single hits). Classification and object detection are areas where deep neural networks currently outperform other approaches. In this work, we use the fast object detector networks YOLOv2 and YOLOv3. By exploiting transfer learning, a moderate amount of data is sufficient to train the neural network. We demonstrate here that a convolutional neural network can be successfully used to classify data from SPI experiments. We compare the results of classification for the two different networks, with different depth and architecture, by applying them to the same SPI data with different data representation. The best results are obtained for diffracted intensity represented by color images on a linear scale using YOLOv2 for classification. It shows an accuracy of about 95% with precision and recall of about 50% and 60%, respectively, in comparison to manual data classification.

FleXCT: a flexible X-ray CT scanner with 10 degrees of freedom

Laboratory based X-ray micro-CT is a non-destructive testing method that enables three dimensional visualization and analysis of the internal and external morphology of samples. Although a wide variety of commercial scanners exist, most of them are limited in the number of degrees of freedom to position the source and detector with respect to the object to be scanned. Hence, they are less suited for industrial X-ray imaging settings that require advanced scanning modes, such as laminography, conveyor belt scanning, or time-resolved imaging (4DCT). We introduce a new X-ray scanner FleXCT that consists of a total of ten motorized axes, which allow a wide range of non-standard XCT scans such as tiled and off-centre scans, laminography, helical tomography, conveyor belt, dynamic zooming, and X-ray phase contrast imaging. Additionally, a new software tool 'FlexRayTools' was created that enables reconstruction of non-standard XCT projection data of the FleXCT instrument using the ASTRA Toolbox, a highly efficient and open source set of tools for tomographic projection and reconstruction.

PtyNAMi: ptychographic nano-analytical microscope

Ptychographic X-ray imaging at the highest spatial resolution requires an optimal experimental environment, providing a high coherent flux, excellent mechanical stability and a low background in the measured data. This requires, for example, a stable performance of all optical components along the entire beam path, high temperature stability, a robust sample and optics tracking system, and a scatter-free environment. This contribution summarizes the efforts along these lines to transform the nanoprobe station on beamline P06 (PETRA III) into the ptychographic nano-analytical microscope (PtyNAMi).