A streaming multi-GPU implementation of image simulation algorithms for scanning transmission electron microscopy

Revealing the three-dimensional arrangement of polar topology in nanoparticles

In the early 2000s, low dimensional ferroelectric systems were predicted to have topologically nontrivial polar structures, such as vortices or skyrmions, depending on mechanical or electrical boundary conditions. A few variants of these structures have been experimentally observed in thin film model systems, where they are engineered by balancing electrostatic charge and elastic distortion energies. However, the measurement and classification of topological textures for general ferroelectric nanostructures have remained elusive, as it requires mapping the local polarization at the atomic scale in three dimensions. Here we unveil topological polar structures in ferroelectric BaTiO3 nanoparticles via atomic electron tomography, which enables us to reconstruct the full three-dimensional arrangement of cation atoms at an individual atom level. Our three-dimensional polarization maps reveal clear topological orderings, along with evidence of size-dependent topological transitions from a single vortex structure to multiple vortices, consistent with theoretical predictions. The discovery of the predicted topological polar ordering in nanoscale ferroelectrics, independent of epitaxial strain, widens the research perspective and offers potential for practical applications utilizing contact-free switchable toroidal moments.

Giant energy storage and power density negative capacitance superlattices

Dielectric electrostatic capacitors1, because of their ultrafast charge-discharge, are desirable for high-power energy storage applications. Along with ultrafast operation, on-chip integration can enable miniaturized energy storage devices for emerging autonomous microelectronics and microsystems2-5. Moreover, state-of-the-art miniaturized electrochemical energy storage systems-microsupercapacitors and microbatteries-currently face safety, packaging, materials and microfabrication challenges preventing on-chip technological readiness2,3,6, leaving an opportunity for electrostatic microcapacitors. Here we report record-high electrostatic energy storage density (ESD) and power density, to our knowledge, in HfO2-ZrO2-based thin film microcapacitors integrated into silicon, through a three-pronged approach. First, to increase intrinsic energy storage, atomic-layer-deposited antiferroelectric HfO2-ZrO2 films are engineered near a field-driven ferroelectric phase transition to exhibit amplified charge storage by the negative capacitance effect7-12, which enhances volumetric ESD beyond the best-known back-end-of-the-line-compatible dielectrics (115 J cm-3) (ref. 13). Second, to increase total energy storage, antiferroelectric superlattice engineering14 scales the energy storage performance beyond the conventional thickness limitations of HfO2-ZrO2-based (anti)ferroelectricity15 (100-nm regime). Third, to increase the storage per footprint, the superlattices are conformally integrated into three-dimensional capacitors, which boosts the areal ESD nine times and the areal power density 170 times that of the best-known electrostatic capacitors: 80 mJ cm-2 and 300 kW cm-2, respectively. This simultaneous demonstration of ultrahigh energy density and power density overcomes the traditional capacity-speed trade-off across the electrostatic-electrochemical energy storage hierarchy1,16. Furthermore, the integration of ultrahigh-density and ultrafast-charging thin films within a back-end-of-the-line-compatible process enables monolithic integration of on-chip microcapacitors5, which can unlock substantial energy storage and power delivery performance for electronic microsystems17-19.

InFluence: An Open-Source Python Package to Model Images Captured with Direct Electron Detectors

The high detection efficiencies of direct electron detectors facilitate the routine collection of low fluence electron micrographs and diffraction patterns. Low dose and low fluence electron microscopy experiments are the only practical way to acquire useful data from beam sensitive pharmaceutical and biological materials. Appropriate modeling of low fluence images acquired using direct electron detectors is, therefore, paramount for quantitative analysis of the experimental images. We have developed a new open-source Python package to accurately model any single layer direct electron detector for low and high fluence imaging conditions, including a means to validate against experimental data through computation of modulation transfer function and detective quantum efficiency.

Denoising Autoencoder Trained on Simulation-Derived Structures for Noise Reduction in Chromatin Scanning Transmission Electron Microscopy

Scanning transmission electron microscopy tomography with ChromEM staining (ChromSTEM), has allowed for the three-dimensional study of genome organization. By leveraging convolutional neural networks and molecular dynamics simulations, we have developed a denoising autoencoder (DAE) capable of postprocessing experimental ChromSTEM images to provide nucleosome-level resolution. Our DAE is trained on synthetic images generated from simulations of the chromatin fiber using the 1-cylinder per nucleosome (1CPN) model of chromatin. We find that our DAE is capable of removing noise commonly found in high-angle annular dark field (HAADF) STEM experiments and is able to learn structural features driven by the physics of chromatin folding. The DAE outperforms other well-known denoising algorithms without degradation of structural features and permits the resolution of α-tetrahedron tetranucleosome motifs that induce local chromatin compaction and mediate DNA accessibility. Notably, we find no evidence for the 30 nm fiber, which has been suggested to serve as the higher-order structure of the chromatin fiber. This approach provides high-resolution STEM images that allow for the resolution of single nucleosomes and organized domains within chromatin dense regions comprising of folding motifs that modulate the accessibility of DNA to external biological machinery.

CyRSoXS: a GPU-accelerated virtual instrument for polarized resonant soft X-ray scattering

Polarized resonant soft X-ray scattering (P-RSoXS) has emerged as a powerful synchrotron-based tool that combines the principles of X-ray scattering and X-ray spectroscopy. P-RSoXS provides unique sensitivity to molecular orientation and chemical heterogeneity in soft materials such as polymers and biomaterials. Quantitative extraction of orientation information from P-RSoXS pattern data is challenging, however, because the scattering processes originate from sample properties that must be represented as energy-dependent three-dimensional tensors with heterogeneities at nanometre to sub-nanometre length scales. This challenge is overcome here by developing an open-source virtual instrument that uses graphical processing units (GPUs) to simulate P-RSoXS patterns from real-space material representations with nanoscale resolution. This computational framework - called CyRSoXS (https://github.com/usnistgov/cyrsoxs) - is designed to maximize GPU performance, including algorithms that minimize both communication and memory footprints. The accuracy and robustness of the approach are demonstrated by validating against an extensive set of test cases, which include both analytical solutions and numerical comparisons, demonstrating an acceleration of over three orders of magnitude relative to the current state-of-the-art P-RSoXS simulation software. Such fast simulations open up a variety of applications that were previously computationally unfeasible, including pattern fitting, co-simulation with the physical instrument for operando analytics, data exploration and decision support, data creation and integration into machine learning workflows, and utilization in multi-modal data assimilation approaches. Finally, the complexity of the computational framework is abstracted away from the end user by exposing CyRSoXS to Python using Pybind. This eliminates input/output requirements for large-scale parameter exploration and inverse design, and democratizes usage by enabling seamless integration with a Python ecosystem (https://github.com/usnistgov/nrss) that can include parametric morphology generation, simulation result reduction, comparison with experiment and data fitting approaches.

Fast generation of calculated ADF-EDX scattering cross-sections under channelling conditions

Advanced materials often consist of multiple elements which are arranged in a complicated structure. Quantitative scanning transmission electron microscopy is useful to determine the composition and thickness of nanostructures at the atomic scale. However, significant difficulties remain to quantify mixed columns by comparing the resulting atomic resolution images and spectroscopy data with multislice simulations where dynamic scattering needs to be taken into account. The combination of the computationally intensive nature of these simulations and the enormous amount of possible mixed column configurations for a given composition indeed severely hamper the quantification process. To overcome these challenges, we here report the development of an incoherent non-linear method for the fast prediction of ADF-EDX scattering cross-sections of mixed columns under channelling conditions. We first explain the origin of the ADF and EDX incoherence from scattering physics suggesting a linear dependence between those two signals in the case of a high-angle ADF detector. Taking EDX as a perfect incoherent reference mode, we quantitatively examine the ADF longitudinal incoherence under different microscope conditions using multislice simulations. Based on incoherent imaging, the atomic lensing model previously developed for ADF is now expanded to EDX, which yields ADF-EDX scattering cross-section predictions in good agreement with multislice simulations for mixed columns in a core-shell nanoparticle and a high entropy alloy. The fast and accurate prediction of ADF-EDX scattering cross-sections opens up new opportunities to explore the wide range of ordering possibilities of heterogeneous materials with multiple elements.

Towards real-time STEM simulations through targeted subsampling strategies

Scanning transmission electron microscopy images can be complex to interpret on the atomic scale as the contrast is sensitive to multiple factors such as sample thickness, composition, defects and aberrations. Simulations are commonly used to validate or interpret real experimental images, but they come at a cost of either long computation times or specialist hardware such as graphics processing units. Recent works in compressive sensing for experimental STEM images have shown that it is possible to significantly reduce the amount of acquired signal and still recover the full image without significant loss of image quality, and therefore it is proposed here that similar methods can be applied to STEM simulations. In this paper, we demonstrate a method that can significantly increase the efficiency of STEM simulations through a targeted sampling strategy, along with a new approach to independently subsample each frozen phonon layer. We show the effectiveness of this method by simulating a SrTiO₃ grain boundary and monolayer 2H-MoS₂ containing a sulphur vacancy using the abTEM software. We also show how this method is not limited to only traditional multislice methods, but also increases the speed of the PRISM simulation method. Furthermore, we discuss the possibility for STEM simulations to seed the acquisition of real data, to potentially lead the way to self-driving (correcting) STEM.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

Direct strain correlations at the single-atom level in three-dimensional core-shell interface structures

Nanomaterials with core-shell architectures are prominent examples of strain-engineered materials. The lattice mismatch between the core and shell materials can cause strong interface strain, which affects the surface structures. Therefore, surface functional properties such as catalytic activities can be designed by fine-tuning the misfit strain at the interface. To precisely control the core-shell effect, it is essential to understand how the surface and interface strains are related at the atomic scale. Here, we elucidate the surface-interface strain relations by determining the full 3D atomic structure of Pd@Pt core-shell nanoparticles at the single-atom level via atomic electron tomography. Full 3D displacement fields and strain profiles of core-shell nanoparticles were obtained, which revealed a direct correlation between the surface and interface strain. The strain distributions show a strong shape-dependent anisotropy, whose nature was further corroborated by molecular statics simulations. From the observed surface strains, the surface oxygen reduction reaction activities were predicted. These findings give a deep understanding of structure-property relationships in strain-engineerable core-shell systems, which can lead to direct control over the resulting catalytic properties.

Understanding 3D interfacial strain at the atomic level has been a long-sought challenge in the field of core-shell nanomaterials. Here, the authors address this challenge by revealing the full 3D atomic structures of Pd@Pt core-shell nanoparticles

Understanding Micro and Atomic Structures of Secondary Phases in Cu-Doped SnTe

Highly efficient thermoelectric materials require, including point defects within the host matrix, secondary phases generating positive effects on lowering lattice thermal conductivity (κ_L ). Amongst effective dopants for a functional thermoelectric material, SnTe, Cu doping realizes the ultra-low κ_L approaching the SnTe amorphous limit. Such effective κ_L reduction is first attributed to strong phonon scattering by substitutional Cu atoms at Sn sites and interstitial defects in the host SnTe. However, other crystallographic defects in secondary phases have been unfocused. Here, this work reports micro- to atomic-scale characterization on secondary phases of Cu-doped SnTe using advanced microscopes. It is found that Cu-rich secondary phases begin precipitation ≈1.7 at% Cu (x = 0.034 where Sn_{1- x} Cu_x Te). The Cu-rich secondary phases encapsulate two distinct solids: Cu₂ SnTe₃ ( F 4 ¯ 3 m $F\bar{4}3m$ ) has semi-coherent interfaces with SnTe ( F m 3 ¯ m $Fm\bar{3}{\rm{m}}$ ) such that they minimize lattice mismatch to favor the thermoelectric transport; the other resembles a stoichiometric Cu₂ Te model, yet is so meta-stable that it demonstrates not only various defects such as dislocation cores and ordered/disordered Cu vacancies, but also dynamic grain-boundary migration with heating and a subsequent phase transition ≈350 °C. The atomic-scale analysis on the Cu-rich secondary phases offers viable strategies for reducing κ_L through Cu addition to SnTe.

SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation

Recently it has been shown that precise dose control and an increase in the overall acquisition speed of atomic resolution scanning transmission electron microscope (STEM) images can be achieved by acquiring only a small fraction of the pixels in the image experimentally and then reconstructing the full image using an inpainting algorithm. In this paper, we apply the same inpainting approach (a form of compressed sensing) to simulated, sub-sampled atomic resolution STEM images. We find that it is possible to significantly sub-sample the area that is simulated, the number of g-vectors contributing the image, and the number of frozen phonon configurations contributing to the final image while still producing an acceptable fit to a fully sampled simulation. Here we discuss the parameters that we use and how the resulting simulations can be quantifiably compared to the full simulations. As with any Compressed Sensing methodology, care must be taken to ensure that isolated events are not excluded from the process, but the observed increase in simulation speed provides significant opportunities for real time simulations, image classification and analytics to be performed as a supplement to experiments on a microscope to be developed in the future.

Scanning Transmission Electron Microscopy in a Scanning Electron Microscope for the High-Throughput Imaging of Biological Assemblies

Electron microscopy of soft and biological materials, or "soft electron microscopy", is essential to the characterization of macromolecules. Soft microscopy is governed by enhancing contrast while maintaining low electron doses, and sample preparation and imaging methodologies are driven by the length scale of features of interest. While cryo-electron microscopy offers the highest resolution, larger structures can be characterized efficiently and with high contrast using low-voltage electron microscopy by performing scanning transmission electron microscopy in a scanning electron microscope (STEM-in-SEM). Here, STEM-in-SEM is demonstrated for a four-lobed protein assembly where the arrangement of the proteins in the construct must be examined. STEM image simulations show the theoretical contrast enhancement at SEM-level voltages for unstained structures, and experimental images with multiple STEM modes exhibit the resolution possible for negative-stained proteins. This technique can be extended to complex protein assemblies, larger structures such as cell sections, and hybrid materials, making STEM-in-SEM a valuable high-throughput imaging method.

ToTEM: A software for fast TEM image simulation

ToTEM, a multislice based image simulation software is developed for transmission electron microscope (TEM). This software implements the following major features: i) capability of assigning three-dimensional potentials of atom into multiple slices and precise introduction of phase shift caused by the sub-pixel atomic position, ii) employing CUDA coding and graphical processing units (GPU) with multi-threading parallel algorithm based on the powerful batch (inverse) fast Fourier transform (FFT), which is especially beneficial for image simulation of scanning transmission electron microscopy (STEM) or (integrated) differential phase contrast (i)DPC, iii) design for efficiently generating large batch of dataset of high resolution transmission electron microscopy (HRTEM) images. Image simulation acceleration for STEM has been verified by simulating a large-scale SrTiO₃ . Additionally, iDPC image of MFI-type zeolites with xylene molecules encapsulated in straight channels demonstrates the advantage of iDPC in detecting light molecules. This article is protected by copyright. All rights reserved.

Emergent ferroelectricity in subnanometer binary oxide films on silicon

The critical size limit of voltage-switchable electric dipoles has extensive implications for energy-efficient electronics, underlying the importance of ferroelectric order stabilized at reduced dimensionality. We report on the thickness-dependent antiferroelectric-to-ferroelectric phase transition in zirconium dioxide (ZrO2) thin films on silicon. The emergent ferroelectricity and hysteretic polarization switching in ultrathin ZrO2, conventionally a paraelectric material, notably persists down to a film thickness of 5 angstroms, the fluorite-structure unit-cell size. This approach to exploit three-dimensional centrosymmetric materials deposited down to the two-dimensional thickness limit, particularly within this model fluorite-structure system that possesses unconventional ferroelectric size effects, offers substantial promise for electronics, demonstrated by proof-of-principle atomic-scale nonvolatile ferroelectric memory on silicon. Additionally, it is also indicative of hidden electronic phenomena that are achievable across a wide class of simple binary materials.

Ingrained: An Automated Framework for Fusing Atomic-Scale Image Simulations into Experiments

To fully leverage the power of image simulation to corroborate and explain patterns and structures in atomic resolution microscopy, an initial correspondence between the simulation and experimental image must be established at the outset of further high accuracy simulations or calculations. Furthermore, if simulation is to be used in context of highly automated processes or high-throughput optimization, the process of finding this correspondence itself must be automated. In this work, "ingrained," an open-source automation framework which solves for this correspondence and fuses atomic resolution image simulations into the experimental images to which they correspond, is introduced. Herein, the overall "ingrained" workflow, focusing on its application to interface structure approximations, and the development of an experimentally rationalized forward model for scanning tunneling microscopy simulation are described.

TopoTEM: A Python Package for Quantifying and Visualizing Scanning Transmission Electron Microscopy Data of Polar Topologies

The exotic internal structure of polar topologies in multiferroic materials offers a rich landscape for materials science research. As the spatial scale of these entities is often subatomic in nature, aberration-corrected transmission electron microscopy (TEM) is the ideal characterization technique. Software to quantify and visualize the slight shifts in atomic placement within unit cells is of paramount importance due to the now routine acquisition of images at such resolution. In the previous ~decade since the commercialization of aberration-corrected TEM, many research groups have written their own code to visualize these polar entities. More recently, open-access Python packages have been developed for the purpose of TEM atomic position quantification. Building on these packages, we introduce the TEMUL Toolkit: a Python package for analysis and visualization of atomic resolution images. Here, we focus specifically on the TopoTEM module of the toolkit where we show an easy to follow, streamlined version of calculating the atomic displacements relative to the surrounding lattice and thus plotting polarization. We hope this toolkit will benefit the rapidly expanding field of topology-based nano-electronic and quantum materials research, and we invite the electron microscopy community to contribute to this open-access project.

ReciPro: free and open-source multipurpose crystallographic software integrating a crystal model database and viewer, diffraction and microscopy simulators, and diffraction data analysis tools

ReciPro is a comprehensive multipurpose crystallographic program equipped with an intuitive graphical user interface (GUI), and it is completely free and open source. This software has a built-in crystal database consisting of over 20 000 crystal models, and the visualization system can seamlessly display a specified crystal model as an attractive three-dimensional graphic. The comprehensive features are not confined to these crystal model databases and viewers. It can smoothly and quantitatively simulate not only single-crystal and/or polycrystalline (powder) diffraction patterns of X-ray, electron and neutron diffraction of a selected crystal model, based on the kinematic scattering theory, but also various electron diffraction patterns and high-resolution transmission electron microscopy (TEM) images, based on the dynamical scattering theory. The features of stereographic projection of crystal planes/axes to explore crystal orientation relationships and the semi-automatic diffraction spot indexing function for experimental diffraction patterns assist diffraction experiments and analyses. These features are linked through a user-friendly GUI, and the results can be synchronously displayed almost in real time. ReciPro will assist a wide range of crystallographers (including beginners) using X-ray, electron and neutron diffraction crystallography and TEM.

Three-dimensional atomic packing in amorphous solids with liquid-like structure

Liquids and solids are two fundamental states of matter. However, our understanding of their three-dimensional atomic structure is mostly based on physical models. Here we use atomic electron tomography to experimentally determine the three-dimensional atomic positions of monatomic amorphous solids, namely a Ta thin film and two Pd nanoparticles. We observe that pentagonal bipyramids are the most abundant atomic motifs in these amorphous materials. Instead of forming icosahedra, the majority of pentagonal bipyramids arrange into pentagonal bipyramid networks with medium-range order. Molecular dynamics simulations further reveal that pentagonal bipyramid networks are prevalent in monatomic metallic liquids, which rapidly grow in size and form more icosahedra during the quench from the liquid to the glass state. These results expand our understanding of the atomic structures of amorphous solids and will encourage future studies on amorphous-crystalline phase and glass transitions in non-crystalline materials with three-dimensional atomic resolution.

Aluminum depletion induced by co-segregation of carbon and boron in a bcc-iron grain boundary

The local variation of grain boundary atomic structure and chemistry caused by segregation of impurities influences the macroscopic properties of polycrystalline materials. Here, the effect of co-segregation of carbon and boron on the depletion of aluminum at a Σ5 (3 1 0 )[0 0 1] tilt grain boundary in a α - Fe-4 at%Al bicrystal is studied by combining atomic resolution scanning transmission electron microscopy, atom probe tomography and density functional theory calculations. The atomic grain boundary structural units mostly resemble kite-type motifs and the structure appears disrupted by atomic scale defects. Atom probe tomography reveals that carbon and boron impurities are co-segregating to the grain boundary reaching levels of >1.5 at%, whereas aluminum is locally depleted by approx. 2 at.%. First-principles calculations indicate that carbon and boron exhibit the strongest segregation tendency and their repulsive interaction with aluminum promotes its depletion from the grain boundary. It is also predicted that substitutional segregation of boron atoms may contribute to local distortions of the kite-type structural units. These results suggest that the co-segregation and interaction of interstitial impurities with substitutional solutes strongly influences grain boundary composition and with this the properties of the interface.

Prismatic 2.0 - Simulation software for scanning and high resolution transmission electron microscopy (STEM and HRTEM)

Scanning transmission electron microscopy (STEM), where a converged electron probe is scanned over a sample's surface and an imaging, diffraction, or spectroscopic signal is measured as a function of probe position, is an extremely powerful tool for materials characterization. The widespread adoption of hardware aberration correction, direct electron detectors, and computational imaging methods have made STEM one of the most important tools for atomic-resolution materials science. Many of these imaging methods rely on accurate imaging and diffraction simulations in order to interpret experimental results. However, STEM simulations have traditionally required large calculation times, as modeling the electron scattering requires a separate simulation for each of the typically millions of probe positions. We have created the Prismatic simulation code for fast simulation of STEM experiments with support for multi-CPU and multi-GPU (graphics processing unit) systems, using both the conventional multislice and our recently-introduced PRISM method. In this paper, we introduce Prismatic version 2.0, which adds many new algorithmic improvements, an updated graphical user interface (GUI), post-processing of simulation data, and additional operating modes such as plane-wave TEM. We review various aspects of the simulation methods and codes in detail and provide various simulation examples. Prismatic 2.0 is freely available both as an open-source package that can be run using a C++ or Python command line interface, or GUI, as well within a Docker container environment.

py4DSTEM: A Software Package for Four-Dimensional Scanning Transmission Electron Microscopy Data Analysis

Scanning transmission electron microscopy (STEM) allows for imaging, diffraction, and spectroscopy of materials on length scales ranging from microns to atoms. By using a high-speed, direct electron detector, it is now possible to record a full two-dimensional (2D) image of the diffracted electron beam at each probe position, typically a 2D grid of probe positions. These 4D-STEM datasets are rich in information, including signatures of the local structure, orientation, deformation, electromagnetic fields, and other sample-dependent properties. However, extracting this information requires complex analysis pipelines that include data wrangling, calibration, analysis, and visualization, all while maintaining robustness against imaging distortions and artifacts. In this paper, we present py4DSTEM, an analysis toolkit for measuring material properties from 4D-STEM datasets, written in the Python language and released with an open-source license. We describe the algorithmic steps for dataset calibration and various 4D-STEM property measurements in detail and present results from several experimental datasets. We also implement a simple and universal file format appropriate for electron microscopy data in py4DSTEM, which uses the open-source HDF5 standard. We hope this tool will benefit the research community and help improve the standards for data and computational methods in electron microscopy, and we invite the community to contribute to this ongoing project.

A Fast Algorithm for Scanning Transmission Electron Microscopy Imaging and 4D-STEM Diffraction Simulations

Scanning transmission electron microscopy (STEM) is an extremely versatile method for studying materials on the atomic scale. Many STEM experiments are supported or validated with electron scattering simulations. However, using the conventional multislice algorithm to perform these simulations can require extremely large calculation times, particularly for experiments with millions of probe positions as each probe position must be simulated independently. Recently, the plane-wave reciprocal-space interpolated scattering matrix (PRISM) algorithm was developed to reduce calculation times for large STEM simulations. Here, we introduce a new method for STEM simulation: partitioning of the STEM probe into “beamlets,” given by a natural neighbor interpolation of the parent beams. This idea is compatible with PRISM simulations and can lead to even larger improvements in simulation time, as well requiring significantly less computer random access memory (RAM). We have performed various simulations to demonstrate the advantages and disadvantages of partitioned PRISM STEM simulations. We find that this new algorithm is particularly useful for 4D-STEM simulations of large fields of view. We also provide a reference implementation of the multislice, PRISM, and partitioned PRISM algorithms.

Spin-valley locking and bulk quantum Hall effect in a noncentrosymmetric Dirac semimetal BaMnSb2

Spin-valley locking in monolayer transition metal dichalcogenides has attracted enormous interest, since it offers potential for valleytronic and optoelectronic applications. Such an exotic electronic state has sparsely been seen in bulk materials. Here, we report spin-valley locking in a Dirac semimetal BaMnSb2. This is revealed by comprehensive studies using first principles calculations, tight-binding and effective model analyses, angle-resolved photoemission spectroscopy measurements. Moreover, this material also exhibits a stacked quantum Hall effect (QHE). The spin-valley degeneracy extracted from the QHE is close to 2. This result, together with the Landau level spin splitting, further confirms the spin-valley locking picture. In the extreme quantum limit, we also observed a plateau in the z-axis resistance, suggestive of a two-dimensional chiral surface state present in the quantum Hall state. These findings establish BaMnSb2 as a rare platform for exploring coupled spin and valley physics in bulk single crystals and accessing 3D interacting topological states.

The abTEM code: transmission electron microscopy from first principles

Simulation of transmission electron microscopy (TEM) images or diffraction patterns is often required to interpret experimental data. Since nuclear cores dominate electron scattering, the scattering potential is typically described using the independent atom model, which completely neglects valence bonding and its effect on the transmitting electrons. As instrumentation has advanced, new measurements have revealed subtle details of the scattering potential that were previously not accessible to experiment. We have created an open-source simulation code designed to meet these demands by integrating the ability to calculate the potential via density functional theory (DFT) with a flexible modular software design. abTEM can simulate most standard imaging modes and incorporates the latest algorithmic developments. The development of new techniques requires a program that is accessible to domain experts without extensive programming experience. abTEM is written purely in Python and designed for easy modification and extension. The effective use of modern open-source libraries makes the performance of abTEM highly competitive with existing optimized codes on both CPUs and GPUs and allows us to leverage an extensive ecosystem of libraries, such as the Atomic Simulation Environment and the DFT code GPAW. abTEM is designed to work in an interactive Python notebook, creating a seamless and reproducible workflow from defining an atomic structure, calculating molecular dynamics (MD) and electrostatic potentials, to the analysis of results, all in a single, easy-to-read document.  This article provides ongoing documentation of abTEM development. In this first version, we show use cases for hexagonal boron nitride, where valence bonding can be detected, a 4D-STEM simulation of molybdenum disulfide including ptychographic phase reconstruction, a comparison of MD and frozen phonon modeling for convergent-beam electron diffraction of a 2.6-million-atom silicon system, and a performance comparison of our fast implementation of the PRISM algorithm for a decahedral 20000-atom gold nanoparticle.

The abTEM code: transmission electron microscopy from first principles

Simulation of transmission electron microscopy (TEM) images or diffraction patterns is often required to interpret experimental data. Since nuclear cores dominate electron scattering, the scattering potential is typically described using the independent atom model, which completely neglects valence bonding and its effect on the transmitting electrons. As instrumentation has advanced, new measurements have revealed subtle details of the scattering potential that were previously not accessible to experiment. We have created an open-source simulation code designed to meet these demands by integrating the ability to calculate the potential via density functional theory (DFT) with a flexible modular software design. abTEM can simulate most standard imaging modes and incorporates the latest algorithmic developments. The development of new techniques requires a program that is accessible to domain experts without extensive programming experience. abTEM is written purely in Python and designed for easy modification and extension. The effective use of modern open-source libraries makes the performance of abTEM highly competitive with existing optimized codes on both CPUs and GPUs and allows us to leverage an extensive ecosystem of libraries, such as the Atomic Simulation Environment and the DFT code GPAW. abTEM is designed to work in an interactive Python notebook, creating a seamless and reproducible workflow from defining an atomic structure, calculating molecular dynamics (MD) and electrostatic potentials, to the analysis of results, all in a single, easy-to-read document.  This article provides ongoing documentation of abTEM development. In this first version, we show use cases for hexagonal boron nitride, where valence bonding can be detected, a 4D-STEM simulation of molybdenum disulfide including ptychographic phase reconstruction, a comparison of MD and frozen phonon modeling for convergent-beam electron diffraction of a 2.6-million-atom silicon system, and a performance comparison of our fast implementation of the PRISM algorithm for a decahedral 20000-atom gold nanoparticle.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Determining the three-dimensional atomic structure of an amorphous solid

Amorphous solids such as glass, plastics and amorphous thin films are ubiquitous in our daily life and have broad applications ranging from telecommunications to electronics and solar cells^1-4. However, owing to the lack of long-range order, the three-dimensional (3D) atomic structure of amorphous solids has so far eluded direct experimental determination^5-15. Here we develop an atomic electron tomography reconstruction method to experimentally determine the 3D atomic positions of an amorphous solid. Using a multi-component glass-forming alloy as proof of principle, we quantitatively characterize the short- and medium-range order of the 3D atomic arrangement. We observe that, although the 3D atomic packing of the short-range order is geometrically disordered, some short-range-order structures connect with each other to form crystal-like superclusters and give rise to medium-range order. We identify four types of crystal-like medium-range order-face-centred cubic, hexagonal close-packed, body-centred cubic and simple cubic-coexisting in the amorphous sample, showing translational but not orientational order. These observations provide direct experimental evidence to support the general framework of the efficient cluster packing model for metallic glasses^10,12-14,16. We expect that this work will pave the way for the determination of the 3D structure of a wide range of amorphous solids, which could transform our fundamental understanding of non-crystalline materials and related phenomena.

Single-atom level determination of 3-dimensional surface atomic structure via neural network-assisted atomic electron tomography

Functional properties of nanomaterials strongly depend on their surface atomic structures, but they often become largely different from their bulk structures, exhibiting surface reconstructions and relaxations. However, most of the surface characterization methods are either limited to 2D measurements or not reaching to true 3D atomic-scale resolution, and single-atom level determination of the 3D surface atomic structure for general 3D nanomaterials still remains elusive. Here we demonstrate the measurement of 3D atomic structure at 15 pm precision using a Pt nanoparticle as a model system. Aided by a deep learning-based missing data retrieval combined with atomic electron tomography, the surface atomic structure was reliably measured. We found that <\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$100$$\end{document}100> and <\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$111$$\end{document}111> facets contribute differently to the surface strain, resulting in anisotropic strain distribution as well as compressive support boundary effect. The capability of single-atom level surface characterization will not only deepen our understanding of the functional properties of nanomaterials but also open a new door for fine tailoring of their performance.

Precise determination of surface atomic structure of metallic nanoparticles is key to unlock their surface/interface properties. Here the authors introduce a neural network-assisted atomic electron tomography approach that provides a three-dimensional reconstruction of metallic nanoparticles at individual atom level.

ab initio description of bonding for transmission electron microscopy

The simulation of transmission electron microscopy (TEM) images or diffraction patterns is often required to interpret their contrast and extract specimen features. This is especially true for high-resolution phase-contrast imaging of materials, but electron scattering simulations based on atomistic models are widely used in materials science and structural biology. Since electron scattering is dominated by the nuclear cores, the scattering potential is typically described by the widely applied independent atom model. This approximation is fast and fairly accurate, especially for scanning TEM (STEM) annular dark-field contrast, but it completely neglects valence bonding and its effect on the transmitting electrons. However, an emerging trend in electron microscopy is to use new instrumentation and methods to extract the maximum amount of information from each electron. This is evident in the increasing popularity of techniques such as 4D-STEM combined with ptychography in materials science, and cryogenic microcrystal electron diffraction in structural biology, where subtle differences in the scattering potential may be both measurable and contain additional insights. Thus, there is increasing interest in electron scattering simulations based on electrostatic potentials obtained from first principles, mainly via density functional theory, which was previously mainly required for holography. In this Review, we discuss the motivation and basis for these developments, survey the pioneering work that has been published thus far, and give our outlook for the future. We argue that a physically better justified ab initio description of the scattering potential is both useful and viable for an increasing number of systems, and we expect such simulations to steadily gain in popularity and importance.

Epitaxial Ferroelectric Hf0.5 Zr0.5 O2 with Metallic Pyrochlore Oxide Electrodes

The synthesis of fully epitaxial ferroelectric Hf_0.5 Zr_0.5 O₂ (HZO) thin films through the use of a conducting pyrochlore oxide electrode that acts as a structural and chemical template is reported. Such pyrochlores, exemplified by Pb₂ Ir₂ O₇ (PIO) and Bi₂ Ru₂ O₇ (BRO), exhibit metallic conductivity with room-temperature resistivity of <1 mΩ cm and are closely lattice matched to yttria-stabilized zirconia substrates as well as the HZO layers grown on top of them. Evidence for epitaxy and domain formation is established with X-ray diffraction and scanning transmission electron microscopy, which show that the c-axis of the HZO film is normal to the substrate surface. The emergence of the non-polar-monoclinic phase from the polar-orthorhombic phase is observed when the HZO film thickness is ≥≈30 nm. Thermodynamic analyses reveal the role of epitaxial strain and surface energy in stabilizing the polar phase as well as its coexistence with the non-polar-monoclinic phase as a function of film thickness.

Multibeam Electron Diffraction

One of the primary uses for transmission electron microscopy (TEM) is to measure diffraction pattern images in order to determine a crystal structure and orientation. In nanobeam electron diffraction (NBED), we scan a moderately converged electron probe over the sample to acquire thousands or even millions of sequential diffraction images, a technique that is especially appropriate for polycrystalline samples. However, due to the large Ewald sphere of TEM, excitation of Bragg peaks can be extremely sensitive to sample tilt, varying strongly for even a few degrees of sample tilt for crystalline samples. In this paper, we present multibeam electron diffraction (MBED), where multiple probe-forming apertures are used to create multiple scanning transmission electron microscopy (STEM) probes, all of which interact with the sample simultaneously. We detail designs for MBED experiments, and a method for using a focused ion beam to produce MBED apertures. We show the efficacy of the MBED technique for crystalline orientation mapping using both simulations and proof-of-principle experiments. We also show how the angular information in MBED can be used to perform 3D tomographic reconstruction of samples without needing to tilt or scan the sample multiple times. Finally, we also discuss future opportunities for the MBED method.

A symmetry-derived mechanism for atomic resolution imaging

We introduce an image-contrast mechanism for scanning transmission electron microscopy (STEM) that derives from the local symmetry within the specimen. For a given position of the electron probe on the specimen, the image intensity is determined by the degree of similarity between the exit electron-intensity distribution and a chosen symmetry operation applied to that distribution. The contrast mechanism detects both light and heavy atomic columns and is robust with respect to specimen thickness, electron-probe energy, and defocus. Atomic columns appear as sharp peaks that can be significantly narrower than for STEM images using conventional disk and annular detectors. This fundamentally different contrast mechanism complements conventional imaging modes and can be acquired simultaneously with them, expanding the power of STEM for materials characterization.

Denoising atomic resolution 4D scanning transmission electron microscopy data with tensor singular value decomposition

Tensor singular value decomposition (SVD) is a method to find a low-dimensional representation of data with meaningful structure in three or more dimensions. Tensor SVD has been applied to denoise atomic-resolution 4D scanning transmission electron microscopy (4D STEM) data. On data simulated from a SrTiO₃ [100] perfect crystal and a Si [110] edge dislocation, tensor SVD achieved an average peak signal-to-noise ratio (PSNR) of ~40 dB, which matches or exceeds the performance of other denoising methods, with processing times at least 100 times shorter. On experimental data from SrTiO₃ [100] and LiZnSb [112¯0]/GaSb [110] samples, tensor SVD denoises multiple GB 4D STEM data sets in ten minutes on a typical personal computer. Denoising with tensor SVD improves both convergent beam electron diffraction patterns and virtual-aperture annular dark field images.

Correlating the three-dimensional atomic defects and electronic properties of two-dimensional transition metal dichalcogenides

The electronic, optical and chemical properties of two-dimensional transition metal dichalcogenides strongly depend on their three-dimensional atomic structure and crystal defects. Using Re-doped MoS₂ as a model system, here we present scanning atomic electron tomography as a method to determine three-dimensional atomic positions as well as positions of crystal defects such as dopants, vacancies and ripples with a precision down to 4 pm. We measure the three-dimensional bond distortion and local strain tensor induced by single dopants. By directly providing these experimental three-dimensional atomic coordinates to density functional theory, we obtain more accurate electronic band structures than derived from conventional density functional theory calculations that relies on relaxed three-dimensional atomic coordinates. We anticipate that scanning atomic electron tomography not only will be generally applicable to determine the three-dimensional atomic coordinates of two-dimensional materials, but also will enable ab initio calculations to better predict the physical, chemical and electronic properties of these materials.

2D Electrets of Ultrathin MoO2 with Apparent Piezoelectricity

Since graphene, a variety of 2D materials have been fabricated in a quest for a tantalizing combination of properties and desired physiochemical behavior. 2D materials that are piezoelectric, i.e., that allow for a facile conversion of electrical energy into mechanical and vice versa, offer applications for sensors, actuators, energy harvesting, stretchable and flexible electronics, and energy storage, among others. Unfortunately, materials must satisfy stringent symmetry requirements to be classified as piezoelectric. Here, 2D ultrathin single-crystal molybdenum oxide (MoO₂ ) flakes that exhibit unexpected piezoelectric-like response are fabricated, as MoO₂ is centrosymmetric and should not exhibit intrinsic piezoelectricity. However, it is demonstrated that the apparent piezoelectricity in 2D MoO₂ emerges from an electret-like behavior induced by the trapping and stabilization of charges around defects in the material. Arguably, the material represents the first 2D electret material and suggests a route to artificially engineer piezoelectricity in 2D crystals. Specifically, it is found that the maximum out-of-plane piezoresponse is 0.56 pm V^-1 , which is as strong as that observed in conventional 2D piezoelectric materials. The charges are found to be highly stable at room temperature with a trapping energy barrier of ≈2 eV.

Atomic-Resolution EDX, HAADF, and EELS Study of GaAs1-xBix Alloys

The distribution of alloyed atoms in semiconductors often deviates from a random distribution which can have significant effects on the properties of the materials. In this study, scanning transmission electron microscopy techniques are employed to analyze the distribution of Bi in several distinctly MBE grown GaAs1-xBix alloys. Statistical quantification of atomic-resolution HAADF images, as well as numerical simulations, are employed to interpret the contrast from Bi-containing columns at atomically abrupt (001) GaAs-GaAsBi interface and the onset of CuPt-type ordering. Using monochromated EELS mapping, bulk plasmon energy red-shifts are examined in a sample exhibiting phase-separated domains. This suggests a simple method to investigate local GaAsBi unit-cell volume expansions and to complement standard X-ray-based lattice-strain measurements. Also, a single-variant CuPt-ordered GaAsBi sample grown on an offcut substrate is characterized with atomic scale compositional EDX mappings, and the order parameter is estimated. Finally, a GaAsBi alloy with a vertical Bi composition modulation is synthesized using a low substrate rotation rate. Atomically, resolved EDX and HAADF imaging shows that the usual CuPt-type ordering is further modulated along the [001] growth axis with a period of three lattice constants. These distinct GaAsBi samples exemplify the variety of Bi distributions that can be achieved in this alloy, shedding light on the incorporation mechanisms of Bi atoms and ways to further develop Bi-containing III-V semiconductors.

Observations of grain-boundary phase transformations in an elemental metal

The theory of grain boundary (the interface between crystallites, GB) structure has a long history¹ and the concept of GBs undergoing phase transformations was proposed 50 years ago^2,3. The underlying assumption was that multiple stable and metastable states exist for different GB orientations^4-6. The terminology 'complexion' was recently proposed to distinguish between interfacial states that differ in any equilibrium thermodynamic property⁷. Different types of complexion and transitions between complexions have been characterized, mostly in binary or multicomponent systems^8-19. Simulations have provided insight into the phase behaviour of interfaces and shown that GB transitions can occur in many material systems^20-24. However, the direct experimental observation and transformation kinetics of GBs in an elemental metal have remained elusive. Here we demonstrate atomic-scale GB phase coexistence and transformations at symmetric and asymmetric [Formula: see text] tilt GBs in elemental copper. Atomic-resolution imaging reveals the coexistence of two different structures at Σ19b GBs (where Σ19 is the density of coincident sites and b is a GB variant), in agreement with evolutionary GB structure search and clustering analysis^21,25,26. We also use finite-temperature molecular dynamics simulations to explore the coexistence and transformation kinetics of these GB phases. Our results demonstrate how GB phases can be kinetically trapped, enabling atomic-scale room-temperature observations. Our work paves the way for atomic-scale in situ studies of metallic GB phase transformations, which were previously detected only indirectly^9,15,27-29, through their influence on abnormal grain growth, non-Arrhenius-type diffusion or liquid metal embrittlement.

Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks

Two types of convolutional neural network (CNN) models, a discrete classification network and a continuous regression network, were trained to determine local sample thickness from convergent beam diffraction (CBED) patterns of SrTiO₃ collected in a scanning transmission electron microscope (STEM) at atomic column resolution. Acquisition of atomic resolution CBED patterns for this purpose requires careful balancing of CBED feature size in pixels, acquisition speed, and detector dynamic range. The training datasets were derived from multislice simulations, which must be convolved with incoherent source broadening. Sample thicknesses were also determined using quantitative high-angle annular dark-field (HAADF) STEM images acquired simultaneously. The regression CNN performed well on sample thinner than 35 nm, with 70% of the CNN results within 1 nm of HAADF thickness, and 1.0 nm overall root mean square error between the two measurements. The classification CNN was trained for a thicknesses up to 100 nm and yielded 66% of CNN results within one classification increment of 2 nm of HAADF thickness. Our approach depends on methods from computer vision including transfer learning and image augmentation.

Size Induced Structural Changes in Molybdenum Oxide Nanoparticles

Nanosizing of metal oxide particles is a common strategy for improving materials properties; however, small particles often take structures different from the bulk material. MoO₂ nanoparticles show a structure that is distinct from the bulk distorted rutile structure and which has not yet been determined. Here, we present a model for nanostructured MoO₂ obtained through detailed atomic pair distribution function analysis combined with high-resolution electron microscopy. Defects occur in the arrangement of [MoO₆] octahedra, in both large (40-100 nm) nanoparticles, where the overall distorted rutile structure is preserved, and in small nanoparticles (<5 nm), where a new nanostructure is formed. The study provides a piece in the puzzle of understanding the structure/properties relationship of molybdenum oxides and further our understanding of the origin of structural changes taking place upon nanosizing in oxide materials.

Four-Dimensional Scanning Transmission Electron Microscopy (4D-STEM): From Scanning Nanodiffraction to Ptychography and Beyond

Scanning transmission electron microscopy (STEM) is widely used for imaging, diffraction, and spectroscopy of materials down to atomic resolution. Recent advances in detector technology and computational methods have enabled many experiments that record a full image of the STEM probe for many probe positions, either in diffraction space or real space. In this paper, we review the use of these four-dimensional STEM experiments for virtual diffraction imaging, phase, orientation and strain mapping, measurements of medium-range order, thickness and tilt of samples, and phase contrast imaging methods, including differential phase contrast, ptychography, and others.

Segregation-Induced Nanofaceting Transition at an Asymmetric Tilt Grain Boundary in Copper

We show that chemistry can be used to trigger a nanofaceting transition. In particular, the segregation of Ag to an asymmetric tilt grain boundary in Cu is investigated. Aberration-corrected electron microscopy reveals that annealing the bicrystal results in the formation of nanometer-sized facets composed of preferentially Ag-segregated symmetric Σ5{210} segments and Ag-depleted {230}/{100} asymmetric segments. Our observations oppose an anticipated trend to form coarse facets. Atomistic simulations confirm the nanofacet formation observed in the experiment and demonstrate a concurrent grain boundary phase transition induced by the anisotropic segregation of Ag.

An open-source software ecosystem for the interactive exploration of ultrafast electron scattering data

This paper details a software ecosystem comprising three free and open-source Python packages for processing raw ultrafast electron scattering (UES) data and interactively exploring the processed data. The first package, iris, is graphical user-interface program and library for interactive exploration of UES data. Under the hood, iris makes use of npstreams, an extensions of numpy to streaming array-processing, for high-throughput parallel data reduction. Finally, we present scikit-ued, a library of reusable routines and data structures for analysis of UES data, including specialized image processing algorithms, simulation routines, and crystal structure manipulation operations. In this paper, some of the features or all three packages are highlighted, such as parallel data reduction, image registration, interactive exploration. The packages are fully tested and documented and are released under permissive licenses.

STEMcl-A multi-GPU multislice algorithm for simulation of large structure and imaging parameter series

Electron microscopy images are interference patterns and can generally not be interpreted in a straight forward manner. Typically, time consuming numerical simulations have to be employed to separate specimen features from imaging artifacts. Directly comparing numerical predictions to experimental results, realistic simulation box sizes and varying imaging parameters are needed. In this work, we introduce an accelerated multislice algorithm, named STEMcl, that is capable of simulating series of large super cells typical for defective and amorphous systems, in addition to parameter series using the massive parallelization accessible in today's commercial PC-hardware, e.g. graphics processing units (GPUs). A new numerical approach is used to overcome the memory constraint limiting the maximum computable system size. This approach creates the possibility to study systematically the contrast formation arising by structural differences. STEM simulations of structure series of a crystalline Si and an amorphous CuZr system are presented and the contrast formation of vacancies/voids are studied. The detectability of vacancies/voids in STEM experiments is discussed in terms of density changes.