Adaptive probe trajectory scanning probe microscopy for multiresolution measurements of interface geometry

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. The discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.

Towards automating structural discovery in scanning transmission electron microscopy *

Scanning transmission electron microscopy is now the primary tool for exploring functional materials on the atomic level. Often, features of interest are highly localized in specific regions in the material, such as ferroelectric domain walls, extended defects, or second phase inclusions. Selecting regions to image for structural and chemical discovery via atomically resolved imaging has traditionally proceeded via human operators making semi-informed judgements on sampling locations and parameters. Recent efforts at automation for structural and physical discovery have pointed towards the use of ‘active learning’ methods that utilize Bayesian optimization with surrogate models to quickly find relevant regions of interest. Yet despite the potential importance of this direction, there is a general lack of certainty in selecting relevant control algorithms and how to balance a priori knowledge of the material system with knowledge derived during experimentation. Here we address this gap by developing the automated experiment workflows with several combinations to both illustrate the effects of these choices and demonstrate the tradeoffs associated with each in terms of accuracy, robustness, and susceptibility to hyperparameters for structural discovery. We discuss possible methods to build descriptors using the raw image data and deep learning based semantic segmentation, as well as the implementation of variational autoencoder based representation. Furthermore, each workflow is applied to a range of feature sizes including NiO pillars within a La:SrMnO3 matrix, ferroelectric domains in BiFeO3, and topological defects in graphene. The code developed in this manuscript is open sourced and will be released at github.com/nccreang/AE_Workflows.

Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics to self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiments (AE) in imaging. Here, we aim to analyze the major pathways toward AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment and consider the latencies, biases, and prior knowledge of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities, and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning. Overall, we argue that ML/AI can dramatically alter the (S)TEM and SPM fields; however, this process is likely to be highly nontrivial and initiated by combined human-ML workflows and will bring challenges both from the microscope and ML/AI sides. At the same time, these methods will enable opportunities and paradigms for scientific discovery and nanostructure fabrication.

Autonomous Experiments in Scanning Probe Microscopy and Spectroscopy: Choosing Where to Explore Polarization Dynamics in Ferroelectrics

Polarization dynamics in ferroelectric materials are explored via automated experiment in piezoresponse force microscopy/spectroscopy (PFM/S). A Bayesian optimization (BO) framework for imaging is developed, and its performance for a variety of acquisition and pathfinding functions is explored using previously acquired data. The optimized algorithm is then deployed on an operational scanning probe microscope (SPM) for finding areas of large electromechanical response in a thin film of PbTiO₃, with results showing that, with just 20% of the area sampled, most high-response clusters were captured. This approach can allow performing more complex spectroscopies in SPM that were previously not possible due to time constraints and sample stability. Improvements to the framework to enable the incorporation of more prior information and improve efficiency further are modeled and discussed.

Fast Scanning Probe Microscopy via Machine Learning: Non-Rectangular Scans with Compressed Sensing and Gaussian Process Optimization

Fast scanning probe microscopy enabled via machine learning allows for a broad range of nanoscale, temporally resolved physics to be uncovered. However, such examples for functional imaging are few in number. Here, using piezoresponse force microscopy (PFM) as a model application, a factor of 5.8 reduction in data collection using a combination of sparse spiral scanning with compressive sensing and Gaussian process regression reconstruction is demonstrated. It is found that even extremely sparse spiral scans offer strong reconstructions with less than 6% error for Gaussian process regression reconstructions. Further, the error associated with each reconstructive technique per reconstruction iteration is analyzed, finding the error is similar past ≈15 iterations, while at initial iterations Gaussian process regression outperforms compressive sensing. This study highlights the capabilities of reconstruction techniques when applied to sparse data, particularly sparse spiral PFM scans, with broad applications in scanning probe and electron microscopies.

Big, Deep, and Smart Data in Scanning Probe Microscopy

Scanning probe microscopy (SPM) techniques have opened the door to nanoscience and nanotechnology by enabling imaging and manipulation of the structure and functionality of matter at nanometer and atomic scales. Here, we analyze the scientific discovery process in SPM by following the information flow from the tip-surface junction, to knowledge adoption by the wider scientific community. We further discuss the challenges and opportunities offered by merging SPM with advanced data mining, visual analytics, and knowledge discovery technologies.

Dynamic scan control in STEM: spiral scans

Scanning transmission electron microscopy (STEM) has emerged as one of the foremost techniques to analyze materials at atomic resolution. However, two practical difficulties inherent to STEM imaging are: radiation damage imparted by the electron beam, which can potentially damage or otherwise modify the specimen and slow-scan image acquisition, which limits the ability to capture dynamic changes at high temporal resolution. Furthermore, due in part to scan flyback corrections, typical raster scan methods result in an uneven distribution of dose across the scanned area. A method to allow extremely fast scanning with a uniform residence time would enable imaging at low electron doses, ameliorating radiation damage and at the same time permitting image acquisition at higher frame-rates while maintaining atomic resolution. The practical complication is that rastering the STEM probe at higher speeds causes significant image distortions. Non-square scan patterns provide a solution to this dilemma and can be tailored for low dose imaging conditions. Here, we develop a method for imaging with alternative scan patterns and investigate their performance at very high scan speeds. A general analysis for spiral scanning is presented here for the following spiral scan functions: Archimedean, Fermat, and constant linear velocity spirals, which were tested for STEM imaging. The quality of spiral scan STEM images is generally comparable with STEM images from conventional raster scans, and the dose uniformity can be improved.

Note: Multiscale scanning probe microscopy

Combining the nanoscopic and macroscopic worlds is a serious challenge common to numerous scientific fields, from physics to biology. In this paper, we demonstrate nanometric resolution over a millimeter range by means of atomic-force microscopy using metrological stage. Nanometric repeatability and millimeter range open up the possibility of probing components and materials combining multiscale properties i.e., engineered nanomaterials. Multiscale probing is not limited to atomic-force microscopy and can be extended to any type of scanning probe technique in nanotechnology, including piezoforce microscopy, electrostatic-force microscopy, and scanning near-field optical microscopy.