Automated Tip Conditioning for Scanning Tunneling Spectroscopy

Automated Structure Discovery for Scanning Tunneling Microscopy

Scanning tunneling microscopy (STM) with a functionalized tip apex reveals the geometric and electronic structures of a sample within the same experiment. However, the complex nature of the signal makes images difficult to interpret and has so far limited most research to planar samples with a known chemical composition. Here, we present automated structure discovery for STM (ASD-STM), a machine learning tool for predicting the atomic structure directly from an STM image, by building upon successful methods for structure discovery in noncontact atomic force microscopy (nc-AFM). We apply the method on various organic molecules and achieve good accuracy on structure predictions and chemical identification on a qualitative level while highlighting future development requirements for ASD-STM. This method is directly applicable to experimental STM images of organic molecules, making structure discovery available for a wider scanning probe microscopy audience outside of nc-AFM. This work also allows more advanced machine learning methods to be developed for STM structure discovery.

Automated Scanning Probe Tip State Classification without Machine Learning

The manual identification and in situ correction of the state of the scanning probe tip is one of the most time-consuming and tedious processes in atomic-resolution scanning probe microscopy. This is due to the random nature of the probe tip on the atomic level, and the requirement for a human operator to compare the probe quality via manual inspection of the topographical images after any change in the probe. Previous attempts to automate the classification of the scanning probe state have focused on the use of machine learning techniques, but the training of these models relies on large, labeled data sets for each surface being studied. These data sets are extremely time-consuming to create and are not always available, especially when considering a new substrate or adsorbate system. In this paper, we show that the problem of tip classification from a topographical image can be solved by using only a single image of the surface along with a small amount of prior knowledge of the appearance of the system in question with a method utilizing template matching (TM). We find that by using these TM methods, comparable accuracy and precision can be achieved to values obtained with the use of machine learning. We demonstrate the efficacy of this technique by training a machine learning-based classifier and comparing the classifications with the TM classifier for two prototypical silicon-based surfaces. We also apply the TM classifier to a number of other systems where supervised machine learning-based training was not possible due to the nature of the training data sets. Finally, the applicability of the TM method to surfaces used in the literature, which have been classified using machine learning-based methods, is considered.

Autonomous Molecular Structure Imaging with High-Resolution Atomic Force Microscopy for Molecular Mixture Discovery

Due to its single-molecule sensitivity, high-resolution atomic force microscopy (HR-AFM) has proved to be a valuable and uniquely advantageous tool to study complex molecular mixtures, which hold promise for developing clean energy and achieving environmental sustainability. However, significant challenges remain to achieve the full potential of the sophisticated and time-consuming experiments. Automation combined with machine learning (ML) and artificial intelligence (AI) is key to overcoming these challenges. Here we present Auto-HR-AFM, an AI tool to automatically collect HR-AFM images of petroleum-based mixtures. We trained an instance segmentation model to teach Auto-HR-AFM how to recognize features in HR-AFM images. Auto-HR-AFM then uses that information to optimize the imaging by adjusting the probe-molecule distance for each molecule in the run. Auto-HR-AFM is the initial tool that will lead to fully automated scanning probe microscopy (SPM) experiments, from start to finish. This automation will allow SPM to become a mainstream characterization technique for complex mixtures, an otherwise unattainable target.

Engineering Robust Metallic Zero-Mode States in Olympicene Graphene Nanoribbons

Metallic graphene nanoribbons (GNRs) represent a critical component in the toolbox of low-dimensional functional materials technology serving as 1D interconnects capable of both electronic and quantum information transport. The structural constraints imposed by on-surface bottom-up GNR synthesis protocols along with the limited control over orientation and sequence of asymmetric monomer building blocks during the radical step-growth polymerization have plagued the design and assembly of metallic GNRs. Here, we report the regioregular synthesis of GNRs hosting robust metallic states by embedding a symmetric zero-mode (ZM) superlattice along the backbone of a GNR. Tight-binding electronic structure models predict a strong nearest-neighbor electron hopping interaction between adjacent ZM states, resulting in a dispersive metallic band. First-principles density functional theory-local density approximation calculations confirm this prediction, and the robust, metallic ZM band of olympicene GNRs is experimentally corroborated by scanning tunneling spectroscopy.

Autonomous Single-Molecule Manipulation Based on Reinforcement Learning

Building nanostructures one-by-one requires precise control of single molecules over many manipulation steps. The ideal scenario for machine learning algorithms is complex, repetitive, and time-consuming. Here, we show a reinforcement learning algorithm that learns how to control a single dipolar molecule in the electric field of a scanning tunneling microscope. Using about 2250 iterations to train, the algorithm learned to manipulate the molecule toward specific positions on the surface. Simultaneously, it generates physical insights into the movement as well as orientation of the molecule, based on the position where the electric field is applied relative to the molecule. This reveals that molecular movement is strongly inhibited in some directions, and the torque is not symmetric around the dipole moment.

Machine learning in electron microscopy for advanced nanocharacterization: current developments, available tools and future outlook

In the last few years, electron microscopy has experienced a new methodological paradigm aimed to fix the bottlenecks and overcome the challenges of its analytical workflow. Machine learning and artificial intelligence are answering this call providing powerful resources towards automation, exploration, and development. In this review, we evaluate the state-of-the-art of machine learning applied to electron microscopy (and obliquely, to materials and nano-sciences). We start from the traditional imaging techniques to reach the newest higher-dimensionality ones, also covering the recent advances in spectroscopy and tomography. Additionally, the present review provides a practical guide for microscopists, and in general for material scientists, but not necessarily advanced machine learning practitioners, to straightforwardly apply the offered set of tools to their own research. To conclude, we explore the state-of-the-art of other disciplines with a broader experience in applying artificial intelligence methods to their research (e.g., high-energy physics, astronomy, Earth sciences, and even robotics, videogames, or marketing and finances), in order to narrow down the incoming future of electron microscopy, its challenges and outlook.

High-Throughput Preparation of Supramolecular Nanostructures on Metal Surfaces

One of the contemporary challenges in materials science lies in the rapid materials screening and discovery. Experimental sample libraries can be generated by high-throughput parallel synthesis to map the composition space for rapid material discoveries. Molecular self-assembly on surfaces has proved a useful way to construct nanostructures with interesting topologies or properties. Despite the strong dependence of molecular stoichiometry on the structures, high-throughput preparations of supramolecular surface nanostructures have been far less explored. Here, by integrating a physical mask into the standard ultra-high-vacuum (UHV) molecular preparation system we show a high-throughput approach for preparing supramolecular nanostructures of continuous composition spreads on metal surfaces. The spatially addressable sample libraries of supramolecular self-assemblies are characterized by high-resolution scanning probe microscopy. We could explore different binary nanostructures of varying molecular ratios on one single substrate. Moreover, we use the minimum spanning tree approach to qualitatively and quantitatively study the structural properties of the formed nanostructures. This high-throughput approach may accelerate the screening and exploration of surface-supported, low-dimensional nanostructures not limited to supramolecular interactions.

Extraction of Hidden Science from Nanoscale Images

Scanning probe microscopies and spectroscopies enable investigation of surfaces and even buried interfaces down to the scale of chemical-bonding interactions, and this capability has been enhanced with the support of computational algorithms for data acquisition and image processing to explore physical, chemical, and biological phenomena. Here, we describe how scanning probe techniques have been enhanced by some of these recent algorithmic improvements. One improvement to the data acquisition algorithm is to advance beyond a simple rastering framework by using spirals at constant angular velocity then switching to constant linear velocity, which limits the piezo creep and hysteresis issues seen in traditional acquisition methods. One can also use image-processing techniques to model the distortions that appear from tip motion effects and to make corrections to these images. Another image-processing algorithm we discuss enables researchers to segment images by domains and subdomains, thereby highlighting reactive and interesting disordered sites at domain boundaries. Lastly, we discuss algorithms used to examine the dipole direction of individual molecules and surface domains, hydrogen bonding interactions, and molecular tilt. The computational algorithms used for scanning probe techniques are still improving rapidly and are incorporating machine learning at the next level of iteration. That said, the algorithms are not yet able to perform live adjustments during data recording that could enhance the microscopy and spectroscopic imaging methods significantly.

Spin splitting of dopant edge state in magnetic zigzag graphene nanoribbons

Spin-ordered electronic states in hydrogen-terminated zigzag nanographene give rise to magnetic quantum phenomena^1,2 that have sparked renewed interest in carbon-based spintronics^3,4. Zigzag graphene nanoribbons (ZGNRs)-quasi one-dimensional semiconducting strips of graphene bounded by parallel zigzag edges-host intrinsic electronic edge states that are ferromagnetically ordered along the edges of the ribbon and antiferromagnetically coupled across its width^1,2,5. Despite recent advances in the bottom-up synthesis of GNRs featuring symmetry protected topological phases^6-8 and even metallic zero mode bands⁹, the unique magnetic edge structure of ZGNRs has long been obscured from direct observation by a strong hybridization of the zigzag edge states with the surface states of the underlying support^10-15. Here, we present a general technique to thermodynamically stabilize and electronically decouple the highly reactive spin-polarized edge states by introducing a superlattice of substitutional N-atom dopants along the edges of a ZGNR. First-principles GW calculations and scanning tunnelling spectroscopy reveal a giant spin splitting of low-lying nitrogen lone-pair flat bands by an exchange field (~850 tesla) induced by the ferromagnetically ordered edge states of ZGNRs. Our findings directly corroborate the nature of the predicted emergent magnetic order in ZGNRs and provide a robust platform for their exploration and functional integration into nanoscale sensing and logic devices^15-21.

Increasing Efficiency of Nonadiabatic Molecular Dynamics by Hamiltonian Interpolation with Kernel Ridge Regression

Nonadiabatic (NA) molecular dynamics (MD) goes beyond the adiabatic Born-Oppenheimer approximation to account for transitions between electronic states. Such processes are common in molecules and materials used in solar energy, optoelectronics, sensing, and many other fields. NA-MD simulations are much more expensive compared to adiabatic MD due to the need to compute excited state properties and NA couplings (NACs). Similarly, application of machine learning (ML) to NA-MD is more challenging compared with adiabatic MD. We develop an NA-MD simulation strategy in which an adiabatic MD trajectory, which can be generated with a ML force field, is used to sample excitation energies and NACs for a small fraction of geometries, while the properties for the remaining geometries are interpolated with kernel ridge regression (KRR). This ML strategy allows for one to perform NA-MD under the classical path approximation, increasing the computational efficiency by over an order of magnitude. Compared to neural networks, KRR requires little parameter tuning, saving efforts on model building. The developed strategy is demonstrated with two metal halide perovskites that exhibit complicated MD and are actively studied for various applications.

Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy

Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of physics research, with domain applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, the recent successes in applying ML/AI methods for autonomous systems from robotics to self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiments (AE) in imaging. Here, we aim to analyze the major pathways toward AE in imaging methods with sequential image formation mechanisms, focusing on scanning probe microscopy (SPM) and (scanning) transmission electron microscopy ((S)TEM). We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment. As such, this analysis should explore the human and ML/AI roles prior to and during the experiment and consider the latencies, biases, and prior knowledge of the decision-making process. Similarly, such discussion should include the limitations of the existing imaging systems, including intrinsic latencies, non-idealities, and drifts comprising both correctable and stochastic components. We further pose that the role of the AE in microscopy is not the exclusion of human operators (as is the case for autonomous driving), but rather automation of routine operations such as microscope tuning, etc., prior to the experiment, and conversion of low latency decision making processes on the time scale spanning from image acquisition to human-level high-order experiment planning. Overall, we argue that ML/AI can dramatically alter the (S)TEM and SPM fields; however, this process is likely to be highly nontrivial and initiated by combined human-ML workflows and will bring challenges both from the microscope and ML/AI sides. At the same time, these methods will enable opportunities and paradigms for scientific discovery and nanostructure fabrication.

Machine Vision Automated Chiral Molecule Detection and Classification in Molecular Imaging

Scanning probe microscopy (SPM) is recognized as an essential characterization tool in a broad range of applications, allowing for real-space atomic imaging of solid surfaces, nanomaterials, and molecular systems. Recently, the imaging of chiral molecular nanostructures via SPM has become a matter of increased scientific and technological interest due to their imminent use as functional platforms in a wide scope of applications, including nonlinear chiroptics, enantioselective catalysis, and enantiospecific sensing. Due to the time-consuming and error-prone image analysis process, a highly efficient analytic framework capable of identifying complex chiral patterns in SPM images is needed. Here, we adopted a state-of-the-art machine vision algorithm to develop a one-image-one-system deep learning framework for the analysis of SPM images. To demonstrate its accuracy and versatility, we employed it to determine the chirality of the molecules comprising two supramolecular self-assemblies with two distinct chiral organization patterns. Our framework accurately detected the position and labeled the chirality of each molecule. This framework underpins the tremendous potential of machine learning algorithms for the automated recognition of complex SPM image patterns in a wide range of research disciplines.