Development of a deep learning-based method to identify "good" regions of a cryo-electron microscopy grid

Chirality Analysis of Complex Microparticles using Deep Learning on Realistic Sets of Microscopy Images

Nanoscale chirality is an actively growing research field spurred by the giant chiroptical activity, enantioselective biological activity, and asymmetric catalytic activity of chiral nanostructures. Compared to chiral molecules, the handedness of chiral nano- and microstructures can be directly established via electron microscopy, which can be utilized for the automatic analysis of chiral nanostructures and prediction of their properties. However, chirality in complex materials may have multiple geometric forms and scales. Computational identification of chirality from electron microscopy images rather than optical measurements is convenient but is fundamentally challenging, too, because (1) image features differentiating left- and right-handed particles can be ambiguous and (2) three-dimensional structure essential for chirality is 'flattened' into two-dimensional projections. Here, we show that deep learning algorithms can identify twisted bowtie-shaped microparticles with nearly 100% accuracy and classify them as left- and right-handed with as high as 99% accuracy. Importantly, such accuracy was achieved with as few as 30 original electron microscopy images of bowties. Furthermore, after training on bowtie particles with complex nanostructured features, the model can recognize other chiral shapes with different geometries without retraining for their specific chiral geometry with 93% accuracy, indicating the true learning abilities of the employed neural networks. These findings indicate that our algorithm trained on a practically feasible set of experimental data enables automated analysis of microscopy data for the accelerated discovery of chiral particles and their complex systems for multiple applications.

Fully automated multi-grid cryoEM screening using Smart Leginon

Single-particle cryo-electron microscopy (cryoEM) is a swiftly growing method for understanding protein structure. With increasing demand for high-throughput, high-resolution cryoEM services comes greater demand for rapid and automated cryoEM grid and sample screening. During screening, optimal grids and sample conditions are identified for subsequent high-resolution data collection. Screening is a major bottleneck for new cryoEM projects because grids must be optimized for several factors, including grid type, grid hole size, sample concentration, buffer conditions, ice thickness and particle behavior. Even for mature projects, multiple grids are commonly screened to select a subset for high-resolution data collection. Here, machine learning and novel purpose-built image-processing and microscope-handling algorithms are incorporated into the automated data-collection software Leginon, to provide an open-source solution for fully automated high-throughput grid screening. This new version, broadly called Smart Leginon, emulates the actions of an operator in identifying areas on the grid to explore as potentially useful for data collection. Smart Leginon Autoscreen sequentially loads and examines grids from an automated specimen-exchange system to provide completely unattended grid screening across a set of grids. Comparisons between a multi-grid autoscreen session and conventional manual screening by 5 expert microscope operators are presented. On average, Autoscreen reduces operator time from ∼6 h to <10 min and provides a percentage of suitable images for evaluation comparable to the best operator. The ability of Smart Leginon to target holes that are particularly difficult to identify is analyzed. Finally, the utility of Smart Leginon is illustrated with three real-world multi-grid user screening/collection sessions, demonstrating the efficiency and flexibility of the software package. The fully automated functionality of Smart Leginon significantly reduces the burden on operator screening time, improves the throughput of screening and recovers idle microscope time, thereby improving availability of cryoEM services.

Learning to automate cryo-electron microscopy data collection with Ptolemy

Over the past decade, cryo-electron microscopy (cryoEM) has emerged as an important method for determining near-native, near-atomic resolution 3D structures of biological macromolecules. To meet the increasing demand for cryoEM, automated methods that improve throughput and efficiency of microscope operation are needed. Currently, the targeting algorithms provided by most data-collection software require time-consuming manual tuning of parameters for each grid, and, in some cases, operators must select targets completely manually. However, the development of fully automated targeting algorithms is non-trivial, because images often have low signal-to-noise ratios and optimal targeting strategies depend on a range of experimental parameters and macromolecule behaviors that vary between projects and collection sessions. To address this, Ptolemy provides a pipeline to automate low- and medium-magnification targeting using a suite of purpose-built computer vision and machine-learning algorithms, including mixture models, convolutional neural networks and U-Nets. Learned models in this pipeline are trained on a large set of images from real-world cryoEM data-collection sessions, labeled with locations selected by human operators. These models accurately detect and classify regions of interest in low- and medium-magnification images, and generalize to unseen sessions, as well as to images collected on different microscopes at another facility. This open-source, modular pipeline can be integrated with existing microscope control software to enable automation of cryoEM data collection and can serve as a foundation for future cryoEM automation software.

Ice thickness monitoring for cryo-EM grids by interferometry imaging

While recent technological developments contributed to breakthrough advances in single particle cryo-electron microscopy (cryo-EM), sample preparation remains a significant bottleneck for the structure determination of macromolecular complexes. A critical time factor is sample optimization that requires the use of an electron microscope to screen grids prepared under different conditions to achieve the ideal vitreous ice thickness containing the particles. Evaluating sample quality requires access to cryo-electron microscopes and a strong expertise in EM. To facilitate and accelerate the selection procedure of probes suitable for high-resolution cryo-EM, we devised a method to assess the vitreous ice layer thickness of sample coated grids. The experimental setup comprises an optical interferometric microscope equipped with a cryogenic stage and image analysis software based on artificial neural networks (ANN) for an unbiased sample selection. We present and validate this approach for different protein complexes and grid types, and demonstrate its performance for the assessment of ice quality. This technique is moderate in cost and can be easily performed on a laboratory bench. We expect that its throughput and its versatility will contribute to facilitate the sample optimization process for structural biologists.

Automated systematic evaluation of cryo-EM specimens with SmartScope

Finding the conditions to stabilize a macromolecular target for imaging remains the most critical barrier to determining its structure by cryo-electron microscopy (cryo-EM). While automation has significantly increased the speed of data collection, specimens are still screened manually, a laborious and subjective task that often determines the success of a project. Here, we present SmartScope, the first framework to streamline, standardize, and automate specimen evaluation in cryo-EM. SmartScope employs deep-learning-based object detection to identify and classify features suitable for imaging, allowing it to perform thorough specimen screening in a fully automated manner. A web interface provides remote control over the automated operation of the microscope in real time and access to images and annotation tools. Manual annotations can be used to re-train the feature recognition models, leading to improvements in performance. Our automated tool for systematic evaluation of specimens streamlines structure determination and lowers the barrier of adoption for cryo-EM.

Developing Graphene Grids for Cryoelectron Microscopy

Cryogenic electron microscopy (cryo-EM) single particle analysis has become one of the major techniques used to study high-resolution 3D structures of biological macromolecules. Specimens are generally prepared in a thin layer of vitrified ice using a holey carbon grid. However, the sample quality using this type of grid is not always ideal for high-resolution imaging even when the specimens in the test tube behave ideally. Various problems occur during a vitrification procedure, including poor/nonuniform distribution of particles, preferred orientation of particles, specimen denaturation/degradation, high background from thick ice, and beam-induced motion, which have become important bottlenecks in high-resolution structural studies using cryo-EM in many projects. In recent years, grids with support films made of graphene and its derivatives have been developed to efficiently solve these problems. Here, the various advantages of graphene grids over conventional holey carbon film grids, functionalization of graphene support films, production methods of graphene grids, and origins of pristine graphene contamination are reviewed and discussed.

Better, Faster, Cheaper: Recent Advances in Cryo-Electron Microscopy

Cryo-electron microscopy (cryo-EM) continues its remarkable growth as a method for visualizing biological objects, which has been driven by advances across the entire pipeline. Developments in both single-particle analysis and in situ tomography have enabled more structures to be imaged and determined to better resolutions, at faster speeds, and with more scientists having improved access. This review highlights recent advances at each stageof the cryo-EM pipeline and provides examples of how these techniques have been used to investigate real-world problems, including antibody development against the SARS-CoV-2 spike during the recent COVID-19 pandemic.

IceBreaker: Software for high-resolution single-particle cryo-EM with non-uniform ice

Despite the abundance of available software tools, optimal particle selection is still a vital issue in single-particle cryoelectron microscopy (cryo-EM). Regardless of the method used, most pickers struggle when ice thickness varies on a micrograph. IceBreaker allows users to estimate the relative ice gradient and flatten it by equalizing the local contrast. It allows the differentiation of particles from the background and improves overall particle picking performance. Furthermore, we introduce an additional parameter corresponding to local ice thickness for each particle. Particles with a defined ice thickness can be grouped and filtered based on this parameter during processing. These functionalities are especially valuable for on-the-fly processing to automatically pick as many particles as possible from each micrograph and to select optimal regions for data collection. Finally, estimated ice gradient distributions can be stored separately and used to inspect the quality of prepared samples.

Applications of deep learning in electron microscopy

We review the growing use of machine learning in electron microscopy (EM) driven in part by the availability of fast detectors operating at kiloHertz frame rates leading to large data sets that cannot be processed using manually implemented algorithms. We summarize the various network architectures and error metrics that have been applied to a range of EM-related problems including denoising and inpainting. We then provide a review of the application of these in both physical and life sciences, highlighting how conventional networks and training data have been specifically modified for EM.

Machine Learning for Structure Determination in Single-Particle Cryo-Electron Microscopy: A Systematic Review

Recently, single-particle cryo-electron microscopy (cryo-EM) has become an indispensable method for determining macromolecular structures at high resolution to deeply explore the relevant molecular mechanism. Its recent breakthrough is mainly because of the rapid advances in hardware and image processing algorithms, especially machine learning. As an essential support of single-particle cryo-EM, machine learning has powered many aspects of structure determination and greatly promoted its development. In this article, we provide a systematic review of the applications of machine learning in this field. Our review begins with a brief introduction of single-particle cryo-EM, followed by the specific tasks and challenges of its image processing. Then, focusing on the workflow of structure determination, we describe relevant machine learning algorithms and applications at different steps, including particle picking, 2-D clustering, 3-D reconstruction, and other steps. As different tasks exhibit distinct characteristics, we introduce the evaluation metrics for each task and summarize their dynamics of technology development. Finally, we discuss the open issues and potential trends in this promising field.

Fluorescence sensors for imaging membrane lipid domains and cholesterol

Lipid membrane domains are supramolecular lateral heterogeneities of biological membranes. Of nanoscopic dimensions, they constitute specialized hubs used by the cell as transient signaling platforms for a great variety of biologically important mechanisms. Their property to form and dissolve in the bulk lipid bilayer endow them with the ability to engage in highly dynamic processes, and temporarily recruit subpopulations of membrane proteins in reduced nanometric compartments that can coalesce to form larger mesoscale assemblies. Cholesterol is an essential component of these lipid domains; its unique molecular structure is suitable for interacting intricately with crevices and cavities of transmembrane protein surfaces through its rough β face while "talking" to fatty acid acyl chains of glycerophospholipids and sphingolipids via its smooth α face. Progress in the field of membrane domains has been closely associated with innovative improvements in fluorescence microscopy and new fluorescence sensors. These advances enabled the exploration of the biophysical properties of lipids and their supramolecular platforms. Here I review the rationale behind the use of biosensors over the last few decades and their contributions towards elucidation of the in-plane and transbilayer topography of cholesterol-enriched lipid domains and their molecular constituents. The challenges introduced by super-resolution optical microscopy are discussed, as well as possible scenarios for future developments in the field, including virtual ("no staining") staining.

Application of Monolayer Graphene and Its Derivative in Cryo-EM Sample Preparation

Cryo-electron microscopy (Cryo-EM) has become a routine technology for resolving the structure of biological macromolecules due to the resolution revolution in recent years. The specimens are typically prepared in a very thin layer of vitrified ice suspending in the holes of the perforated amorphous carbon film. However, the samples prepared by directly applying to the conventional support membranes may suffer from partial or complete denaturation caused by sticking to the air-water interface (AWI). With the application in materials, graphene has also been used recently to improve frozen sample preparation instead of a suspended conventional amorphous thin carbon. It has been proven that graphene or graphene oxide and various chemical modifications on its surface can effectively prevent particles from adsorbing to the AWI, which improves the dispersion, adsorbed number, and orientation preference of frozen particles in the ice layer. Their excellent properties and thinner thickness can significantly reduce the background noise, allowing high-resolution three-dimensional reconstructions using a minimum data set.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Biophysical Reviews: 2020-looking back, going forward

After first describing the issue contents (Biophysical Reviews-Volume 12 Issue 6), this Editorial goes on to provide a short round-up of the activities of the journal in 2020. Directly following this Editorial are two obituaries marking the recent deaths of Prof. Fumio Oosawa (Japan) and Dr. Herbert Tabor (USA)-two major figures in Biophysical/Biochemical science from the last 100 years.

Biophysical Reviews' national biophysical society partnership program

This Special Issue is focused on the Biophysical Society of Japan. It represents the first in a series tasked with introducing an individual national biophysical society to the wider biophysical community. In this Editorial for Volume 12 Issue 2, I first outline the nature and goals of this program before going on to describe the contents of the Special Issue that relate to the activities organized by the Biophysical Society of Japan and the scope of the research performed by its members.