Particle picking by segmentation: a comparative study with SPIDER-based manual particle picking

Artificial Intelligence in Cryo-Electron Microscopy

Cryo-electron microscopy (cryo-EM) has become an unrivaled tool for determining the structure of macromolecular complexes. The biological function of macromolecular complexes is inextricably tied to the flexibility of these complexes. Single particle cryo-EM can reveal the conformational heterogeneity of a biochemically pure sample, leading to well-founded mechanistic hypotheses about the roles these complexes play in biology. However, the processing of increasingly large, complex datasets using traditional data processing strategies is exceedingly expensive in both user time and computational resources. Current innovations in data processing capitalize on artificial intelligence (AI) to improve the efficiency of data analysis and validation. Here, we review new tools that use AI to automate the data analysis steps of particle picking, 3D map reconstruction, and local resolution determination. We discuss how the application of AI moves the field forward, and what obstacles remain. We also introduce potential future applications of AI to use cryo-EM in understanding protein communities in cells.

DeepCryoPicker: fully automated deep neural network for single protein particle picking in cryo-EM

BACKGROUND

Cryo-electron microscopy (Cryo-EM) is widely used in the determination of the three-dimensional (3D) structures of macromolecules. Particle picking from 2D micrographs remains a challenging early step in the Cryo-EM pipeline due to the diversity of particle shapes and the extremely low signal-to-noise ratio of micrographs. Because of these issues, significant human intervention is often required to generate a high-quality set of particles for input to the downstream structure determination steps.

RESULTS

Here we propose a fully automated approach (DeepCryoPicker) for single particle picking based on deep learning. It first uses automated unsupervised learning to generate particle training datasets. Then it trains a deep neural network to classify particles automatically. Results indicate that the DeepCryoPicker compares favorably with semi-automated methods such as DeepEM, DeepPicker, and RELION, with the significant advantage of not requiring human intervention.

CONCLUSIONS

Our framework combing supervised deep learning classification with automated un-supervised clustering for generating training data provides an effective approach to pick particles in cryo-EM images automatically and accurately.

Macromolecule Particle Picking and Segmentation of a KLH Database by Unsupervised Cryo-EM Image Processing

Segmentation is one of the most important stages in the 3D reconstruction of macromolecule structures in cryo-electron microscopy. Due to the variability of macromolecules and the low signal-to-noise ratio of the structures present, there is no generally satisfactory solution to this process. This work proposes a new unsupervised particle picking and segmentation algorithm based on the composition of two well-known image filters: Anisotropic (Perona-Malik) diffusion and non-negative matrix factorization. This study focused on keyhole limpet hemocyanin (KLH) macromolecules which offer both a top view and a side view. Our proposal was able to detect both types of views and separate them automatically. In our experiments, we used 30 images from the KLH dataset of 680 positive classified regions. The true positive rate was 95.1% for top views and 77.8% for side views. The false negative rate was 14.3%. Although the false positive rate was high at 21.8%, it can be lowered with a supervised classification technique.

Robustness of signal detection in cryo-electron microscopy via a bi-objective-function approach

Background

The detection of weak signals and selection of single particles from low-contrast micrographs of frozen hydrated biomolecules by cryo-electron microscopy (cryo-EM) represents a major practical bottleneck in cryo-EM data analysis. Template-based particle picking by an objective function using fast local correlation (FLC) allows computational extraction of a large number of candidate particles from micrographs. Another independent objective function based on maximum likelihood estimates (MLE) can be used to align the images and verify the presence of a signal in the selected particles. Despite the widespread applications of the two objective functions, an optimal combination of their utilities has not been exploited. Here we propose a bi-objective function (BOF) approach that combines both FLC and MLE and explore the potential advantages and limitations of BOF in signal detection from cryo-EM data.

Results

The robustness of the BOF strategy in particle selection and verification was systematically examined with both simulated and experimental cryo-EM data. We investigated how the performance of the BOF approach is quantitatively affected by the signal-to-noise ratio (SNR) of cryo-EM data and by the choice of initialization for FLC and MLE. We quantitatively pinpointed the critical SNR (~ 0.005), at which the BOF approach starts losing its ability to select and verify particles reliably. We found that the use of a Gaussian model to initialize the MLE suppresses the adverse effects of reference dependency in the FLC function used for template-matching.

Conclusion

The BOF approach, which combines two distinct objective functions, provides a sensitive way to verify particles for downstream cryo-EM structure analysis. Importantly, reference dependency of the FLC does not necessarily transfer to the MLE, enabling the robust detection of weak signals. Our insights into the numerical behavior of the BOF approach can be used to improve automation efficiency in the cryo-EM data processing pipeline for high-resolution structural determination.

Electronic supplementary material

The online version of this article (10.1186/s12859-019-2714-8) contains supplementary material, which is available to authorized users.

Deep Consensus, a deep learning-based approach for particle pruning in cryo-electron microscopy

Single-particle cryo-electron microscopy (cryo-EM) has recently become a mainstream technique for the structural determination of macromolecules. Typical cryo-EM workflows collect hundreds of thousands of single-particle projections from thousands of micrographs using particle-picking algorithms. However, the number of false positives selected by these algorithms is large, so that a number of different 'cleaning steps' are necessary to decrease the false-positive ratio. Most commonly employed techniques for the pruning of false-positive particles are time-consuming and require user intervention. In order to overcome these limitations, a deep learning-based algorithm named Deep Consensus is presented in this work. Deep Consensus works by computing a smart consensus over the output of different particle-picking algorithms, resulting in a set of particles with a lower false-positive ratio than the initial set obtained by the pickers. Deep Consensus is based on a deep convolutional neural network that is trained on a semi-automatically generated data set. The performance of Deep Consensus has been assessed on two well known experimental data sets, virtually eliminating user intervention for pruning, and enhances the reproducibility and objectivity of the whole process while achieving precision and recall figures above 90%.

A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy

BACKGROUND

Single-particle cryo-electron microscopy (cryo-EM) has become a mainstream tool for the structural determination of biological macromolecular complexes. However, high-resolution cryo-EM reconstruction often requires hundreds of thousands of single-particle images. Particle extraction from experimental micrographs thus can be laborious and presents a major practical bottleneck in cryo-EM structural determination. Existing computational methods for particle picking often use low-resolution templates for particle matching, making them susceptible to reference-dependent bias. It is critical to develop a highly efficient template-free method for the automatic recognition of particle images from cryo-EM micrographs.

RESULTS

We developed a deep learning-based algorithmic framework, DeepEM, for single-particle recognition from noisy cryo-EM micrographs, enabling automated particle picking, selection and verification in an integrated fashion. The kernel of DeepEM is built upon a convolutional neural network (CNN) composed of eight layers, which can be recursively trained to be highly "knowledgeable". Our approach exhibits an improved performance and accuracy when tested on the standard KLH dataset. Application of DeepEM to several challenging experimental cryo-EM datasets demonstrated its ability to avoid the selection of un-wanted particles and non-particles even when true particles contain fewer features.

CONCLUSIONS

The DeepEM methodology, derived from a deep CNN, allows automated particle extraction from raw cryo-EM micrographs in the absence of a template. It demonstrates an improved performance, objectivity and accuracy. Application of this novel method is expected to free the labor involved in single-particle verification, significantly improving the efficiency of cryo-EM data processing.

Automated particle picking for low-contrast macromolecules in cryo-electron microscopy

Cryo-electron microscopy is an increasingly popular tool for studying the structure and dynamics of biological macromolecules at high resolution. A crucial step in automating single-particle reconstruction of a biological sample is the selection of particle images from a micrograph. We present a novel algorithm for selecting particle images in low-contrast conditions; it proves more effective than the human eye on close-to-focus micrographs, yielding improved or comparable resolution in reconstructions of two macromolecular complexes.

gEMpicker: a highly parallel GPU-accelerated particle picking tool for cryo-electron microscopy

Background

Picking images of particles in cryo-electron micrographs is an important step in solving the 3D structures of large macromolecular assemblies. However, in order to achieve sub-nanometre resolution it is often necessary to capture and process many thousands or even several millions of 2D particle images. Thus, a computational bottleneck in reaching high resolution is the accurate and automatic picking of particles from raw cryo-electron micrographs.

Results

We have developed “gEMpicker”, a highly parallel correlation-based particle picking tool. To our knowledge, gEMpicker is the first particle picking program to use multiple graphics processor units (GPUs) to accelerate the calculation. When tested on the publicly available keyhole limpet hemocyanin dataset, we find that gEMpicker gives similar results to the FindEM program. However, compared to calculating correlations on one core of a contemporary central processor unit (CPU), running gEMpicker on a modern GPU gives a speed-up of about 27 ×. To achieve even higher processing speeds, the basic correlation calculations are accelerated considerably by using a hierarchy of parallel programming techniques to distribute the calculation over multiple GPUs and CPU cores attached to multiple nodes of a computer cluster. By using a theoretically optimal reduction algorithm to collect and combine the cluster calculation results, the speed of the overall calculation scales almost linearly with the number of cluster nodes available.

Conclusions

The very high picking throughput that is now possible using GPU-powered workstations or computer clusters will help experimentalists to achieve higher resolution 3D reconstructions more rapidly than before.

Three-dimensional reconstruction of icosahedral particles from single micrographs in real time at the microscope

Single particle analysis is a valuable tool in cryo-electron microscopy for determining the structure of biological complexes. However, the conformational state and the preparation of the sample are factors that play a critical role in the ultimate attainable resolution. In some cases extensive analysis at the microscope of a sample under different conditions is required to derive the optimal acquisition conditions. Currently this analysis is limited to raw micrographs, thus conveying only limited information on the structure of the complex. We are developing a computing system that generates a three-dimensional reconstruction from a single micrograph acquired under cryogenic and low dose conditions, and containing particles with icosahedral symmetry. The system provides the microscopist with immediate structural information from a sample while it is in the microscope and during the preliminary acquisition stage. The system is designed to run without user intervention on a multi-processor computing resource and integrates all the processing steps required for the analysis. Tests performed on experimental data sets show that the probability of obtaining a reliable reconstruction from one micrograph is primarily determined by the quality of the sample, with success rates close to 100% when sample conditions are optimal, and decreasing to about 60% when conditions are sub-optimal. The time required to generate a reconstruction depends significantly on the diameter of the particles, and in most instances takes about 1min. The proposed approach can provide valuable three-dimensional information, albeit at low resolution, on conformational states, epitope binding, and stoichiometry of icosahedral multi-protein complexes.

A 3D image filter for parameter-free segmentation of macromolecular structures from electron tomograms

3D image reconstruction of large cellular volumes by electron tomography (ET) at high (≤ 5 nm) resolution can now routinely resolve organellar and compartmental membrane structures, protein coats, cytoskeletal filaments, and macromolecules. However, current image analysis methods for identifying in situ macromolecular structures within the crowded 3D ultrastructural landscape of a cell remain labor-intensive, time-consuming, and prone to user-bias and/or error. This paper demonstrates the development and application of a parameter-free, 3D implementation of the bilateral edge-detection (BLE) algorithm for the rapid and accurate segmentation of cellular tomograms. The performance of the 3D BLE filter has been tested on a range of synthetic and real biological data sets and validated against current leading filters-the pseudo 3D recursive and Canny filters. The performance of the 3D BLE filter was found to be comparable to or better than that of both the 3D recursive and Canny filters while offering the significant advantage that it requires no parameter input or optimisation. Edge widths as little as 2 pixels are reproducibly detected with signal intensity and grey scale values as low as 0.72% above the mean of the background noise. The 3D BLE thus provides an efficient method for the automated segmentation of complex cellular structures across multiple scales for further downstream processing, such as cellular annotation and sub-tomogram averaging, and provides a valuable tool for the accurate and high-throughput identification and annotation of 3D structural complexity at the subcellular level, as well as for mapping the spatial and temporal rearrangement of macromolecular assemblies in situ within cellular tomograms.

Reference-free particle selection enhanced with semi-supervised machine learning for cryo-electron microscopy

Reference-based methods have dominated the approaches to the particle selection problem, proving fast, and accurate on even the most challenging micrographs. A reference volume, however, is not always available and compiling a set of reference projections from the micrographs themselves requires significant effort to attain the same level of accuracy. We propose a reference-free method to quickly extract particles from the micrograph. The method is augmented with a new semi-supervised machine-learning algorithm to accurately discriminate particles from contaminants and noise.

A clarification of the terms used in comparing semi-automated particle selection algorithms in cryo-EM

Many cyro-EM datasets are heterogeneous stemming from molecules undergoing conformational changes. The need to characterize each of the substrates with sufficient resolution entails a large increase in the data flow and motivates the development of more effective automated particle selection algorithms. Concepts and procedures from the machine-learning field are increasingly employed toward this end. However, a review of recent literature has revealed a discrepancy in terminology of the performance scores used to compare particle selection algorithms, and this has subsequently led to ambiguities in the meaning of claimed performance. In an attempt to curtail the perpetuation of this confusion and to disentangle past mistakes, we review the performance of published particle selection efforts with a set of explicitly defined performance scores using the terminology established and accepted within the field of machine learning.

Cryo-electron microscopy structure of an adenovirus-integrin complex indicates conformational changes in both penton base and integrin

A structure of adenovirus type 12 (HAdV12) complexed with a soluble form of integrin alphavbeta5 was determined by cryo-electron microscopy (cryoEM) image reconstruction. Subnanometer resolution (8 A) was achieved for the icosahedral capsid with moderate resolution (27 A) for integrin density above each penton base. Modeling with alphavbeta3 and alpha(IIb)beta3 crystal structures indicates that a maximum of four integrins fit over the pentameric penton base. The close spacing (approximately 60 A) of the RGD protrusions on penton base precludes integrin binding in the same orientation to neighboring RGD sites. Flexible penton-base RGD loops and incoherent averaging of bound integrin molecules explain the moderate resolution observed for the integrin density. A model with four integrins bound to a penton base suggests that integrin might extend one RGD-loop in the direction that could induce a conformational change in the penton base involving clockwise untwisting of the pentamer. A global conformational change in penton base could be one step on the way to the release of Ad vertex proteins during cell entry. Comparison of the cryoEM structure with bent and extended models for the integrin ectodomain reveals that integrin adopts an extended conformation when bound to the Ad penton base, a multivalent viral ligand. These findings shed further light on the structural basis of integrin binding to biologically relevant ligands, as well as on the molecular events leading to HAdV cell entry.

Cryo-electron microscopy structure of adenovirus type 2 temperature-sensitive mutant 1 reveals insight into the cell entry defect

The structure of the adenovirus type 2 temperature-sensitive mutant 1 (Ad2ts1) was determined to a resolution of 10 A by cryo-electron microscopy single-particle reconstruction. Ad2ts1 was prepared at a nonpermissive temperature and contains the precursor forms of the capsid proteins IIIa, VI, and VIII; the core proteins VII, X (mu), and terminal protein (TP); and the L1-52K protein. Cell entry studies have shown that although Ad2ts1 can bind the coxsackievirus and Ad receptor and undergo internalization via alphav integrins, this mutant does not escape from the early endosome and is targeted for degradation. Comparison of the Ad2ts1 structure to that of mature Ad indicates that Ad2ts1 has a different core architecture. The Ad2ts1 core is closely associated with the icosahedral capsid, a connection which may be mediated by preproteins IIIa and VI. Density within hexon cavities is assigned to preprotein VI, and membrane disruption assays show that hexon shields the lytic activity of both the mature and precursor forms of protein VI. The internal surface of the penton base in Ad2ts1 appears to be anchored to the core by interactions with preprotein IIIa. Our structural analyses suggest that these connections to the core inhibit the release of the vertex proteins and lead to the cell entry defect of Ad2ts1.

DoG Picker and TiltPicker: software tools to facilitate particle selection in single particle electron microscopy

Solving the structure of macromolecular complexes using transmission electron microscopy can be an arduous task. Many of the steps in this process rely strongly on the aid of pre-existing structural knowledge, and are greatly complicated when this information is unavailable. Here, we present two software tools meant to facilitate particle picking, an early stage in the single-particle processing of unknown macromolecules. The first tool, DoG Picker, is an efficient and reasonably general, particle picker based on the Difference of Gaussians (DoG) image transform. It can function alone, as a reference-free particle picker with the unique ability to sort particles based on size, or it can also be used as a way to bootstrap the creation of templates or training datasets for other particle pickers. The second tool is TiltPicker, an interactive graphical interface application designed to streamline the selection of particle pairs from tilted-pair datasets. In many respects, TiltPicker is a re-implementation of the SPIDER WEB tilted-particle picker, but built on modern computer frameworks making it easier to deploy and maintain. The TiltPicker program also includes several useful new features beyond those of its predecessor.

Adenovirus serotype 5 hexon is critical for virus infection of hepatocytes in vivo

Human species C adenovirus serotype 5 (Ad5) is the most common viral vector used in clinical studies worldwide. Ad5 vectors infect liver cells in vivo with high efficiency via a poorly defined mechanism, which involves virus binding to vitamin K-dependent blood coagulation factors. Here, we report that the major Ad5 capsid protein, hexon, binds human coagulation factor X (FX) with an affinity of 229 pM. This affinity is 40-fold stronger than the reported affinity of Ad5 fiber for the cellular receptor coxsackievirus and adenovirus receptor, CAR. Cryoelectron microscopy and single-particle image reconstruction revealed that the FX attachment site is localized to the central depression at the top of the hexon trimer. Hexon-mutated virus bearing a large insertion in hexon showed markedly reduced FX binding in vitro and failed to deliver a transgene to hepatocytes in vivo. This study describes the mechanism of FX binding to Ad5 and demonstrates the critical role of hexon for virus infection of hepatocytes in vivo.

Bilateral edge filter: photometrically weighted, discontinuity based edge detection

Edge-detection algorithms have the potential to play an increasingly important role both in single particle analysis (for the detection of randomly oriented particles), and in tomography (for the segmentation of 3D volumes). However, the majority of traditional linear filters are significantly affected by noise as well as artefacts, and offer limited selectivity. The Bilateral edge filter presented here is an adaptation of the Bilateral filter [Jiang, W., Baker, M.L., Wu, Q., Bajaj, C., Chiu, W., 2003. Applications of a bilateral denoising filter in biological electron microscopy. J. Struct. Biol. 144, 114-122] designed for enhanced edge detection. It uses photometric weighting to identify significant discontinuities (representing edges), minimizing artefacts and noise. Compared with common edge-detectors (LoG, Marr-Hildreth) the Bilateral edge filter yielded significantly better results. Indeed data was of a similar quality to that of the Canny edge-detector, which is considered as a leading standard in edge detection [Basu, M., 2002. Gaussian-based edge-detection methods-a survey. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 32, 252-260]. Compared to the Canny edge-detector the Bilateral edge-detector has the advantages that it only requires the adjustment of a single parameter, is theoretically faster for reasonably sized images, and can be used in selective contrast enhancement of images. The simplicity and speed of the filter for single particle and tomographic analysis are discussed.

Imaging Atomic Structure and Dynamics with Ultrafast X-ray Scattering

Measuring atomic-resolution images of materials with x-ray photons during chemical reactions or physical transformations resides at the technological forefront of x-ray science. New x-ray-based experimental capabilities have been closely linked with advances in x-ray sources, a trend that will continue with the impending arrival of x-ray-free electron lasers driven by electron accelerators. We discuss recent advances in ultrafast x-ray science and coherent imaging made possible by linear-accelerator-based light sources. These studies highlight the promise of ultrafast x-ray lasers, as well as the technical challenges and potential range of applications that will accompany these transformative x-ray light sources.

SCA: Symmetry-based center assignment of 2D projections of symmetric 3D objects

A method for finding the center of cryo-EM images which correspond to the projections of a symmetric 3D structure, based on mathematical properties of symmetry adapted functions and the Fourier-Bessel transform, is presented. It is a model independent one-step procedure with no parameters to be chosen by the user. The proposed method is tested in one synthetic tetrahedral case with different noise levels and in two real cases with D7 and icosahedral symmetries.

Visualization of alpha-helices in a 6-angstrom resolution cryoelectron microscopy structure of adenovirus allows refinement of capsid protein assignments

The structure of adenovirus was determined to a resolution of 6 A by cryoelectron microscopy (cryoEM) single-particle image reconstruction. Docking of the hexon and penton base crystal structures into the cryoEM density established that alpha-helices of 10 or more residues are resolved as rods. A difference map was calculated by subtracting a pseudoatomic capsid from the cryoEM reconstruction. The resulting density was analyzed in terms of observed alpha-helices and secondary structure predictions for the additional capsid proteins that currently lack atomic resolution structures (proteins IIIa, VI, VIII, and IX). Protein IIIa, which is predicted to be highly alpha-helical, is assigned to a cluster of helices observed below the penton base on the inner capsid surface. Protein VI is present in approximately 1.5 copies per hexon trimer and is predicted to have two long alpha-helices, one of which appears to lie inside the hexon cavity. Protein VIII is cleaved by the adenovirus protease into two fragments of 7.6 and 12.1 kDa, and the larger fragment is predicted to have one long alpha-helix, in agreement with the observed density for protein VIII on the inner capsid surface. Protein IX is predicted to have one long alpha-helix, which also has a strongly indicated propensity for coiled-coil formation. A region of density near the facet edge is now resolved as a four-helix bundle and is assigned to four copies of the C-terminal alpha-helix from protein IX.

SwarmPS: rapid, semi-automated single particle selection software

Single particle analysis (SPA) coupled with high-resolution electron cryo-microscopy is emerging as a powerful technique for the structure determination of membrane protein complexes and soluble macromolecular assemblies. Current estimates suggest that approximately 10(4)-10(5) particle projections are required to attain a 3A resolution 3D reconstruction (symmetry dependent). Selecting this number of molecular projections differing in size, shape and symmetry is a rate-limiting step for the automation of 3D image reconstruction. Here, we present Swarm(PS), a feature rich GUI based software package to manage large scale, semi-automated particle picking projects. The software provides cross-correlation and edge-detection algorithms. Algorithm-specific parameters are transparently and automatically determined through user interaction with the image, rather than by trial and error. Other features include multiple image handling (approximately 10(2)), local and global particle selection options, interactive image freezing, automatic particle centering, and full manual override to correct false positives and negatives. Swarm(PS) is user friendly, flexible, extensible, fast, and capable of exporting boxed out projection images, or particle coordinates, compatible with downstream image processing suites.