Estimating alignment errors in sets of 2-D images

Transformations between rotational and translational invariants formulated in reciprocal spaces

Correlation functions play an important role in the theoretical underpinnings of many disparate areas of the physical sciences: in particular, scattering theory. More recently, they have become useful in the classification of objects in areas such as computer vision and our area of cryoEM. Our primary classification scheme in the cryoEM image processing system, EMAN2, is now based on third order invariants formulated in Fourier space. This allows a factor of 8 speed up in the two classification procedures inherent in our software pipeline, because it allows for classification without the need for computationally costly alignment procedures. In this work, we address several formal and practical aspects of such multispectral invariants. We show that we can formulate such invariants in the representation in which the original signal is most compact. We explicitly construct transformations between invariants in different orientations for arbitrary order of correlation functions and dimension. We demonstrate that third order invariants distinguish 2D mirrored patterns (unlike the radial power spectrum), which is a fundamental aspects of its classification efficacy. We show the limitations of 3rd order invariants also, by giving an example of a wide family of patterns with identical (vanishing) set of 3rd order invariants. For sufficiently rich patterns, the third order invariants should distinguish typical images, textures and patterns.

Cryo-EM image alignment: From pair-wise to joint with deep unsupervised difference learning

Cryo-electron microscopy (cryo-EM) single-particle analysis is a revolutionary imaging technique to resolve and visualize biomacromolecules. Image alignment in cryo-EM is an important and basic step to improve the precision of the image distance calculation. However, it is a very challenging task due to high noise and low signal-to-noise ratio. Therefore, we propose a new deep unsupervised difference learning (UDL) strategy with novel pseudo-label guided learning network architecture and apply it to pair-wise image alignment in cryo-EM. The training framework is fully unsupervised. Furthermore, a variant of UDL called joint UDL (JUDL), is also proposed, which is capable of utilizing the similarity information of the whole dataset and thus further increase the alignment precision. Assessments on both real-world and synthetic cryo-EM single-particle image datasets suggest the new unsupervised joint alignment method can achieve more accurate alignment results. Our method is highly efficient by taking advantages of GPU devices. The source code of our methods is publicly available at "http://www.csbio.sjtu.edu.cn/bioinf/JointUDL/" for academic use.

Non-uniformity of projection distributions attenuates resolution in Cryo-EM

Virtually all single-particle cryo-EM experiments currently suffer from specimen adherence to the air-water interface, leading to a non-uniform distribution in the set of projection views. Whereas it is well accepted that uniform projection distributions can lead to high-resolution reconstructions, non-uniform (anisotropic) distributions can negatively affect map quality, elongate structural features, and in some cases, prohibit interpretation altogether. Although some consequences of non-uniform sampling have been described qualitatively, we know little about how sampling quantitatively affects resolution in cryo-EM. Here, we show how inhomogeneity in any projection distribution scheme attenuates the global Fourier Shell Correlation (FSC) in relation to the number of particles and a single geometrical parameter, which we term the sampling compensation factor (SCF). The reciprocal of the SCF is defined as the average over Fourier shells of the reciprocal of the per-particle sampling and normalized to unity for uniform distributions. The SCF therefore ranges from one to zero, with values close to the latter implying large regions of poorly sampled or completely missing data in Fourier space. Using two synthetic test cases, influenza hemagglutinin and human apoferritin, we demonstrate how any amount of sampling inhomogeneity always attenuates the FSC compared to a uniform distribution. We advocate quantitative evaluation of the SCF criterion to approximate the effect of non-uniform sampling on resolution within experimental single-particle cryo-EM reconstructions.

Single-particle reconstruction statistics: a diagnostic tool in solving biomolecular structures by cryo-EM

In single-particle analysis (SPA), the aim is to obtain a 3D reconstruction of a biological molecule from 2D electron micrographs to the highest level of detail or resolution as possible. Current practice is to collect large volumes of data, hoping to reach high-resolution maps through sheer numbers. However, adding more particles from a specific data set eventually leads to diminishing improvements in resolution. Understanding what these resolution limits are and how to deal with them are important in optimization and automation of SPA. This study revisits the theory of 3D reconstruction and demonstrates how the associated statistics can provide a diagnostic tool to improve SPA. Small numbers of images already give sufficient information on micrograph quality and the amount of data required to reach high resolution. Such feedback allows the microscopist to improve sample-preparation and imaging parameters before committing to extensive data collection. Once a larger data set is available, a B factor can be determined describing the suppression of the signal owing to one or more causes, such as specimen movement, radiation damage, alignment inaccuracy and structural variation. Insight into the causes of signal suppression can then guide the user to consider appropriate actions to obtain better reconstructions.

Visualization and quality assessment of the contrast transfer function estimation

The contrast transfer function (CTF) describes an undesirable distortion of image data from a transmission electron microscope. Many users of full-featured processing packages are often new to electron microscopy and are unfamiliar with the CTF concept. Here we present a common graphical output to clearly demonstrate the CTF fit quality independent of estimation software. Separately, many software programs exist to estimate the four CTF parameters, but their results are difficult to compare across multiple runs and it is all but impossible to select the best parameters to use for further processing. A new measurement is presented based on the correlation falloff of the calculated CTF oscillations against the normalized oscillating signal of the data, called the CTF resolution. It was devised to provide a robust numerical quality metric of every CTF estimation for high-throughput screening of micrographs and to select the best parameters for each micrograph. These new CTF visualizations and quantitative measures will help users better assess the quality of their CTF parameters and provide a mechanism to choose the best CTF tool for their data.

Validation of 3D EM Reconstructions: The Phantom in the Noise

Validation is a necessity to trust the structures solved by electron microscopy by single particle techniques. The impressive achievements in single particle reconstruction fuel its expansion beyond a small community of image processing experts. This poses the risk of inappropriate data processing with dubious results. Nowhere is it more clearly illustrated than in the recovery of a reference density map from pure noise aligned to that map—a phantom in the noise. Appropriate use of existing validating methods such as resolution-limited alignment and the processing of independent data sets (“gold standard”) avoid this pitfall. However, these methods can be undermined by biases introduced in various subtle ways. How can we test that a map is a coherent structure present in the images selected from the micrographs? In stead of viewing the phantom emerging from noise as a cautionary tale, it should be used as a defining baseline. Any map is always recoverable from noise images, provided a sufficient number of images are aligned and used in reconstruction. However, with smaller numbers of images, the expected coherence in the real particle images should yield better reconstructions than equivalent numbers of noise or background images, even without masking or imposing resolution limits as potential biases. The validation test proposed is therefore a simple alignment of a limited number of micrograph and noise images against the final reconstruction as reference, demonstrating that the micrograph images yield a better reconstruction. I examine synthetic cases to relate the resolution of a reconstruction to the alignment error as a function of the signal-to-noise ratio. I also administered the test to real cases of publicly available data. Adopting such a test can aid the microscopist in assessing the usefulness of the micrographs taken before committing to lengthy processing with questionable outcomes.

Joint registration and super-resolution with omnidirectional images

This paper addresses the reconstruction of high-resolution omnidirectional images from multiple low-resolution images with inexact registration. When omnidirectional images from low-resolution vision sensors can be uniquely mapped on the 2-sphere, such a reconstruction can be described as a transform-domain super-resolution problem in a spherical imaging framework. We describe how several spherical images with arbitrary rotations in the SO(3) rotation group contribute to the reconstruction of a high-resolution image with help of the spherical Fourier transform (SFT). As low-resolution images might not be perfectly registered in practice, the impact of inaccurate alignment on the transform coefficients is analyzed. We then cast the joint registration and super-resolution problem as a total least-squares norm minimization problem in the SFT domain. A l(1)-regularized total least-squares problem is considered and solved efficiently by interior point methods. Experiments with synthetic and natural images show that the proposed methods lead to effective reconstruction of high-resolution images even when large registration errors exist in the low-resolution images. The quality of the reconstructed images also increases rapidly with the number of low-resolution images, which demonstrates the benefits of the proposed solution in super-resolution schemes. Finally, we highlight the benefit of the additional regularization constraint that clearly leads to reduced noise and improved reconstruction quality.

A Stochastic Kinematic Model of Class Averaging in Single-Particle Electron Microscopy

Single-particle electron microscopy is an experimental technique that is used to determine the 3D structure of biological macromolecules and the complexes that they form. In general, image processing techniques and reconstruction algorithms are applied to micrographs, which are two-dimensional (2D) images taken by electron microscopes. Each of these planar images can be thought of as a projection of the macromolecular structure of interest from an a priori unknown direction. A class is defined as a collection of projection images with a high degree of similarity, presumably resulting from taking projections along similar directions. In practice, micrographs are very noisy and those in each class are aligned and averaged in order to reduce the background noise. Errors in the alignment process are inevitable due to noise in the electron micrographs. This error results in blurry averaged images. In this paper, we investigate how blurring parameters are related to the properties of the background noise in the case when the alignment is achieved by matching the mass centers and the principal axes of the experimental images. We observe that the background noise in micrographs can be treated as Gaussian. Using the mean and variance of the background Gaussian noise, we derive equations for the mean and variance of translational and rotational misalignments in the class averaging process. This defines a Gaussian probability density on the Euclidean motion group of the plane. Our formulation is validated by convolving the derived blurring function representing the stochasticity of the image alignments with the underlying noiseless projection and comparing with the original blurry image.

Resolution measures in molecular electron microscopy

Resolution measures in molecular electron microscopy provide means to evaluate quality of macromolecular structures computed from sets of their two-dimensional (2D) line projections. When the amount of detail in the computed density map is low there are no external standards by which the resolution of the result can be judged. Instead, resolution measures in molecular electron microscopy evaluate consistency of the results in reciprocal space and present it as a one-dimensional (1D) function of the modulus of spatial frequency. Here we provide description of standard resolution measures commonly used in electron microscopy. We point out that the organizing principle is the relationship between these measures and the spectral signal-to-noise ratio (SSNR) of the computed density map. Within this framework it becomes straightforward to describe the connection between the outcome of resolution evaluations and the quality of electron microscopy maps, in particular, the optimum filtration, in the Wiener sense, of the computed map. We also provide a discussion of practical difficulties of evaluation of resolution in electron microscopy, particularly in terms of its sensitivity to data processing operations used during structure determination process in single particle analysis and in electron tomography (ET).

Automation in single-particle electron microscopy connecting the pieces

Throughout the history of single-particle electron microscopy (EM), automated technologies have seen varying degrees of emphasis and development, usually depending upon the contemporary demands of the field. We are currently faced with increasingly sophisticated devices for specimen preparation, vast increases in the size of collected data sets, comprehensive algorithms for image processing, sophisticated tools for quality assessment, and an influx of interested scientists from outside the field who might lack the skills of experienced microscopists. This situation places automated techniques in high demand. In this chapter, we provide a generic definition of and discuss some of the most important advances in automated approaches to specimen preparation, grid handling, robotic screening, microscope calibrations, data acquisition, image processing, and computational infrastructure. Each section describes the general problem and then provides examples of how that problem has been addressed through automation, highlighting available processing packages, and sometimes describing the particular approach at the National Resource for Automated Molecular Microscopy (NRAMM). We contrast the more familiar manual procedures with automated approaches, emphasizing breakthroughs as well as current limitations. Finally, we speculate on future directions and improvements in automated technologies. Our overall goal is to present automation as more than simply a tool to save time. Rather, we aim to illustrate that automation is a comprehensive and versatile strategy that can deliver biological information on an unprecedented scale beyond the scope available with classical manual approaches.

Image restoration in cryo-electron microscopy

Image restoration techniques are used to obtain, given experimental measurements, the best possible approximation of the original object within the limits imposed by instrumental conditions and noise level in the data. In molecular electron microscopy (EM), we are mainly interested in linear methods that preserve the respective relationships between mass densities within the restored map. Here, we describe the methodology of image restoration in structural EM, and more specifically, we will focus on the problem of the optimum recovery of Fourier amplitudes given electron microscope data collected under various defocus settings. We discuss in detail two classes of commonly used linear methods, the first of which consists of methods based on pseudoinverse restoration, and which is further subdivided into mean-square error, chi-square error, and constrained based restorations, where the methods in the latter two subclasses explicitly incorporates non-white distribution of noise in the data. The second class of methods is based on the Wiener filtration approach. We show that the Wiener filter-based methodology can be used to obtain a solution to the problem of amplitude correction (or "sharpening") of the EM map that makes it visually comparable to maps determined by X-ray crystallography, and thus amenable to comparative interpretation. Finally, we present a semiheuristic Wiener filter-based solution to the problem of image restoration given sets of heterogeneous solutions. We conclude the chapter with a discussion of image restoration protocols implemented in commonly used single particle software packages.

Estimation of variance in single-particle reconstruction using the bootstrap technique

Density maps of a molecule obtained by single-particle reconstruction from thousands of molecule projections exhibit strong changes in local definition and reproducibility, as a consequence of conformational variability of the molecule and non-stoichiometry of ligand binding. These changes complicate the interpretation of density maps in terms of molecular structure. A three-dimensional (3-D) variance map provides an effective tool to assess the structural definition in each volume element. In this work, the different contributions to the 3-D variance in a single-particle reconstruction are discussed, and an effective method for the estimation of the 3-D variance map is proposed, using a bootstrap technique of sampling. Computations with test data confirm the viability, computational efficiency, and accuracy of the method under conditions encountered in practical circumstances.

Transform-based backprojection for volume reconstruction of large format electron microscope tilt series

Alignment of the individual images of a tilt series is a critical step in obtaining high-quality electron microscope reconstructions. We report on general methods for producing good alignments, and utilizing the alignment data in subsequent reconstruction steps. Our alignment techniques utilize bundle adjustment. Bundle adjustment is the simultaneous calculation of the position of distinguished markers in the object space and the transforms of these markers to their positions in the observed images, along the bundle of particle trajectories along which the object is projected to each EM image. Bundle adjustment techniques are general enough to encompass the computation of linear, projective or nonlinear transforms for backprojection, and can compensate for curvilinear trajectories through the object, sample warping, and optical aberration. We will also report on new reconstruction codes and describe our results using these codes.

Fast rotational matching of single-particle images

The presence of noise and absence of contrast in electron micrographs lead to a reduced resolution of the final 3D reconstruction, due to the inherent limitations of single-particle image alignment. The fast rotational matching (FRM) algorithm was introduced recently for an accurate alignment of 2D images under such challenging conditions. Here, we implemented this algorithm for the first time in a standard 3D reconstruction package used in electron microscopy. This allowed us to carry out exhaustive tests of the robustness and reliability in iterative orientation determination, classification, and 3D reconstruction on simulated and experimental image data. A classification test on GroEL chaperonin images demonstrates that FRM assigns up to 13% more images to their correct reference orientation, compared to the classical self-correlation function method. Moreover, at sub-nanometer resolution, GroEL and rice dwarf virus reconstructions exhibit a remarkable resolution gain of 10-20% that is attributed to the novel image alignment kernel.