An optimized locally adaptive non-local means denoising filter for cryo-electron microscopy data

A review of denoising methods in single-particle cryo-EM

Cryo-EM has become a vital technique for resolving macromolecular structures at near-atomic resolution, enabling the visualization of proteins and large molecular complexes. However, the images are often accompanied by extremely low SNR, which poses significant challenges for subsequent processes such as particle picking and 3D reconstruction. Effective denoising methods can substantially improve SNR, making downstream analyzes more accurate and reliable. Thus, image denoising is an essential step in cryo-EM data processing. This paper comprehensively reviews recent advances in image denoising methods for single-particle analysis, covering approaches from traditional filtering methods to the latest deep learning-based strategies. By analyzing and comparing mainstream denoising methods, this review aims to advance the field of single-particle cryo-EM denoising, facilitate the acquisition of higher-quality images, and offer valuable insights for researchers new to the field.

Adaptive differentiable grids for cryo-electron tomography reconstruction and denoising

Motivation

Tilt-series cryo-electron tomography is a powerful tool widely used in structural biology to study 3D structures of micro-organisms, macromolecular complexes, etc. Still, the reconstruction process remains an arduous task due to several challenges: The missing-wedge acquisition, sample misalignment and motion, the need to process large data, and, especially, a low signal-to-noise ratio.

Results

Inspired by the recently introduced neural representations, we propose an adaptive learning-based representation of the density field of the captured sample. This representation consists of an octree structure, where each node represents a 3D density grid optimized from the captured projections during the training process. This optimization is performed using a loss that combines a differentiable image formation model with different regularization terms: total variation, boundary consistency, and a cross-nodes non-local constraint. The final reconstruction is obtained by interpolating the learned density grid at the desired voxel positions. The evaluation of our approach using captured data of viruses and cells shows that our proposed representation is well adapted to handle missing wedges, and improves the signal-to-noise ratio of the reconstructed tomogram. The reconstruction quality is highly improved in comparison to the state-of-the-art methods, while using the lowest computing time footprint.

Availability and implementation

The code is available on Github at https://github.com/yuanhaowang1213/adaptivediffgrid_ex.

It's noisy out there! A review of denoising techniques in cryo-electron tomography

Cryo-electron tomography is the only technique that can provide sub-nanometer resolved images of cell regions or even whole cells, without the need of labeling or staining methods. Technological advances over the past decade in electron microscope stability, cameras, stage precision and software have resulted in faster acquisition speeds and considerably improved resolution. In pursuit of even better image resolution, researchers seek to reduce noise - a crucial factor affecting the reliability of the tomogram interpretation and ultimately limiting the achieved resolution. Sub-tomogram averaging is the method of choice for reducing noise in repetitive objects. However, when averaging is not applicable, a trade-off between reducing noise and conserving genuine image details must be achieved. Thus, denoising is an important process that improves the interpretability of the tomogram not only directly but also by facilitating other downstream tasks, such as segmentation and 3D visualization. Here, I review contemporary denoising techniques for cryo-electron tomography by taking into account noise-specific properties of both reconstruction and detector noise. The outcomes of different techniques are compared, in order to help researchers select the most appropriate for each dataset and to achieve better and more reliable interpretation of the tomograms.

[Progress in filters for denoising cryo-electron microscopy images]

Cryo-electron microscopy (cryo-EM) imaging has the unique potential to bridge the gap between cellular and molecular biology. Therefore, cryo-EM three-dimensional (3D) reconstruction has been rapidly developed in recent several years and applied widely in life science research to reveal the structures of large macromolecular assemblies and cellular complexes, which is critical to understanding their functions at all scales. Although the technical breakthrough in recent years, for example, the introduction of the direct detection device (DDD) camera and the development of cryo-EM software tools, made the three cryo-EM pioneers share the 2017 Nobel Prize, several bottleneck problems still exist that hamper the further increase of the resolution of single-particle reconstruction and hold back the application of in situ subnanometer structure determination by cryo-tomography. Radiation damage is still the key limiting factor in cryo-EM. In order to minimize the radiation damage and preserve as much resolution as possible, the imaging conditions of a low dose and weak contrast make cryo-EM images extremely noisy with very low signal-to-noise ratios (SNR), generally about 0.1. The high noise will obscure the fine details in cryo-EM images or reconstructed maps. Thus, a method to reduce the level of noise and improve the resolution has become an important issue. In this paper, we systematically reviewed and compared some robust filters in the cryo-EM field of two aspects, single-particle analysis (SPA) and cryo-electron tomography (cryo-ET), and especially studied their applications, such as, 3D reconstruction, visualization, structural analysis, and interpretation. Conventional approaches to noise reduction in cryo-EM imaging include the use of Gaussian, median, and bilateral filters, among other means. A Gaussian filter selects an appropriate filter kernel to conduct spatial convolution with a noisy image. Although noise with larger standard deviations in cryo-EM images can be suppressed and satisfactory performance is achieved in certain cases, this filter also blurs the images and over-smooths small-scale image features. This is especially detrimental when precise quantitative information needs to be extracted. Unlike a Gaussian filter, a median filter is based on the order statistics of the image and selects the median intensity in a window of the adjacent pixels to denoise the image. Although this filter is robust to outliers, it suffers from aliasing problems that possibly result in incorrect information for cryo-EM structure interpretation. A bilateral filter is a nonlinear filter that performs spatial weighted averaging and is more selective in the pixels allowing to contribute to the weighted sum, excluding the high frequency noise from the smoothing process. Thus, this filter can be used to smooth out noise while maintaining the edge details, which is similar to an anisotropic diffusion filter, and distinct from a Gaussian filter but its utility will be limited when the SNR of a cryo-EM image is very low. Generally, spatial filtering methods have the disadvantage of losing image resolution when reducing noise. A wavelet transform can exploit the wavelet's natural ability to separate a signal from noise at multiple image scales to allow for joint resolution in both the spatial and frequency domains, and thus has the potential to outperform existing methods. The modified wavelet shrinkage filter we developed can offer a remarkable improvement in image quality with a good compromise between detail preservation and noise smoothing. We expect that our review study on different filters can provide benefits to cryo-EM applications and the interpretation of biological structures.

An interactive ImageJ plugin for semi-automated image denoising in electron microscopy

The recent advent of 3D in electron microscopy (EM) has allowed for detection of nanometer resolution structures. This has caused an explosion in dataset size, necessitating the development of automated workflows. Moreover, large 3D EM datasets typically require hours to days to be acquired and accelerated imaging typically results in noisy data. Advanced denoising techniques can alleviate this, but tend to be less accessible to the community due to low-level programming environments, complex parameter tuning or a computational bottleneck. We present DenoisEM: an interactive and GPU accelerated denoising plugin for ImageJ that ensures fast parameter tuning and processing through parallel computing. Experimental results show that DenoisEM is one order of magnitude faster than related software and can accelerate data acquisition by a factor of 4 without significantly affecting data quality. Lastly, we show that image denoising benefits visualization and (semi-)automated segmentation and analysis of ultrastructure in various volume EM datasets.

Large 3D electron microscopy data sets frequently contain noisy data due to accelerated imaging, and denoising techniques require specialised skill sets. Here the authors introduce DenoisEM, an ImageJ plugin that democratises denoising EM data sets, enabling fast parameter tuning and processing through parallel computing.

A Monte Carlo framework for missing wedge restoration and noise removal in cryo-electron tomography

We propose a statistical method to address an important issue in cryo-electron tomography image analysis: reduction of a high amount of noise and artifacts due to the presence of a missing wedge (MW) in the spectral domain. The method takes as an input a 3D tomogram derived from limited-angle tomography, and gives as an output a 3D denoised and artifact compensated volume. The artifact compensation is achieved by filling up the MW with meaningful information. To address this inverse problem, we compute a Minimum Mean Square Error (MMSE) estimator of the uncorrupted image. The underlying high-dimensional integral is computed by applying a dedicated Markov Chain Monte-Carlo (MCMC) sampling procedure based on the Metropolis-Hasting (MH) algorithm. The proposed MWR (Missing Wedge Restoration) algorithm can be used to enhance visualization or as a pre-processing step for image analysis, including segmentation and classification of macromolecules. Results are presented for both synthetic data and real 3D cryo-electron images.

An overview of state-of-the-art image restoration in electron microscopy

In Life Science research, electron microscopy (EM) is an essential tool for morphological analysis at the subcellular level as it allows for visualization at nanometer resolution. However, electron micrographs contain image degradations such as noise and blur caused by electromagnetic interference, electron counting errors, magnetic lens imperfections, electron diffraction, etc. These imperfections in raw image quality are inevitable and hamper subsequent image analysis and visualization. In an effort to mitigate these artefacts, many electron microscopy image restoration algorithms have been proposed in the last years. Most of these methods rely on generic assumptions on the image or degradations and are therefore outperformed by advanced methods that are based on more accurate models. Ideally, a method will accurately model the specific degradations that fit the physical acquisition settings. In this overview paper, we discuss different electron microscopy image degradation solutions and demonstrate that dedicated artefact regularisation results in higher quality restoration and is applicable through recently developed probabilistic methods.

Bayesian deconvolution of scanning electron microscopy images using point-spread function estimation and non-local regularization

Microscopy is one of the most essential imaging techniques in life sciences. High-quality images are required in order to solve (potentially life-saving) biomedical research problems. Many microscopy techniques do not achieve sufficient resolution for these purposes, being limited by physical diffraction and hardware deficiencies. Electron microscopy addresses optical diffraction by measuring emitted or transmitted electrons instead of photons, yielding nanometer resolution. Despite pushing back the diffraction limit, blur should still be taken into account because of practical hardware imperfections and remaining electron diffraction. Deconvolution algorithms can remove some of the blur in post-processing but they depend on knowledge of the point-spread function (PSF) and should accurately regularize noise. Any errors in the estimated PSF or noise model will reduce their effectiveness. This paper proposes a new procedure to estimate the lateral component of the point spread function of a 3D scanning electron microscope more accurately. We also propose a Bayesian maximum a posteriori deconvolution algorithm with a non-local image prior which employs this PSF estimate and previously developed noise statistics. We demonstrate visual quality improvements and show that applying our method improves the quality of subsequent segmentation steps.

Denoising of high-resolution single-particle electron-microscopy density maps by their approximation using three-dimensional Gaussian functions

Cryo-electron microscopy (cryo-EM) of frozen-hydrated preparations of isolated macromolecular complexes is the method of choice to obtain the structure of complexes that cannot be easily studied by other experimental methods due to their flexibility or large size. An increasing number of macromolecular structures are currently being obtained at subnanometer resolution but the interpretation of structural details in such EM-derived maps is often difficult because of noise at these high-frequency signal components that reduces their contrast. In this paper, we show that the method for EM density-map approximation using Gaussian functions can be used for denoising of single-particle EM maps of high (typically subnanometer) resolution. We show its denoising performance using simulated and experimental EM density maps of several complexes.

Computation in electron microscopy

Some uses of the computer and computation in high-resolution transmission electron microscopy are reviewed. The theory of image calculation using Bloch wave and multislice methods with and without aberration correction is reviewed and some applications are discussed. The inverse problem of reconstructing the specimen structure from an experimentally measured electron microscope image is discussed. Some future directions of software development are given.

Poisson noise removal from high-resolution STEM images based on periodic block matching

Scanning transmission electron microscopy (STEM) provides sub-ångstrom, atomic resolution images of crystalline structures. However, in many applications, the ability to extract information such as atom positions, from such electron micrographs, is severely obstructed by low signal-to-noise ratios of the acquired images resulting from necessary limitations to the electron dose. We present a denoising strategy tailored to the special features of atomic-resolution electron micrographs of crystals limited by Poisson noise based on the block-matching and 3D-filtering (BM3D) algorithm by Dabov et al. We also present an economized block-matching strategy that exploits the periodic structure of the observed crystals. On simulated single-shot STEM images of inorganic materials, with incident electron doses below 4 C/cm 2, our new method achieves precisions of 7 to 15 pm and an increase in peak signal-to-noise ratio (PSNR) of 15 to 20 dB compared to noisy images and 2 to 4 dB compared to images denoised with the original BM3D.

A Zernike-moment-based non-local denoising filter for cryo-EM images

Cryo-electron microscopy (cryo-EM) plays an important role in determining the structure of proteins, viruses, and even the whole cell. It can capture dynamic structural changes of large protein complexes, which other methods such as X-ray crystallography and nuclear magnetic resonance analysis find difficult. The signal-to-noise ratio of cryo-EM images is low and the contrast is very weak, and therefore, the images are very noisy and require filtering. In this paper, a filtering method based on non-local means and Zernike moments is proposed. The method takes into account the rotational symmetry of some biological molecules to enhance the signal-to-noise ratio of cryo-EM images. The method may be useful in cryo-EM image processing such as the automatic selection of particles, orientation determination, and the building of initial models.

Developing a denoising filter for electron microscopy and tomography data in the cloud

The low radiation conditions and the predominantly phase-object image formation of cryo-electron microscopy (cryo-EM) result in extremely high noise levels and low contrast in the recorded micrographs. The process of single particle or tomographic 3D reconstruction does not completely eliminate this noise and is even capable of introducing new sources of noise during alignment or when correcting for instrument parameters. The recently developed Digital Paths Supervised Variance (DPSV) denoising filter uses local variance information to control regional noise in a robust and adaptive manner. The performance of the DPSV filter was evaluated in this review qualitatively and quantitatively using simulated and experimental data from cryo-EM and tomography in two and three dimensions. We also assessed the benefit of filtering experimental reconstructions for visualization purposes and for enhancing the accuracy of feature detection. The DPSV filter eliminates high-frequency noise artifacts (density gaps), which would normally preclude the accurate segmentation of tomography reconstructions or the detection of alpha-helices in single-particle reconstructions. This collaborative software development project was carried out entirely by virtual interactions among the authors using publicly available development and file sharing tools.

A CANDLE for a deeper in vivo insight

A new Collaborative Approach for eNhanced Denoising under Low-light Excitation (CANDLE) is introduced for the processing of 3D laser scanning multiphoton microscopy images. CANDLE is designed to be robust for low signal-to-noise ratio (SNR) conditions typically encountered when imaging deep in scattering biological specimens. Based on an optimized non-local means filter involving the comparison of filtered patches, CANDLE locally adapts the amount of smoothing in order to deal with the noise inhomogeneity inherent to laser scanning fluorescence microscopy images. An extensive validation on synthetic data, images acquired on microspheres and in vivo images is presented. These experiments show that the CANDLE filter obtained competitive results compared to a state-of-the-art method and a locally adaptive optimized non-local means filter, especially under low SNR conditions (PSNR<8dB). Finally, the deeper imaging capabilities enabled by the proposed filter are demonstrated on deep tissue in vivo images of neurons and fine axonal processes in the Xenopus tadpole brain.

Electron tomography of cells

The electron microscope has contributed deep insights into biological structure since its invention nearly 80 years ago. Advances in instrumentation and methodology in recent decades have now enabled electron tomography to become the highest resolution three-dimensional (3D) imaging technique available for unique objects such as cells. Cells can be imaged either plastic-embedded or frozen-hydrated. Then the series of projection images are aligned and back-projected to generate a 3D reconstruction or 'tomogram'. Here, we review how electron tomography has begun to reveal the molecular organization of cells and how the existing and upcoming technologies promise even greater insights into structural cell biology.