Comparing cryo-EM structures

Community recommendations on cryoEM data archiving and validation

In January 2020, a workshop was held at EMBL-EBI (Hinxton, UK) to discuss data requirements for the deposition and validation of cryoEM structures, with a focus on single-particle analysis. The meeting was attended by 47 experts in data processing, model building and refinement, validation, and archiving of such structures. This report describes the workshop's motivation and history, the topics discussed, and the resulting consensus recommendations. Some challenges for future methods-development efforts in this area are also highlighted, as is the implementation to date of some of the recommendations.

Predictive modeling and cryo-EM: A synergistic approach to modeling macromolecular structure

Over the last 15 years, structural biology has seen unprecedented development and improvement in two areas: electron cryo-microscopy (cryo-EM) and predictive modeling. Once relegated to low resolutions, single-particle cryo-EM is now capable of achieving near-atomic resolutions of a wide variety of macromolecular complexes. Ushered in by AlphaFold, machine learning has powered the current generation of predictive modeling tools, which can accurately and reliably predict models for proteins and some complexes directly from the sequence alone. Although they offer new opportunities individually, there is an inherent synergy between these techniques, allowing for the construction of large, complex macromolecular models. Here, we give a brief overview of these approaches in addition to illustrating works that combine these techniques for model building. These examples provide insight into model building, assessment, and limitations when integrating predictive modeling with cryo-EM density maps. Together, these approaches offer the potential to greatly accelerate the generation of macromolecular structural insights, particularly when coupled with experimental data.

Community recommendations on cryoEM data archiving and validation: Outcomes of a wwPDB/EMDB workshop on cryoEM data management, deposition and validation

In January 2020, a workshop was held at EMBL-EBI (Hinxton, UK) to discuss data requirements for deposition and validation of cryoEM structures, with a focus on single-particle analysis. The meeting was attended by 47 experts in data processing, model building and refinement, validation, and archiving of such structures. This report describes the workshop's motivation and history, the topics discussed, and consensus recommendations resulting from the workshop. Some challenges for future methods-development efforts in this area are also highlighted, as is the implementation to date of some of the recommendations.

Outcomes of the EMDataResource Cryo-EM Ligand Modeling Challenge

The EMDataResource Ligand Model Challenge aimed to assess the reliability and reproducibility of modeling ligands bound to protein and protein/nucleic-acid complexes in cryogenic electron microscopy (cryo-EM) maps determined at near-atomic (1.9-2.5 Å) resolution. Three published maps were selected as targets: E. coli beta-galactosidase with inhibitor, SARS-CoV-2 RNA-dependent RNA polymerase with covalently bound nucleotide analog, and SARS-CoV-2 ion channel ORF3a with bound lipid. Sixty-one models were submitted from 17 independent research groups, each with supporting workflow details. We found that (1) the quality of submitted ligand models and surrounding atoms varied, as judged by visual inspection and quantification of local map quality, model-to-map fit, geometry, energetics, and contact scores, and (2) a composite rather than a single score was needed to assess macromolecule+ligand model quality. These observations lead us to recommend best practices for assessing cryo-EM structures of liganded macromolecules reported at near-atomic resolution.

Electron microscopy holdings of the Protein Data Bank: the impact of the resolution revolution, new validation tools, and implications for the future

As a discipline, structural biology has been transformed by the three-dimensional electron microscopy (3DEM) "Resolution Revolution" made possible by convergence of robust cryo-preservation of vitrified biological materials, sample handling systems, and measurement stages operating a liquid nitrogen temperature, improvements in electron optics that preserve phase information at the atomic level, direct electron detectors (DEDs), high-speed computing with graphics processing units, and rapid advances in data acquisition and processing software. 3DEM structure information (atomic coordinates and related metadata) are archived in the open-access Protein Data Bank (PDB), which currently holds more than 11,000 3DEM structures of proteins and nucleic acids, and their complexes with one another and small-molecule ligands (~ 6% of the archive). Underlying experimental data (3DEM density maps and related metadata) are stored in the Electron Microscopy Data Bank (EMDB), which currently holds more than 21,000 3DEM density maps. After describing the history of the PDB and the Worldwide Protein Data Bank (wwPDB) partnership, which jointly manages both the PDB and EMDB archives, this review examines the origins of the resolution revolution and analyzes its impact on structural biology viewed through the lens of PDB holdings. Six areas of focus exemplifying the impact of 3DEM across the biosciences are discussed in detail (icosahedral viruses, ribosomes, integral membrane proteins, SARS-CoV-2 spike proteins, cryogenic electron tomography, and integrative structure determination combining 3DEM with complementary biophysical measurement techniques), followed by a review of 3DEM structure validation by the wwPDB that underscores the importance of community engagement.

Using deep-learning predictions of inter-residue distances for model validation

Determination of protein structures typically entails building a model that satisfies the collected experimental observations and its deposition in the Protein Data Bank. Experimental limitations can lead to unavoidable uncertainties during the process of model building, which result in the introduction of errors into the deposited model. Many metrics are available for model validation, but most are limited to consideration of the physico-chemical aspects of the model or its match to the experimental data. The latest advances in the field of deep learning have enabled the increasingly accurate prediction of inter-residue distances, an advance which has played a pivotal role in the recent improvements observed in the field of protein ab initio modelling. Here, new validation methods are presented based on the use of these precise inter-residue distance predictions, which are compared with the distances observed in the protein model. Sequence-register errors are particularly clearly detected and the register shifts required for their correction can be reliably determined. The method is available in the ConKit package (https://www.conkit.org).

Overview and applications of map and model validation tools in the CCP-EM software suite

Cryogenic electron microscopy (cryo-EM) has recently been established as a powerful technique for solving macromolecular structures. Although the best resolutions achievable are improving, a significant majority of data are still resolved at resolutions worse than 3 Å, where it is non-trivial to build or fit atomic models. The map reconstructions and atomic models derived from the maps are also prone to errors accumulated through the different stages of data processing. Here, we highlight the need to evaluate both model geometry and fit to data at different resolutions. Assessment of cryo-EM structures from SARS-CoV-2 highlights a bias towards optimising the model geometry to agree with the most common conformations, compared to the agreement with data. We present the CoVal web service which provides multiple validation metrics to reflect the quality of atomic models derived from cryo-EM data of structures from SARS-CoV-2. We demonstrate that further refinement can lead to improvement of the agreement with data without the loss of geometric quality. We also discuss the recent CCP-EM developments aimed at addressing some of the current shortcomings.

Artificial Intelligence in Cryo-Electron Microscopy

Cryo-electron microscopy (cryo-EM) has become an unrivaled tool for determining the structure of macromolecular complexes. The biological function of macromolecular complexes is inextricably tied to the flexibility of these complexes. Single particle cryo-EM can reveal the conformational heterogeneity of a biochemically pure sample, leading to well-founded mechanistic hypotheses about the roles these complexes play in biology. However, the processing of increasingly large, complex datasets using traditional data processing strategies is exceedingly expensive in both user time and computational resources. Current innovations in data processing capitalize on artificial intelligence (AI) to improve the efficiency of data analysis and validation. Here, we review new tools that use AI to automate the data analysis steps of particle picking, 3D map reconstruction, and local resolution determination. We discuss how the application of AI moves the field forward, and what obstacles remain. We also introduce potential future applications of AI to use cryo-EM in understanding protein communities in cells.

Atomic model validation using the CCP-EM software suite

Recently, there has been a dramatic improvement in the quality and quantity of data derived using cryogenic electron microscopy (cryo-EM). This is also associated with a large increase in the number of atomic models built. Although the best resolutions that are achievable are improving, often the local resolution is variable, and a significant majority of data are still resolved at resolutions worse than 3 Å. Model building and refinement is often challenging at these resolutions, and hence atomic model validation becomes even more crucial to identify less reliable regions of the model. Here, a graphical user interface for atomic model validation, implemented in the CCP-EM software suite, is presented. It is aimed to develop this into a platform where users can access multiple complementary validation metrics that work across a range of resolutions and obtain a summary of evaluations. Based on the validation estimates from atomic models associated with cryo-EM structures from SARS-CoV-2, it was observed that models typically favor adopting the most common conformations over fitting the observations when compared with the model agreement with data. At low resolutions, the stereochemical quality may be favored over data fit, but care should be taken to ensure that the model agrees with the data in terms of resolvable features. It is demonstrated that further re-refinement can lead to improvement of the agreement with data without the loss of geometric quality. This also highlights the need for improved resolution-dependent weight optimization in model refinement and an effective test for overfitting that would help to guide the refinement process.

Three-dimensional Structure Databases of Biological Macromolecules

Databases of three-dimensional structures of proteins (and their associated molecules) provide: (a) Curated repositories of coordinates of experimentally determined structures, including extensive metadata; for instance information about provenance, details about data collection and interpretation, and validation of results. (b) Information-retrieval tools to allow searching to identify entries of interest and provide access to them. (c) Links among databases, especially to databases of amino-acid and genetic sequences, and of protein function; and links to software for analysis of amino-acid sequence and protein structure, and for structure prediction. (d) Collections of predicted three-dimensional structures of proteins. These will become more and more important after the breakthrough in structure prediction achieved by AlphaFold2. The single global archive of experimentally determined biomacromolecular structures is the Protein Data Bank (PDB). It is managed by wwPDB, a consortium of five partner institutions: the Protein Data Bank in Europe (PDBe), the Research Collaboratory for Structural Bioinformatics (RCSB), the Protein Data Bank Japan (PDBj), the BioMagResBank (BMRB), and the Electron Microscopy Data Bank (EMDB). In addition to jointly managing the PDB repository, the individual wwPDB partners offer many tools for analysis of protein and nucleic acid structures and their complexes, including providing computer-graphic representations. Their collective and individual websites serve as hubs of the community of structural biologists, offering newsletters, reports from Task Forces, training courses, and "helpdesks," as well as links to external software.Many specialized projects are based on the information contained in the PDB. Especially important are SCOP, CATH, and ECOD, which present classifications of protein domains.

Challenges in sample preparation and structure determination of amyloids by cryo-EM

Amyloids share a common architecture but play disparate biological roles in processes ranging from bacterial defense mechanisms to protein misfolding diseases. Their structures are highly polymorphic, which makes them difficult to study by X-ray diffraction or NMR spectroscopy. Our understanding of amyloid structures is due in large part to recent advances in the field of cryo-EM, which allows for determining the polymorphs separately. In this review, we highlight the main stepping stones leading to the substantial number of high-resolution amyloid fibril structures known today as well as recent developments regarding automation and software in cryo-EM. We discuss that sample preparation should move closer to physiological conditions to understand how amyloid aggregation and disease are linked. We further highlight new approaches to address heterogeneity and polymorphism of amyloid fibrils in EM image processing and give an outlook to the upcoming challenges in researching the structural biology of amyloids.

Host-Pathogen Adhesion as the Basis of Innovative Diagnostics for Emerging Pathogens

Infectious diseases are an existential health threat, potentiated by emerging and re-emerging viruses and increasing bacterial antibiotic resistance. Targeted treatment of infectious diseases requires precision diagnostics, especially in cases where broad-range therapeutics such as antibiotics fail. There is thus an increasing need for new approaches to develop sensitive and specific in vitro diagnostic (IVD) tests. Basic science and translational research are needed to identify key microbial molecules as diagnostic targets, to identify relevant host counterparts, and to use this knowledge in developing or improving IVD. In this regard, an overlooked feature is the capacity of pathogens to adhere specifically to host cells and tissues. The molecular entities relevant for pathogen–surface interaction are the so-called adhesins. Adhesins vary from protein compounds to (poly-)saccharides or lipid structures that interact with eukaryotic host cell matrix molecules and receptors. Such interactions co-define the specificity and sensitivity of a diagnostic test. Currently, adhesin-receptor binding is typically used in the pre-analytical phase of IVD tests, focusing on pathogen enrichment. Further exploration of adhesin–ligand interaction, supported by present high-throughput “omics” technologies, might stimulate a new generation of broadly applicable pathogen detection and characterization tools. This review describes recent results of novel structure-defining technologies allowing for detailed molecular analysis of adhesins, their receptors and complexes. Since the host ligands evolve slowly, the corresponding adhesin interaction is under selective pressure to maintain a constant receptor binding domain. IVD should exploit such conserved binding sites and, in particular, use the human ligand to enrich the pathogen. We provide an inventory of methods based on adhesion factors and pathogen attachment mechanisms, which can also be of relevance to currently emerging pathogens, including SARS-CoV-2, the causative agent of COVID-19.

Cryo-EM Map-Based Model Validation Using the False Discovery Rate Approach

Significant technological developments and increasing scientific interest in cryogenic electron microscopy (cryo-EM) has resulted in a rapid increase in the amount of data generated by these experiments and the derived atomic models. Robust measures for the validation of 3D reconstructions and atomic models are essential for appropriate interpretation of the data. The resolution of data and availability of software tools that work across a range of resolutions often limit the quality of derived models. Hence, the final atomic model is often incomplete or contains regions where atomic positions are less reliable or incorrectly built. Extensive manual pruning and local adjustments or rebuilding are usually required to address these issues. The presented research introduces a software tool for the validation of the backbone trace of atomic models built in the cryo-EM density maps. In this study, we use the false discovery rate analysis, which can be used to segregate molecular signals from the background. Each atomic position in the model can be associated with an FDR backbone validation score, which can be used to identify potential mistraced residues. We demonstrate that the proposed validation score is complementary to existing validation metrics and is useful especially in cases where the model is built in the maps having varying local resolution. We also discuss the application of the score for automated pruning of atomic models built ab-initio during the iterative model building process in Buccaneer. We have implemented this score in the CCP-EM software suite.

Evolution of standardization and dissemination of cryo-EM structures and data jointly by the community, PDB, and EMDB

Cryogenic electron microscopy (cryo-EM) methods began to be used in the mid-1970s to study thin and periodic arrays of proteins. Following a half-century of development in cryo-specimen preparation, instrumentation, data collection, data processing, and modeling software, cryo-EM has become a routine method for solving structures from large biological assemblies to small biomolecules at near to true atomic resolution. This review explores the critical roles played by the Protein Data Bank (PDB) and Electron Microscopy Data Bank (EMDB) in partnership with the community to develop the necessary infrastructure to archive cryo-EM maps and associated models. Public access to cryo-EM structure data has in turn facilitated better understanding of structure–function relationships and advancement of image processing and modeling tool development. The partnership between the global cryo-EM community and PDB and EMDB leadership has synergistically shaped the standards for metadata, one-stop deposition of maps and models, and validation metrics to assess the quality of cryo-EM structures. The advent of cryo-electron tomography (cryo-ET) for in situ molecular cell structures at a broad resolution range and their correlations with other imaging data introduce new data archival challenges in terms of data size and complexity in the years to come.

Structural interpretation of cryo-EM image reconstructions

The productivity of single-particle cryo-EM as a structure determination method has rapidly increased as many novel biological structures are being elucidated. The ultimate result of the cryo-EM experiment is an atomic model that should faithfully represent the computed image reconstruction. Although the principal approach of atomic model building and refinement from maps resembles that of the X-ray crystallographic methods, there are important differences due to the unique properties resulting from the 3D image reconstructions. In this review, we discuss the practiced work-flow from the cryo-EM image reconstruction to the atomic model. We give an overview of (i) resolution determination methods in cryo-EM including local and directional resolution variation, (ii) cryo-EM map contrast optimization including complementary map types that can help in identifying ambiguous density features, (iii) atomic model building and (iv) refinement in various resolution regimes including (v) their validation and (vi) discuss differences between X-ray and cryo-EM maps. Based on the methods originally developed for X-ray crystallography, the path from 3D image reconstruction to atomic coordinates has become an integral and important part of the cryo-EM structure determination work-flow that routinely delivers atomic models.

Protein storytelling through physics

Every protein has a story-how it folds, what it binds, its biological actions, and how it misbehaves in aging or disease. Stories are often inferred from a protein's shape (i.e., its structure). But increasingly, stories are told using computational molecular physics (CMP). CMP is rooted in the principled physics of driving forces and reveals granular detail of conformational populations in space and time. Recent advances are accessing longer time scales, larger actions, and blind testing, enabling more of biology's stories to be told in the language of atomistic physics.

Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge

This paper describes outcomes of the 2019 Cryo-EM Model Challenge. The goals were to (1) assess the quality of models that can be produced from cryogenic electron microscopy (cryo-EM) maps using current modeling software, (2) evaluate reproducibility of modeling results from different software developers and users and (3) compare performance of current metrics used for model evaluation, particularly Fit-to-Map metrics, with focus on near-atomic resolution. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived by 13 participating teams from four benchmark maps, including three forming a resolution series (1.8 to 3.1 Å). The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual experiments and structure data archives such as the Protein Data Bank. We recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed cryo-EM map density.

High-Throughput Cryo-EM Enabled by User-Free Preprocessing Routines

Single-particle cryoelectron microscopy (cryo-EM) continues to grow into a mainstream structural biology technique. Recent developments in data collection strategies alongside new sample preparation devices herald a future where users will collect multiple datasets per microscope session. To make cryo-EM data processing more automatic and user-friendly, we have developed an automatic pipeline for cryo-EM data preprocessing and assessment using a combination of deep-learning and image-analysis tools. We have verified the performance of this pipeline on a number of datasets and extended its scope to include sample screening by the user-free assessment of the qualities of a series of datasets under different conditions. We propose that our workflow provides a decision-free solution for cryo-EM, making data preprocessing more generalized and robust in the high-throughput era as well as more convenient for users from a range of backgrounds.

The data universe of structural biology

The Protein Data Bank (PDB) has grown from a small data resource for crystallographers to a worldwide resource serving structural biology. The history of the growth of the PDB and the role that the community has played in developing standards and policies are described. This article also illustrates how other biophysics communities are collaborating with the worldwide PDB to create a network of interoperating data resources. This network will expand the capabilities of structural biology and enable the determination and archiving of increasingly complex structures.

Flexible Fitting of Small Molecules into Electron Microscopy Maps Using Molecular Dynamics Simulations with Neural Network Potentials

Despite significant advances in resolution, the potential for cryo-electron microscopy (EM) to be used in determining the structures of protein-drug complexes remains unrealized. Determination of accurate structures and coordination of bound ligands necessitates simultaneous fitting of the models into the density envelopes, exhaustive sampling of the ligand geometries, and, most importantly, concomitant rearrangements in the side chains to optimize the binding energy changes. In this article, we present a flexible-fitting pipeline where molecular dynamics flexible fitting (MDFF) is used to refine structures of protein-ligand complexes from 3 to 5 Å electron density data. Enhanced sampling is employed to explore the binding pocket rearrangements. To provide a model that can accurately describe the conformational dynamics of the chemically diverse set of small-molecule drugs inside MDFF, we use QM/MM and neural-network potential (NNP)/MM models of protein-ligand complexes, where the ligand is represented using the QM or NNP model, and the protein is represented using established molecular mechanical force fields (e.g., CHARMM). This pipeline offers structures commensurate to or better than recently submitted high-resolution cryo-EM or X-ray models, even when given medium to low-resolution data as input. The use of the NNPs makes the algorithm more robust to the choice of search models, offering a radius of convergence of 6.5 Å for ligand structure determination. The quality of the predicted structures was also judged by density functional theory calculations of ligand strain energy. This strain potential energy is found to systematically decrease with better fitting to density and improved ligand coordination, indicating correct binding interactions. A computationally inexpensive protocol for computing strain energy is reported as part of the model analysis protocol that monitors both the ligand fit as well as model quality.

Comparing Cryo-EM Reconstructions and Validating Atomic Model Fit Using Difference Maps

Cryogenic electron microscopy (cryo-EM) is a powerful technique for determining structures of multiple conformational or compositional states of macromolecular assemblies involved in cellular processes. Recent technological developments have led to a leap in the resolution of many cryo-EM data sets, making atomic model building more common for data interpretation. We present a method for calculating differences between two cryo-EM maps or a map and a fitted atomic model. The proposed approach works by scaling the maps using amplitude matching in resolution shells. To account for variability in local resolution of cryo-EM data, we include a procedure for local amplitude scaling that enables appropriate scaling of local map contrast. The approach is implemented as a user-friendly tool in the CCP-EM software package. To obtain clean and interpretable differences, we propose a protocol involving steps to process the input maps and output differences. We demonstrate the utility of the method for identifying conformational and compositional differences including ligands. We also highlight the use of difference maps for evaluating atomic model fit in cryo-EM maps.

Evolving data standards for cryo-EM structures

Electron cryo-microscopy (cryo-EM) is increasingly being used to determine 3D structures of a broad spectrum of biological specimens from molecules to cells. Anticipating this progress in the early 2000s, an international collaboration of scientists with expertise in both cryo-EM and structure data archiving was established (EMDataResource, previously known as EMDataBank). The major goals of the collaboration have been twofold: to develop the necessary infrastructure for archiving cryo-EM-derived density maps and models, and to promote development of cryo-EM structure validation standards. We describe how cryo-EM data archiving and validation have been developed and jointly coordinated for the Electron Microscopy Data Bank and Protein Data Bank archives over the past two decades, as well as the impact of evolving technology on data standards. Just as for X-ray crystallography and nuclear magnetic resonance, engaging the scientific community via workshops and challenging activities has played a central role in developing recommendations and requirements for the cryo-EM structure data archives.

Cryo-electron microscopy targets in CASP13: Overview and evaluation of results

Structures of seven CASP13 targets were determined using cryo-electron microscopy (cryo-EM) technique with resolution between 3.0 and 4.0 Å. We provide an overview of the experimentally derived structures and describe results of the numerical evaluation of the submitted models. The evaluation is carried out by comparing coordinates of models to those of reference structures (CASP-style evaluation), as well as checking goodness-of-fit of modeled structures to the cryo-EM density maps. The performance of contributing research groups in the CASP-style evaluation is measured in terms of backbone accuracy, all-atom local geometry and similarity of inter-subunit interfaces. The results on the cryo-EM targets are compared with those on the whole set of eighty CASP13 targets. A posteriori refinement of the best models in their corresponding cryo-EM density maps resulted in structures that are very close to the reference structure, including some regions with better fit to the density.

Federating Structural Models and Data: Outcomes from A Workshop on Archiving Integrative Structures

Structures of biomolecular systems are increasingly computed by integrative modeling. In this approach, a structural model is constructed by combining information from multiple sources, including varied experimental methods and prior models. In 2019, a Workshop was held as a Biophysical Society Satellite Meeting to assess progress and discuss further requirements for archiving integrative structures. The primary goal of the Workshop was to build consensus for addressing the challenges involved in creating common data standards, building methods for federated data exchange, and developing mechanisms for validating integrative structures. The summary of the Workshop and the recommendations that emerged are presented here.

Building de novo cryo-electron microscopy structures collaboratively with citizen scientists

With the rapid improvement of cryo-electron microscopy (cryo-EM) resolution, new computational tools are needed to assist and improve upon atomic model building and refinement options. This communication demonstrates that microscopists can now collaborate with the players of the computer game Foldit to generate high-quality de novo structural models. This development could greatly speed the generation of excellent cryo-EM structures when used in addition to current methods.