Bilingual language model for protein sequence and structure

Large-Scale Multi-omic Biosequence Transformers for Modeling Protein-Nucleic Acid Interactions

The transformer architecture has revolutionized bioinformatics and driven progress in the understanding and prediction of the properties of biomolecules. To date, most biosequence transformers have been trained on a single omic-either proteins or nucleic acids and have seen incredible success in downstream tasks in each domain with particularly noteworthy breakthroughs in protein structural modeling. However, single-omic pre-training limits the ability of these models to capture cross-modal interactions. Here we present OmniBioTE, the largest open-source multi-omic model trained on over 250 billion tokens of mixed protein and nucleic acid data. We show that despite only being trained on unlabelled sequence data, OmniBioTE learns joint representations consistent with the central dogma of molecular biology. We further demonstrate that OmbiBioTE achieves state-of-the-art results predicting the change in Gibbs free energy ({\Delta}G) of the binding interaction between a given nucleic acid and protein. Remarkably, we show that multi-omic biosequence transformers emergently learn useful structural information without any a priori structural training, allowing us to predict which protein residues are most involved in the protein-nucleic acid binding interaction. Lastly, compared to single-omic controls trained with identical compute, OmniBioTE demonstrates superior performance-per-FLOP and absolute accuracy across both multi-omic and single-omic benchmarks, highlighting the power of a unified modeling approach for biological sequences.

The impact of metagenomic analysis on the discovery of novel endolysins

Metagenomics has revolutionized enzyme discovery by enabling the study of genetic material directly from environmental samples, bypassing the need for microbial cultivation. This approach is particularly effective for identifying novel endolysins, phage-derived enzymes with antibacterial properties suited for therapeutic and industrial applications. Diverse ecosystems, such as biofilms, human microbiome, hot springs, and geothermal areas, serve as rich reservoirs for endolysins with traits like thermostability, broad-spectrum activity, specificity and resistance to harsh conditions. Functional metagenomics, complemented by bioinformatics, enables the discovery and annotation of previously uncharacterized endolysins. Examples of endolysins discovered from metagenomics analysis are discussed. Despite the challenges of analyzing complex microbial ecosystems and isolating target genes, metagenomics holds immense potential for uncovering innovative endolysins, paving the way for developing new biotechnological applications. KEY POINTS: • Endolysins offer antibacterial potential for therapeutic and industrial use. • Metagenomics enables discovery of novel endolysins from diverse ecosystems. • Advances in tools and methods have accelerated novel endolysins discovery.

StructMAn 2.0 Web: a web server for structural annotation of protein sequences and mutations

StructMAn is a method for protein structural annotation. It describes each position of a protein sequence or specific variants in it in terms of their importance for the three-dimensional (3D) structure of the protein and its interactions with other molecules. StructMAn maps, aligns, and aggregates data from experimentally resolved and predicted 3D structures of proteins and their homologs for any given protein sequence and/or a combination of mutations in it. The results provide structural annotation for every amino acid position allowing a detailed structural analysis. Furthermore, StructMAn enables generation of a wide variety of position-specific high-quality structural features that can be leveraged in machine learning applications. With the new web server StructMAn 2.0 Web, we provide a user-friendly way to use StructMAn offering an easy-to-use input interface and a comprehensive visualization for the various results of StructMAn. StructMAn 2.0 Web is available at https://tools.helmholtz-hips.de/structman.

PLM-DBPs: enhancing plant DNA-binding protein prediction by integrating sequence-based and structure-aware protein language models

DNA-binding proteins (DBPs) play a crucial role in gene regulation, development, and environmental responses across plants, animals, and microorganisms. Existing DBP prediction methods are largely limited to sequence information, whether through handcrafted features or sequence-based protein language models (PLMs), overlooking structural cues critical to protein function. In addition, most existing tools are trained for general DBP predictions, which are often not accurate for plant-specific DBPs due to the unique structural and functional properties of plant proteins. Our work introduces PLM-DBPs, a deep learning framework that integrates both sequence-based and structure-aware representations to enhance DBP prediction in plants. We evaluated several state-of-the-art PLMs to extract high-dimensional protein representations and experimented with various fusion strategies to validate the complementary information between the various representations. Our final model, a fusion of sequence-based and structure-aware ANN models, achieves a notable improvement in predicting DBPs in plants outperforming previous state-of-the-art models. Although sequence-based PLMs already demonstrate strong performance in DBP prediction, our findings show that the integration of structural information further enhances predictive accuracy. This underscores the complementary nature of structural representations and establishes PLM-DBPs as a robust tool for advancing plant research and agricultural innovation. The proposed model and other resources are publicly available at https://github.com/suresh-pokharel/PLM-DBPs.

Computational pipeline for sustainable enzyme discovery through (re)use of metagenomic data

Enzymes derived from extremophilic organisms, also known as extremozymes, offer sustainable and efficient solutions for industrial applications. Valued for their resilience and low environmental impact, extremozymes have found use as catalysts in various processes, ranging from dairy production to pharmaceutical manufacturing. However, discovery of novel extremozymes is often hindered by challenges such as culturing difficulties, underrepresentation of extreme environments in reference databases, and limitations of traditional sequence-based screening methods. In this work, we present a computational pipeline designed to discover novel enzymes from metagenomic data derived from extreme environments. This pipeline represents a versatile and sustainable approach that promotes reuse and recycling of existing datasets and minimises the need for additional environmental sampling. In its core, the algorithm integrates both traditional bioinformatic techniques and recent advances in structural prediction, enabling rapid and accurate identification of enzymes. However, due to its design, the algorithm relies heavily on existing databases, which can limit its effectiveness in situations where reference data is scarce or when encountering novel protein families. As a proof-of-concept, we applied the pipeline to metagenomic data from deep-sea hydrothermal vents, with a focus on β-galactosidases. The pipeline identified 11 potential candidate proteins, out of which 10 showed in vitro activity. One of the selected enzymes, βGal_UW07, showed strong potential for industrial applications. The enzyme exhibited optimal activity at 70 °C and was exceptionally resistant to high pH and the presence of metal ions and reducing agents. Overall, our results indicate that the pipeline is highly accurate and can play a key role in sustainable bioprospecting, leveraging existing metagenomic datasets and minimising in situ interventions in pristine regions.

ATP-Pred: Prediction of Protein-ATP Binding Residues via Fusion of Residue-Level Embeddings and Kolmogorov-Arnold Network

Accurately identifying protein-ATP binding residues is essential for understanding biological processes and designing drugs. However, current sequence-based methods have limitations, such as difficulties in extracting discriminative features and the need for more efficient algorithms. Additionally, methods based on multiple sequence alignments often face challenges in handling large-scale predictions. To address these issues, we developed ATP-Pred, a sequence-based method for predicting ATP-binding residues in proteins. This model applies transfer learning by using two recently developed pretrain protein language models, Ankh and ProstT5, to extract residue-level embeddings that capture protein functionality. ATP-Pred also integrates a CNN-BiLSTM network and a Kolmogorov-Arnold network to build the prediction model. To handle data imbalance, we introduced a weighted focal loss function. Experimental results on three independent test data sets showed that ATP-Pred outperforms most existing methods. Its generalizability was further validated on four protein-mononucleotide binding residue data sets, where it delivered promising results. These findings suggest that ATP-Pred is a robust and reliable predictor.

Foundation models of protein sequences: A brief overview

Protein sequence models have evolved from simple statistics of aligned families to versatile foundation models of evolutionary scale. Enabled by self-supervised learning and an abundance of protein sequence data, such foundation models now play a central role in protein science. They facilitate rich representations, powerful generative design, and fine-tuning across diverse domains. In this review, we trace modeling developments and categorize them into methodological trends over the modalities they describe and the contexts they condition upon. Following a brief historical overview, we focus our attention on the most recent trends and outline future perspectives.

DeepMVD: A Novel Multiview Dynamic Feature Fusion Model for Accurate Protein Function Prediction

Proteins, as the fundamental macromolecules of life, play critical roles in various biological processes. Recent advancements in intelligent protein function prediction methods leverage sequences, structures, and biomedical literature data. Among them, function prediction methods for protein sequences remain an enduring and popular research direction. Existing studies have failed to effectively utilize the multilevel attribute features reflected in protein sequences. This limitation hinders the enrichment of protein descriptions needed for high-precision prediction of protein functions. To address this, we propose DeepMVD, a novel deep learning model that enhances prediction accuracy by dynamically fusing multiview features. DeepMVD employs specialized modules to extract unique features from each view and utilizes an adaptive fusion mechanism for optimal integration. Evaluation of the CAFA4 data set shows that DeepMVD significantly outperforms existing state-of-the-art models in terms of BP, MF, and CC terminology, all obtaining the highest Fmax (0.523, 0.712, 0.740). Ablation studies confirm the model's robustness. Source code and data sets are available at http://swanhub.co/scl/DeepMVD.

AllohubPy: Detecting Allosteric Signals Through An Information-theoretic Approach

Allosteric regulation is crucial for biological processes like signal transduction, transcriptional regulation, and metabolism, yet the mechanisms and macromolecular properties that govern it are still not well understood. Several methods have been developed over the years to study allosterism through different angles. Among the possible ways to study allosterism, information-theoretic approaches, like AlloHubMat or GSAtools, can be particularly effective due to their use of robust statistics and the possibility to be combined with graph analysis. These methods capture local conformational changes associated with global motions from molecular dynamics simulations through the use of a Structural Alphabet, which simplifies the complexity of the Cartesian space by reducing the dimensionality down to a string of encoded fragments, representing sets of internal coordinates that still capture the overall conformation changes. In this work, we present "AllohubPy," an improved and standardized methodology of AlloHubMat and GSAtools coded in Python. We analyse the performance, limitations and sampling requirements of AllohubPy by using extensive molecular dynamics simulations of model allosteric systems and apply convergence analysis techniques to estimate result reliability. Additionally, we expand the methodology to use different dimensionality reduction Structural Alphabets, such as the 3DI alphabet, and integrate Protein Language Models (PLMs) to refine allosteric hub communication detection by monitoring the detected evolutionary constraints. Overall, AllohubPy expands its preceding methods and simplifies the use and reliability of the method to effectively capture dynamic allosteric motions and residue pathways. AllohubPy is freely available on GitHub (https://github.com/Fraternalilab/AlloHubPy) as a package and as a Jupyter Notebook.

A deep learning method for predicting interactions for intrinsically disordered regions of proteins

Intrinsically disordered proteins or regions (IDPs/IDRs) adopt diverse binding modes with different partners, ranging from ordered to multivalent to fuzzy conformations in the bound state. Characterizing IDR interfaces is challenging experimentally and computationally. Alphafold-multimer and Alphafold3, the state-of-the-art structure prediction methods, are less accurate at predicting IDR binding sites at their benchmarked confidence cutoffs. Their performance improves upon lowering the confidence cutoffs. Here, we developed Disobind, a deep-learning method that predicts inter-protein contact maps and interface residues for an IDR and a partner protein, given their sequences. It outperforms AlphaFold-multimer and AlphaFold3 at multiple confidence cutoffs. Combining the Disobind and AlphaFold-multimer predictions further improves the performance. In contrast to most current methods, Disobind considers the context of the binding partner and does not depend on structures and multiple sequence alignments. Its predictions can be used to localize IDRs in integrative structures of large assemblies and characterize and modulate IDR-mediated interactions.

Evaluating the advancements in protein language models for encoding strategies in protein function prediction: a comprehensive review

Protein function prediction is crucial in several key areas such as bioinformatics and drug design. With the rapid progress of deep learning technology, applying protein language models has become a research focus. These models utilize the increasing amount of large-scale protein sequence data to deeply mine its intrinsic semantic information, which can effectively improve the accuracy of protein function prediction. This review comprehensively combines the current status of applying the latest protein language models in protein function prediction. It provides an exhaustive performance comparison with traditional prediction methods. Through the in-depth analysis of experimental results, the significant advantages of protein language models in enhancing the accuracy and depth of protein function prediction tasks are fully demonstrated.

Domainator, a flexible software suite for domain-based annotation and neighborhood analysis, identifies proteins involved in antiviral systems

The availability of large databases of biological sequences presents an opportunity for in-depth exploration of gene diversity and function. Bacterial defense systems are a rich source of diverse but difficult to annotate genes with biotechnological applications. In this work, we present Domainator, a flexible and modular software suite for domain-based gene neighborhood and protein search, extraction and clustering. We demonstrate the utility of Domainator through three examples related to bacterial defense systems. First, we cluster CRISPR-associated Rossman fold (CARF) containing proteins with difficult to annotate effector domains, classifying most of them as likely transcriptional regulators and a subset as likely RNases. Second, we extract and cluster P4-like phage satellite defense hotspots, identify an abundant variant of Lamassu defense systems and demonstrate its in vivo activity against several T-even phages. Third, we integrate a protein language model into Domainator and use it to identify restriction endonucleases with low similarity to known reference sequences, validating the activity of one example in vitro. Domainator is made available as an open-source package with detailed documentation and usage examples.

PHIStruct: improving phage-host interaction prediction at low sequence similarity settings using structure-aware protein embeddings

MOTIVATION

Recent computational approaches for predicting phage-host interaction have explored the use of sequence-only protein language models to produce embeddings of phage proteins without manual feature engineering. However, these embeddings do not directly capture protein structure information and structure-informed signals related to host specificity.

RESULTS

We present PHIStruct, a multilayer perceptron that takes in structure-aware embeddings of receptor-binding proteins, generated via the structure-aware protein language model SaProt, and then predicts the host from among the ESKAPEE genera. Compared against recent tools, PHIStruct exhibits the best balance of precision and recall, with the highest and most stable F1 score across a wide range of confidence thresholds and sequence similarity settings. The margin in performance is most pronounced when the sequence similarity between the training and test sets drops below 40%, wherein, at a relatively high-confidence threshold of above 50%, PHIStruct presents a 7%-9% increase in class-averaged F1 over machine learning tools that do not directly incorporate structure information, as well as a 5%-6% increase over BLASTp.

AVAILABILITY AND IMPLEMENTATION

The data and source code for our experiments and analyses are available at https://github.com/bioinfodlsu/PHIStruct.

Do protein language models learn phylogeny?

Deep machine learning demonstrates a capacity to uncover evolutionary relationships directly from protein sequences, in effect internalising notions inherent to classical phylogenetic tree inference. We connect these two paradigms by assessing the capacity of protein-based language models (pLMs) to discern phylogenetic relationships without being explicitly trained to do so. We evaluate ESM2, ProtTrans, and MSA-Transformer relative to classical phylogenetic methods, while also considering sequence insertions and deletions (indels) across 114 Pfam datasets. The largest ESM2 model tends to outperform other pLMs (including the multimodal ESM3) by recovering phylogenetic relationships among homologous protein sequences in both low- and high-gap settings. pLMs agree with conventional phylogenetic methods in general, but more so for protein families with fewer implied indels, highlighting indels as a key factor differentiating classical phylogenetics from pLMs. We find that pLMs preferentially capture broader as opposed to finer evolutionary relationships within a specific protein family, where ESM2 has a sweet spot for highly divergent sequences, at remote distance. Less than 10% of neurons are sufficient to broadly recapitulate classical phylogenetic distances; when used in isolation, the difference between the paradigms is further diminished. We show these neurons are polysemantic, shared among different homologous families but never fully overlapping. We highlight the potential of ESM2 as a complementary tool for phylogenetic analysis, especially when extending to remote homologs that are difficult to align and imply complex histories of insertions and deletions. Implementations of analyses are available at https://github.com/santule/pLMEvo.

Fast protein structure searching using structure graph embeddings

Comparing and searching protein structures independent of primary sequence has proved useful for remote homology detection, function annotation, and protein classification. Fast and accurate methods to search with structures will be essential to make use of the vast databases that have recently become available, in the same way that fast protein sequence searching underpins much of bioinformatics. We train a simple graph neural network using supervised contrastive learning to learn a low-dimensional embedding of protein domains.

Availability and implementation

The method, called Progres, is available as software at https://github.com/greener-group/progres and as a web server at https://progres.mrc-lmb.cam.ac.uk. It has accuracy comparable to the best current methods and can search the AlphaFold database TED domains in a 10th of a second per query on CPU.