SaLT&PepPr is an interface-predicting language model for designing peptide-guided protein degraders

PTM-Mamba: a PTM-aware protein language model with bidirectional gated Mamba blocks

Current protein language models (LMs) accurately encode protein properties but have yet to represent post-translational modifications (PTMs), which are crucial for proteomic diversity and influence protein structure, function and interactions. To address this gap, we develop PTM-Mamba, a PTM-aware protein LM that integrates PTM tokens using bidirectional Mamba blocks fused with ESM-2 protein LM embeddings via a newly developed gating mechanism. PTM-Mamba uniquely models both wild-type and PTM sequences, enabling downstream tasks such as disease association and druggability prediction, PTM effect prediction on protein-protein interactions and zero-shot PTM discovery. In total, our work establishes PTM-Mamba as a foundational tool for PTM-aware protein modeling and design.

Programmable protein stabilization with language model-derived peptide guides

Dysregulated protein degradation via the ubiquitin-proteasomal pathway can induce numerous disease phenotypes, including cancer, neurodegeneration, and diabetes. While small molecule-based targeted protein degradation (TPD) and targeted protein stabilization (TPS) platforms can address this dysregulation, they rely on structured and stable binding pockets, which do not exist to classically "undruggable" targets. Here, we expand the TPS target space by engineering "deubiquibodies" (duAbs) via fusion of computationally-designed peptide binders to the catalytic domain of the potent OTUB1 deubiquitinase. In human cells, duAbs effectively stabilize exogenous and endogenous proteins in a DUB-dependent manner. Using protein language models to generate target-binding peptides, we engineer duAbs to conformationally diverse target proteins, including key tumor suppressor proteins p53 and WEE1, and heavily-disordered fusion oncoproteins, such as PAX3::FOXO1. We further encapsulate p53-targeting duAbs as mRNA in lipid nanoparticles and demonstrate effective intracellular delivery, p53 stabilization, and apoptosis activation, motivating further in vivo translation.

Teaching AI to speak protein

Large Language Models for proteins, namely protein Language Models (pLMs), have begun to provide an important alternative to capturing the information encoded in a protein sequence in computers. Arguably, pLMs have advanced importantly to understanding aspects of the language of life as written in proteins, and through this understanding, they are becoming an increasingly powerful means of advancing protein prediction, e.g., in the prediction of molecular function as expressed by identifying binding residues or variant effects. While benefitting from the same technology, protein structure prediction remains one of the few applications for which only using pLM embeddings from single sequences appears not to improve over or match the state-of-the-art. Fine-tuning foundation pLMs enhances efficiency and accuracy of solutions, in particular in cases with few experimental annotations. pLMs facilitate the integration of computational and experimental biology, of AI and wet-lab, in particular toward a new era of protein design.

Peptide-functionalized nanoparticles for brain-targeted therapeutics

Despite the rapid development of nanoparticle (NP)-based drug delivery systems, intravenous delivery of drugs to the brain remains a major challenge due to various biological barriers. To achieve therapeutic effects, NP-encapsulated drugs must avoid accumulation in off-target organs and selectively deliver to the brain, successfully cross the blood-brain barrier (BBB), and reach the target cells in the brain. Conjugating receptor-specific ligands to the surface of NPs is a promising technique for engineering NPs to overcome these barriers. Specifically, peptides as brain-targeting ligands have been of increasing interest given their ease of synthesis, low cytotoxicity, and strong affinity to target proteins. The success of peptides as targeting ligands is largely due to the diverse strategies of designing and modifying peptides with favorable properties, including membrane permeability and multi-receptor targeting. Here, we review the design and implementation of peptide-functionalized NP systems for neurological disease applications. We also explore advances in rational peptide design strategies for brain targeting, including using generative deep-learning models to computationally design new peptides.

Protein-Based Degraders: From Chemical Biology Tools to Neo-Therapeutics

The nascent field of targeted protein degradation (TPD) could revolutionize biomedicine due to the ability of degrader molecules to selectively modulate disease-relevant proteins. A key limitation to the broad application of TPD is its dependence on small-molecule ligands to target proteins of interest. This leaves unstructured proteins or those lacking defined cavities for small-molecule binding out of the scope of many TPD technologies. The use of proteins, peptides, and nucleic acids (otherwise known as "biologics") as the protein-targeting moieties in degraders addresses this limitation. In the following sections, we provide a comprehensive and critical review of studies that have used proteins and peptides to mediate the degradation and hence the functional control of otherwise challenging disease-relevant protein targets. We describe existing platforms for protein/peptide-based ligand identification and the drug delivery systems that might be exploited for the delivery of biologic-based degraders. Throughout the Review, we underscore the successes, challenges, and opportunities of using protein-based degraders as chemical biology tools to spur discoveries, elucidate mechanisms, and act as a new therapeutic modality.

FusOn-pLM: a fusion oncoprotein-specific language model via adjusted rate masking

Fusion oncoproteins, a class of chimeric proteins arising from chromosomal translocations, are major drivers of various pediatric cancers. These proteins are intrinsically disordered and lack druggable pockets, making them highly challenging therapeutic targets for both small molecule-based and structure-based approaches. Protein language models (pLMs) have recently emerged as powerful tools for capturing physicochemical and functional protein features but have yet to be trained on fusion oncoprotein sequences. We introduce FusOn-pLM, a fine-tuned pLM trained on a newly curated, comprehensive set of fusion oncoprotein sequences, FusOn-DB. Employing a unique cosine-scheduled masked language modeling strategy, FusOn-pLM dynamically adjusts masking rates (15%-40%) to optimize feature extraction and representation quality, surpassing baseline embeddings in fusion-specific tasks, including localization, puncta formation, and disorder prediction. FusOn-pLM uniquely predicts drug-resistant mutations, providing insights for therapeutic design that anticipates resistance mechanisms. In total, FusOn-pLM provides biologically relevant representations for advancing therapeutic discovery in fusion-driven cancers.

De novo design of peptide binders to conformationally diverse targets with contrastive language modeling

Designing binders to target undruggable proteins presents a formidable challenge in drug discovery. In this work, we provide an algorithmic framework to design short, target-binding linear peptides, requiring only the amino acid sequence of the target protein. To do this, we propose a process to generate naturalistic peptide candidates through Gaussian perturbation of the peptidic latent space of the ESM-2 protein language model and subsequently screen these novel sequences for target-selective interaction activity via a contrastive language-image pretraining (CLIP)-based contrastive learning architecture. By integrating these generative and discriminative steps, we create a Peptide Prioritization via CLIP (PepPrCLIP) pipeline and validate highly ranked, target-specific peptides experimentally, both as inhibitory peptides and as fusions to E3 ubiquitin ligase domains. PepPrCLIP-derived constructs demonstrate functionally potent binding and degradation of conformationally diverse, disease-driving targets in vitro. In total, PepPrCLIP empowers the modulation of previously inaccessible proteins without reliance on stable and ordered tertiary structures.

Leveraging large language models for peptide antibiotic design

Large language models (LLMs) have significantly impacted various domains of our society, including recent applications in complex fields such as biology and chemistry. These models, built on sophisticated neural network architectures and trained on extensive datasets, are powerful tools for designing, optimizing, and generating molecules. This review explores the role of LLMs in discovering and designing antibiotics, focusing on peptide molecules. We highlight advancements in drug design and outline the challenges of applying LLMs in these areas.

AlphaMut: A Deep Reinforcement Learning Model to Suggest Helix-Disrupting Mutations

Helices are important secondary structural motifs within proteins and are pivotal in numerous physiological processes. While amino acids (AA) such as alanine and leucine are known to promote helix formation, proline and glycine disfavor it. Helical structure formation, however, also depends on its environment, and hence, prior prediction of a mutational effect on a helical structure is difficult. Here, we employ a reinforcement learning algorithm to develop a predictive model for helix-disrupting mutations. We start with a model to disrupt helices independent of their protein environment. Our results show that only a few mutations lead to a drastic disruption of the target helix. We further extend our approach to helices in proteins and validate the results using rigorous free energy calculations. Our strategy identifies amino acids crucial for maintaining structural integrity and predicts key mutations that could alter protein structure. Through our work, we present a new use case for reinforcement learning in protein structure disruption.

Accurate de novo design of high-affinity protein binding macrocycles using deep learning

The development of macrocyclic binders to therapeutic proteins typically relies on large-scale screening methods that are resource-intensive and provide little control over binding mode. Despite considerable progress in physics-based methods for peptide design and deep-learning methods for protein design, there are currently no robust approaches for de novo design of protein-binding macrocycles. Here, we introduce RFpeptides, a denoising diffusion-based pipeline for designing macrocyclic peptide binders against protein targets of interest. We test 20 or fewer designed macrocycles against each of four diverse proteins and obtain medium to high-affinity binders against all selected targets. Designs against MCL1 and MDM2 demonstrate KD between 1-10 μM, and the best anti-GABARAP macrocycle binds with a KD of 6 nM and a sub-nanomolar IC50 in vitro. For one of the targets, RbtA, we obtain a high-affinity binder with KD < 10 nM despite starting from the target sequence alone due to the lack of an experimentally determined target structure. X-ray structures determined for macrocycle-bound MCL1, GABARAP, and RbtA complexes match very closely with the computational design models, with three out of the four structures demonstrating Ca RMSD of less than 1.5 Å to the design models. In contrast to library screening approaches for which determining binding mode can be a major bottleneck, the binding modes of RFpeptides-generated macrocycles are known by design, which should greatly facilitate downstream optimization. RFpeptides thus provides a powerful framework for rapid and custom design of macrocyclic peptides for diagnostic and therapeutic applications.

Programmable protein degraders enable selective knockdown of pathogenic β-catenin subpopulations in vitro and in vivo

Aberrant activation of Wnt signaling results in unregulated accumulation of cytosolic β-catenin, which subsequently enters the nucleus and promotes transcription of genes that contribute to cellular proliferation and malignancy. Here, we sought to eliminate pathogenic β-catenin from the cytosol using designer ubiquibodies (uAbs), chimeric proteins composed of an E3 ubiquitin ligase and a target-binding domain that redirect intracellular proteins to the proteasome for degradation. To accelerate uAb development, we leveraged a protein language model (pLM)-driven algorithm called SaLT&PepPr to computationally design "guide" peptides with affinity for β-catenin, which were subsequently fused to the catalytic domain of a human E3 called C-terminus of Hsp70-interacting protein (CHIP). Expression of the resulting peptide-guided uAbs in colorectal cancer cells led to the identification of several designs that significantly reduced the abnormally stable pool of free β-catenin in the cytosol and nucleus while preserving the normal membrane-associated subpopulation. This selective knockdown of pathogenic β-catenin suppressed Wnt/β-catenin signaling and impaired tumor cell survival and proliferation. Furthermore, one of the best degraders selectively decreased cytosolic but not membrane-associated β-catenin levels in livers of BALB/c mice following delivery as a lipid nanoparticle (LNP)-encapsulated mRNA. Collectively, these findings reveal the unique ability of uAbs to selectively eradicate abnormal proteins in vitro and in vivo and open the door to peptide-programmable biologic modulators of other disease-causing proteins.

PepCA: Unveiling protein-peptide interaction sites with a multi-input neural network model

The protein-peptide interaction plays a pivotal role in fields such as drug development, yet remains underexplored experimentally and challenging to model computationally. Herein, we introduce PepCA, a sequence-based approach for predicting peptide-binding sites on proteins. A primary obstacle in predicting peptide-protein interactions is the difficulty in acquiring precise protein structures, coupled with the uncertainty of polypeptide configurations. To address this, we first encode protein sequences using the Evolutionary Scale Modeling 2 (ESM-2) pre-trained model to extract latent structural information. Additionally, we have developed a multi-input coattention mechanism to concurrently update the encoding of both peptide and protein residues. PepCA integrates this module within an encoder-decoder structure. This model's high precision in identifying binding sites significantly advances the field of computational biology, offering vital insights for peptide drug development and protein science.

moPPIt: De Novo Generation of Motif-Specific Binders with Protein Language Models

The ability to precisely target specific motifs on disease-related proteins, whether conserved epitopes on viral proteins, intrinsically disordered regions within transcription factors, or breakpoint junctions in fusion oncoproteins, is essential for modulating their function while minimizing off-target effects. Current methods struggle to achieve this specificity without reliable structural information. In this work, we introduce a motif-specific PPI targeting algorithm, moPPIt, for de novo generation of motif-specific peptide binders from the target protein sequence alone. At the core of moPPIt is BindEvaluator, a transformer-based model that interpolates protein language model embeddings of two proteins via a series of multi-headed self-attention blocks, with a key focus on local motif features. Trained on over 510,000 annotated PPIs, BindEvaluator accurately predicts target binding sites given protein-protein sequence pairs with a test AUC > 0.94, improving to AUC > 0.96 when fine-tuned on peptide-protein pairs. By combining BindEvaluator with our PepMLM peptide generator and genetic algorithm-based optimization, moPPIt generates peptides that bind specifically to user-defined residues on target proteins. We demonstrate moPPIt's efficacy in computationally designing binders to specific motifs, first on targets with known binding peptides and then extending to structured and disordered targets with no known binders. In total, moPPIt serves as a powerful tool for developing highly specific peptide therapeutics without relying on target structure or structure-dependent latent spaces.

Programmable Protein Stabilization with Language Model-Derived Peptide Guides

Dysregulated protein degradation via the ubiquitin-proteasomal pathway can induce numerous disease phenotypes, including cancer, neurodegeneration, and diabetes. Stabilizing improperly ubiquitinated proteins via target-specific deubiquitination is thus a critical therapeutic goal. Building off the major advances in targeted protein degradation (TPD) using bifunctional small-molecule degraders, targeted protein stabilization (TPS) modalities have been described recently. However, these rely on a limited set of chemical linkers and warheads, which are difficult to generate de novo for new targets and do not exist for classically "undruggable" targets. To address the limited reach of small molecule-based degraders, we previously engineered ubiquibodies (uAbs) by fusing computationally-designed "guide" peptides to E3 ubiquitin ligase domains for modular, CRISPR-analogous TPD. Here, we expand the TPS target space by engineering "deubiquibodies" (duAbs) via fusion of computationally-designed guides to the catalytic domain of the potent OTUB1 deubiquitinase. In human cells, duAbs effectively stabilize exogenous and endogenous proteins in a DUB-dependent manner. To demonstrate duAb modularity, we swap in new target-binding peptides designed via our generative language models to stabilize diverse target proteins, including key tumor suppressor proteins such as p53 and WEE1, as well as heavily-disordered fusion oncoproteins, such as PAX3::FOXO1. In total, our duAb system represents a simple, programmable, CRISPR-analogous strategy for TPS.

FusOn-pLM: A Fusion Oncoprotein-Specific Language Model via Focused Probabilistic Masking

Fusion oncoproteins, a class of chimeric proteins arising from chromosomal translocations, drive and sustain various cancers, particularly those impacting children. Unfortunately, due to their intrinsically disordered nature, large size, and lack of well-defined, druggable pockets, they have been historically challenging to target therapeutically: neither small molecule-based methods nor structure-based approaches for binder design are strong options for this class of molecules. Recently, protein language models (pLMs) have demonstrated success at representing protein sequences with information-rich embeddings, enabling downstream design applications from sequence alone. However, no current pLM has been trained on fusion oncoprotein sequences and thus may not produce optimal representations for these proteins. In this work, we introduce FusOn-pLM, a novel pLM that fine-tunes the state-of-the-art ESM-2 model on fusion oncoprotein sequences. We specifically introduce a novel masked language modeling (MLM) strategy, employing a binding-site probability predictor to focus masking on key amino acid residues, thereby generating more optimal fusion oncoprotein-aware embeddings. Our model improves performance on both fusion oncoprotein-specific benchmarks and disorder prediction tasks in comparison to baseline ESM-2 representations, as well as manually-constructed biophysical embeddings, motivating downstream usage of FusOn-pLM embeddings for therapeutic design tasks targeting these fusions. We have made our model publicly available to the community at https://huggingface.co/ChatterjeeLab/FusOn-pLM.

PTM-Mamba: A PTM-Aware Protein Language Model with Bidirectional Gated Mamba Blocks

Proteins serve as the workhorses of living organisms, orchestrating a wide array of vital functions. Post-translational modifications (PTMs) of their amino acids greatly influence the structural and functional diversity of different protein types and uphold proteostasis, allowing cells to swiftly respond to environmental changes and intricately regulate complex biological processes. To this point, efforts to model the complex features of proteins have involved the training of large and expressive protein language models (pLMs) such as ESM-2 and ProtT5, which accurately encode structural, functional, and physicochemical properties of input protein sequences. However, the over 200 million sequences that these pLMs were trained on merely scratch the surface of proteomic diversity, as they neither input nor account for the effects of PTMs. In this work, we fill this major gap in protein sequence modeling by introducing PTM tokens into the pLM training regime. We then leverage recent advancements in structured state space models (SSMs), specifically Mamba, which utilizes efficient hardware-aware primitives to overcome the quadratic time complexities of Transformers. After adding a comprehensive set of PTM tokens to the model vocabulary, we train bidirectional Mamba blocks whose outputs are fused with state-of-the-art ESM-2 embeddings via a novel gating mechanism. We demonstrate that our resultant PTM-aware pLM, PTM-Mamba, improves upon ESM-2's performance on various PTM-specific tasks. PTM-Mamba is the first and only pLM that can uniquely input and represent both wild-type and PTM sequences, motivating downstream modeling and design applications specific to post-translationally modified proteins. To facilitate PTM-aware protein language modeling applications, we have made our model available at: https://huggingface.co/ChatterjeeLab/PTM-Mamba.