Integrating Pre-Trained protein language model and multiple window scanning deep learning networks for accurate identification of secondary active transporters in membrane proteins

DeepCR: predicting cytokine receptor proteins through pretrained language models and deep learning networks

Cytokine receptors play a pivotal role in mediating the immune response and are critical in cytokine storms, which underlie the pathogenesis of conditions such as acute respiratory distress syndrome (ARDS) and autoimmune disorders. Identifying cytokine receptors is essential for understanding their biological functions, exploring therapeutic targets, and guiding clinical interventions. Traditional biochemical methods to identify cytokine receptors are labor-intensive, costly, and time-consuming, prompting the need for more efficient alternatives. Recent advances in computational biology have enabled the use of machine learning to classify cytokine receptor proteins. Most existing approaches focused on homologous features and protein composition to classify cytokine families, but no dedicated studies have been conducted on cytokine receptor proteins. This gap presents an opportunity to develop a method specifically for classifying cytokine receptors among other membrane proteins. In this study, we present a novel classification framework combining pre-trained language models (PLMs) with a multi-window convolutional neural network (mCNN) architecture for the fast and accurate identification of cytokine receptor proteins. PLMs, such as ProtTrans and ESM variants, capture biochemical context directly from raw protein sequences, while mCNN efficiently extracts local and global sequence patterns using convolutional layers with varying window sizes. Our model achieved an AUC of 0.96 in the training as well as 0.97 and 0.93 in two independent tests, demonstrating its effectiveness in distinguishing cytokine receptors from non-cytokine receptor proteins. By eliminating the need for manual feature extraction, this approach offers a robust and scalable solution for protein classification, paving the way for its application in drug discovery and understanding cytokine-mediated diseases.

HIV multidrug class resistance prediction with a time sliding anchor approach

Motivation

The emergence of multidrug class resistance (MDR) in Human Immunodeficiency Virus (HIV) is a rare but significant challenge in antiretroviral therapy (ART). MDR, which may arise from prolonged drug exposure, treatment failures, or transmission of resistant strains, accelerates disease progression and poses particular challenges in resource-limited settings with restricted access to resistance testing and advanced therapies. Early prediction of future MDR development is important to inform therapeutic decisions and mitigate its occurrence.

Results

In this study, we employ various machine learning classifiers to predict future resistance to all four major antiretroviral drug classes using features extracted from clinical HIV sequence data. We systematically explore several variations of the problem that differ in the pre-existing resistance level and the temporal gap between sample collection and observed MDR occurrence. Our models show the ability to predict multidrug class resistance even in the most challenging variations, albeit at a reduced accuracy. Feature importance analysis reveals that our models primarily utilize known drug resistance mutations for easier classification tasks, but rely on new mutations for the difficult task of distinguishing four class drug resistance from three class drug resistance.

Availability and implementation

All analysis was performed using the Euresist Integrated DataBase (EIDB). Researchers wishing to reproduce, validate or extend these findings can request access to the latest EIDB release via the Euresist Network.

NA_mCNN: Classification of Sodium Transporters in Membrane Proteins by Integrating Multi-Window Deep Learning and ProtTrans for Their Therapeutic Potential

Sodium transporters maintain cellular homeostasis by transporting ions, minerals, and nutrients across the membrane, and Na+/K+ ATPases facilitate the cotransport of solutes in neurons, muscle cells, and epithelial cells. Sodium transporters are important for many physiological processes, and their dysfunction leads to diseases such as hypertension, diabetes, neurological disorders, and cancer. The NA_mCNN computational method highlights the functional diversity and significance of sodium transporters in membrane proteins using protein language model embeddings (PLMs) and multiple-window scanning deep learning models. This work investigates PLMs that include Tape, ProtTrans, ESM-1b-1280, and ESM-2-128 to achieve more accuracy in sodium transporter classification. Five-fold cross-validation and independent testing demonstrate ProtTrans embedding robustness. In cross-validation, ProtTrans achieved an AUC of 0.9939, a sensitivity of 0.9829, and a specificity of 0.9889, demonstrating its ability to distinguish positive and negative samples. In independent testing, ProtTrans maintained a sensitivity of 0.9765, a specificity of 0.9991, and an AUC of 0.9975, which indicates its high level of discrimination. This study advances the understanding of sodium transporter diversity and function, as well as their role in human pathophysiology. Our goal is to use deep learning techniques and protein language models for identifying sodium transporters to accelerate identification and develop new therapeutic interventions.

DeepEpiIL13: Deep Learning for Rapid and Accurate Prediction of IL-13-Inducing Epitopes Using Pretrained Language Models and Multiwindow Convolutional Neural Networks

Accurate prediction of interleukin-13 (IL-13)-inducing epitopes is crucial for advancing targeted therapies against allergic inflammation, the cytokine storm associated with severe COVID-19, and related disorders. Current epitope prediction methods, however, often exhibit limitations in efficiency and accuracy. To address this, we introduce DeepEpilL13, a novel deep learning framework that uniquely synergizes pretrained language models with multiwindow convolutional neural networks (CNNs) for the rapid and accurate identification of IL-13-inducing epitopes from protein sequences. DeepEpilL13 leverages high-dimensional embeddings generated by the pretrained language model, which capture rich contextual information from protein sequences. These embeddings are then processed by a multiwindow CNN architecture, enabling the effective exploration of both local and global sequence patterns pertinent to IL-13 induction. The proposed DeepEpilL13 approach underwent rigorous evaluation using both benchmark data sets and an independent SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) data set. Results demonstrate that DeepEpilL13 achieves superior performance compared with traditional methods. On the benchmark data set, DeepEpilL13 attained a Matthews correlation coefficient (MCC) of 0.52 and an area under the receiver operating characteristic curve (AUC) of 0.86. Notably, when assessed on the independent SARS-CoV-2 data set, DeepEpilL13 exhibited remarkable robustness, achieving an MCC of 0.63 and an AUC of 0.92. These metrics underscore the enhanced predictive capability and robust applicability of DeepEpilL13, particularly within the context of the COVID-19 research and related viral infections. This study presents DeepEpilL13 as a powerful and efficient deep learning framework for accurate epitope prediction. By offering significant improvement in performance and robustness, DeepEpilL13 provides new and promising avenues for the development of epitope-based vaccines and immunotherapies specifically targeting IL-13-mediated disorders. The successful and rapid identification of IL-13-inducing epitopes using DeepEpilL13 paves the way for novel therapeutic interventions against a range of conditions, including allergic diseases, inflammatory conditions, and severe viral infections such as COVID-19, with potential for a significant impact on public health outcomes.

RAG_MCNNIL6: A Retrieval-Augmented Multi-Window Convolutional Network for Accurate Prediction of IL-6 Inducing Epitopes

Interleukin-6 (IL-6) is a critical cytokine involved in immune regulation, inflammation, and the pathogenesis of various diseases, including autoimmune disorders, cancer, and the cytokine storm associated with severe COVID-19. Identifying IL-6 inducing epitopes, the short peptide fragments that trigger IL-6 production, is crucial for developing epitope-based vaccines and immunotherapies. However, traditional methods for epitope prediction often lack accuracy and efficiency. This study presents RAG_MCNNIL6, a novel deep learning framework that integrates Retrieval-augmented generation (RAG) with multiwindow convolutional neural networks (MCNNs) for accurate and rapid prediction of IL-6 inducing epitopes. RAG_MCNNIL6 leverages ProtTrans, a state-of-the-art pretrained protein language model, to generate rich embedding representations of peptide sequences. By incorporating a RAG-based similarity retrieval and embedding augmentation strategy, RAG_MCNNIL6 effectively captures both local and global sequence patterns relevant for IL-6 induction, significantly improving prediction performance compared to existing methods. We demonstrate the superior performance of RAG_MCNNIL6 on benchmark data sets, highlighting its potential for advancing research and therapeutic development for IL-6-mediated diseases.

mCNN-glucose: Identifying families of glucose transporters using a deep convolutional neural network based on multiple-scanning windows

Glucose transporters are essential carrier proteins that function on the phospholipid bilayer to facilitate glucose diffusion across cell membranes. The transporters play many physiological and pathological roles in addition to absorption and metabolism of fructose in food and the pathogenesis of gastrointestinal diseases. These carrier proteins play an important role in diseases of the nervous system, cardiovascular system, digestive system, and urinary system. These essential transporters have been extensively studied as potential therapeutic targets for cancers such as pancreatic, prostate, and hepatocellular carcinoma, which serve as diagnostic and prognostic indicators. The method uses position-specific scoring metrics (PSSM) with multiple-scanning windows-based convolutional neural networks to classify glucose transport proteins based on their functional significance and crucial role in therapy. Convolutional neural networks with multiple window scanning are employed to capture biologically meaningful, significant, and meaningful features from PSSM evolutionary profiles. Our proposed Method obtained Matthews correlation coefficients (MCC) of 0.99, Accuracy (AC) of 99.46, for Glucose facilitative transporters (GLUT), 0.99, 99.46, for Sodium Coupled glucose transporters (SGLT), and 0.92, and 97.3 for Sugars will eventually be exported transporters (SWEET) respectively. This study shows significantly higher performance than our previous study, which could be used to accurately classify novel glucose transporters.

ATP_mCNN: Predicting ATP binding sites through pretrained language models and multi-window neural networks

Adenosine triphosphate plays a vital role in providing energy and enabling key cellular processes through interactions with binding proteins. The increasing amount of protein sequence data necessitates computational methods for identifying binding sites. However, experimental identification of adenosine triphosphate-binding residues remains challenging. To address the challenge, we developed a multi-window convolutional neural network architecture taking pre-trained protein language model embeddings as input features. In particular, multiple parallel convolutional layers scan for motifs localized to different window sizes. Max pooling extracts salient features concatenated across windows into a final multi-scale representation for residue-level classification. On benchmark datasets, our model achieves an area under the ROC curve of 0.95, significantly improving on prior sequence-based models and outperforming convolutional neural network baselines. This demonstrates the utility of pre-trained language models and multi-window convolutional neural networks for advanced sequence-based prediction of adenosine triphosphate-binding residues. Our approach provides a promising new direction for elucidating binding mechanisms and interactions from primary structure.

MCNN_MC: Computational Prediction of Mitochondrial Carriers and Investigation of Bongkrekic Acid Toxicity Using Protein Language Models and Convolutional Neural Networks

Mitochondrial carriers (MCs) are essential proteins that transport metabolites across mitochondrial membranes and play a critical role in cellular metabolism. ADP/ATP (adenosine diphosphate/adenosine triphosphate) is one of the most important carriers as it contributes to cellular energy production and is susceptible to the powerful toxin bongkrekic acid. This toxin has claimed several lives; for example, a recent foodborne outbreak in Taipei, Taiwan, has caused four deaths and sickened 30 people. The issue of bongkrekic acid poisoning has been a long-standing problem in Indonesia, with reports as early as 1895 detailing numerous deaths from contaminated coconut fermented cakes. In bioinformatics, significant advances have been made in understanding biological processes through computational methods; however, no established computational method has been developed for identifying mitochondrial carriers. We propose a computational bioinformatics approach for predicting MCs from a broader class of secondary active transporters with a focus on the ADP/ATP carrier and its interaction with bongkrekic acid. The proposed model combines protein language models (PLMs) with multiwindow scanning convolutional neural networks (mCNNs). While PLM embeddings capture contextual information within proteins, mCNN scans multiple windows to identify potential binding sites and extract local features. Our results show 96.66% sensitivity, 95.76% specificity, 96.12% accuracy, 91.83% Matthews correlation coefficient (MCC), 94.63% F1-Score, and 98.55% area under the curve (AUC). The results demonstrate the effectiveness of the proposed approach in predicting MCs and elucidating their functions, particularly in the context of bongkrekic acid toxicity. This study presents a valuable approach for identifying novel mitochondrial complexes, characterizing their functional roles, and understanding mitochondrial toxicology mechanisms. Our findings, that utilize computational methods to improve our understanding of cellular processes and drug-target interactions, contribute to the development of therapeutic strategies for mitochondrial disorders, reducing the devastating effects of bongkrekic acid poisoning.

MCNN-AAPT: accurate classification and functional prediction of amino acid and peptide transporters in secondary active transporters using protein language models and multi-window deep learning

Secondary active transporters play a crucial role in cellular physiology by facilitating the movement of molecules across cell membranes. Identifying the functional classes of these transporters, particularly amino acid and peptide transporters, is essential for understanding their involvement in various physiological processes and disease pathways, including cancer. This study aims to develop a robust computational framework that integrates pre-trained protein language models and deep learning techniques to classify amino acid and peptide transporters within the secondary active transporter (SAT) family and predict their functional association with solute carrier (SLC) proteins. The study leverages a comprehensive dataset of 448 secondary active transporters, including 36 solute carrier proteins, obtained from UniProt and the Transporter Classification Database (TCDB). Three state-of-the-art protein language models, ProtTrans, ESM-1b, and ESM-2, are evaluated within a deep learning neural network architecture that employs a multi-window scanning technique to capture local and global sequence patterns. The ProtTrans-based feature set demonstrates exceptional performance, achieving a classification accuracy of 98.21% with 87.32% sensitivity and 99.76% specificity for distinguishing amino acid and peptide transporters from other SATs. Furthermore, the model maintains strong predictive ability for SLC proteins, with an overall accuracy of 88.89% and a Matthews Correlation Coefficient (MCC) of 0.7750. This study showcases the power of integrating pre-trained protein language models and deep learning techniques for the functional classification of secondary active transporters and the prediction of associated solute carrier proteins. The findings have significant implications for drug development, disease research, and the broader understanding of cellular transport mechanisms.

VesiMCNN: Using pre-trained protein language models and multiple window scanning convolutional neural networks to identify vesicular transport proteins

Vesicular transport is a critical cellular process responsible for the proper organization and functioning of eukaryotic cells. This mechanism relies on specialized vesicles that shuttle macromolecules, such as proteins, across the cellular landscape, a process pivotal to maintaining cellular homeostasis. Disruptions in vesicular transport have been linked to various disease mechanisms, including cancer and neurodegenerative disorders. In this study, we present vesiMCNN, a novel computational approach that integrates pre-trained protein language models with a multi-window scanning convolutional neural network architecture to accurately identify vesicular transport proteins. To the best of our knowledge, this is the first study to leverage the power of pre-trained language models in combination with the multi-window scanning technique for this task. Our method achieved a Matthews Correlation Coefficient (MCC) of 0.558 and an Area Under the Receiver Operating Characteristic (AUC-ROC) of 0.933, outperforming existing state-of-the-art approaches. Additionally, we have curated a comprehensive benchmark dataset for the study of vesicular transport proteins, which can facilitate further research in this field. The remarkable performance of our model, combined with the comprehensive dataset and novel deep learning model, marks a significant advancement in the field of vesicular transport protein research.

DeepPLM_mCNN: An approach for enhancing ion channel and ion transporter recognition by multi-window CNN based on features from pre-trained language models

Accurate classification of membrane proteins like ion channels and transporters is critical for elucidating cellular processes and drug development. We present DeepPLM_mCNN, a novel framework combining Pretrained Language Models (PLMs) and multi-window convolutional neural networks (mCNNs) for effective classification of membrane proteins into ion channels and ion transporters. Our approach extracts informative features from protein sequences by utilizing various PLMs, including TAPE, ProtT5_XL_U50, ESM-1b, ESM-2_480, and ESM-2_1280. These PLM-derived features are then input into a mCNN architecture to learn conserved motifs important for classification. When evaluated on ion transporters, our best performing model utilizing ProtT5 achieved 90% sensitivity, 95.8% specificity, and 95.4% overall accuracy. For ion channels, we obtained 88.3% sensitivity, 95.7% specificity, and 95.2% overall accuracy using ESM-1b features. Our proposed DeepPLM_mCNN framework demonstrates significant improvements over previous methods on unseen test data. This study illustrates the potential of combining PLMs and deep learning for accurate computational identification of membrane proteins from sequence data alone. Our findings have important implications for membrane protein research and drug development targeting ion channels and transporters. The data and source codes in this study are publicly available at the following link: https://github.com/s1129108/DeepPLM_mCNN.