Deciphering "the language of nature": A transformer-based language model for deleterious mutations in proteins

Ge-SAND: an explainable deep learning-driven framework for disease risk prediction by uncovering complex genetic interactions in parallel

BACKGROUND

Accurate genetic risk prediction and understanding the mechanisms underlying complex diseases are essential for effective intervention and precision medicine. However, current methods often struggle to capture the intricate and subtle genetic interactions contributing to disease risk. This challenge may be further exacerbated by the curse of dimensionality when considering large-scale pairwise genetic combinations with limited samples. Overcoming these limitations could transform biomedicine by providing deeper insights into disease mechanisms, moving beyond black-box models and single-locus analyses, and enabling a more comprehensive understanding of cross-disease patterns.

RESULTS

We developed Ge-SAND (Genomic Embedding Self-Attention Neurodynamic Decoder), an explainable deep learning-driven framework designed to uncover complex genetic interactions at scales exceeding 106 in parallel for accurate disease risk prediction. Ge-SAND leverages genotype and genomic positional information to identify both intra- and interchromosomal interactions associated with disease phenotypes, providing comprehensive insights into pathogenic mechanisms crucial for disease risk prediction. Applied to simulated datasets and UK Biobank cohorts for Crohn's disease, schizophrenia, and Alzheimer's disease, Ge-SAND achieved up to a 20% improvement in AUC-ROC compared to mainstream methods. Beyond its predictive accuracy, through self-attention-based interaction networks, Ge-SAND provided insights into large-scale genotype relationships and revealed genetic mechanisms underlying these complex diseases. For instance, Ge-SAND identified potential genetic interaction pairs, including novel relationships such as ISOC1 and HOMER2, potentially implicating the brain-gut axis in Crohn's and Alzheimer's diseases.

CONCLUSION

Ge-SAND is a novel deep-learning approach designed to address the challenges of capturing large-scale genetic interactions. By integrating disease risk prediction with interpretable insights into genetic mechanisms, Ge-SAND offers a valuable tool for advancing genomic research and precision medicine.

Variant effect predictor correlation with functional assays is reflective of clinical classification performance

BACKGROUND

Understanding the relationship between protein sequence and function is crucial for accurate classification of missense variants. Variant effect predictors (VEPs) play a vital role in deciphering this complex relationship, yet evaluating their performance remains challenging for several reasons, including data circularity, where the same or related data is used for training and assessment. High-throughput experimental strategies like deep mutational scanning (DMS) offer a promising solution.

RESULTS

In this study, we extend upon our previous benchmarking approach, assessing the performance of 97 VEPs using missense DMS measurements from 36 different human proteins. In addition, a new pairwise, VEP-centric approach mitigates the impact of missing predictions on overall performance comparison. We observe a strong correspondence between VEP performance in DMS-based benchmarks and clinical variant classification, especially for predictors that have not been directly trained on human clinical variants.

CONCLUSIONS

Our results suggest that comparing VEP performance against diverse functional assays represents a reliable strategy for assessing their relative performance in clinical variant classification. However, major challenges in clinical interpretation of VEP scores persist, highlighting the need for further research to fully leverage computational predictors for genetic diagnosis. We also address practical considerations for end users in terms of choice of methodology.

Protein structure prediction via deep learning: an in-depth review

The application of deep learning algorithms in protein structure prediction has greatly influenced drug discovery and development. Accurate protein structures are crucial for understanding biological processes and designing effective therapeutics. Traditionally, experimental methods like X-ray crystallography, nuclear magnetic resonance, and cryo-electron microscopy have been the gold standard for determining protein structures. However, these approaches are often costly, inefficient, and time-consuming. At the same time, the number of known protein sequences far exceeds the number of experimentally determined structures, creating a gap that necessitates the use of computational approaches. Deep learning has emerged as a promising solution to address this challenge over the past decade. This review provides a comprehensive guide to applying deep learning methodologies and tools in protein structure prediction. We initially outline the databases related to the protein structure prediction, then delve into the recently developed large language models as well as state-of-the-art deep learning-based methods. The review concludes with a perspective on the future of predicting protein structure, highlighting potential challenges and opportunities.

The pharmacogenomic landscape in the Chinese: An analytics of pharmacogenetic variants in 206,640 individuals

Pharmacogenomic landscapes and related databases are important for identifying the biomarkers of drug response and toxicity. However, these data are still lacking for the Chinese population. In this study, we constructed a pharmacogenomic landscape and an associated database using whole-genome sequencing data generated by non-invasive prenatal testing in 206,640 Chinese individuals. In total, 1,577,513 variants (including 331,610 novel variants) were identified among 3,538 pharmacogenes related to 2,086 drugs. We found that the variant spectrum in the Chinese population differed among the seven major regions. Regional differences also exist among provinces in China. The average numbers of drug enzyme, transporter, and receptor variants were 258, 557, and 632, respectively. Subsequent correlation analysis indicated that the pharmacogenes affecting multiple drugs had fewer variants. Among the 16 categories of drugs, we found that nervous system, cardiovascular system, and genitourinary system/sex hormone drugs were more likely to be affected by variants of pharmacogenes. Characteristics of the variants in the enzyme, transporter, and receptor subfamilies showed specificity. To explore the clinical utility of these data, a genetic association study was conducted on 1,019 lung cancer patients. Two novel variants, AKT2 chr19:40770621 C>G and SLC19A1 chr21:46934171 A>C, were identified as novel platinum response biomarkers. Finally, a pharmacogenomic database, named the Chinese Pharmacogenomic Knowledge Base (CNPKB: http://www.cnpkb.com.cn/), was constructed to collect all the data. In summary, a pharmacogenomic landscape and database for the Chinese population were constructed in this study, which could support personalized Chinese medicine in the future.

Paying attention to the SARS-CoV-2 dialect : a deep neural network approach to predicting novel protein mutations

Predicting novel mutations has long-lasting impacts on life science research. Traditionally, this problem is addressed through wet-lab experiments, which are often expensive and time consuming. The recent advancement in neural language models has provided stunning results in modeling and deciphering sequences. In this paper, we propose a Deep Novel Mutation Search (DNMS) method, using deep neural networks, to model protein sequence for mutation prediction. We use SARS-CoV-2 spike protein as the target and use a protein language model to predict novel mutations. Different from existing research which is often limited to mutating the reference sequence for prediction, we propose a parent-child mutation prediction paradigm where a parent sequence is modeled for mutation prediction. Because mutations introduce changing context to the underlying sequence, DNMS models three aspects of the protein sequences: semantic changes, grammatical changes, and attention changes, each modeling protein sequence aspects from shifting of semantics, grammar coherence, and amino-acid interactions in latent space. A ranking approach is proposed to combine all three aspects to capture mutations demonstrating evolving traits, in accordance with real-world SARS-CoV-2 spike protein sequence evolution. DNMS can be adopted for an early warning variant detection system, creating public health awareness of future SARS-CoV-2 mutations.

GPTrans: A Biological Language Model-Based Approach for Predicting Disease-Associated Mutations in G Protein-Coupled Receptors

Accurately predicting mutations in G protein-coupled receptors (GPCRs) is critical for advancing disease diagnosis and drug discovery. In response to this imperative, GPTrans has emerged as a highly accurate predictor of disease-related mutations in GPCRs. The core innovation of GPTrans resides in the design of a novel feature extraction network, that is capable of integrating features from both wildtype and mutant protein variant sites, utilizing multifeature connections within a transformer framework to ensure comprehensive feature extraction. A key aspect of GPTrans's effectiveness is our introduction of an innovative deep feature integration strategy, which merges embeddings and class tokens from multiple protein language models, including evolutionary scale modeling and ProtTrans, thus shedding light on the biochemical properties of proteins. Leveraging transformer components and a self-attention mechanism, GPTrans captures higher-level representations of protein features. Employing both wildtype and mutation site information for feature fusion not only enriches the predictive feature set but also avoids the common issue of overestimation associated with sequence-based predictions. This approach distinguishes GPTrans, enabling it to significantly outperform existing methods. Our evaluations across diverse GPCR data sets, including ClinVar and MutHTP, demonstrate GPTrans's superior performance, with average AUC values of 0.874 and 0.590 in 10-fold cross-validation. Notably, compared to the AlphaMissense method, GPTrans exhibited a remarkable 38.03% improvement in accuracy when predicting disease-associated mutations in the MutHTP data set. A thorough analysis of the predicted results further validates the model's effectiveness. The source code, data sets, and prediction results for GPTrans are available for academic use at https://github.com/EduardWang/GPTrans.

Genetic variant classification by predicted protein structure: A case study on IRF6

Next-generation genome sequencing has revolutionized genetic testing, identifying numerous rare disease-associated gene variants. However, to impute pathogenicity, computational approaches remain inadequate and functional testing of gene variant is required to provide the highest level of evidence. The emergence of AlphaFold2 has transformed the field of protein structure determination, and here we outline a strategy that leverages predicted protein structure to enhance genetic variant classification. We used the gene IRF6 as a case study due to its clinical relevance, its critical role in cleft lip/palate malformation, and the availability of experimental data on the pathogenicity of IRF6 gene variants through phenotype rescue experiments in irf6-/- zebrafish. We compared results from over 30 pathogenicity prediction tools on 37 IRF6 missense variants. IRF6 lacks an experimentally derived structure, so we used predicted structures to explore associations between mutational clustering and pathogenicity. We found that among these variants, 19 of 37 were unanimously predicted as deleterious by computational tools. Comparing in silico predictions with experimental findings, 12 variants predicted as pathogenic were experimentally determined as benign. Even with the recently published AlphaMissense model, 15/18 (83%) of the predicted pathogenic variants were experimentally determined as benign. In comparison, mapping variants to the protein revealed deleterious mutation clusters around the protein binding domain, whereas N-terminal variants tend to be benign, suggesting the importance of structural information in determining pathogenicity of mutations in this gene. In conclusion, incorporating gene-specific structural features of known pathogenic/benign mutations may provide meaningful insights into pathogenicity predictions in a gene-specific manner and facilitate the interpretation of variant pathogenicity.

ProPept-MT: A Multi-Task Learning Model for Peptide Feature Prediction

In the realm of quantitative proteomics, data-independent acquisition (DIA) has emerged as a promising approach, offering enhanced reproducibility and quantitative accuracy compared to traditional data-dependent acquisition (DDA) methods. However, the analysis of DIA data is currently hindered by its reliance on project-specific spectral libraries derived from DDA analyses, which not only limits proteome coverage but also proves to be a time-intensive process. To overcome these challenges, we propose ProPept-MT, a novel deep learning-based multi-task prediction model designed to accurately forecast key features such as retention time (RT), ion intensity, and ion mobility (IM). Leveraging advanced techniques such as multi-head attention and BiLSTM for feature extraction, coupled with Nash-MTL for gradient coordination, ProPept-MT demonstrates superior prediction performance. Integrating ion mobility alongside RT, mass-to-charge ratio (m/z), and ion intensity forms 4D proteomics. Then, we outline a comprehensive workflow tailored for 4D DIA proteomics research, integrating the use of 4D in silico libraries predicted by ProPept-MT. Evaluation on a benchmark dataset showcases ProPept-MT's exceptional predictive capabilities, with impressive results including a 99.9% Pearson correlation coefficient (PCC) for RT prediction, a median dot product (DP) of 96.0% for fragment ion intensity prediction, and a 99.3% PCC for IM prediction on the test set. Notably, ProPept-MT manifests efficacy in predicting both unmodified and phosphorylated peptides, underscoring its potential as a valuable tool for constructing high-quality 4D DIA in silico libraries.

Application of Transformers in Cheminformatics

By accelerating time-consuming processes with high efficiency, computing has become an essential part of many modern chemical pipelines. Machine learning is a class of computing methods that can discover patterns within chemical data and utilize this knowledge for a wide variety of downstream tasks, such as property prediction or substance generation. The complex and diverse chemical space requires complex machine learning architectures with great learning power. Recently, learning models based on transformer architectures have revolutionized multiple domains of machine learning, including natural language processing and computer vision. Naturally, there have been ongoing endeavors in adopting these techniques to the chemical domain, resulting in a surge of publications within a short period. The diversity of chemical structures, use cases, and learning models necessitate a comprehensive summarization of existing works. In this paper, we review recent innovations in adapting transformers to solve learning problems in chemistry. Because chemical data is diverse and complex, we structure our discussion based on chemical representations. Specifically, we highlight the strengths and weaknesses of each representation, the current progress of adapting transformer architectures, and future directions.

Deep learning for cross-region streamflow and flood forecasting at a global scale

Streamflow and flood forecasting remains one of the long-standing challenges in hydrology. Traditional physically based models are hampered by sparse parameters and complex calibration procedures particularly in ungauged catchments. We propose a novel hybrid deep learning model termed encoder-decoder double-layer long short-term memory (ED-DLSTM) to address streamflow forecasting at global scale for all (gauged and ungauged) catchments. Using historical datasets, ED-DLSTM yields a mean Nash-Sutcliffe efficiency coefficient (NSE) of 0.75 across more than 2,000 catchments from the United States, Canada, Central Europe, and the United Kingdom, highlighting improvements by the state-of-the-art machine learning over traditional hydrologic models. Moreover, ED-DLSTM is applied to 160 ungauged catchments in Chile and 76.9% of catchments obtain NSE >0 in the best situation. The interpretability of cross-region capacities of ED-DLSTM are established through the cell state induced by adding a spatial attribute encoding module, which can spontaneously form hydrological regionalization effects after performing spatial coding for different catchments. The study demonstrates the potential of deep leaning methods to overcome the ubiquitous lack of hydrologic information and deficiencies in physical model structure and parameterization.

Identification and characterization of novel genetic variants in the first Chinese family of mucopolysaccharidosis IIIC (Sanfilippo C syndrome)

Mucopolysaccharidosis type IIIC (MPS IIIC) is one of inherited lysosomal storage disorders, caused by deficiencies in lysosomal hydrolases degrading acidic mucopolysaccharides. The gene responsible for MPS IIIC is HGSNAT, which encodes an enzyme that catalyses the acetylation of the terminal glucosamine residues of heparan sulfate. So far, few studies have focused on the genetic landscape of MPS IIIC in China, where IIIA and IIIB were the major subtypes. In this study, we utilized whole-exome sequencing (WES) to identify novel compound heterozygous variants in the HGSNAT gene from a Chinese patient with typical MPS IIIC symptoms: c.743G>A; p.Gly248Glu and c.1030C>T; p.Arg344Cys. We performed in silico analysis and experimental validation, which confirmed the deleterious pathogenic nature of both variants, as evidenced by the loss of HGSNAT activity and failure of lysosomal localization. To the best of our knowledge, the MPS IIIC is first confirmed by clinical, biochemical and molecular genetic findings in China. Our study thus expands the spectrum of MPS IIIC pathogenic variants, which is of importance to dissect the pathogenesis and to carry out clinical diagnosis of MPS IIIC. Moreover, this study helps to depict the natural history of Chinese MPS IIIC populations.