Deep learning models for predicting RNA degradation via dual crowdsourcing

Generative and predictive neural networks for the design of functional RNA molecules

RNA is a remarkably versatile molecule that has been engineered for applications in therapeutics, diagnostics, and in vivo information-processing systems. However, the complex relationship between the sequence, structure, and function of RNA often necessitates extensive experimental screening of candidate sequences. Here we present a generalized, efficient neural network architecture that utilizes the sequence and structure of RNA molecules (SANDSTORM) to inform functional predictions across a diverse range of settings. We pair these predictive models with generative adversarial RNA design networks (GARDN), allowing the generative modelling of a diverse range of functional RNA molecules with targeted experimental attributes. This approach enables the design of novel sequence candidates that outperform those encountered during training or returned by classical thermodynamic algorithms, and can be deployed using as few as 384 example sequences. SANDSTORM and GARDN thus represent powerful new predictive and generative tools for the development of RNA molecules with improved function.

mRNA vaccine sequence and structure design and optimization: Advances and challenges

Messenger RNA (mRNA) vaccines have emerged as a powerful tool against communicable diseases and cancers, as demonstrated by their huge success during the coronavirus disease 2019 (COVID-19) pandemic. Despite the outstanding achievements, mRNA vaccines still face challenges such as stringent storage requirements, insufficient antigen expression, and unexpected immune responses. Since the intrinsic properties of mRNA molecules significantly impact vaccine performance, optimizing mRNA design is crucial in preclinical development. In this review, we outline four key principles for optimal mRNA sequence design: enhancing ribosome loading and translation efficiency through untranslated region (UTR) optimization, improving translation efficiency via codon optimization, increasing structural stability by refining global RNA sequence and extending in-cell lifetime and expression fidelity by adjusting local RNA structures. We also explore recent advancements in computational models for designing and optimizing mRNA vaccine sequences following these principles. By integrating current mRNA knowledge, addressing challenges, and examining advanced computational methods, this review aims to promote the application of computational approaches in mRNA vaccine development and inspire novel solutions to existing obstacles.

StructmRNA a BERT based model with dual level and conditional masking for mRNA representation

In this study, we introduce StructmRNA, a new BERT-based model that was designed for the detailed analysis of mRNA sequences and structures. The success of DNABERT in understanding the intricate language of non-coding DNA with bidirectional encoder representations is extended to mRNA with StructmRNA. This new model uses a special dual-level masking technique that covers both sequence and structure, along with conditional masking. This enables StructmRNA to adeptly generate meaningful embeddings for mRNA sequences, even in the absence of explicit structural data, by capitalizing on the intricate sequence-structure correlations learned during extensive pre-training on vast datasets. Compared to well-known models like those in the Stanford OpenVaccine project, StructmRNA performs better in important tasks such as predicting RNA degradation. Thus, StructmRNA can inform better RNA-based treatments by predicting the secondary structures and biological functions of unseen mRNA sequences. The proficiency of this model is further confirmed by rigorous evaluations, revealing its unprecedented ability to generalize across various organisms and conditions, thereby marking a significant advance in the predictive analysis of mRNA for therapeutic design. With this work, we aim to set a new standard for mRNA analysis, contributing to the broader field of genomics and therapeutic development.

Advances in the design and delivery of RNA vaccines for infectious diseases

RNA medicines represent a paradigm shift in treatment and prevention of critical diseases of global significance, e.g., infectious diseases. The highly successful messenger RNA (mRNA) vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) were developed at record speed during the coronavirus disease 2019 pandemic. A consequence of this is exceptionally shortened vaccine development times, which in combination with adaptability makes the RNA vaccine technology highly attractive against infectious diseases and for pandemic preparedness. Here, we review state of the art in the design and delivery of RNA vaccines for infectious diseases based on different RNA modalities, including linear mRNA, self-amplifying RNA, trans-amplifying RNA, and circular RNA. We provide an overview of the clinical pipeline of RNA vaccines for infectious diseases, and present analytical procedures, which are paramount for characterizing quality attributes and guaranteeing their quality, and we discuss future perspectives for using RNA vaccines to combat pathogens beyond SARS-CoV-2.

Machine Learning Uncovers Vascular Endothelial Cell Identity Genes by Expression Regulation Features in Single Cells

Deciphering cell identity genes is pivotal to understanding cell differentiation, development, and many diseases involving cell identity dysregulation. Here, we introduce SCIG, a machine-learning method to uncover cell identity genes in single cells. In alignment with recent reports that cell identity genes are regulated with unique epigenetic signatures, we found cell identity genes exhibit distinctive genetic sequence signatures, e.g., unique enrichment patterns of cis-regulatory elements. Using these genetic sequence signatures, along with gene expression information from single-cell RNA-seq data, enables SCIG to uncover the identity genes of a cell without a need for comparison to other cells. Cell identity gene score defined by SCIG surpassed expression value in network analysis to uncover master transcription factors regulating cell identity. Applying SCIG to the human endothelial cell atlas revealed that the tissue microenvironment is a critical supplement to master transcription factors for cell identity refinement. SCIG is publicly available at https://github.com/kaifuchenlab/SCIG , offering a valuable tool for advancing cell differentiation, development, and regenerative medicine research.

CodonBERT large language model for mRNA vaccines

mRNA-based vaccines and therapeutics are gaining popularity and usage across a wide range of conditions. One of the critical issues when designing such mRNAs is sequence optimization. Even small proteins or peptides can be encoded by an enormously large number of mRNAs. The actual mRNA sequence can have a large impact on several properties, including expression, stability, immunogenicity, and more. To enable the selection of an optimal sequence, we developed CodonBERT, a large language model (LLM) for mRNAs. Unlike prior models, CodonBERT uses codons as inputs, which enables it to learn better representations. CodonBERT was trained using more than 10 million mRNA sequences from a diverse set of organisms. The resulting model captures important biological concepts. CodonBERT can also be extended to perform prediction tasks for various mRNA properties. CodonBERT outperforms previous mRNA prediction methods, including on a new flu vaccine data set.

Ribonanza: deep learning of RNA structure through dual crowdsourcing

Prediction of RNA structure from sequence remains an unsolved problem, and progress has been slowed by a paucity of experimental data. Here, we present Ribonanza, a dataset of chemical mapping measurements on two million diverse RNA sequences collected through Eterna and other crowdsourced initiatives. Ribonanza measurements enabled solicitation, training, and prospective evaluation of diverse deep neural networks through a Kaggle challenge, followed by distillation into a single, self-contained model called RibonanzaNet. When fine tuned on auxiliary datasets, RibonanzaNet achieves state-of-the-art performance in modeling experimental sequence dropout, RNA hydrolytic degradation, and RNA secondary structure, with implications for modeling RNA tertiary structure.

Big data and deep learning for RNA biology

The exponential growth of big data in RNA biology (RB) has led to the development of deep learning (DL) models that have driven crucial discoveries. As constantly evidenced by DL studies in other fields, the successful implementation of DL in RB depends heavily on the effective utilization of large-scale datasets from public databases. In achieving this goal, data encoding methods, learning algorithms, and techniques that align well with biological domain knowledge have played pivotal roles. In this review, we provide guiding principles for applying these DL concepts to various problems in RB by demonstrating successful examples and associated methodologies. We also discuss the remaining challenges in developing DL models for RB and suggest strategies to overcome these challenges. Overall, this review aims to illuminate the compelling potential of DL for RB and ways to apply this powerful technology to investigate the intriguing biology of RNA more effectively.

HormoNet: a deep learning approach for hormone-drug interaction prediction

Several experimental evidences have shown that the human endogenous hormones can interact with drugs in many ways and affect drug efficacy. The hormone drug interactions (HDI) are essential for drug treatment and precision medicine; therefore, it is essential to understand the hormone-drug associations. Here, we present HormoNet to predict the HDI pairs and their risk level by integrating features derived from hormone and drug target proteins. To the best of our knowledge, this is one of the first attempts to employ deep learning approach for prediction of HDI prediction. Amino acid composition and pseudo amino acid composition were applied to represent target information using 30 physicochemical and conformational properties of the proteins. To handle the imbalance problem in the data, we applied synthetic minority over-sampling technique technique. Additionally, we constructed novel datasets for HDI prediction and the risk level of their interaction. HormoNet achieved high performance on our constructed hormone-drug benchmark datasets. The results provide insights into the understanding of the relationship between hormone and a drug, and indicate the potential benefit of reducing risk levels of interactions in designing more effective therapies for patients in drug treatments. Our benchmark datasets and the source codes for HormoNet are available in: https://github.com/EmamiNeda/HormoNet .

Development and use of machine learning algorithms in vaccine target selection

Computer-aided discovery of vaccine targets has become a cornerstone of rational vaccine design. In this article, I discuss how Machine Learning (ML) can inform and guide key computational steps in rational vaccine design concerned with the identification of B and T cell epitopes and correlates of protection. I provide examples of ML models, as well as types of data and predictions for which they are built. I argue that interpretable ML has the potential to improve the identification of immunogens also as a tool for scientific discovery, by helping elucidate the molecular processes underlying vaccine-induced immune responses. I outline the limitations and challenges in terms of data availability and method development that need to be addressed to bridge the gap between advances in ML predictions and their translational application to vaccine design.

Nucleic acid degradation as barrier to gene delivery: a guide to understand and overcome nuclease activity

Gene therapy is on its way to revolutionize the treatment of both inherited and acquired diseases, by transferring nucleic acids to correct a disease-causing gene in the target cells of patients. In the fight against infectious diseases, mRNA-based therapeutics have proven to be a viable strategy in the recent Covid-19 pandemic. Although a growing number of gene therapies have been approved, the success rate is limited when compared to the large number of preclinical and clinical trials that have been/are being performed. In this review, we highlight some of the hurdles which gene therapies encounter after administration into the human body, with a focus on nucleic acid degradation by nucleases that are extremely abundant in mammalian organs, biological fluids as well as in subcellular compartments. We overview the available strategies to reduce the biodegradation of gene therapeutics after administration, including chemical modifications of the nucleic acids, encapsulation into vectors and co-administration with nuclease inhibitors and discuss which strategies are applied for clinically approved nucleic acid therapeutics. In the final part, we discuss the currently available methods and techniques to qualify and quantify the integrity of nucleic acids, with their own strengths and limitations.

RNA Metabolism Governs Immune Function and Response

Inflammation is a complex process that protects our body from various insults such as infection, injury, and stress. Proper inflammation is beneficial to eliminate the insults and maintain organ homeostasis, however, it can become detrimental if uncontrolled. To tightly regulate inflammation, post-transcriptional mechanisms governing RNA metabolism play a crucial role in monitoring the expression of immune-related genes, such as tumor necrosis factor (TNF) and interleukin-6 (IL-6). These mechanisms involve the coordinated action of various RNA-binding proteins (RBPs), including the Regnase family, Roquin, and RNA methyltransferases, which are responsible for mRNA decay and/or translation regulation. The collaborative efforts of these RBPs are essential in preventing aberrant immune response activation and consequently safeguarding against inflammatory and autoimmune diseases. This review provides an overview of recent advancements in our understanding of post-transcriptional regulation within the immune system and explores the specific roles of individual RBPs in RNA metabolism and regulation.

The mRNA Vaccine Revolution: COVID-19 Has Launched the Future of Vaccinology

During the COVID-19 pandemic, mRNA (mRNA) vaccines emerged as leading vaccine candidates in a record time. Nonreplicating mRNA (NRM) and self-amplifying mRNA (SAM) technologies have been developed into high-performing and clinically viable vaccines against a range of infectious agents, notably SARS-CoV-2. mRNA vaccines demonstrate efficient in vivo delivery, long-lasting stability, and nonexistent risk of infection. The stability and translational efficiency of in vitro transcription (IVT)-mRNA can be further increased by modulating its structural elements. In this review, we present a comprehensive overview of the recent advances, key applications, and future challenges in the field of mRNA-based vaccinology.

Recent trends in RNA informatics: a review of machine learning and deep learning for RNA secondary structure prediction and RNA drug discovery

Computational analysis of RNA sequences constitutes a crucial step in the field of RNA biology. As in other domains of the life sciences, the incorporation of artificial intelligence and machine learning techniques into RNA sequence analysis has gained significant traction in recent years. Historically, thermodynamics-based methods were widely employed for the prediction of RNA secondary structures; however, machine learning-based approaches have demonstrated remarkable advancements in recent years, enabling more accurate predictions. Consequently, the precision of sequence analysis pertaining to RNA secondary structures, such as RNA-protein interactions, has also been enhanced, making a substantial contribution to the field of RNA biology. Additionally, artificial intelligence and machine learning are also introducing technical innovations in the analysis of RNA-small molecule interactions for RNA-targeted drug discovery and in the design of RNA aptamers, where RNA serves as its own ligand. This review will highlight recent trends in the prediction of RNA secondary structure, RNA aptamers and RNA drug discovery using machine learning, deep learning and related technologies, and will also discuss potential future avenues in the field of RNA informatics.

Generative and predictive neural networks for the design of functional RNA molecules

RNA is a remarkably versatile molecule that has been engineered for applications in therapeutics, diagnostics, and in vivo information-processing systems. However, the complex relationship between the sequence and structural properties of an RNA molecule and its ability to perform specific functions often necessitates extensive experimental screening of candidate sequences. Here we present a generalized neural network architecture that utilizes the sequence and structure of RNA molecules (SANDSTORM) to inform functional predictions. We demonstrate that this approach achieves state-of-the-art performance across several distinct RNA prediction tasks, while learning interpretable abstractions of RNA secondary structure. We paired these predictive models with generative adversarial RNA design networks (GARDN), allowing the generative modelling of novel mRNA 5' untranslated regions and toehold switch riboregulators exhibiting a predetermined fitness. This approach enabled the design of novel toehold switches with a 43-fold increase in experimentally characterized dynamic range compared to those designed using classic thermodynamic algorithms. SANDSTORM and GARDN thus represent powerful new predictive and generative tools for the development of diagnostic and therapeutic RNA molecules with improved function.

In silico methods for predicting functional synonymous variants

Single nucleotide variants (SNVs) contribute to human genomic diversity. Synonymous SNVs are previously considered to be "silent," but mounting evidence has revealed that these variants can cause RNA and protein changes and are implicated in over 85 human diseases and cancers. Recent improvements in computational platforms have led to the development of numerous machine-learning tools, which can be used to advance synonymous SNV research. In this review, we discuss tools that should be used to investigate synonymous variants. We provide supportive examples from seminal studies that demonstrate how these tools have driven new discoveries of functional synonymous SNVs.