Accelerating the Discovery and Design of Antimicrobial Peptides with Artificial Intelligence

Peptides modulate many processes of human physiology targeting ion channels, protein receptors, or enzymes. They represent valuable starting points for the development of new biologics against communicable and non-communicable disorders. However, turning native peptide ligands into druggable materials requires high selectivity and efficacy, predictable metabolism, and good safety profiles. Machine learning models have gradually emerged as cost-effective and time-saving solutions to predict and generate new proteins with optimal properties. In this chapter, we will discuss the evolution and applications of predictive modeling and generative modeling to discover and design safe and effective antimicrobial peptides. We will also present their current limitations and suggest future research directions, applicable to peptide drug design campaigns.

Improved myelin water fraction mapping with deep neural networks using synthetically generated 3D data

We introduce a generative model for synthesis of large scale 3D datasets for quantitative parameter mapping of myelin water fraction (MWF). Our model combines a MR physics signal decay model with an accurate probabilistic multi-component parametric T2 model. We synthetically generate a wide variety of high quality signals and corresponding parameters from a wide range of naturally occurring prior parameter values. To capture spatial variation, the generative signal decay model is combined with a generative spatial model conditioned on generic tissue segmentations. Synthesized 3D datasets can be used to train any convolutional neural network (CNN) based architecture for MWF estimation. Our source code is available at: https://github.com/quin-med-harvard-edu/synthmap Reduction of acquisition time at the expense of lower SNR, as well as accuracy and repeatability of MWF estimation techniques, are key factors that affect the adoption of MWF mapping in clinical practice. We demonstrate that the synthetically trained CNN provides superior accuracy over the competing methods under the constraints of naturally occurring noise levels as well as on the synthetically generated images at low SNR levels. Normalized root mean squared error (nRMSE) is less than 7% on synthetic data, which is significantly lower than competing methods. Additionally, the proposed method yields a coefficient of variation (CoV) that is at least 4x better than the competing method on intra-session test-retest reference dataset.

AB-Gen: Antibody Library Design with Generative Pre-trained Transformer and Deep Reinforcement Learning

Antibody leads must fulfill multiple desirable properties to be clinical candidates. Primarily due to the low throughput in the experimental procedure, the need for such multi-property optimization causes the bottleneck in preclinical antibody discovery and development, because addressing one issue usually causes another. We developed a reinforcement learning (RL) method, named AB-Gen, for antibody library design using a generative pre-trained transformer (GPT) as the policy network of the RL agent. We showed that this model can learn the antibody space of heavy chain complementarity determining region 3 (CDRH3) and generate sequences with similar property distributions. Besides, when using human epidermal growth factor receptor-2 (HER2) as the target, the agent model of AB-Gen was able to generate novel CDRH3 sequences that fulfill multi-property constraints. Totally, 509 generated sequences were able to pass all property filters, and three highly conserved residues were identified. The importance of these residues was further demonstrated by molecular dynamics simulations, consolidating that the agent model was capable of grasping important information in this complex optimization task. Overall, the AB-Gen method is able to design novel antibody sequences with an improved success rate than the traditional propose-then-filter approach. It has the potential to be used in practical antibody design, thus empowering the antibody discovery and development process. The source code of AB-Gen is freely available at Zenodo (https://doi.org/10.5281/zenodo.7657016) and BioCode (https://ngdc.cncb.ac.cn/biocode/tools/BT007341).

Co-evolving dynamics and topology in a coupled oscillator model of resting brain function

Dynamic models of ongoing BOLD fMRI brain dynamics and models of communication strategies have been two important approaches to understanding how brain network structure constrains function. However, dynamic models have yet to widely incorporate one of the most important insights from communication models: the brain may not use all of its connections in the same way or at the same time. Here we present a variation of a phase delayed Kuramoto coupled oscillator model that dynamically limits communication between nodes on each time step. An active subgraph of the empirically derived anatomical brain network is chosen in accordance with the local dynamic state on every time step, thus coupling dynamics and network structure in a novel way. We analyze this model with respect to its fit to empirical time-averaged functional connectivity, finding that, with the addition of only one parameter, it significantly outperforms standard Kuramoto models with phase delays. We also perform analyses on the novel time series of active edges it produces, demonstrating a slowly evolving topology moving through intermittent episodes of integration and segregation. We hope to demonstrate that the exploration of novel modeling mechanisms and the investigation of dynamics of networks in addition to dynamics on networks may advance our understanding of the relationship between brain structure and function.

From Classical Methods to Generative Models: Tackling the Unreliability of Neuroscientific Measures in Mental Health Research

Advances in computational statistics and corresponding shifts in funding initiatives over the past few decades have led to a proliferation of neuroscientific measures being developed in the context of mental health research. Although such measures have undoubtedly deepened our understanding of neural mechanisms underlying cognitive, affective, and behavioral processes associated with various mental health conditions, the clinical utility of such measures remains underwhelming. Recent commentaries point toward the poor reliability of neuroscientific measures to partially explain this lack of clinical translation. Here, we provide a concise theoretical overview of how unreliability impedes clinical translation of neuroscientific measures; discuss how various modeling principles, including those from hierarchical and structural equation modeling frameworks, can help to improve reliability; and demonstrate how to combine principles of hierarchical and structural modeling within the generative modeling framework to achieve more reliable, generalizable measures of brain-behavior relationships for use in mental health research.

DeepCubist: Molecular Generator for Designing Peptidomimetics based on Complex three-dimensional scaffolds

Mimicking bioactive conformations of peptide segments involved in the formation of protein-protein interfaces with small molecules is thought to represent a promising strategy for the design of protein-protein interaction (PPI) inhibitors. For compound design, the use of three-dimensional (3D) scaffolds rich in sp3-centers makes it possible to precisely mimic bioactive peptide conformations. Herein, we introduce DeepCubist, a molecular generator for designing peptidomimetics based on 3D scaffolds. Firstly, enumerated 3D scaffolds are superposed on a target peptide conformation to identify a preferred template structure for designing peptidomimetics. Secondly, heteroatoms and unsaturated bonds are introduced into the template via a deep generative model to produce candidate compounds. DeepCubist was applied to design peptidomimetics of exemplary peptide turn, helix, and loop structures in pharmaceutical targets engaging in PPIs.

Learning Extremal Representations with Deep Archetypal Analysis

Archetypes represent extreme manifestations of a population with respect to specific characteristic traits or features. In linear feature space, archetypes approximate the data convex hull allowing all data points to be expressed as convex mixtures of archetypes. As mixing of archetypes is performed directly on the input data, linear Archetypal Analysis requires additivity of the input, which is a strong assumption unlikely to hold e.g. in case of image data. To address this problem, we propose learning an appropriate latent feature space while simultaneously identifying suitable archetypes. We thus introduce a generative formulation of the linear archetype model, parameterized by neural networks. By introducing the distance-dependent archetype loss, the linear archetype model can be integrated into the latent space of a deep variational information bottleneck and an optimal representation, together with the archetypes, can be learned end-to-end. Moreover, the information bottleneck framework allows for a natural incorporation of arbitrarily complex side information during training. As a consequence, learned archetypes become easily interpretable as they derive their meaning directly from the included side information. Applicability of the proposed method is demonstrated by exploring archetypes of female facial expressions while using multi-rater based emotion scores of these expressions as side information. A second application illustrates the exploration of the chemical space of small organic molecules. By using different kinds of side information we demonstrate how identified archetypes, along with their interpretation, largely depend on the side information provided.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11263-020-01390-3.

Characterization of tissue types in basal cell carcinoma images via generative modeling and concept vectors

The promise of machine learning methods to act as decision support systems for pathologists continues to grow. However, central to their successful adoption must be interpretable implementations so that people can trust and learn from them effectively. Generative modeling, most notable in the form of adversarial generative models, is a naturally interpretable technique because the quality of the model is explicit from the quality of images it generates. Such a model can be further assessed by exploring its latent space, using human-meaningful concepts by defining concept vectors. Motivated by these ideas, we apply for the first time generative methods to histological images of basal cell carcinoma (BCC). By simultaneously learning to generate and encode realistic image patches, we extract feature rich latent vectors that correspond to various tissue morphologies, namely BCC, epidermis, keratin, papillary dermis and inflammation. We show that a logistic regression model trained on these latent vectors can achieve high classification accuracies across 6 binary tasks (86-98%). Further, by projecting the latent vectors onto learned concept vectors we can generate a score for the absence or degree of presence for a given concept, providing semantically accurate "conceptual summaries" of the various tissues types within a patch. This can be extended to generate multi-dimensional heat maps for whole-image specimens, which characterizes the tissue in a similar way to a pathologist. We additionally find that accurate concept vectors can be defined using a small labeled dataset.

Fine-tuning of a generative neural network for designing multi-target compounds

Exploring the origin of multi-target activity of small molecules and designing new multi-target compounds are highly topical issues in pharmaceutical research. We have investigated the ability of a generative neural network to create multi-target compounds. Data sets of experimentally confirmed multi-target, single-target, and consistently inactive compounds were extracted from public screening data considering positive and negative assay results. These data sets were used to fine-tune the REINVENT generative model via transfer learning to systematically recognize multi-target compounds, distinguish them from single-target or inactive compounds, and construct new multi-target compounds. During fine-tuning, the model showed a clear tendency to increasingly generate multi-target compounds and structural analogs. Our findings indicate that generative models can be adopted for de novo multi-target compound design.

A deep-learning-based unsupervised model on esophageal manometry using variational autoencoder

High-resolution manometry (HRM) is the primary method for diagnosing esophageal motility disorders and its interpretation and classification are based on variables (features) from data of each swallow. Modeling and learning the semantics directly from raw swallow data could not only help automate the feature extraction, but also alleviate the bias from pre-defined features. With more than 32-thousand raw swallow data, a generative model using the approach of variational auto-encoder (VAE) was developed, which, to our knowledge, is the first deep-learning-based unsupervised model on raw esophageal manometry data. The VAE model was reformulated to include different types of loss motivated by domain knowledge and tuned with different hyper-parameters. Training of the VAE model was found sensitive on the learning rate and hence the evidence lower bound objective (ELBO) was further scaled by the data dimension. Case studies showed that the dimensionality of latent space have a big impact on the learned semantics. In particular, cases with 4-dimensional latent variables were found to encode various physiologically meaningful contraction patterns, including strength, propagation pattern as well as sphincter relaxation. Cases with so-called hybrid L2 loss seemed to better capture the coherence of contraction/relaxation transition. Discriminating capability was further evaluated using simple linear discriminative analysis (LDA) on predicting swallow type and swallow pressurization, which yields clustering patterns consistent with clinical impression. The current work on modeling and understanding swallow-level data will guide the development of study-level models for automatic diagnosis as the next stage.

High-content image generation for drug discovery using generative adversarial networks

Immense amount of high-content image data generated in drug discovery screening requires computationally driven automated analysis. Emergence of advanced machine learning algorithms, like deep learning models, has transformed the interpretation and analysis of imaging data. However, deep learning methods generally require large number of high-quality data samples, which could be limited during preclinical investigations. To address this issue, we propose a generative modeling based computational framework to synthesize images, which can be used for phenotypic profiling of perturbations induced by drug compounds. We investigated the use of three variants of Generative Adversarial Network (GAN) in our framework, viz., a basic Vanilla GAN, Deep Convolutional GAN (DCGAN) and Progressive GAN (ProGAN), and found DCGAN to be most efficient in generating realistic synthetic images. A pre-trained convolutional neural network (CNN) was used to extract features of both real and synthetic images, followed by a classification model trained on real and synthetic images. The quality of synthesized images was evaluated by comparing their feature distributions with that of real images. The DCGAN-based framework was applied to high-content image data from a drug screen to synthesize high-quality cellular images, which were used to augment the real image data. The augmented dataset was shown to yield better classification performance compared with that obtained using only real images. We also demonstrated the application of proposed method on the generation of bacterial images and computed feature distributions for bacterial images specific to different drug treatments. In summary, our results showed that the proposed DCGAN-based framework can be utilized to generate realistic synthetic high-content images, thus enabling the study of drug-induced effects on cells and bacteria.

Analysis and classification of collective behavior using generative modeling and nonlinear manifold learning

In this paper, we build a framework for the analysis and classification of collective behavior using methods from generative modeling and nonlinear manifold learning. We represent an animal group with a set of finite-sized particles and vary known features of the group structure and motion via a class of generative models to position each particle on a two-dimensional plane. Particle positions are then mapped onto training images that are processed to emphasize the features of interest and match attainable far-field videos of real animal groups. The training images serve as templates of recognizable patterns of collective behavior and are compactly represented in a low-dimensional space called embedding manifold. Two mappings from the manifold are derived: the manifold-to-image mapping serves to reconstruct new and unseen images of the group and the manifold-to-feature mapping allows frame-by-frame classification of raw video. We validate the combined framework on datasets of growing level of complexity. Specifically, we classify artificial images from the generative model, interacting self-propelled particle model, and raw overhead videos of schooling fish obtained from the literature.