Stochastic Neural Network Approach for Learning High-Dimensional Free Energy Surfaces

Unsupervised machine learning in atomistic simulations, between predictions and understanding

Automated analyses of the outcome of a simulation have been an important part of atomistic modeling since the early days, addressing the need of linking the behavior of individual atoms and the collective properties that are usually the final quantity of interest. Methods such as clustering and dimensionality reduction have been used to provide a simplified, coarse-grained representation of the structure and dynamics of complex systems from proteins to nanoparticles. In recent years, the rise of machine learning has led to an even more widespread use of these algorithms in atomistic modeling and to consider different classification and inference techniques as part of a coherent toolbox of data-driven approaches. This perspective briefly reviews some of the unsupervised machine-learning methods-that are geared toward classification and coarse-graining of molecular simulations-seen in relation to the fundamental mathematical concepts that underlie all machine-learning techniques. It discusses the importance of using concise yet complete representations of atomic structures as the starting point of the analyses and highlights the risk of introducing preconceived biases when using machine learning to rationalize and understand structure-property relations. Supervised machine-learning techniques that explicitly attempt to predict the properties of a material given its structure are less susceptible to such biases. Current developments in the field suggest that using these two classes of approaches side-by-side and in a fully integrated mode, while keeping in mind the relations between the data analysis framework and the fundamental physical principles, will be key to realizing the full potential of machine learning to help understand the behavior of complex molecules and materials.

How machine learning can assist the interpretation of ab initio molecular dynamics simulations and conceptual understanding of chemistry

Molecular dynamics simulations are often key to the understanding of the mechanism, rate and yield of chemical reactions. One current challenge is the in-depth analysis of the large amount of data produced by the simulations, in order to produce valuable insight and general trends. In the present study, we propose to employ recent machine learning analysis tools to extract relevant information from simulation data without a priori knowledge on chemical reactions. This is demonstrated by training machine learning models to predict directly a specific outcome quantity of ab initio molecular dynamics simulations - the timescale of the decomposition of 1,2-dioxetane. The machine learning models accurately reproduce the dissociation time of the compound. Keeping the aim of gaining physical insight, it is demonstrated that, in order to make accurate predictions, the models evidence empirical rules that are, today, part of the common chemical knowledge. This opens the way for conceptual breakthroughs in chemistry where machine analysis would provide a source of inspiration to humans.

Nonequilibrium fluctuations of a driven quantum heat engine via machine learning

We propose a machine-learning approach based on artificial neural network to efficiently obtain new insights on the role of geometric contributions to the nonequilibrium fluctuations of an adiabatically temperature-driven quantum heat engine coupled to a cavity. Using the artificial neural network we have explored the interplay between bunched and antibunched photon exchange statistics for different engine parameters. We report that beyond a pivotal cavity temperature, the Fano factor oscillates between giant and low values as a function of phase difference between the driving protocols. We further observe that the standard thermodynamic uncertainty relation is not valid when there are finite geometric contributions to the fluctuations but holds true for zero phase difference even in the presence of coherences.

Assessment and optimization of collective variables for protein conformational landscape: GB1 β-hairpin as a case study

Collective variables (CVs), when chosen judiciously, can play an important role in recognizing rate-limiting processes and rare events in any biomolecular systems. However, high dimensionality and inherent complexities associated with such biochemical systems render the identification of an optimal CV a challenging task, which in turn precludes the elucidation of an underlying conformational landscape in sufficient details. In this context, a relevant model system is presented by a 16-residue β-hairpin of GB1 protein. Despite being the target of numerous theoretical and computational studies for understanding the protein folding, the set of CVs optimally characterizing the conformational landscape of the β-hairpin of GB1 protein has remained elusive, resulting in a lack of consensus on its folding mechanism. Here we address this by proposing a pair of optimal CVs which can resolve the underlying free energy landscape of the GB1 hairpin quite efficiently. Expressed as a linear combination of a number of traditional CVs, the optimal CV for this system is derived by employing the recently introduced time-structured independent component analysis approach on a large number of independent unbiased simulations. By projecting the replica-exchange simulated trajectories along these pair of optimized CVs, the resulting free energy landscape of this system is able to resolve four distinct well-separated metastable states encompassing the extensive ensembles of folded, unfolded, and molten globule states. Importantly, the optimized CVs were found to be capable of automatically recovering a novel partial helical state of this protein, without needing to explicitly invoke helicity as a constituent CV. Furthermore, a quantitative sensitivity analysis of each constituent in the optimized CV provided key insights on the relative contributions of the constituent CVs in the overall free energy landscapes. Finally, the kinetic pathways connecting these metastable states, constructed using a Markov state model, provide an optimum description of the underlying folding mechanism of the peptide. Taken together, this work offers a quantitatively robust approach toward comprehensive mapping of the underlying folding landscape of a quintessential model system along its optimized CV.

Machine-Learned Coarse-Grained Models

Optimizing force-field (FF) parameters to perform molecular dynamics (MD) simulations is a challenging and time-consuming process. We present a novel FF optimization framework that integrates MD simulations with particle swarm optimization (PSO) algorithm and artificial neural network (ANN). This new ANN-assisted PSO framework was used to develop transferable coarse-grained (CG) models for D₂O and DMF as a proof of concept. The PSO algorithm was used to generate the set of input FF parameters for the MD simulations of the CG models of these solvents, which were optimized to reproduce their experimental properties. Herein, for the first time, a reverse approach was employed for on-the-fly training of the ANN model, where results (solvent properties) obtained from the MD simulations and their corresponding FF parameters were used as inputs and outputs, respectively. The ANN model was then required to predict a set of new FF parameters, which were tested for their ability to predict the desired experimental properties. This new framework can be extended to integrate any optimization algorithm with ANN and MD simulations to accelerate the FF development.

DeePCG: Constructing coarse-grained models via deep neural networks

We introduce a general framework for constructing coarse-grained potential models without ad hoc approximations such as limiting the potential to two- and/or three-body contributions. The scheme, called the Deep Coarse-Grained Potential (abbreviated DeePCG), exploits a carefully crafted neural network to construct a many-body coarse-grained potential. The network is trained with full atomistic data in a way that preserves the natural symmetries of the system. The resulting model is very accurate and can be used to sample the configurations of the coarse-grained variables in a much faster way than with the original atomistic model. As an application, we consider liquid water and use the oxygen coordinates as the coarse-grained variables, starting from a full atomistic simulation of this system at the ab initio molecular dynamics level. We find that the two-body, three-body, and higher-order oxygen correlation functions produced by the coarse-grained and full atomistic models agree very well with each other, illustrating the effectiveness of the DeePCG model on a rather challenging task.

Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics

Inspired by the success of deep learning techniques in the physical and chemical sciences, we apply a modification of an autoencoder type deep neural network to the task of dimension reduction of molecular dynamics data. We can show that our time-lagged autoencoder reliably finds low-dimensional embeddings for high-dimensional feature spaces which capture the slow dynamics of the underlying stochastic processes-beyond the capabilities of linear dimension reduction techniques.

Adaptive enhanced sampling by force-biasing using neural networks

A machine learning assisted method is presented for molecular simulation of systems with rugged free energy landscapes. The method is general and can be combined with other advanced sampling techniques. In the particular implementation proposed here, it is illustrated in the context of an adaptive biasing force approach where, rather than relying on discrete force estimates, one can resort to a self-regularizing artificial neural network to generate continuous, estimated generalized forces. By doing so, the proposed approach addresses several shortcomings common to adaptive biasing force and other algorithms. Specifically, the neural network enables (1) smooth estimates of generalized forces in sparsely sampled regions, (2) force estimates in previously unexplored regions, and (3) continuous force estimates with which to bias the simulation, as opposed to biases generated at specific points of a discrete grid. The usefulness of the method is illustrated with three different examples, chosen to highlight the wide range of applicability of the underlying concepts. In all three cases, the new method is found to enhance considerably the underlying traditional adaptive biasing force approach. The method is also found to provide improvements over previous implementations of neural network assisted algorithms.

Simulations meet machine learning in structural biology

Classical molecular dynamics (MD) simulations will be able to reach sampling in the second timescale within five years, producing petabytes of simulation data at current force field accuracy. Notwithstanding this, MD will still be in the regime of low-throughput, high-latency predictions with average accuracy. We envisage that machine learning (ML) will be able to solve both the accuracy and time-to-prediction problem by learning predictive models using expensive simulation data. The synergies between classical, quantum simulations and ML methods, such as artificial neural networks, have the potential to drastically reshape the way we make predictions in computational structural biology and drug discovery.