Unifying machine learning and quantum chemistry with a deep neural network for molecular wavefunctions

In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back

Computational chemistry is an indispensable tool for understanding molecules and predicting chemical properties. However, traditional computational methods face significant challenges due to the difficulty of solving the Schrödinger equations and the increasing computational cost with the size of the molecular system. In response, there has been a surge of interest in leveraging artificial intelligence (AI) and machine learning (ML) techniques to in silico experiments. Integrating AI and ML into computational chemistry increases the scalability and speed of the exploration of chemical space. However, challenges remain, particularly regarding the reproducibility and transferability of ML models. This review highlights the evolution of ML in learning from, complementing, or replacing traditional computational chemistry for energy and property predictions. Starting from models trained entirely on numerical data, a journey set forth toward the ideal model incorporating or learning the physical laws of quantum mechanics. This paper also reviews existing computational methods and ML models and their intertwining, outlines a roadmap for future research, and identifies areas for improvement and innovation. Ultimately, the goal is to develop AI architectures capable of predicting accurate and transferable solutions to the Schrödinger equation, thereby revolutionizing in silico experiments within chemistry and materials science.

Development of scoring-assisted generative exploration (SAGE) and its application to dual inhibitor design for acetylcholinesterase and monoamine oxidase B

De novo molecular design is the process of searching chemical space for drug-like molecules with desired properties, and deep learning has been recognized as a promising solution. In this study, I developed an effective computational method called Scoring-Assisted Generative Exploration (SAGE) to enhance chemical diversity and property optimization through virtual synthesis simulation, the generation of bridged bicyclic rings, and multiple scoring models for drug-likeness. In six protein targets, SAGE generated molecules with high scores within reasonable numbers of steps by optimizing target specificity without a constraint and even with multiple constraints such as synthetic accessibility, solubility, and metabolic stability. Furthermore, I suggested a top-ranked molecule with SAGE as dual inhibitors of acetylcholinesterase and monoamine oxidase B through multiple desired property optimization. Therefore, SAGE can generate molecules with desired properties by optimizing multiple properties simultaneously, indicating the importance of de novo design strategies in the future of drug discovery and development. SCIENTIFIC CONTRIBUTION: The scientific contribution of this study lies in the development of the Scoring-Assisted Generative Exploration (SAGE) method, a novel computational approach that significantly enhances de novo molecular design. SAGE uniquely integrates virtual synthesis simulation, the generation of complex bridged bicyclic rings, and multiple scoring models to optimize drug-like properties comprehensively. By efficiently generating molecules that meet a broad spectrum of pharmacological criteria-including target specificity, synthetic accessibility, solubility, and metabolic stability-within a reasonable number of steps, SAGE represents a substantial advancement over traditional methods. Additionally, the application of SAGE to discover dual inhibitors for acetylcholinesterase and monoamine oxidase B not only demonstrates its potential to streamline and enhance the drug development process but also highlights its capacity to create more effective and precisely targeted therapies. This study emphasizes the critical and evolving role of de novo design strategies in reshaping the future of drug discovery and development, providing promising avenues for innovative therapeutic discoveries.

Spiers Memorial Lecture: New directions in molecular scattering

The field of molecular scattering is reviewed as it pertains to gas-gas as well as gas-surface chemical reaction dynamics. We emphasize the importance of collaboration of experiment and theory, from which new directions of research are being pursued on increasingly complex problems. We review both experimental and theoretical advances that provide the modern toolbox available to molecular-scattering studies. We distinguish between two classes of work. The first involves simple systems and uses experiment to validate theory so that from the validated theory, one may learn far more than could ever be measured in the laboratory. The second class involves problems of great complexity that would be difficult or impossible to understand without a partnership of experiment and theory. Key topics covered in this review include crossed-beams reactive scattering and scattering at extremely low energies, where quantum effects dominate. They also include scattering from surfaces, reactive scattering and kinetics at surfaces, and scattering work done at liquid surfaces. The review closes with thoughts on future promising directions of research.

Machine-Learned Kohn-Sham Hamiltonian Mapping for Nonadiabatic Molecular Dynamics

In this work, we report a simple, efficient, and scalable machine-learning (ML) approach for mapping non-self-consistent Kohn-Sham Hamiltonians constructed with one kind of density functional to the nearly self-consistent Hamiltonians constructed with another kind of density functional. This approach is designed as a fast surrogate Hamiltonian calculator for use in long nonadiabatic dynamics simulations of large atomistic systems. In this approach, the input and output features are Hamiltonian matrices computed from different levels of theory. We demonstrate that the developed ML-based Hamiltonian mapping method (1) speeds up the calculations by several orders of magnitude, (2) is conceptually simpler than alternative ML approaches, (3) is applicable to different systems and sizes and can be used for mapping Hamiltonians constructed with arbitrary density functionals, (4) requires a modest training data, learns fast, and generates molecular orbitals and their energies with the accuracy nearly matching that of conventional calculations, and (5) when applied to nonadiabatic dynamics simulation of excitation energy relaxation in large systems yields the corresponding time scales within the margin of error of the conventional calculations. Using this approach, we explore the excitation energy relaxation in C60 fullerene and Si75H64 quantum dot structures and derive qualitative and quantitative insights into dynamics in these systems.

An Unsupervised Machine Learning Approach for the Automatic Construction of Local Chemical Descriptors

Condensing the many physical variables defining a chemical system into a fixed-size array poses a significant challenge in the development of chemical Machine Learning (ML). Atom Centered Symmetry Functions (ACSFs) offer an intuitive featurization approach by means of a tedious and labor-intensive selection of tunable parameters. In this work, we implement an unsupervised ML strategy relying on a Gaussian Mixture Model (GMM) to automatically optimize the ACSF parameters. GMMs effortlessly decompose the vastness of the chemical and conformational spaces into well-defined radial and angular clusters, which are then used to build tailor-made ACSFs. The unsupervised exploration of the space has demonstrated general applicability across a diverse range of systems, spanning from various unimolecular landscapes to heterogeneous databases. The impact of the sampling technique and temperature on space exploration is also addressed, highlighting the particularly advantageous role of high-temperature Molecular Dynamics (MD) simulations. The reliability of the resulting features is assessed through the estimation of the atomic charges of a prototypical capped amino acid and a heterogeneous collection of CHON molecules. The automatically constructed ACSFs serve as high-quality descriptors, consistently yielding typical prediction errors below 0.010 electrons bound for the reported atomic charges. Altering the spatial distribution of the functions with respect to the cluster highlights the critical role of symmetry rupture in achieving significantly improved features. More specifically, using two separate functions to describe the lower and upper tails of the cluster results in the best performing models with errors as low as 0.006 electrons. Finally, the effectiveness of finely tuned features was checked across different architectures, unveiling the superior performance of Gaussian Process (GP) models over Feed Forward Neural Networks (FFNNs), particularly in low-data regimes, with nearly a 2-fold increase in prediction quality. Altogether, this approach paves the way toward an easier construction of local chemical descriptors, while providing valuable insights into how radial and angular spaces should be mapped. Finally, this work opens the possibility of encoding many-body information beyond angular terms into upcoming ML features.

Biomolecular dynamics with machine-learned quantum-mechanical force fields trained on diverse chemical fragments

The GEMS method enables molecular dynamics simulations of large heterogeneous systems at ab initio quality.

Electronic Excited States from Physically Constrained Machine Learning

Data-driven techniques are increasingly used to replace electronic-structure calculations of matter. In this context, a relevant question is whether machine learning (ML) should be applied directly to predict the desired properties or combined explicitly with physically grounded operations. We present an example of an integrated modeling approach in which a symmetry-adapted ML model of an effective Hamiltonian is trained to reproduce electronic excitations from a quantum-mechanical calculation. The resulting model can make predictions for molecules that are much larger and more complex than those on which it is trained and allows for dramatic computational savings by indirectly targeting the outputs of well-converged calculations while using a parametrization corresponding to a minimal atom-centered basis. These results emphasize the merits of intertwining data-driven techniques with physical approximations, improving the transferability and interpretability of ML models without affecting their accuracy and computational efficiency and providing a blueprint for developing ML-augmented electronic-structure methods.

AI in computational chemistry through the lens of a decade-long journey

This article gives a perspective on the progress of AI tools in computational chemistry through the lens of the author's decade-long contributions put in the wider context of the trends in this rapidly expanding field. This progress over the last decade is tremendous: while a decade ago we had a glimpse of what was to come through many proof-of-concept studies, now we witness the emergence of many AI-based computational chemistry tools that are mature enough to make faster and more accurate simulations increasingly routine. Such simulations in turn allow us to validate and even revise experimental results, deepen our understanding of the physicochemical processes in nature, and design better materials, devices, and drugs. The rapid introduction of powerful AI tools gives rise to unique challenges and opportunities that are discussed in this article too.

Depicted simulation model for removal of second-generation antipsychotic drugs adsorbed on Zn-MOF: adsorption locator assessment

CONTEXT

Electronic durable behavior on the material surface was accompanied by a class of antipsychotic drugs (APD) to describe the surface modification in the designed adsorption model. Hierarchically Zn-MOF system was utilized for estimating its capacity for drug molecule removal. Geometrically optimized strategy on the studied systems was performed using DFT/GGA/PBE. FMOs analysis was depicted based on the same level of calculations, and molecular electrostatic potential surface (MEP) was generated for unadsorbed and adsorbed systems to illustrate the variation in the surface-active sites. By interpreting the electronic density of states (DOS), the atomic orbital can be identified as a major or minor electronic distribution by PDOS graph. Adsorption locating behavior was considered to detect the significant surface interaction mode between APD and Zn-MOF surface based on lower adsorption energy. The stability of the adsorbed model was best described through dynamic simulation analysis with time through elevated temperatures. The non-covalent interactions were described using RDG/NCI analysis to show the major favorable surface interaction predicting the highly stable adsorption system.

METHODS

The most accurate geometrical computations were performed using the materials studio software followed by surface cleavage and vacuum slab generation. The first principle of DFT was used to apply CASTEP module with GGA/PBE method for band structure and DOS calculations. Three systems of antipsychotic drugs were computationally studied using CASTEP simulation package and adsorbed on an optimized Zn-MOF surface. Adsorption locator module predicted the preferred adsorption mechanistic models, in which the first model was arranged to be more stable, to confirm the occurrence of some interactions in the adsorption mechanism.

Exploration of the Two-Electron Excitation Space with Data-Driven Coupled Cluster

Computational cost limits the applicability of post-Hartree-Fock methods such as coupled-cluster on larger molecular systems. The data-driven coupled-cluster (DDCC) method applies machine learning to predict the coupled-cluster two-electron amplitudes (t2) using data from second-order perturbation theory (MP2). One major limitation of the DDCC models is the size of training sets that increases exponentially with the system size. Effective sampling of the amplitude space can resolve this issue. Five different amplitude selection techniques that reduce the amount of data used for training were evaluated, an approach that also prevents model overfitting and increases the portability of data-driven coupled-cluster singles and doubles to more complex molecules or larger basis sets. In combination with a localized orbital formalism to predict the CCSD t2 amplitudes, we have achieved a 10-fold error reduction for energy calculations.

SPAHM(a,b): Encoding the Density Information from Guess Hamiltonian in Quantum Machine Learning Representations

Recently, we introduced a class of molecular representations for kernel-based regression methods─the spectrum of approximated Hamiltonian matrices (SPAHM)─that takes advantage of lightweight one-electron Hamiltonians traditionally used as a self-consistent field initial guess. The original SPAHM variant is built from occupied-orbital energies (i.e., eigenvalues) and naturally contains all of the information about nuclear charges, atomic positions, and symmetry requirements. Its advantages were demonstrated on data sets featuring a wide variation of charge and spin, for which traditional structure-based representations commonly fail. SPAHM(a,b), as introduced here, expand the eigenvalue SPAHM into local and transferable representations. They rely upon one-electron density matrices to build fingerprints from atomic and bond density overlap contributions inspired from preceding state-of-the-art representations. The performance and efficiency of SPAHM(a,b) is assessed on the predictions for data sets of prototypical organic molecules (QM7) of different charges and azoheteroarene dyes in an excited state. Overall, both SPAHM(a) and SPAHM(b) outperform state-of-the-art representations on difficult prediction tasks such as the atomic properties of charged open-shell species and of π-conjugated systems.

Employing neural density functionals to generate potential energy surfaces

CONTEXT

With the union of machine learning (ML) and quantum chemistry, amid the debate between machine-learned functionals and human-designed functionals in density functional theory (DFT), this paper aims to demonstrate the generation of potential energy surfaces using computations with machine-learned density functional approximation (ML-DFA). A recent research trend is the application of ML in quantum sciences in the design of density functionals such as DeepMind's Deep Learning model (DeepMind21, DM21). Though science reported the state-of-the-art performance of DM21, the opportunity to utilize DeepMind's pretrained DM21 neural networks in computations in quantum chemistry has not yet been tapped. So far in the literature, the Deep Learning density functionals (DM21) have not been applied to generate potential energy surfaces. While the superior accuracy of DM21 has been reported, there is still a scarcity of publications that apply DM21 in calculations in the field. In this context, for the first time in literature, neural density functionals inferring 2D potential energy surfaces (ML-DFA-PES) based on machine-learned DFA-based computational method is contributed in this paper. This paper reports the ML-DFA-generated PES for C4H8, H2O, H2, and H2+ by employing a pretrained DM21m TensorFlow model with cc-pVDZ basis set. In addition, we also analyze the long-range behavior of DM21 based PES to investigate the ability to describe a system at long ranges. Furthermore, we compare PES diagrams from DM21 with popular DFT functionals (b3lyp/ PW6B95) and CCSD(T).

METHODS

In this method, 2D potential energy surfaces are obtained using a method that relies upon the neural network's ability to accurately learn the mapping between 3D electron density and exchange-correlation potential. By inserting Deep Learning inference in DFT with a pretrained neural network, self-consistent field (SCF) energy at different geometries along the coordinates of interest is computed, and then, potential energy surfaces are plotted. In this method, first, the electron density is computed mathematically, and this computed 3D electron density is used as a ML feature vector to predict the exchange correlation potential as a ML inference computed by a forward pass of pre-trained DM21 TensorFlow computational graph, followed by the computation of self-consistent field energy at multiple geometries, and then, SCF energies at different bond lengths/angles are plotted as 2D PES. We implement this in a python source code using frameworks such as PySCF and DM21. This paper contributes this implementation in open source. The source code and DM21-DFA-based PES are contributed at https://sites.google.com/view/MLfunctionals-DeepMind-PES .

Improved machine learning algorithm for predicting ground state properties

Finding the ground state of a quantum many-body system is a fundamental problem in quantum physics. In this work, we give a classical machine learning (ML) algorithm for predicting ground state properties with an inductive bias encoding geometric locality. The proposed ML model can efficiently predict ground state properties of an n-qubit gapped local Hamiltonian after learning from only [Formula: see text] data about other Hamiltonians in the same quantum phase of matter. This improves substantially upon previous results that require [Formula: see text] data for a large constant c. Furthermore, the training and prediction time of the proposed ML model scale as [Formula: see text] in the number of qubits n. Numerical experiments on physical systems with up to 45 qubits confirm the favorable scaling in predicting ground state properties using a small training dataset.

All-Purpose Measure of Electron Correlation for Multireference Diagnostics

We present an analytical relationship between two natural orbital occupancy-based indices, I N D ¯ and INDmax, and two established electron correlation metrics: the leading term of a configuration interaction expansion, c0, and the D2 diagnostic. Numerical validation revealed that I N D ¯ and INDmax can effectively substitute for c0 and D2, respectively. These indices offer three distinct advantages: (i) they are universally applicable across all electronic structure methods, (ii) their interpretation is more intuitive, and (iii) they can be readily incorporated into the development of hybrid electronic structure methods. Additionally, we draw a distinction between correlation measures and correlation diagnostics, establishing MP2 and CCSD numerical thresholds for INDmax, which are to be used as a multireference diagnostic. Our findings further demonstrate that establishing thresholds for other electronic structure methods can be easily accomplished using small data sets.

Reinforcement Learning for Traversing Chemical Structure Space: Optimizing Transition States and Minimum Energy Paths of Molecules

In recent years, deep learning has made remarkable strides, surpassing human capabilities in tasks, such as strategy games, and it has found applications in complex domains, including protein folding. In the realm of quantum chemistry, machine learning methods have primarily served as predictive tools or design aids using generative models, while reinforcement learning remains in its early stages of exploration. This work introduces an actor-critic reinforcement learning framework suitable for diverse optimization tasks, such as searching for molecular structures with specific properties within conformational spaces. As an example, we show an implementation of this scheme for calculating minimum energy pathways of a Claisen rearrangement reaction and a number of SN2 reactions. The results show that the algorithm is able to accurately predict minimum energy pathways and, thus, transition states, providing the first steps in using actor-critic methods to study chemical reactions.

Two-Step Machine Learning Approach for Charge-Transfer Coupling with Structurally Diverse Data

Electronic coupling is important in determining charge-transfer rates and dynamics. Coupling strength is sensitive to both intermolecular, e.g., orientation or distance, and intramolecular degrees of freedom. Hence, it is challenging to build an accurate machine learning model to predict electronic coupling of molecular pairs, especially for those derived from the amorphous phase, for which intermolecular configurations are much more diverse than those derived from crystals. In this work, we devise a new prediction algorithm that employs two consecutive KRR models. The first model predicts molecular orbitals (MOs) from structural variation for each fragment, and coupling is further predicted by using the overlap integral included in a second model. With our two-step procedure, we achieved mean absolute errors of 0.27 meV for an ethylene dimer and 1.99 meV for a naphthalene pair, much improved accuracy amounting to 14-fold and 3-fold error reductions, respectively. In addition, MOs from the first model can also be the starting point to obtain other quantum chemical properties from atomistic structures. This approach is also compatible with a MO predictor with sufficient accuracy.

Machine Learning Many-Body Green's Functions for Molecular Excitation Spectra

We present a machine learning (ML) framework for predicting Green's functions of molecular systems, from which photoemission spectra and quasiparticle energies at quantum many-body level can be obtained. Kernel ridge regression is adopted to predict self-energy matrix elements on compact imaginary frequency grids from static and dynamical mean-field electronic features, which gives direct access to real-frequency many-body Green's functions through analytic continuation and Dyson's equation. Feature and self-energy matrices are represented in a symmetry-adapted intrinsic atomic orbital plus projected atomic orbital basis to enforce rotational invariance. We demonstrate good transferability and high data efficiency of the proposed ML method across molecular sizes and chemical species by showing accurate predictions of density of states (DOS) and quasiparticle energies at the level of many-body perturbation theory (GW) or full configuration interaction. For the ML model trained on 48 out of 1995 molecules randomly sampled from the QM7 and QM9 data sets, we report the mean absolute errors of ML-predicted highest occupied and lowest unoccupied molecular orbital energies to be 0.13 and 0.10 eV, respectively, compared to GW@PBE0. We further showcase the capability of this method by applying the same ML model to predict DOS for significantly larger organic molecules with up to 44 heavy atoms.

Towards a transferable fermionic neural wavefunction for molecules

Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schrödinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.

Leveraging Deep Neural Networks for Estimating Vickers Hardness from Nanoindentation Hardness

This research presents a comprehensive analysis of deep neural network models (DNNs) for the precise prediction of Vickers hardness (HV) in nitrided and carburized M50NiL steel samples, with hardness values spanning from 400 to 1000 HV. By conducting rigorous experimentation and obtaining corresponding nanoindentation data, we evaluated the performance of four distinct neural network architectures: Multilayer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory network (LSTM), and Transformer. Our findings reveal that MLP and LSTM models excel in predictive accuracy and efficiency, with MLP showing exceptional iteration efficiency and predictive precision. The study validates models for broad application in various steel types and confirms nanoindentation as an effective direct measure for HV hardness in thin films and gradient-variable regions. This work contributes a validated and versatile approach to the hardness assessment of thin-film materials and those with intricate microstructures, enhancing material characterization and potential application in advanced material engineering.

From intuition to AI: evolution of small molecule representations in drug discovery

Within drug discovery, the goal of AI scientists and cheminformaticians is to help identify molecular starting points that will develop into safe and efficacious drugs while reducing costs, time and failure rates. To achieve this goal, it is crucial to represent molecules in a digital format that makes them machine-readable and facilitates the accurate prediction of properties that drive decision-making. Over the years, molecular representations have evolved from intuitive and human-readable formats to bespoke numerical descriptors and fingerprints, and now to learned representations that capture patterns and salient features across vast chemical spaces. Among these, sequence-based and graph-based representations of small molecules have become highly popular. However, each approach has strengths and weaknesses across dimensions such as generality, computational cost, inversibility for generative applications and interpretability, which can be critical in informing practitioners' decisions. As the drug discovery landscape evolves, opportunities for innovation continue to emerge. These include the creation of molecular representations for high-value, low-data regimes, the distillation of broader biological and chemical knowledge into novel learned representations and the modeling of up-and-coming therapeutic modalities.

Physics-Inspired Equivariant Descriptors of Nonbonded Interactions

One essential ingredient in many machine learning (ML) based methods for atomistic modeling of materials and molecules is the use of locality. While allowing better system-size scaling, this systematically neglects long-range (LR) effects such as electrostatic or dispersion interactions. We present an extension of the long distance equivariant (LODE) framework that can handle diverse LR interactions in a consistent way and seamlessly integrates with preexisting methods by building new sets of atom centered features. We provide a direct physical interpretation of these using the multipole expansion, which allows for simpler and more efficient implementations. The framework is applied to simple toy systems as proof of concept and a heterogeneous set of molecular dimers to push the method to its limits. By generalizing LODE to arbitrary asymptotic behaviors, we provide a coherent approach to treat arbitrary two- and many-body nonbonded interactions in the data-driven modeling of matter.

Machine learning electronic structure methods based on the one-electron reduced density matrix

The theorems of density functional theory (DFT) establish bijective maps between the local external potential of a many-body system and its electron density, wavefunction and, therefore, one-particle reduced density matrix. Building on this foundation, we show that machine learning models based on the one-electron reduced density matrix can be used to generate surrogate electronic structure methods. We generate surrogates of local and hybrid DFT, Hartree-Fock and full configuration interaction theories for systems ranging from small molecules such as water to more complex compounds like benzene and propanol. The surrogate models use the one-electron reduced density matrix as the central quantity to be learned. From the predicted density matrices, we show that either standard quantum chemistry or a second machine-learning model can be used to compute molecular observables, energies, and atomic forces. The surrogate models can generate essentially anything that a standard electronic structure method can, ranging from band gaps and Kohn-Sham orbitals to energy-conserving ab-initio molecular dynamics simulations and infrared spectra, which account for anharmonicity and thermal effects, without the need to employ computationally expensive algorithms such as self-consistent field theory. The algorithms are packaged in an efficient and easy to use Python code, QMLearn, accessible on popular platforms.

Automatic identification of chemical moieties

In recent years, the prediction of quantum mechanical observables with machine learning methods has become increasingly popular. Message-passing neural networks (MPNNs) solve this task by constructing atomic representations, from which the properties of interest are predicted. Here, we introduce a method to automatically identify chemical moieties (molecular building blocks) from such representations, enabling a variety of applications beyond property prediction, which otherwise rely on expert knowledge. The required representation can either be provided by a pretrained MPNN, or be learned from scratch using only structural information. Beyond the data-driven design of molecular fingerprints, the versatility of our approach is demonstrated by enabling the selection of representative entries in chemical databases, the automatic construction of coarse-grained force fields, as well as the identification of reaction coordinates.

Ab initio quantum chemistry with neural-network wavefunctions

Deep learning methods outperform human capabilities in pattern recognition and data processing problems and now have an increasingly important role in scientific discovery. A key application of machine learning in molecular science is to learn potential energy surfaces or force fields from ab initio solutions of the electronic Schrödinger equation using data sets obtained with density functional theory, coupled cluster or other quantum chemistry (QC) methods. In this Review, we discuss a complementary approach using machine learning to aid the direct solution of QC problems from first principles. Specifically, we focus on quantum Monte Carlo methods that use neural-network ansatzes to solve the electronic Schrödinger equation, in first and second quantization, computing ground and excited states and generalizing over multiple nuclear configurations. Although still at their infancy, these methods can already generate virtually exact solutions of the electronic Schrödinger equation for small systems and rival advanced conventional QC methods for systems with up to a few dozen electrons.

Langevin Cooling for Unsupervised Domain Translation

Domain translation is the task of finding correspondence between two domains. Several deep neural network (DNN) models, e.g., CycleGAN and cross-lingual language models, have shown remarkable successes on this task under the unsupervised setting-the mappings between the domains are learned from two independent sets of training data in both domains (without paired samples). However, those methods typically do not perform well on a significant proportion of test samples. In this article, we hypothesize that many of such unsuccessful samples lie at the fringe-relatively low-density areas-of data distribution, where the DNN was not trained very well, and propose to perform the Langevin dynamics to bring such fringe samples toward high-density areas. We demonstrate qualitatively and quantitatively that our strategy, called Langevin cooling (L-Cool), enhances state-of-the-art methods in image translation and language translation tasks.

Treating Semiempirical Hamiltonians as Flexible Machine Learning Models Yields Accurate and Interpretable Results

Quantum chemistry provides chemists with invaluable information, but the high computational cost limits the size and type of systems that can be studied. Machine learning (ML) has emerged as a means to dramatically lower the cost while maintaining high accuracy. However, ML models often sacrifice interpretability by using components such as the artificial neural networks of deep learning that function as black boxes. These components impart the flexibility needed to learn from large volumes of data but make it difficult to gain insight into the physical or chemical basis for the predictions. Here, we demonstrate that semiempirical quantum chemical (SEQC) models can learn from large volumes of data without sacrificing interpretability. The SEQC model is that of density-functional-based tight binding (DFTB) with fixed atomic orbital energies and interactions that are one-dimensional functions of the interatomic distance. This model is trained to ab initio data in a manner that is analogous to that used to train deep learning models. Using benchmarks that reflect the accuracy of the training data, we show that the resulting model maintains a physically reasonable functional form while achieving an accuracy, relative to coupled cluster energies with a complete basis set extrapolation (CCSD(T)*/CBS), that is comparable to that of density functional theory (DFT). This suggests that trained SEQC models can achieve a low computational cost and high accuracy without sacrificing interpretability. Use of a physically motivated model form also substantially reduces the amount of ab initio data needed to train the model compared to that required for deep learning models.

Synergy of semiempirical models and machine learning in computational chemistry

Catalyzed by enormous success in the industrial sector, many research programs have been exploring data-driven, machine learning approaches. Performance can be poor when the model is extrapolated to new regions of chemical space, e.g., new bonding types, new many-body interactions. Another important limitation is the spatial locality assumption in model architecture, and this limitation cannot be overcome with larger or more diverse datasets. The outlined challenges are primarily associated with the lack of electronic structure information in surrogate models such as interatomic potentials. Given the fast development of machine learning and computational chemistry methods, we expect some limitations of surrogate models to be addressed in the near future; nevertheless spatial locality assumption will likely remain a limiting factor for their transferability. Here, we suggest focusing on an equally important effort-design of physics-informed models that leverage the domain knowledge and employ machine learning only as a corrective tool. In the context of material science, we will focus on semi-empirical quantum mechanics, using machine learning to predict corrections to the reduced-order Hamiltonian model parameters. The resulting models are broadly applicable, retain the speed of semiempirical chemistry, and frequently achieve accuracy on par with much more expensive ab initio calculations. These early results indicate that future work, in which machine learning and quantum chemistry methods are developed jointly, may provide the best of all worlds for chemistry applications that demand both high accuracy and high numerical efficiency.

Thermodynamic and kinetic modeling of electrocatalytic reactions using a first-principles approach

The computational modeling of electrochemical interfaces and their applications in electrocatalysis has attracted great attention in recent years. While tremendous progress has been made in this area, however, the accurate atomistic descriptions at the electrode/electrolyte interfaces remain a great challenge. The Computational Hydrogen Electrode (CHE) method and continuum modeling of the solvent and electrolyte interactions form the basis for most of these methodological developments. Several posterior corrections have been added to the CHE method to improve its accuracy and widen its applications. The most recently developed grand canonical potential approaches with the embedded diffuse layer models have shown considerable improvement in defining interfacial interactions at electrode/electrolyte interfaces over the state-of-the-art computational models for electrocatalysis. In this Review, we present an overview of these different computational models developed over the years to quantitatively probe the thermodynamics and kinetics of electrochemical reactions in the presence of an electrified catalyst surface under various electrochemical environments. We begin our discussion by giving a brief picture of the different continuum solvation approaches, implemented within the ab initio method to effectively model the solvent and electrolyte interactions. Next, we present the thermodynamic and kinetic modeling approaches to determine the activity and stability of the electrocatalysts. A few applications to these approaches are also discussed. We conclude by giving an outlook on the different machine learning models that have been integrated with the thermodynamic approaches to improve their efficiency and widen their applicability.

Combining Deep Learning Neural Networks with Genetic Algorithms to Map Nanocluster Configuration Spaces with Quantum Accuracy at Low Computational Cost

The configuration spaces for bimetallic AuPd nanoclusters of various sizes are explored efficiently and analyzed accurately by combining genetic algorithms with neural networks trained on density functional theory. The methodology demonstrated herein provides an optimizable solution to the problem of searching vast configuration spaces with quantum accuracy in a way that is computationally practical. We implement a machine learning algorithm which learns the density functional theory potential with increasing performance while simultaneously generating and relaxing structures within the system's global configuration space unbiasedly. As a result, the algorithm naturally converges onto the system's energy minima while mapping the configuration space as a function of energy. The algorithm's simple design applies not only to nanocluster configurations, as demonstrated, but to bulk, substrate, and adsorption sites as well, and it is designed to scale. To demonstrate its computational efficiency, we work with AuPd nanoclusters of sizes 15, 20, and 25 atoms. Results focus primarily on evaluating the algorithm's performance; however, several physical insights into possible configurations for these nanoclusters naturally emerge as well, such as geometric Au surface segregation and stoichiometric Au minimization as a function of stability.

Material symmetry recognition and property prediction accomplished by crystal capsule representation

Learning the global crystal symmetry and interpreting the equivariant information is crucial for accurately predicting material properties, yet remains to be fully accomplished by existing algorithms based on convolution networks. To overcome this challenge, here we develop a machine learning (ML) model, named symmetry-enhanced equivariance network (SEN), to build material representation with joint structure-chemical patterns, to encode important clusters embedded in the crystal structure, and to learn pattern equivariance in different scales via capsule transformers. Quantitative analyses of the intermediate matrices demonstrate that the intrinsic crystal symmetries and interactions between clusters have been exactly perceived by the SEN model and critically affect the prediction performances by reducing effective feature space. The mean absolute errors (MAEs) of 0.181 eV and 0.0161 eV/atom are obtained for predicting bandgap and formation energy in the MatBench dataset. The general and interpretable SEN model reveals the potential to design ML models by implicitly encoding feature relationship based on physical mechanisms.

Low-Data Deep Quantum Chemical Learning for Accurate MP2 and Coupled-Cluster Correlations

Accurate ab initio prediction of electronic energies is very expensive for macromolecules by explicitly solving post-Hartree-Fock equations. We here exploit the physically justified local correlation feature in a compact basis of small molecules and construct an expressive low-data deep neural network (dNN) model to obtain machine-learned electron correlation energies on par with MP2 and CCSD levels of theory for more complex molecules and different datasets that are not represented in the training set. We show that our dNN-powered model is data efficient and makes highly transferable predictions across alkanes of various lengths, organic molecules with non-covalent and biomolecular interactions, as well as water clusters of different sizes and morphologies. In particular, by training 800 (H2O)8 clusters with the local correlation descriptors, accurate MP2/cc-pVTZ correlation energies up to (H2O)128 can be predicted with a small random error within chemical accuracy from exact values, while a majority of prediction deviations are attributed to an intrinsically systematic error. Our results reveal that an extremely compact local correlation feature set, which is poor for any direct post-Hartree-Fock calculations, has however a prominent advantage in reserving important electron correlation patterns for making accurate transferable predictions across distinct molecular compositions, bond types, and geometries.

Machine Learning Frontier Orbital Energies of Nanodiamonds

Nanodiamonds have a wide range of applications including catalysis, sensing, tribology, and biomedicine. To leverage nanodiamond design via machine learning, we introduce the new data set ND5k, consisting of 5089 diamondoid and nanodiamond structures and their frontier orbital energies. ND5k structures are optimized via tight-binding density functional theory (DFTB) and their frontier orbital energies are computed using density functional theory (DFT) with the PBE0 hybrid functional. From this data set we derive a qualitative design suggestion for nanodiamonds in photocatalysis. We also compare recent machine learning models for predicting frontier orbital energies for similar structures as they have been trained on (interpolation on ND5k), and we test their abilities to extrapolate predictions to larger structures. For both the interpolation and extrapolation task, we find the best performance using the equivariant message passing neural network PaiNN. The second best results are achieved with a message passing neural network using a tailored set of atomic descriptors proposed here.

The central role of density functional theory in the AI age

Density functional theory (DFT) plays a pivotal role in chemical and materials science because of its relatively high predictive power, applicability, versatility, and computational efficiency. We review recent progress in machine learning (ML) model developments, which have relied heavily on DFT for synthetic data generation and for the design of model architectures. The general relevance of these developments is placed in a broader context for chemical and materials sciences. DFT-based ML models have reached high efficiency, accuracy, scalability, and transferability and pave the way to the routine use of successful experimental planning software within self-driving laboratories.

General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian

The combination of deep learning and ab initio calculation has shown great promise in revolutionizing future scientific research, but how to design neural network models incorporating a priori knowledge and symmetry requirements is a key challenging subject. Here we propose an E(3)-equivariant deep-learning framework to represent density functional theory (DFT) Hamiltonian as a function of material structure, which can naturally preserve the Euclidean symmetry even in the presence of spin-orbit coupling. Our DeepH-E3 method enables efficient electronic structure calculation at ab initio accuracy by learning from DFT data of small-sized structures, making the routine study of large-scale supercells (>10⁴ atoms) feasible. The method can reach sub-meV prediction accuracy at high training efficiency, showing state-of-the-art performance in our experiments. The work is not only of general significance to deep-learning method development but also creates opportunities for materials research, such as building a Moiré-twisted material database.

Physics-inspired machine learning of localized intensive properties

Machine learning (ML) has been widely applied to chemical property prediction, most prominently for the energies and forces in molecules and materials. The strong interest in predicting energies in particular has led to a 'local energy'-based paradigm for modern atomistic ML models, which ensures size-extensivity and a linear scaling of computational cost with system size. However, many electronic properties (such as excitation energies or ionization energies) do not necessarily scale linearly with system size and may even be spatially localized. Using size-extensive models in these cases can lead to large errors. In this work, we explore different strategies for learning intensive and localized properties, using HOMO energies in organic molecules as a representative test case. In particular, we analyze the pooling functions that atomistic neural networks use to predict molecular properties, and suggest an orbital weighted average (OWA) approach that enables the accurate prediction of orbital energies and locations.

Harnessing Data Augmentation and Normalization Preprocessing to Improve the Performance of Chemical Reaction Predictions of Data-Driven Model

As a template-free, data-driven methodology, the molecular transformer model provides an alternative by which to predict the outcome of chemical reactions and design the route of the retrosynthetic plane in the field of organic synthesis and polymer chemistry. However, in consideration of the small datasets of chemical reactions, the data-driven model suffers from the difficulty of low accuracy in the prediction tasks of chemical reactions. In this contribution, we integrate the molecular transformer model with the strategies of data augmentation and normalization preprocessing to accomplish the three tasks of chemical reactions, including the forward predictions of chemical reactions, and single-step retrosynthetic predictions with and without the reaction classes. It is clearly demonstrated that the prediction accuracy of the molecular transformer model can be significantly raised by the use of proposed strategies for the three tasks of chemical reactions. Notably, after the introduction of the 40-level data augmentation and normalization preprocessing, the top-1 accuracy of the forward prediction increases markedly from 71.6% to 84.2% and the top-1 accuracy of the single-step retrosynthetic prediction with additional reaction class increases from 53.2% to 63.4%. Furthermore, it is found that the superior performance of the data-driven model originates from the correction of the grammatical errors of the SMILES strings, especially for the case of the reaction classes with small datasets.

SchNetPack 2.0: A neural network toolbox for atomistic machine learning

SchNetPack is a versatile neural network toolbox that addresses both the requirements of method development and the application of atomistic machine learning. Version 2.0 comes with an improved data pipeline, modules for equivariant neural networks, and a PyTorch implementation of molecular dynamics. An optional integration with PyTorch Lightning and the Hydra configuration framework powers a flexible command-line interface. This makes SchNetPack 2.0 easily extendable with a custom code and ready for complex training tasks, such as the generation of 3D molecular structures.

Deep-learning electronic-structure calculation of magnetic superstructures

Ab initio studies of magnetic superstructures are indispensable to research on emergent quantum materials, but are currently bottlenecked by the formidable computational cost. Here, to break this bottleneck, we have developed a deep equivariant neural network framework to represent the density functional theory Hamiltonian of magnetic materials for efficient electronic-structure calculation. A neural network architecture incorporating a priori knowledge of fundamental physical principles, especially the nearsightedness principle and the equivariance requirements of Euclidean and time-reversal symmetries ([Formula: see text]), is designed, which is critical to capture the subtle magnetic effects. Systematic experiments on spin-spiral, nanotube and moiré magnets were performed, making the challenging study of magnetic skyrmions feasible.

Electronic Descriptors for Supervised Spectroscopic Predictions

Spectroscopic properties of molecules hold great importance for the description of the molecular response under the effect of UV/vis electromagnetic radiation. Computationally expensive ab initio (e.g., MultiConfigurational SCF, Coupled Cluster) or TDDFT methods are commonly used by the quantum chemistry community to compute these properties. In this work, we propose a (supervised) Machine Learning approach to model the absorption spectra of organic molecules. Several supervised ML methods have been tested such as Kernel Ridge Regression (KRR), Multiperceptron Neural Networs (MLP), and Convolutional Neural Networks. [Ramakrishnan et al. J. Chem. Phys. 2015, 143, 084111. Ghosh et al. Adv. Sci. 2019, 6, 1801367.] The use of only geometrical-atomic number descriptors (e.g., Coulomb Matrix) proved to be insufficient for an accurate training. [Ramakrishnan et al. J. Chem. Phys. 2015, 143, 084111.] Inspired by the TDDFT theory, we propose to use a set of electronic descriptors obtained from low-cost DFT methods: orbital energy differences (Δϵ_ia = ϵ_a - ϵ_i), transition dipole moment between occupied and unoccupied Kohn-Sham orbitals (⟨ϕ_i|r|ϕ_a⟩), and when relevant, charge-transfer character of monoexcitations (R_ia). We demonstrate that with these electronic descriptors and the use of Neural Networks we can predict not only a density of excited states but also get a very good estimation of the absorption spectrum and charge-transfer character of the electronic excited states, reaching results close to chemical accuracy (∼2 kcal/mol or ∼0.1 eV).

Accurate and Interpretable Dipole Interaction Model-Based Machine Learning for Molecular Polarizability

Polarizabilities play significant roles in describing dispersive and inductive interactions of the atom and molecular systems. However, an accurate prediction of molecular polarizabilities from first principles is computationally prohibitive. Although physical models or statistical machine learning models have been proposed, either a lack of accurate description of local chemical environments or demanding a large number of samples for training has limited their practical applications. In this study, we combine a physically inspired dipole interaction model and an accurate neural network method for predicting the polarizability tensors of molecules. With the local chemical environment precisely described and the requirement of rotational covariance naturally fulfilled, this hybrid model is proven to give an accurate molecular polarizability prediction, essentially reducing the number of training samples. The atomic polarizabilities are physically interpretable and transferable to larger molecules unseen in the training set. This promising method may find its wide range of applications, such as spectroscopic simulations and the construction of polarizable force fields.

An Imbalance in the Force: The Need for Standardized Benchmarks for Molecular Simulation

Force fields (FFs) for molecular simulation have been under development for more than half a century. As with any predictive model, rigorous testing and comparisons of models critically depends on the availability of standardized data sets and benchmarks. While such benchmarks are rather common in the fields of quantum chemistry, this is not the case for empirical FFs. That is, few benchmarks are reused to evaluate FFs, and development teams rather use their own training and test sets. Here we present an overview of currently available tests and benchmarks for computational chemistry, focusing on organic compounds, including halogens and common ions, as FFs for these are the most common ones. We argue that many of the benchmark data sets from quantum chemistry can in fact be reused for evaluating FFs, but new gas phase data is still needed for compounds containing phosphorus and sulfur in different valence states. In addition, more nonequilibrium interaction energies and forces, as well as molecular properties such as electrostatic potentials around compounds, would be beneficial. For the condensed phases there is a large body of experimental data available, and tools to utilize these data in an automated fashion are under development. If FF developers, as well as researchers in artificial intelligence, would adopt a number of these data sets, it would become easier to compare the relative strengths and weaknesses of different models and to, eventually, restore the balance in the force.

Accurate global machine learning force fields for molecules with hundreds of atoms

Global machine learning force fields, with the capacity to capture collective interactions in molecular systems, now scale up to a few dozen atoms due to considerable growth of model complexity with system size. For larger molecules, locality assumptions are introduced, with the consequence that nonlocal interactions are not described. Here, we develop an exact iterative approach to train global symmetric gradient domain machine learning (sGDML) force fields (FFs) for several hundred atoms, without resorting to any potentially uncontrolled approximations. All atomic degrees of freedom remain correlated in the global sGDML FF, allowing the accurate description of complex molecules and materials that present phenomena with far-reaching characteristic correlation lengths. We assess the accuracy and efficiency of sGDML on a newly developed MD22 benchmark dataset containing molecules from 42 to 370 atoms. The robustness of our approach is demonstrated in nanosecond path-integral molecular dynamics simulations for supramolecular complexes in the MD22 dataset.

Machine learning in the evaluation and prediction models of biochar application: A review

This article reviews recent studies applying machine learning (ML) approaches to biochar applications. We first briefly introduce the general biochar production process. Various aspects are contained, including the biochar application in the elimination of heavy metals and/or organic compounds and the biochar application in environmental and economic scopes, for instance, food security, energy, and carbon emission. The utilization of ML methods, including ANN, RF, and NN, plays a vital role in evaluating and predicting the efficiency of biochar absorption. It has been proved that ML methods can validly predict the adsorption effectiveness of biochar for water heavy metals with higher accuracy. Moreover, the literature proposed a comprehensive data-driven model to forecast biochar yield and compositions under various biomass input feedstock and different pyrolysis criteria. They said a 12.7% improvement in prediction accuracy compared to the existing literature. However, it might need further optimization in this direction. In summary, this review concludes increasing studies that a well-trained ML method can sufficiently reduce the number of experiment trials and working times associated with higher prediction accuracy. Moreover, further studies on ML applications are needed to optimize the trade-off between biochar yield and its composition.

Electronic-Structure Properties from Atom-Centered Predictions of the Electron Density

The electron density of a molecule or material has recently received major attention as a target quantity of machine-learning models. A natural choice to construct a model that yields transferable and linear-scaling predictions is to represent the scalar field using a multicentered atomic basis analogous to that routinely used in density fitting approximations. However, the nonorthogonality of the basis poses challenges for the learning exercise, as it requires accounting for all the atomic density components at once. We devise a gradient-based approach to directly minimize the loss function of the regression problem in an optimized and highly sparse feature space. In so doing, we overcome the limitations associated with adopting an atom-centered model to learn the electron density over arbitrarily complex data sets, obtaining very accurate predictions using a comparatively small training set. The enhanced framework is tested on 32-molecule periodic cells of liquid water, presenting enough complexity to require an optimal balance between accuracy and computational efficiency. We show that starting from the predicted density a single Kohn-Sham diagonalization step can be performed to access total energy components that carry an error of just 0.1 meV/atom with respect to the reference density functional calculations. Finally, we test our method on the highly heterogeneous QM9 benchmark data set, showing that a small fraction of the training data is enough to derive ground-state total energies within chemical accuracy.

Graph neural networks for materials science and chemistry

Machine learning plays an increasingly important role in many areas of chemistry and materials science, being used to predict materials properties, accelerate simulations, design new structures, and predict synthesis routes of new materials. Graph neural networks (GNNs) are one of the fastest growing classes of machine learning models. They are of particular relevance for chemistry and materials science, as they directly work on a graph or structural representation of molecules and materials and therefore have full access to all relevant information required to characterize materials. In this Review, we provide an overview of the basic principles of GNNs, widely used datasets, and state-of-the-art architectures, followed by a discussion of a wide range of recent applications of GNNs in chemistry and materials science, and concluding with a road-map for the further development and application of GNNs.

Machine learning the Hohenberg-Kohn map for molecular excited states

The Hohenberg-Kohn theorem of density-functional theory establishes the existence of a bijection between the ground-state electron density and the external potential of a many-body system. This guarantees a one-to-one map from the electron density to all observables of interest including electronic excited-state energies. Time-Dependent Density-Functional Theory (TDDFT) provides one framework to resolve this map; however, the approximations inherent in practical TDDFT calculations, together with their computational expense, motivate finding a cheaper, more direct map for electronic excitations. Here, we show that determining density and energy functionals via machine learning allows the equations of TDDFT to be bypassed. The framework we introduce is used to perform the first excited-state molecular dynamics simulations with a machine-learned functional on malonaldehyde and correctly capture the kinetics of its excited-state intramolecular proton transfer, allowing insight into how mechanical constraints can be used to control the proton transfer reaction in this molecule. This development opens the door to using machine-learned functionals for highly efficient excited-state dynamics simulations.

Selected machine learning of HOMO-LUMO gaps with improved data-efficiency

Despite their relevance for organic electronics, quantum machine learning (QML) models of molecular electronic properties, such as HOMO-LUMO-gaps, often struggle to achieve satisfying data-efficiency as measured by decreasing prediction errors for increasing training set sizes. We demonstrate that partitioning training sets into different chemical classes prior to training results in independently trained QML models with overall reduced training data needs. For organic molecules drawn from previously published QM7 and QM9-data-sets we have identified and exploited three relevant classes corresponding to compounds containing either aromatic rings and carbonyl groups, or single unsaturated bonds, or saturated bonds The selected QML models of band-gaps (considered at GW and hybrid DFT levels of theory) reach mean absolute prediction errors of ∼0.1 eV for up to an order of magnitude fewer training molecules than for QML models trained on randomly selected molecules. Comparison to Δ-QML models of band-gaps indicates that selected QML exhibit superior data-efficiency. Our findings suggest that selected QML, e.g. based on simple classifications prior to training, could help to successfully tackle challenging quantum property screening tasks of large libraries with high fidelity and low computational burden.

Obtaining Electronic Properties of Molecules through Combining Density Functional Tight Binding with Machine Learning

We have introduced a machine learning workflow that allows for optimizing electronic properties in the density functional tight binding method (DFTB). The workflow allows for the optimization of electronic properties by generating two-center integrals, either by training basis function parameters directly or by training a spline model for the diatomic integrals, which are then used to build the Hamiltonian and the overlap matrices. Using our workflow, we have managed to obtain improved electronic properties, such as charge distributions, dipole moments, and approximated polarizabilities. While both machine learning approaches enabled us to improve on the electronic properties of the molecules as compared with existing DFTB parametrizations, only by training on the basis function parameters we were able to obtain consistent Hamiltonians and overlap matrices in the physically reasonable ranges or to improve on multiple electronic properties simultaneously.