Highly Efficient, Linear-Scaling Seminumerical Exact-Exchange Method for Graphic Processing Units

Combining low-cost electronic structure theory and low-cost parallel computing architecture

The computational efficiency of low-cost electronic structure methods can be further improved by leveraging heterogenous computing architectures. The software package TeraChem has been developed since 2008 to make use of graphical processing units (GPUs), particularly their strong single-precision performance, for the acceleration of quantum chemical calculations. Here, we present the implementation of three low-cost methods, namely HF-3c, PBEh-3c, and the recently introduced ωB97X-3c. We show that these can benefit in terms of performance when combined with "consumer grade" GPUs by leveraging the mixed precision integral handling in TeraChem. The current limitation of the latter's GPU integral library is that Gaussian integrals only for functions with angular momentum l < 3 can be computed, which generally restricts the achievable accuracy in terms of the one-particle basis set. Particularly, the implementation of the ωB97X-3c method now enables higher accuracy with this setting which, in turn, provides the most efficient implementation accessible with consumer-grade hardware. We furthermore show that the implemented 3c methods can be combined with the hh-TDA formalism. This gives new and efficient low-cost multi-configurational excited states methods, which are benchmarked for the description of lowest vertical excitation energies in this work. All in all, the combination of these efficient electronic structure theory methods with affordable highly parallelized computing hardware provides an optimal computational and monetary cost to accuracy ratio.

Efficient Exploitation of Numerical Quadrature with Distance-Dependent Integral Screening in Explicitly Correlated F12 Theory: Linear Scaling Evaluation of the Most Expensive RI-MP2-F12 Term

We present a linear scaling atomic orbital based algorithm for the computation of the most expensive exchange-type RI-MP2-F12 term by employing numerical quadrature in combination with CABS-RI to avoid six-center-three-electron integrals. Furthermore, a robust distance-dependent integral screening scheme, based on integral partition bounds [Thompson, T. H.; Ochsenfeld, C. J. Chem. Phys. 2019, 150, 044101], is used to drastically reduce the number of the required three-center-one-electron integrals substantially. The accuracy of our numerical quadrature/CABS-RI approach and the corresponding integral screening is thoroughly assessed for interaction and isomerization energies across a variety of numerical integration grids. Our method outperforms the standard density fitting/CABS-RI approach with errors below 1 μEh even for small grid sizes and moderate screening thresholds. The choice of the grid size and screening threshold allows us to tailor our ansatz to a desired accuracy and computational efficiency. We showcase the approach's effectiveness for the chemically relevant system valinomycin, employing a triple-ζ F12 basis set combination (C54H90N6O18, 5757 AO basis functions, 10,266 CABS basis functions, 735,783 grid points). In this context, our ansatz achieves higher accuracy combined with a 135× speedup compared to the classical density fitting based variant, requiring notably less computation time than the corresponding RI-MP2 calculation. Additionally, we demonstrate near-linear scaling through calculations on linear alkanes. We achieved an 817-fold acceleration for C80H162 and an extrapolated 28,765-fold acceleration for C200H402, resulting in a substantially reduced computational time for the latter─from 229 days to just 11.5 min. Our ansatz may also be adapted to the remaining MP2-F12 terms, which will be the subject of future work.

Systematic QM/MM Study for Predicting 31P NMR Chemical Shifts of Adenosine Nucleotides in Solution and Stages of ATP Hydrolysis in a Protein Environment

NMR (nuclear magnetic resonance) spectroscopy allows for important atomistic insights into the structure and dynamics of biological macromolecules; however, reliable assignments of experimental spectra are often difficult. Herein, quantum mechanical/molecular mechanical (QM/MM) calculations can provide crucial support. A major problem for the simulations is that experimental NMR signals are time-averaged over much longer time scales, and since computed chemical shifts are highly sensitive to local changes in the electronic and structural environment, sufficiently large averages over representative structural ensembles are essential. This entails high computational demands for reliable simulations. For NMR measurements in biological systems, a nucleus of major interest is 31P since it is both highly present (e.g., in nucleic acids) and easily observable. The focus of our present study is to develop a robust and computationally cost-efficient framework for simulating 31P NMR chemical shifts of nucleotides. We apply this scheme to study the different stages of the ATP hydrolysis reaction catalyzed by p97. Our methodology is based on MM molecular dynamics (MM-MD) sampling, followed by QM/MM structure optimizations and NMR calculations. Overall, our study is one of the most comprehensive QM-based 31P studies in a protein environment and the first to provide computed NMR chemical shifts for multiple nucleotide states in a protein environment. This study sheds light on a process that is challenging to probe experimentally and aims to bridge the gap between measured and calculated NMR spectroscopic properties.

Exploring Chemical Space Using Ab Initio Hyperreactor Dynamics

In recent years, first-principles exploration of chemical reaction space has provided valuable insights into intricate reaction networks. Here, we introduce ab initio hyperreactor dynamics, which enables rapid screening of the accessible chemical space from a given set of initial molecular species, predicting new synthetic routes that can potentially guide subsequent experimental studies. For this purpose, different hyperdynamics derived bias potentials are applied along with pressure-inducing spherical confinement of the molecular system in ab initio molecular dynamics simulations to efficiently enhance reactivity under mild conditions. To showcase the advantages and flexibility of the hyperreactor approach, we present a systematic study of the method's parameters on a HCN toy model and apply it to a recently introduced experimental model for the prebiotic formation of glycinal and acetamide in interstellar ices, which yields results in line with experimental findings. In addition, we show how the developed framework enables the study of complicated transitions like the first step of a nonenzymatic DNA nucleoside synthesis in an aqueous environment, where the molecular fragmentation problem of earlier nanoreactor approaches is avoided.

Improved Sampling of Adaptive Path Collective Variables by Stabilized Extended-System Dynamics

Because of the complicated multistep nature of many biocatalytic reactions, an a priori definition of reaction coordinates is difficult. Therefore, we apply enhanced sampling algorithms along with adaptive path collective variables (PCVs), which converge to the minimum free energy path (MFEP) during the simulation. We show how PCVs can be combined with the highly efficient well-tempered metadynamics extended-system adaptive biasing force (WTM-eABF) hybrid sampling algorithm, offering dramatically increased sampling efficiency due to its fast adaptation to path updates. For this purpose, we address discontinuities of PCVs that can arise due to path shortcutting or path updates with a novel stabilization algorithm for extended-system methods. In addition, we show how the convergence of simulations can be further accelerated by utilizing the multistate Bennett's acceptance ratio (MBAR) estimator. These methods are applied to the first step of the enzymatic reaction mechanism of pseudouridine synthases, where the ability of path WTM-eABF to efficiently explore intricate molecular transitions is demonstrated.

Zero-field splitting parameters within exact two-component theory and modern density functional theory using seminumerical integration

An efficient implementation of zero-field splitting parameters based on the work of Schmitt et al. [J. Chem. Phys. 134, 194113 (2011)] is presented. Seminumerical integration techniques are used for the two-electron spin-dipole contribution and the response equations of the spin-orbit perturbation. The original formulation is further generalized. First, it is extended to meta-generalized gradient approximations and local hybrid functionals. For these functional classes, the response of the paramagnetic current density is considered in the coupled-perturbed Kohn-Sham equations for the spin-orbit perturbation term. Second, the spin-orbit perturbation is formulated within relativistic exact two-component theory and the screened nuclear spin-orbit (SNSO) approximation. The accuracy of the implementation is demonstrated for transition-metal and diatomic main-group compounds. The efficiency is assessed for Mn and Mo complexes. Here, it is found that coarse integration grids for the seminumerical schemes lead to drastic speedups while introducing clearly negligible errors. In addition, the SNSO approximation substantially reduces the computational demands and leads to very similar results as the spin-orbit mean field Ansatz.

Highly accurate σ- and τ-functionals for beyond-RPA methods with approximate exchange kernels

σ-Functionals are promising new developments for the Kohn-Sham correlation energy based upon the direct Random Phase Approximation (dRPA) within the adiabatic connection formalism, providing impressive improvements over dRPA for a broad range of benchmarks. However, σ-functionals exhibit a high amount of self-interaction inherited from the approximations made within dRPA. Inclusion of an exchange kernel in deriving the coupling-strength-dependent density-density response function leads to so-called τ-functionals, which - apart from a fourth-order Taylor series expansion - have only been realized in an approximate fashion so far to the best of our knowledge, most notably in the form of scaled σ-functionals. In this work, we derive, optimize, and benchmark three types of σ- and τ-functionals including approximate exchange effects in the form of an antisymmetrized Hartree kernel. These functionals, based on a second-order screened exchange type contribution in the adiabatic connection formalism, the electron-hole time-dependent Hartree-Fock kernel (eh-TDHF) otherwise known as RPA with exchange (RPAx), and an approximation thereof known as approximate exchange kernel (AXK), are optimized on the ASCDB database using two new parametrizations named A1 and A2. In addition, we report a first full evaluation of σ- and τ-functionals on the GMTKN55 database, revealing our exchange-including functionals to considerably outperform existing σ-functionals while being highly competitive with some of the best double-hybrid functionals of the original GMTKN55 publication. In particular, the σ-functionals based on AXK and τ-functionals based on RPAx with PBE0 reference stand out as highly accurate approaches for a wide variety of chemically relevant problems.

Corrected density functional theory and the random phase approximation: Improved accuracy at little extra cost

We recently introduced an efficient methodology to perform density-corrected Hartree-Fock density functional theory [DC(HF)-DFT] calculations and an extension to it we called "corrected" HF DFT [C(HF)-DFT] [Graf and Thom, J. Chem. Theory Comput. 19 5427-5438 (2023)]. In this work, we take a further step and combine C(HF)-DFT, augmented with a straightforward orbital energy correction, with the random phase approximation (RPA). We refer to the resulting methodology as corrected HF RPA [C(HF)-RPA]. We evaluate the proposed methodology across various RPA methods: direct RPA (dRPA), RPA with an approximate exchange kernel, and RPA with second-order screened exchange. C(HF)-dRPA demonstrates very promising performance; for RPA with exchange methods, on the other hand, we often find over-corrections.

Simple and Efficient Route toward Improved Energetics within the Framework of Density-Corrected Density Functional Theory

The crucial step in density-corrected Hartree-Fock density functional theory (DC(HF)-DFT) is to decide whether the density produced by the density functional for a specific calculation is erroneous and, hence, should be replaced by, in this case, the HF density. We introduce an indicator, based on the difference in noninteracting kinetic energies between DFT and HF calculations, to determine when the HF density is the better option. Our kinetic energy indicator directly compares the self-consistent density of the analyzed functional with the HF density, is size-intensive, reliable, and most importantly highly efficient. Moreover, we present a procedure that makes best use of the computed quantities necessary for DC(HF)-DFT by additionally evaluating a related hybrid functional and, in that way, not only "corrects" the density but also the functional itself; we call that procedure corrected Hartree-Fock density functional theory (C(HF)-DFT).

Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory

With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. In the present work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn-Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.

Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units

We have ported and optimized the graphics processing unit (GPU)-accelerated QUICK and AMBER-based ab initio quantum mechanics/molecular mechanics (QM/MM) implementation on AMD GPUs. This encompasses the entire Fock matrix build and force calculation in QUICK including one-electron integrals, two-electron repulsion integrals, exchange-correlation quadrature, and linear algebra operations. General performance improvements to the QUICK GPU code are also presented. Benchmarks carried out on NVIDIA V100 and AMD MI100 cards display similar performance on both hardware for standalone HF/DFT calculations with QUICK and QM/MM molecular dynamics simulations with QUICK/AMBER. Furthermore, with respect to the QUICK/AMBER release version 21, significant speedups are observed for QM/MM molecular dynamics simulations. This significantly increases the range of scientific problems that can be addressed with open-source QM/MM software on state-of-the-art computer hardware.

Analytic gradients for local density fitting Hartree-Fock and Kohn-Sham methods

We present analytic gradients for local density fitting Hartree-Fock (HF) and hybrid Kohn-Sham (KS) density functional methods. Due to the non-variational nature of the local fitting algorithm, the method of Lagrange multipliers is used to avoid the solution of the coupled perturbed HF and KS equations. We propose efficient algorithms for the solution of the arising Z-vector equations and the gradient calculation that preserve the third-order scaling and low memory requirement of the original local fitting algorithm. In order to demonstrate the speed and accuracy of our implementation, gradient calculations and geometry optimizations are presented for various molecular systems. Our results show that significant speedups can be achieved compared to conventional density fitting calculations without sacrificing accuracy.

Fast Exchange with Gaussian Basis Set Using Robust Pseudospectral Method

In this article, we present an algorithm to efficiently evaluate the exchange matrix in periodic systems when a Gaussian basis set with pseudopotentials is used. The usual algorithm for evaluating exchange matrix scales cubically with the system size because one has to perform O(N²) fast Fourier transform (FFT). Here, we introduce an algorithm that retains the cubic scaling but reduces the prefactor significantly by eliminating the need to do FFTs during each exchange build. This is accomplished by representing the products of Gaussian basis function using a linear combination of an auxiliary basis the number of which scales linearly with the size of the system. We store the potential due to these auxiliary functions in memory, which allows us to obtain the exchange matrix without the need to do FFT, albeit at the cost of additional memory requirement. Although the basic idea of using auxiliary functions is not new, our algorithm is cheaper due to a combination of three ingredients: (a) we use a robust pseudospectral method that allows us to use a relatively small number of auxiliary basis to obtain high accuracy; (b) we use occ-RI exchange, which eliminates the need to construct the full exchange matrix; and (c) we use the (interpolative separable density fitting) ISDF algorithm to construct these auxiliary basis sets that are used in the robust pseudospectral method. The resulting algorithm is accurate, and we note that the error in the final energy decreases exponentially rapidly with the number of auxiliary functions.

Fully Automated Generation of Prebiotically Relevant Reaction Networks from Optimized Nanoreactor Simulations

The nanoreactor approach first introduced by the group of Martı́nez [Wang et al. Nat. Chem. 2014, 6, 1044-1048] has recently attracted much attention because of its ability to accelerate the discovery of reaction pathways. Here, we provide a comprehensive study of various simulation parameters and present an alternative implementation for the reactivity-enhancing spherical constraint function, as well as for the detection of reaction events. In this context, a fully automated postsimulation evaluation procedure based on RDKit and NetworkX analysis is introduced. The chemical and physical robustness of the procedure is examined by investigating the reactivity of selected homogeneous systems. The optimized procedure is applied at the GFN2-xTB level of theory to a system composed of HCN molecules and argon atoms, acting as a buffer, yielding prebiotically plausible primary and secondary precursors for the synthesis of RNA. Furthermore, the formose reaction network is explored leading to numerous sugar precursors. The discovered compounds reflect experimental findings; however, new synthetic routes and a large collection of exotic, highly reactive molecules are observed, highlighting the predictive power of the nanoreactor approach for unraveling the reactive manifold.

Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language

The computation of two-electron repulsion integrals (ERIs) is often the most expensive step of integral-direct self-consistent field methods. Formally it scales as O(N⁴), where N is the number of Gaussian basis functions used to represent the molecular wave function. In practice, this scaling can be reduced to O(N²) or less by neglecting small integrals with screening methods. The contributions of the ERIs to the Fock matrix are of Coulomb (J) and exchange (K) type and require separate algorithms to compute matrix elements efficiently. We previously implemented highly efficient GPU-accelerated J-matrix and K-matrix algorithms in the electronic structure code TeraChem. Although these implementations supported the use of multiple GPUs on a node, they did not support the use of multiple nodes. This presents a key bottleneck to cutting-edge ab initio simulations of large systems, e.g., excited state dynamics of photoactive proteins. We present our implementation of multinode multi-GPU J- and K-matrix algorithms in TeraChem using the Regent programming language. Regent directly supports distributed computation in a task-based model and can generate code for a variety of architectures, including NVIDIA GPUs. We demonstrate multinode scaling up to 45 GPUs (3 nodes) and benchmark against hand-coded TeraChem integral code. We also outline our metaprogrammed Regent implementation, which enables flexible code generation for integrals of different angular momenta.

Accelerating Hybrid Density Functional Theory Molecular Dynamics Simulations by Seminumerical Integration, Resolution-of-the-Identity Approximation, and Graphics Processing Units

The computationally very demanding evaluation of the 4-center-2-electron (4c2e) integrals and their respective integral derivatives typically represents the major bottleneck within hybrid Kohn-Sham density functional theory molecular dynamics simulations. Building upon our previous works on seminumerical exact-exchange (sn-LinK) [Laqua, H., Thompsons, T. H., Kussmann, J., Ochsenfeld, C., J. Chem. Theory Comput. 2020, 16, 1465] and resolution-of-the-identity Coulomb (RI-J) [Kussmann, J., Laqua, H., Ochsenfeld, C., J. Chem. Theory Comput. 2021, 17, 1512], the expensive 4c2e integral evaluation can be avoided entirely, resulting in a highly efficient electronic structure theory method, allowing for fast ab initio molecular dynamics (AIMD) simulations even with large basis sets. Moreover, we propose to combine the final self-consistent field (SCF) step with the subsequent nuclear forces evaluation, providing the forces at virtually no additional cost after a converged SCF calculation, reducing the total runtime of an AIMD simulation by about another 25%. In addition, multiple independent MD trajectories can be computed concurrently on a single node, leading to a greatly increased utilization of the available hardware─especially when combined with graphics processing unit acceleration─improving the overall throughput by up to another 5 times in this way. With all of those optimizations combined, our proposed method provides nearly 3 orders of magnitude faster execution times than traditional 4c2e integral-based methods. To demonstrate the practical utility of the approach, quantum-mechanical/molecular-mechanical dynamics simulations on double-stranded DNA were performed, investigating the relative hydrogen bond strength between adenine-thymine and guanine-cytosine base pairs. In addition, this illustrative application also contains a general accuracy assessment of the introduced approximations (integration grids, resolution-of-the-identity) within AIMD simulations, serving as a protocol on how to apply these new methods to practical problems.

An effective sub-quadratic scaling atomic-orbital reformulation of the scaled opposite-spin RI-CC2 ground-state model using Cholesky-decomposed densities and an attenuated Coulomb metric

An atomic-orbital reformulation of the Laplace-transformed scaled opposite-spin (SOS) coupled cluster singles and doubles (CC2) model within the resolution of the identity (RI) approximation (SOS-RI-CC2) is presented that extends its applicability to molecules with several hundreds of atoms and triple-zeta basis sets. We exploit sparse linear algebra and an attenuated Coulomb metric to decrease the disk space demands and the computational efforts. In this way, an effective sub-quadratic computational scaling is achieved with our ω-SOS-CDD-RI-CC2 model. Moreover, Cholesky decomposition of the ground-state one-electron density matrix reduces the prefactor, allowing for an early crossover with the molecular orbital formulation. The accuracy and performance of the presented method are investigated for various molecular systems.

Tensor-Hypercontracted MP2 First Derivatives: Runtime and Memory Efficient Computation of Hyperfine Coupling Constants

We employ our recently introduced tensor-hypercontracted (THC) second-order Møller-Plesset perturbation theory (MP2) method [Bangerter, F. H., Glasbrenner, M., Ochsenfeld, C. J. Chem. Theory Comput. 2021, 17, 211-221] for the computation of hyperfine coupling constants (HFCCs). The implementation leverages the tensor structure of the THC factorized electron repulsion integrals for an efficient formation of the integral-based intermediates. The computational complexity of the most expensive and formally quintic scaling exchange-like contribution is reduced to effectively subquadratic, by making use of the intrinsic, exponentially decaying coupling between tensor indices through screening based on natural blocking. Overall, this yields an effective subquadratic scaling with a low prefactor for the presented THC-based AO-MP2 method for the computation of isotropic HFCCs on DNA fragments with up to 500 atoms and 5000 basis functions. Furthermore, the implementation achieves considerable speedups with up to a factor of roughly 600-1000 compared to previous implementations [Vogler, S., Ludwig, M., Maurer, M., Ochsenfeld, C. J. Chem. Phys. 2017, 147, 024101] for medium-sized organic radicals, while also significantly reducing storage requirements.

Statistically optimal analysis of the extended-system adaptive biasing force (eABF) method

The extended-system adaptive biasing force (eABF) method and its newer variants offer rapid exploration of the configuration space of chemical systems. Instead of directly applying the ABF bias to collective variables, they are harmonically coupled to fictitious particles, which separates the problem of enhanced sampling from that of free energy estimation. The prevalent analysis method to obtain the potential of mean force (PMF) from eABF is thermodynamic integration. However, besides the PMF, most information is lost as the unbiased probability of visited configurations is never recovered. In this contribution, we show how statistical weights of individual frames can be computed using the Multistate Bennett's Acceptance Ratio (MBAR), putting the post-processing of eABF on one level with other frequently used sampling methods. In addition, we apply this formalism to the prediction of nuclear magnetic resonance shieldings, which are very sensitive to molecular geometries and often require extensive sampling. The results show that the combination of enhanced sampling by means of extended-system dynamics with the MBAR estimator is a highly useful tool for the calculation of ensemble properties. Furthermore, the extension of the presented scheme to the recently published Gaussian-accelerated molecular dynamics eABF hybrid is straightforward and approximation free.

Highly Efficient and Accurate Computation of Multiple Orbital Spaces Spanning Fock Matrix Elements on Central and Graphics Processing Units for Application in F12 Theory

We employ our recently published highly efficient seminumerical exchange (sn-LinK) [Laqua, H.; Thompson, T. H.; Kussmann, J.; Ochsenfeld, C. J. Chem. Theory Comput. 2020, 16, 1456-1468] and integral-direct resolution of the identity Coulomb (RI-J) [Kussmann, J.; Laqua, H.; Ochsenfeld, C. J. Chem. Theory Comput. 2021, 17, 1512-1521] methods to significantly accelerate the computation of the demanding multiple orbital spaces spanning Fock matrix elements present in R12/F12 theory on central and graphics processing units. The errors introduced by RI-J and sn-LinK into the RI-MP2-F12 energy are thoroughly assessed for a variety of basis sets and integration grids. We find that these numerical errors are always below "chemical accuracy" (∼1 mH) even for the coarsest settings and can easily be reduced below 1 μH by employing only moderately large integration grids and RI-J basis sets. Since the number of basis functions of the multiple orbital spaces is notably larger compared with conventional Hartree-Fock theory, the efficiency gains from the superior basis scaling of RI-J and sn-LinK (O(N_bas²) instead of O(N_bas⁴) for both) are even more significant, with maximum speedup factors of 37 000 for RI-J and 4500 for sn-LinK. In total, the multiple orbital spaces spanning Fock matrix evaluation of the largest tested structure using a triple-ζ F12 basis set (5058 AO basis functions, 9267 CABS basis functions) is accelerated over 1575× using CPUs and over 4155× employing GPUs.

Excited-State Properties for Extended Systems: Efficient Hybrid Density Functional Methods

Time-dependent density functional theory has become state-of-the-art for describing photophysical and photochemical processes in extended materials because of its affordable cost. The inclusion of exact exchange was shown to be essential for the correct description of the long-range asymptotics of electronic interactions and thus a well-balanced description of valence, Rydberg, and charge-transfer excitations. Several approaches for an efficient treatment of exact exchange have been established for the ground state, while implementations for excited-state properties are rare. Furthermore, the high computational costs required for excited-state properties in comparison to ground-state computations often hinder large-scale applications on periodic systems with hybrid functional accuracy. We therefore propose two approximate schemes for improving computational efficiency for the treatment of exact exchange. Within the auxiliary density matrix method (ADMM), exact exchange is estimated using a relatively small auxiliary basis and the introduced basis set incompleteness error is compensated by an exchange density functional correction term. Benchmark results for a test set of 35 molecules demonstrate that the mean absolute error introduced by ADMM is smaller than 0.3 pm for excited-state bond lengths and in the range of 0.02-0.04 eV for vertical excitation, adiabatic excitation, and fluorescence energies. Computational timings for a series of covalent-organic frameworks demonstrate that a speed-up of at least 1 order of magnitude can be achieved for excited-state geometry optimizations in comparison to conventional hybrid functionals. The second method is to use a semiempirical tight binding approximation for both Coulomb and exchange contributions to the excited-state kernel. This simplified Tamm-Dancoff approximation (sTDA) achieves an accuracy comparable to approximated hybrid density functional theory when referring to highly accurate coupled-cluster reference data. We find that excited-state bond lengths deviate by 1.1 pm on average and mean absolute errors in vertical excitation, adiabatic excitation, and fluorescence energies are in the range of 0.2-0.5 eV. In comparison to ADMM-approximated hybrid functional theory, sTDA accelerates the computation of broad-band excitation spectra by 1 order of magnitude, suggesting its potential use for large-scale screening purposes.

Factors That Determine the Variation of Equilibrium and Kinetic Properties of QM/MM Enzyme Simulations: QM Region, Conformation, and Boundary Condition

To analyze the impact of various technical details on the results of quantum mechanical (QM)/molecular mechanical (MM) enzyme simulations, including the QM region size, catechol-O-methyltransferase (COMT) is studied as a model system using an approximate QM/MM method (DFTB3/CHARMM). The results show that key equilibrium and kinetic properties for methyl transfer in COMT exhibit limited variations with respect to the size of the QM region, which ranges from ∼100 to ∼500 atoms in this study. With extensive sampling, local and global structural characteristics of the enzyme are largely conserved across the studied QM regions, while the nature of the transition state (e.g., secondary kinetic isotope effect) and reaction exergonicity are largely maintained. Deviations in the free energy profile with different QM region sizes are similar in magnitude to those observed with changes in other simulation protocols, such as different initial enzyme conformations and boundary conditions. Electronic structural properties, such as the covariance matrix of residual charge fluctuations, appear to exhibit rather long-range correlations, especially when the peptide backbone is included in the QM region; this observation holds when a range-separated DFT approach is used as the QM region, suggesting that delocalization error is unlikely the origin. Overall, the analyses suggest that multiple simulation details determine the results of QM/MM enzyme simulations with comparable contributions.

Fermi.jl: A Modern Design for Quantum Chemistry

Approximating molecular wave functions involves heavy numerical effort; therefore, codes for such tasks are written completely or partially in efficient languages such as C, C++, and Fortran. While these tools are dominant throughout quantum chemistry packages, the efficient development of new methods is often hindered by the complexity associated with code development. In order to ameliorate this scenario, some software packages take a dual approach where a simpler, higher-level language, such as Python, substitutes the traditional ones wherever performance is not critical. Julia is a novel, dynamically typed, programming language that aims to solve this two-language problem. It gained attention because of its modern and intuitive design, while still being highly optimized to compete with "low-level" languages. Recently, some chemistry-related projects have emerged exploring the capabilities of Julia. Herein, we introduce the quantum chemistry package Fermi.jl, which contains the first implementations of post-Hartree-Fock methods written in Julia. Its design makes use of many Julia core features, including multiple dispatch, metaprogramming, and interactive usage. Fermi.jl is a modular package, where new methods and implementations can be easily added to the existing code. Furthermore, it is designed to maximize code reusability by relying on general functions with specialized methods for particular cases. The feasibility of the project is explored through evaluating the performance of popular ab initio methods. It is our hope that this project motivates the usage of Julia within the community and brings new contributions into Fermi.jl.

Resolving Chemical Dynamics in Biological Energy Conversion: Long-Range Proton-Coupled Electron Transfer in Respiratory Complex I

Biological energy conversion is catalyzed by membrane-bound proteins that transduce chemical or light energy into energy forms that power endergonic processes in the cell. At a molecular level, these catalytic processes involve elementary electron-, proton-, charge-, and energy-transfer reactions that take place in the intricate molecular machineries of cell respiration and photosynthesis. Recent developments in structural biology, particularly cryo-electron microscopy (cryoEM), have resolved the molecular architecture of several energy transducing proteins, but detailed mechanistic principles of their charge transfer reactions still remain poorly understood and a major challenge for modern biochemical research. To this end, multiscale molecular simulations provide a powerful approach to probe mechanistic principles on a broad range of time scales (femtoseconds to milliseconds) and spatial resolutions (10¹–10⁶ atoms), although technical challenges also require balancing between the computational accuracy, cost, and approximations introduced within the model. Here we discuss how the combination of atomistic (aMD) and hybrid quantum/classical molecular dynamics (QM/MM MD) simulations with free energy (FE) sampling methods can be used to probe mechanistic principles of enzymes responsible for biological energy conversion. We present mechanistic explorations of long-range proton-coupled electron transfer (PCET) dynamics in the highly intricate respiratory chain enzyme Complex I, which functions as a redox-driven proton pump in bacterial and mitochondrial respiratory chains by catalyzing a 300 Å fully reversible PCET process. This process is initiated by a hydride (H^–) transfer between NADH and FMN, followed by long-range (>100 Å) electron transfer along a wire of 8 FeS centers leading to a quinone biding site. The reduction of the quinone to quinol initiates dissociation of the latter to a second membrane-bound binding site, and triggers proton pumping across the membrane domain of complex I, in subunits up to 200 Å away from the active site. Our simulations across different size and time scales suggest that transient charge transfer reactions lead to changes in the internal hydration state of key regions, local electric fields, and the conformation of conserved ion pairs, which in turn modulate the dynamics of functional steps along the reaction cycle. Similar functional principles, which operate on much shorter length scales, are also found in some unrelated proteins, suggesting that enzymes may employ conserved principles in the catalysis of biological energy transduction processes.

Efficient low-scaling computation of NMR shieldings at the second-order Møller-Plesset perturbation theory level with Cholesky-decomposed densities and an attenuated Coulomb metric

A method for the computation of nuclear magnetic resonance (NMR) shieldings with second-order Møller-Plesset perturbation theory (MP2) is presented which allows to efficiently compute the entire set of shieldings for a given molecular structure. The equations are derived using Laplace-transformed atomic orbital second-order Møller-Plesset perturbation theory as a starting point. The Z-vector approach is employed for minimizing the number of coupled-perturbed self-consistent-field equations that need to be solved. In addition, the method uses the resolution-of-the-identity approximation with an attenuated Coulomb metric and Cholesky decomposition of pseudo-density matrices. The sparsity in the three-center integrals is exploited with sparse linear algebra approaches, leading to reduced computational cost and memory demands. Test calculations show that the deviations from NMR shifts obtained with canonical MP2 are small if appropriate thresholds are used. The performance of the method is illustrated in calculations on DNA strands and on glycine chains with up to 283 atoms and 2864 basis functions.

Faster Self-Consistent Field (SCF) Calculations on GPU Clusters

A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of one-electron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.

An improved chain of spheres for exchange algorithm

In the present work, we describe a more accurate and efficient variant of the chain-of-spheres algorithm (COSX) for exchange matrix computations. Higher accuracy for the numerical integration is obtained with new grids that were developed using global optimization techniques. With our new default grids, the average absolute energy errors are much lower than 0.1 kcal/mol, which is desirable to achieve "chemical accuracy." Although the size of the new grids is increased by roughly a factor of 2.5, the excellent efficiency of the original COSX implementation is still further improved in most cases. The evaluation of the analytic electrostatic potential integrals was significantly accelerated by a new implementation of rolled-out versions of the Dupuis-Rys-King and Head-Gordon-Pople algorithms. Compared to our earlier implementation, a twofold speedup is obtained for the frequently used triple-ζ basis sets, while up to a 16-fold speedup is observed for quadruple-ζ basis sets. These large gains are a consequence of both the more efficient integral evaluation and the intermediate exchange matrix computation in a partially contracted basis when generally contracted shells occur. With our new RIJCOSX implementation, we facilitate accurate self-consistent field (SCF) binding energy calculations on a large supra-molecular complex composed of 320 atoms. The binding-energy errors with respect to the fully analytic results are well below 0.1 kcal/mol for the cc-pV(T/Q)Z basis sets and even smaller than for RIJ with fully analytic exchange. At the same time, our RIJCOSX SCF calculation even with the cc-pVQZ basis and the finest grid is 21 times faster than the fully analytic calculation.

Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program

We report a new multi-GPU capable ab initio Hartree-Fock/density functional theory implementation integrated into the open source QUantum Interaction Computational Kernel (QUICK) program. Details on the load balancing algorithms for electron repulsion integrals and exchange correlation quadrature across multiple GPUs are described. Benchmarking studies carried out on up to four GPU nodes, each containing four NVIDIA V100-SXM2 type GPUs demonstrate that our implementation is capable of achieving excellent load balancing and high parallel efficiency. For representative medium to large size protein/organic molecular systems, the observed parallel efficiencies remained above 82% for the Kohn-Sham matrix formation and above 90% for nuclear gradient calculations. The accelerations on NVIDIA A100, P100, and K80 platforms also have realized parallel efficiencies higher than 68% in all tested cases, paving the way for large-scale ab initio electronic structure calculations with QUICK.

Accelerating seminumerical Fock-exchange calculations using mixed single- and double-precision arithmethic

We investigate the applicability of single-precision (fp32) floating point operations within our linear-scaling, seminumerical exchange method sn-LinK [Laqua et al., J. Chem. Theory Comput. 16, 1456 (2020)] and find that the vast majority of the three-center-one-electron (3c1e) integrals can be computed with reduced numerical precision with virtually no loss in overall accuracy. This leads to a near doubling in performance on central processing units (CPUs) compared to pure fp64 evaluation. Since the cost of evaluating the 3c1e integrals is less significant on graphic processing units (GPUs) compared to CPU, the performance gains from accelerating 3c1e integrals alone is less impressive on GPUs. Therefore, we also investigate the possibility of employing only fp32 operations to evaluate the exchange matrix within the self-consistent-field (SCF) followed by an accurate one-shot evaluation of the exchange energy using mixed fp32/fp64 precision. This still provides very accurate (1.8 µEh maximal error) results while providing a sevenfold speedup on a typical "gaming" GPU (GTX 1080Ti). We also propose the use of incremental exchange-builds to further reduce these errors. The proposed SCF scheme (i-sn-LinK) requires only one mixed-precision exchange matrix calculation, while all other exchange-matrix builds are performed with only fp32 operations. Compared to pure fp64 evaluation, this leads to 4-7× speedups for the whole SCF procedure without any significant deterioration of the results or the convergence behavior.

Assessing locally range-separated hybrid functionals from a gradient expansion of the exchange energy density

Locally range-separated hybrid (LRSH) functionals feature a real-space-dependent range separation function (RSF) instead of a system-independent range-separation parameter, which thus enables a more flexible admixture of exact exchange than conventional range-separated hybrid functionals. In particular, the development of suitable RSF models and exploring the capabilities of the LRSH approach, in general, are tasks that require further investigations and will be addressed in this work. We propose a non-empirical scheme based on a detailed scaling analysis with respect to a uniform coordinate scaling and on a short-range expansion of the range-separated exchange energy density to derive new RSF models from a gradient expansion of the exchange energy density. After optimizing a small set of empirical parameters introduced to enhance their flexibility, the resulting second- and fourth-order RSFs are evaluated with respect to atomic exchange energies, atomization energies, and transition barrier heights.

Replacing hybrid density functional theory: motivation and recent advances

Density functional theory (DFT) is the most widely-used electronic structure approximation across chemistry, physics, and materials science. Every year, thousands of papers report hybrid DFT simulations of chemical structures, mechanisms, and spectra. Unfortunately, hybrid DFT's accuracy is ultimately limited by tradeoffs between over-delocalization and under-binding. This review summarizes these tradeoffs, and introduces six modern attempts to go beyond them while maintaining hybrid DFT's relatively low computational cost: DFT+U, self-interaction corrections, localized orbital scaling corrections, local hybrid functionals, real-space nondynamical correlation, and our rung-3.5 approach. The review concludes with practical suggestions for DFT users to identify and mitigate these tradeoffs' impact on their simulations.

Open-Source Multi-GPU-Accelerated QM/MM Simulations with AMBER and QUICK

The quantum mechanics/molecular mechanics (QM/MM) approach is an essential and well-established tool in computational chemistry that has been widely applied in a myriad of biomolecular problems in the literature. In this publication, we report the integration of the QUantum Interaction Computational Kernel (QUICK) program as an engine to perform electronic structure calculations in QM/MM simulations with AMBER. This integration is available through either a file-based interface (FBI) or an application programming interface (API). Since QUICK is an open-source GPU-accelerated code with multi-GPU parallelization, users can take advantage of "free of charge" GPU-acceleration in their QM/MM simulations. In this work, we discuss implementation details and give usage examples. We also investigate energy conservation in typical QM/MM simulations performed at the microcanonical ensemble. Finally, benchmark results for two representative systems in bulk water, the N-methylacetamide (NMA) molecule and the photoactive yellow protein (PYP), show the performance of QM/MM simulations with QUICK and AMBER using a varying number of CPU cores and GPUs. Our results highlight the acceleration obtained from a single or multiple GPUs; we observed speedups of up to 53× between a single GPU vs a single CPU core and of up to 2.6× when comparing four GPUs to a single GPU. Results also reveal speedups of up to 3.5× when the API is used instead of FBI.

Speeding up Hartree-Fock and Kohn-Sham calculations with first-order corrections

Several approaches are presented to improve the efficiency of Hartree-Fock and Kohn-Sham self-consistent field (SCF) calculations relying on a simple first-order energy correction reminiscent of the scheme used in dual-basis SCF methods. The basic idea is to perform an initial SCF calculation computing approximate Fock-matrices and, in the final iteration step, to use a more complete Fock-matrix builder together with the energy correction to diminish the error. The approximation is tested for conventional and local density fitting (DF) SCF approaches combining various auxiliary basis sets, fitting metrics, and Fock-matrix construction algorithms in the initial and final iterations as well as for seminumerical SCF methods combining integration grids of different qualities. We also report the implementation of the occupied orbital resolution of identity exchange construction algorithm with local DF approximations. Benchmark calculations are presented for total energies, reaction energies, and molecular geometries. Our results show that speedups of up to 80% can be expected utilizing the new approaches without significant loss of accuracy.

From NWChem to NWChemEx: Evolving with the Computational Chemistry Landscape

Since the advent of the first computers, chemists have been at the forefront of using computers to understand and solve complex chemical problems. As the hardware and software have evolved, so have the theoretical and computational chemistry methods and algorithms. Parallel computers clearly changed the common computing paradigm in the late 1970s and 80s, and the field has again seen a paradigm shift with the advent of graphical processing units. This review explores the challenges and some of the solutions in transforming software from the terascale to the petascale and now to the upcoming exascale computers. While discussing the field in general, NWChem and its redesign, NWChemEx, will be highlighted as one of the early codesign projects to take advantage of massively parallel computers and emerging software standards to enable large scientific challenges to be tackled.

Low-Scaling Tensor Hypercontraction in the Cholesky Molecular Orbital Basis Applied to Second-Order Møller-Plesset Perturbation Theory

We employ various reduced scaling techniques to accelerate the recently developed least-squares tensor hypercontraction (LS-THC) approximation [Parrish, R. M., Hohenstein, E. G., Martínez, T. J., Sherrill, C. D. J. Chem. Phys. 137, 224106 (2012)] for electron repulsion integrals (ERIs) and apply it to second-order Møller-Plesset perturbation theory (MP2). The grid-projected ERI tensors are efficiently constructed using a localized Cholesky molecular orbital basis from density-fitted integrals with an attenuated Coulomb metric. Additionally, rigorous integral screening and the natural blocking matrix format are applied to reduce the complexity of this step. By recasting the equations to form the quantized representation of the 1/r operator Z into the form of a system of linear equations, the bottleneck of inverting the grid metric via pseudoinversion is removed. This leads to a reduced scaling THC algorithm and application to MP2 yields the (sub-)quadratically scaling THC-ω-RI-CDD-SOS-MP2 method. The efficiency of this method is assessed for various systems including DNA fragments with over 8000 basis functions and the subquadratic scaling is illustrated.

On the Efficient Evaluation of the Exchange Correlation Potential on Graphics Processing Unit Clusters

The predominance of Kohn–Sham density functional theory (KS-DFT) for the theoretical treatment of large experimentally relevant systems in molecular chemistry and materials science relies primarily on the existence of efficient software implementations which are capable of leveraging the latest advances in modern high-performance computing (HPC). With recent trends in HPC leading toward increasing reliance on heterogeneous accelerator-based architectures such as graphics processing units (GPU), existing code bases must embrace these architectural advances to maintain the high levels of performance that have come to be expected for these methods. In this work, we purpose a three-level parallelism scheme for the distributed numerical integration of the exchange-correlation (XC) potential in the Gaussian basis set discretization of the Kohn–Sham equations on large computing clusters consisting of multiple GPUs per compute node. In addition, we purpose and demonstrate the efficacy of the use of batched kernels, including batched level-3 BLAS operations, in achieving high levels of performance on the GPU. We demonstrate the performance and scalability of the implementation of the purposed method in the NWChemEx software package by comparing to the existing scalable CPU XC integration in NWChem.