Calculations for millions of atoms with density functional theory: linear scaling shows its potential

Machine learning of kinetic energy densities with target and feature smoothing: Better results with fewer training data

Machine learning (ML) of kinetic energy functionals (KEFs), in particular kinetic energy density (KED) functionals, is a promising way to construct KEFs for orbital-free density functional theory (DFT). Neural networks and kernel methods including Gaussian process regression (GPR) have been used to learn Kohn-Sham (KS) KED from density-based descriptors derived from KS DFT calculations. The descriptors are typically expressed as functions of different powers and derivatives of the electron density. This can generate large and extremely unevenly distributed datasets, which complicates effective application of ML techniques. Very uneven data distributions require many training datapoints, can cause overfitting, and can ultimately lower the quality of an ML KED model. We show that one can produce more accurate ML models from fewer data by working with smoothed density-dependent variables and KED. Smoothing palliates the issue of very uneven data distributions and associated difficulties of sampling while retaining enough spatial structure necessary for working within the paradigm of KEDF. We use GPR as a function of smoothed terms of the fourth order gradient expansion and KS effective potential and obtain accurate and stable (with respect to different random choices of training points) kinetic energy models for Al, Mg, and Si simultaneously from as few as 2000 samples (about 0.3% of the total KS DFT data). In particular, accuracies on the order of 1% in a measure of the quality of energy-volume dependence B'=EV0-ΔV-2EV0+E(V0+ΔV)ΔV/V02 (where V0 is the equilibrium volume and ΔV is a deviation from it) are obtained simultaneously for all three materials.

PyConSolv: A Python Package for Conformer Generation of (Metal-Containing) Systems in Explicit Solvent

We introduce PyConSolv, a freely available Python package that automates the generation of conformers of metal- and nonmetal-containing complexes in explicit solvent, through classical molecular dynamics simulations. Using a streamlined workflow and interfacing with widely used computational chemistry software, PyConSolv is an all-in-one tool for the generation of conformers in any solvent. Input requirements are minimal; only the geometry of the structure and the desired solvent in xyz (XMOL) format are needed. The package can also account for charged systems, by including arbitrary counterions in the simulation. A bonded model parametrization is performed automatically, utilizing AmberTools, ORCA, and Multiwfn software packages. PyConSolv provides a selection of preparametrized solvents and counterions for use in classical molecular dynamics simulations. We show the applicability of our package on a number of (transition-metal-containing) systems. The software is provided open source and free of charge.

Exact exchange with range-separated algorithm for thermodynamic limit of periodic Hartree-Fock theory

The expensive cost of computing exact exchange in periodic systems limits the application range of density functional theory with hybrid functionals. To reduce the computational cost of exact change, we present a range-separated algorithm to compute electron repulsion integrals for Gaussian-type crystal basis. The algorithm splits the full-range Coulomb interactions into short-range and long-range parts, which are, respectively, computed in real and reciprocal space. This approach significantly reduces the overall computational cost, as integrals can be efficiently computed in both regions. The algorithm can efficiently handle large numbers of k points with limited central processing unit (CPU) and memory resources. As a demonstration, we performed an all-electron k-point Hartree-Fock calculation for LiH crystal with one million Gaussian basis functions, which was completed on a desktop computer in 1400 CPU hours.

Hybrid Density Functional Tight Binding (DFTB)─Molecular Mechanics Approach for a Low-Cost Expansion of DFTB Applicability

The density functional-based tight binding (DFTB) method has seen a rise in adoption for materials modeling, as it offers significant improvement in scalability with accuracy comparable to the density functional theory (DFT) when good parameterizations exist. The cost reduction in DFTB compared to DFT is achieved by the pre-parameterization of the elements of the Hamiltonian matrix as well as the repulsion potential between all pairs of atoms. Parameterization for new systems with accuracies competitive with DFT in specific applications requires specialized manpower and computational resources. This prevents the application of the DFTB method to systems for which it was not parameterized. In this work, we explore an approach to address the problem of missing parameters of DFTB by modeling the interactions with missing Slater-Koster parameters with an interatomic interaction potential. When the distance between two atoms modeled at the force-field level is sufficiently large, the approach results in accurate structural and electronic properties. The resulting calculation is therefore a hybrid between DFTB and molecular mechanics, a pure DFTB for atoms with a complete set of interatomic parameterizations, and a mix between DFTB and molecular mechanics for atoms with a missing interatomic parameterization. The approach is expected to be particularly useful for hybrid materials and interfaces. The method is tested on the examples of 2D materials, mixed oxides, and a large-scale calculation of an oxide-oxide interface.

Emerging Trends of Computational Chemistry and Molecular Modeling in Froth Flotation: A Review

Froth flotation is the most versatile process in mineral beneficiation, extensively used to concentrate a wide range of minerals. This process comprises mixtures of more or less liberated minerals, water, air, and various chemical reagents, involving a series of intermingled multiphase physical and chemical phenomena in the aqueous environment. Today's main challenge facing the froth flotation process is to gain atomic-level insights into the properties of its inherent phenomena governing the process performance. While it is often challenging to determine these phenomena via trial-and-error experimentations, molecular modeling approaches not only elicit a deeper understanding of froth flotation but can also assist experimental studies in saving time and budget. Thanks to the rapid development of computer science and advances in high-performance computing (HPC) infrastructures, theoretical/computational chemistry has now matured enough to successfully and gainfully apply to tackle the challenges of complex systems. In mineral processing, however, advanced applications of computational chemistry are increasingly gaining ground and demonstrating merit in addressing these challenges. Accordingly, this contribution aims to encourage mineral scientists, especially those interested in rational reagent design, to become familiarized with the necessary concepts of molecular modeling and to apply similar strategies when studying and tailoring properties at the molecular level. This review also strives to deliver the state-of-the-art integration and application of molecular modeling in froth flotation studies to assist either active researchers in this field to disclose new directions for future research or newcomers to the field to initiate innovative works.

Gaussian-Approximated Poisson Preconditioner for Iterative Diagonalization in Real-Space Density Functional Theory

Various real-space methods optimized on massive parallel computers have been developed for efficient large-scale density functional theory (DFT) calculations of materials and biomolecules. The iterative diagonalization of the Hamiltonian matrix is a computational bottleneck in real-space DFT calculations. Despite the development of various iterative eigensolvers, the absence of efficient real-space preconditioners has hindered their overall efficiency. An efficient preconditioner must satisfy two conditions: appropriate acceleration of the convergence of the iterative process and inexpensive computation. This study proposed a Gaussian-approximated Poisson preconditioner (GAPP) that satisfied both conditions and was suitable for real-space methods. A low computational cost was realized through the Gaussian approximation of a Poisson Green's function. Fast convergence was achieved through the proper determination of Gaussian coefficients to fit the Coulomb energies. The performance of GAPP was evaluated for several molecular and extended systems, and it showed the highest efficiency among the existing preconditioners adopted in real-space codes.

Modular implementation of the linear- and cubic-scaling orbital minimization methods in electronic structure codes using atomic orbitals

We present a code modularization approach to design efficient and massively parallel cubic- and linear-scaling solvers for electronic structure calculations using atomic orbitals. The modular implementation of the orbital minimization method, in which linear algebra and parallelization issues are handled via external libraries, is demonstrated in the SIESTA code. The distributed block compressed sparse row (DBCSR) and scalable linear algebra package (ScaLAPACK) libraries are used for algebraic operations with sparse and dense matrices, respectively. The MatrixSwitch and libOMM libraries, recently developed within the Electronic Structure Library, facilitate switching between different matrix formats and implement the energy minimization. We show results comparing the performance of several cubic-scaling algorithms, and also demonstrate the parallel performance of the linear-scaling solvers, and their supremacy over the cubic-scaling solvers for insulating systems with sizes of several hundreds of atoms.

Accelerating the density-functional tight-binding method using graphical processing units

Acceleration of the density-functional tight-binding (DFTB) method on single and multiple graphical processing units (GPUs) was accomplished using the MAGMA linear algebra library. Two major computational bottlenecks of DFTB ground-state calculations were addressed in our implementation: the Hamiltonian matrix diagonalization and the density matrix construction. The code was implemented and benchmarked on two different computer systems: (1) the SUMMIT IBM Power9 supercomputer at the Oak Ridge National Laboratory Leadership Computing Facility with 1-6 NVIDIA Volta V100 GPUs per computer node and (2) an in-house Intel Xeon computer with 1-2 NVIDIA Tesla P100 GPUs. The performance and parallel scalability were measured for three molecular models of 1-, 2-, and 3-dimensional chemical systems, represented by carbon nanotubes, covalent organic frameworks, and water clusters.

Large Scale Quantum Chemistry with Tensor Processing Units

We demonstrate the use of Googles cloud-based Tensor Processing Units (TPUs) to accelerate and scale up conventional (cubic-scaling) density functional theory (DFT) calculations. Utilizing 512 TPU cores, we accomplish the largest such DFT computation to date, with 247848 orbitals, corresponding to a cluster of 10327 water molecules with 103270 electrons, all treated explicitly. Our work thus paves the way toward accessible and systematic use of conventional DFT, free of any system-specific constraints, at unprecedented scales.

Machine Learning of Coupled Cluster (T)-Energy Corrections via Delta (Δ)-Learning

Accurate thermochemistry is essential in many chemical disciplines, such as astro-, atmospheric, or combustion chemistry. These areas often involve fleetingly existent intermediates whose thermochemistry is difficult to assess. Whenever direct calorimetric experiments are infeasible, accurate computational estimates of relative molecular energies are required. However, high-level computations, often using coupled cluster theory, are generally resource-intensive. To expedite the process using machine learning techniques, we generated a database of energies for small organic molecules at the CCSD(T)/cc-pVDZ, CCSD(T)/aug-cc-pVDZ, and CCSD(T)/cc-pVTZ levels of theory. Leveraging the power of deep learning by employing graph neural networks, we are able to predict the effect of perturbatively included triples (T), that is, the difference between CCSD and CCSD(T) energies, with a mean absolute error of 0.25, 0.25, and 0.28 kcal mol^-1 (R² of 0.998, 0.997, and 0.998) with the cc-pVDZ, aug-cc-pVDZ, and cc-pVTZ basis sets, respectively. Our models were further validated by application to three validation sets taken from the S22 Database as well as to a selection of known theoretically challenging cases.

The Middle Science: Traversing Scale In Complex Many-Body Systems

A roadmap is developed that integrates simulation methodology and data science methods to target new theories that traverse the multiple length- and time-scale features of many-body phenomena.

Deep-Learning Approach to First-Principles Transport Simulations

Large-scale first-principles transport calculations, while essential for device modeling, remain computationally demanding. To overcome this bottle neck, we combine first-principles transport calculations with machine learning-based nonlinear regression. We calculate the electronic conductance through first-principles based nonequilibrium Green's function techniques for small systems and map the transport properties onto local properties using local descriptors. We show that using the local descriptor as input features for deep learning-based nonlinear regression allows us to build a robust neural network that can predict the conductance of large systems beyond that of the current state-of-the-art first-principles calculation algorithms. Our protocol is applied to alkali metal nanowires, i.e., potassium, which have unique geometrical and electronic properties and hence nontrivial transport properties. We demonstrate that within our approach we can achieve qualitative agreement with experiment at a fraction of the computational effort as compared to the direct calculation of the transport properties using conventional first-principles methods.

From NWChem to NWChemEx: Evolving with the Computational Chemistry Landscape

Since the advent of the first computers, chemists have been at the forefront of using computers to understand and solve complex chemical problems. As the hardware and software have evolved, so have the theoretical and computational chemistry methods and algorithms. Parallel computers clearly changed the common computing paradigm in the late 1970s and 80s, and the field has again seen a paradigm shift with the advent of graphical processing units. This review explores the challenges and some of the solutions in transforming software from the terascale to the petascale and now to the upcoming exascale computers. While discussing the field in general, NWChem and its redesign, NWChemEx, will be highlighted as one of the early codesign projects to take advantage of massively parallel computers and emerging software standards to enable large scientific challenges to be tackled.

Computational Methods in Heterogeneous Catalysis

The unprecedented ability of computations to probe atomic-level details of catalytic systems holds immense promise for the fundamentals-based bottom-up design of novel heterogeneous catalysts, which are at the heart of the chemical and energy sectors of industry. Here, we critically analyze recent advances in computational heterogeneous catalysis. First, we will survey the progress in electronic structure methods and atomistic catalyst models employed, which have enabled the catalysis community to build increasingly intricate, realistic, and accurate models of the active sites of supported transition-metal catalysts. We then review developments in microkinetic modeling, specifically mean-field microkinetic models and kinetic Monte Carlo simulations, which bridge the gap between nanoscale computational insights and macroscale experimental kinetics data with increasing fidelity. We finally review the advancements in theoretical methods for accelerating catalyst design and discovery. Throughout the review, we provide ample examples of applications, discuss remaining challenges, and provide our outlook for the near future.

GPU algorithms for density matrix methods on MOPAC: linear scaling electronic structure calculations for large molecular systems

Purification of the density matrix methods should be employed when dealing with complex chemical systems containing many atoms. The running times for these methods scale linearly with the number of atoms if we consider the sparsity from the density matrix. Since the efficiency expected from those methods is closely tied to the underlying parallel implementations of the linear algebra operations (e.g., P² = P × P), we proposed a central processing unit (CPU) and graphics processing unit (GPU) parallel matrix-matrix multiplication in SVBR (symmetrical variable block row) format for energy calculations through the SP2 algorithm. This algorithm was inserted in MOPAC's MOZYME method, using the original LMO Fock matrix assembly, and the atomic integral calculation implemented on it. Correctness and performance tests show that the implemented SP2 is accurate and fast, as the GPU is able to achieve speedups up to 40 times for a water cluster system with 42,312 orbitals running in one NVIDIA K40 GPU card compared to the single-threaded version. The GPU-accelerated SP2 algorithm using the MOZYME LMO framework enables the calculations of semiempirical wavefunction with stricter SCF criteria for localized charged molecular systems, as well as the single-point energies of molecules with more than 100.000 LMO orbitals in less than 1 h. Graphical abstract Parallel CPU and GPU purification algorithms for electronic structure calculations were implemented in MOPAC's MOZYME method. Some matrices in these calculations, e.g., electron density P, are compressed, and the developed linear algebra operations deal with non-zero entries only. We employed the NVIDIA/CUDA platform to develop GPU algorithms, and accelerations up to 40 times for larger systems were achieved.

Machine Learning Adaptive Basis Sets for Efficient Large Scale Density Functional Theory Simulation

It is chemically intuitive that an optimal atom centered basis set must adapt to its atomic environment, for example by polarizing toward nearby atoms. Adaptive basis sets of small size can be significantly more accurate than traditional atom centered basis sets of the same size. The small size and well conditioned nature of these basis sets leads to large saving in computational cost, in particular in a linear scaling framework. Here, it is shown that machine learning can be used to predict such adaptive basis sets using local geometrical information only. As a result, various properties of standard DFT calculations can be easily obtained at much lower costs, including nuclear gradients. In our approach, a rotationally invariant parametrization of the basis is obtained by employing a potential anchored on neighboring atoms to ultimately construct a rotation matrix that turns a traditional atom centered basis set into a suitable adaptive basis set. The method is demonstrated using MD simulations of liquid water, where it is shown that minimal basis sets yield structural properties in fair agreement with basis set converged results, while reducing the computational cost in the best case by a factor of 200 and the required flops by 4 orders of magnitude. Already a very small training set yields satisfactory results as the variational nature of the method provides robustness.

Two-Level Chebyshev Filter Based Complementary Subspace Method: Pushing the Envelope of Large-Scale Electronic Structure Calculations

We describe a novel iterative strategy for Kohn-Sham density functional theory calculations aimed at large systems (>1,000 electrons), applicable to metals and insulators alike. In lieu of explicit diagonalization of the Kohn-Sham Hamiltonian on every self-consistent field (SCF) iteration, we employ a two-level Chebyshev polynomial filter based complementary subspace strategy to (1) compute a set of vectors that span the occupied subspace of the Hamiltonian; (2) reduce subspace diagonalization to just partially occupied states; and (3) obtain those states in an efficient, scalable manner via an inner Chebyshev filter iteration. By reducing the necessary computation to just partially occupied states and obtaining these through an inner Chebyshev iteration, our approach reduces the cost of large metallic calculations significantly, while eliminating subspace diagonalization for insulating systems altogether. We describe the implementation of the method within the framework of the discontinuous Galerkin (DG) electronic structure method and show that this results in a computational scheme that can effectively tackle bulk and nano systems containing tens of thousands of electrons, with chemical accuracy, within a few minutes or less of wall clock time per SCF iteration on large-scale computing platforms. We anticipate that our method will be instrumental in pushing the envelope of large-scale ab initio molecular dynamics. As a demonstration of this, we simulate a bulk silicon system containing 8,000 atoms at finite temperature, and obtain an average SCF step wall time of 51 s on 34,560 processors; thus allowing us to carry out 1.0 ps of ab initio molecular dynamics in approximately 28 h (of wall time).

Efficient Computation of Sparse Matrix Functions for Large-Scale Electronic Structure Calculations: The CheSS Library

We present CheSS, the "Chebyshev Sparse Solvers" library, which has been designed to solve typical problems arising in large-scale electronic structure calculations using localized basis sets. The library is based on a flexible and efficient expansion in terms of Chebyshev polynomials and presently features the calculation of the density matrix, the calculation of matrix powers for arbitrary powers, and the extraction of eigenvalues in a selected interval. CheSS is able to exploit the sparsity of the matrices and scales linearly with respect to the number of nonzero entries, making it well-suited for large-scale calculations. The approach is particularly adapted for setups leading to small spectral widths of the involved matrices and outperforms alternative methods in this regime. By coupling CheSS to the DFT code BigDFT, we show that such a favorable setup is indeed possible in practice. In addition, the approach based on Chebyshev polynomials can be massively parallelized, and CheSS exhibits excellent scaling up to thousands of cores even for relatively small matrix sizes.

Quantum crystallographic charge density of urea

Standard X-ray crystallography methods use free-atom models to calculate mean unit-cell charge densities. Real molecules, however, have shared charge that is not captured accurately using free-atom models. To address this limitation, a charge density model of crystalline urea was calculated using high-level quantum theory and was refined against publicly available ultra-high-resolution experimental Bragg data, including the effects of atomic displacement parameters. The resulting quantum crystallographic model was compared with models obtained using spherical atom or multipole methods. Despite using only the same number of free parameters as the spherical atom model, the agreement of the quantum model with the data is comparable to the multipole model. The static, theoretical crystalline charge density of the quantum model is distinct from the multipole model, indicating the quantum model provides substantially new information. Hydrogen thermal ellipsoids in the quantum model were very similar to those obtained using neutron crystallography, indicating that quantum crystallography can increase the accuracy of the X-ray crystallographic atomic displacement parameters. The results demonstrate the feasibility and benefits of integrating fully periodic quantum charge density calculations into ultra-high-resolution X-ray crystallographic model building and refinement.

Canonical density matrix perturbation theory

Density matrix perturbation theory [Niklasson and Challacombe, Phys. Rev. Lett. 92, 193001 (2004)] is generalized to canonical (NVT) free-energy ensembles in tight-binding, Hartree-Fock, or Kohn-Sham density-functional theory. The canonical density matrix perturbation theory can be used to calculate temperature-dependent response properties from the coupled perturbed self-consistent field equations as in density-functional perturbation theory. The method is well suited to take advantage of sparse matrix algebra to achieve linear scaling complexity in the computational cost as a function of system size for sufficiently large nonmetallic materials and metals at high temperatures.

Linear scaling density matrix real time TDDFT: Propagator unitarity and matrix truncation

Real time, density matrix based, time dependent density functional theory (TDDFT) proceeds through the propagation of the density matrix, as opposed to the Kohn-Sham orbitals. It is possible to reduce the computational workload by imposing spatial cutoff radii on sparse matrices, and the propagation of the density matrix in this manner provides direct access to the optical response of very large systems, which would be otherwise impractical to obtain using the standard formulations of TDDFT. Following a brief summary of our implementation, along with several benchmark tests illustrating the validity of the method, we present an exploration of the factors affecting the accuracy of the approach. In particular, we investigate the effect of basis set size and matrix truncation, the key approximation used in achieving linear scaling, on the propagator unitarity and optical spectra. Finally, we illustrate that, with an appropriate density matrix truncation range applied, the computational load scales linearly with the system size and discuss the limitations of the approach.

Recent advances in QM/MM free energy calculations using reference potentials

Background

Recent years have seen enormous progress in the development of methods for modeling (bio)molecular systems. This has allowed for the simulation of ever larger and more complex systems. However, as such complexity increases, the requirements needed for these models to be accurate and physically meaningful become more and more difficult to fulfill. The use of simplified models to describe complex biological systems has long been shown to be an effective way to overcome some of the limitations associated with this computational cost in a rational way.

Scope of review

Hybrid QM/MM approaches have rapidly become one of the most popular computational tools for studying chemical reactivity in biomolecular systems. However, the high cost involved in performing high-level QM calculations has limited the applicability of these approaches when calculating free energies of chemical processes. In this review, we present some of the advances in using reference potentials and mean field approximations to accelerate high-level QM/MM calculations. We present illustrative applications of these approaches and discuss challenges and future perspectives for the field.

Major conclusions

The use of physically-based simplifications has shown to effectively reduce the cost of high-level QM/MM calculations. In particular, lower-level reference potentials enable one to reduce the cost of expensive free energy calculations, thus expanding the scope of problems that can be addressed.

General significance

As was already demonstrated 40 years ago, the usage of simplified models still allows one to obtain cutting edge results with substantially reduced computational cost. This article is part of a Special Issue entitled Recent developments of molecular dynamics.

•

We present some of the advances to accelerate high-level QM/MM calculations.

•

Quantitative limitations of low-level methods can be overcome by these approaches.

•

Reference potentials make free energy simulations feasible for large systems.

•

Automated fitting reduces the need of expensive sampling of high-level approaches.

•

Application of reference potentials can be extended to a wide range of processes.

Performance analysis of electronic structure codes on HPC systems: a case study of SIESTA

We report on scaling and timing tests of the SIESTA electronic structure code for ab initio molecular dynamics simulations using density-functional theory. The tests are performed on six large-scale supercomputers belonging to the PRACE Tier-0 network with four different architectures: Cray XE6, IBM BlueGene/Q, BullX, and IBM iDataPlex. We employ a systematic strategy for simultaneously testing weak and strong scaling, and propose a measure which is independent of the range of number of cores on which the tests are performed to quantify strong scaling efficiency as a function of simulation size. We find an increase in efficiency with simulation size for all machines, with a qualitatively different curve depending on the supercomputer topology, and discuss the connection of this functional form with weak scaling behaviour. We also analyze the absolute timings obtained in our tests, showing the range of system sizes and cores favourable for different machines. Our results can be employed as a guide both for running SIESTA on parallel architectures, and for executing similar scaling tests of other electronic structure codes.

Density-functional theory study of gramicidin A ion channel geometry and electronic properties

Understanding the mechanisms underlying ion channel function from the atomic-scale requires accurate ab initio modelling as well as careful experiments. Here, we present a density functional theory (DFT) study of the ion channel gramicidin A (gA), whose inner pore conducts only monovalent cations and whose conductance has been shown to depend on the side chains of the amino acids in the channel. We investigate the ground state geometry and electronic properties of the channel in vacuum, focusing on their dependence on the side chains of the amino acids. We find that the side chains affect the ground state geometry, while the electrostatic potential of the pore is independent of the side chains. This study is also in preparation for a full, linear scaling DFT study of gA in a lipid bilayer with surrounding water. We demonstrate that linear scaling DFT methods can accurately model the system with reasonable computational cost. Linear scaling DFT allows ab initio calculations with 10,000-100,000 atoms and beyond, and will be an important new tool for biomolecular simulations.

O(N) methods in electronic structure calculations

Linear-scaling methods, or O(N) methods, have computational and memory requirements which scale linearly with the number of atoms in the system, N, in contrast to standard approaches which scale with the cube of the number of atoms. These methods, which rely on the short-ranged nature of electronic structure, will allow accurate, ab initio simulations of systems of unprecedented size. The theory behind the locality of electronic structure is described and related to physical properties of systems to be modelled, along with a survey of recent developments in real-space methods which are important for efficient use of high-performance computers. The linear-scaling methods proposed to date can be divided into seven different areas, and the applicability, efficiency and advantages of the methods proposed in these areas are then discussed. The applications of linear-scaling methods, as well as the implementations available as computer programs, are considered. Finally, the prospects for and the challenges facing linear-scaling methods are discussed.