1
|
Di Felice R, Mayes ML, Richard RM, Williams-Young DB, Chan GKL, de Jong WA, Govind N, Head-Gordon M, Hermes MR, Kowalski K, Li X, Lischka H, Mueller KT, Mutlu E, Niklasson AMN, Pederson MR, Peng B, Shepard R, Valeev EF, van Schilfgaarde M, Vlaisavljevich B, Windus TL, Xantheas SS, Zhang X, Zimmerman PM. A Perspective on Sustainable Computational Chemistry Software Development and Integration. J Chem Theory Comput 2023; 19:7056-7076. [PMID: 37769271 PMCID: PMC10601486 DOI: 10.1021/acs.jctc.3c00419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Indexed: 09/30/2023]
Abstract
The power of quantum chemistry to predict the ground and excited state properties of complex chemical systems has driven the development of computational quantum chemistry software, integrating advances in theory, applied mathematics, and computer science. The emergence of new computational paradigms associated with exascale technologies also poses significant challenges that require a flexible forward strategy to take full advantage of existing and forthcoming computational resources. In this context, the sustainability and interoperability of computational chemistry software development are among the most pressing issues. In this perspective, we discuss software infrastructure needs and investments with an eye to fully utilize exascale resources and provide unique computational tools for next-generation science problems and scientific discoveries.
Collapse
Affiliation(s)
- Rosa Di Felice
- Departments
of Physics and Astronomy and Quantitative and Computational Biology, University of Southern California, Los Angeles, California 90089, United States
- CNR-NANO
Modena, Modena 41125, Italy
| | - Maricris L. Mayes
- Department
of Chemistry and Biochemistry, University
of Massachusetts Dartmouth, North Dartmouth, Massachusetts 02747, United States
| | | | | | - Garnet Kin-Lic Chan
- Division
of Chemistry and Chemical Engineering, California
Institute of Technology, Pasadena, California 91125, United States
| | - Wibe A. de Jong
- Lawrence
Berkeley National Laboratory, Berkeley, California 94720, United States
| | - Niranjan Govind
- Physical
Sciences Division, Pacific Northwest National
Laboratory, Richland, Washington 99354, United States
| | - Martin Head-Gordon
- Pitzer Center
for Theoretical Chemistry, Department of Chemistry, University of California, Berkeley, California 94720, United States
| | - Matthew R. Hermes
- Department
of Chemistry, Chicago Center for Theoretical Chemistry, University of Chicago, Chicago, Illinois 60637, United States
| | - Karol Kowalski
- Physical
Sciences Division, Pacific Northwest National
Laboratory, Richland, Washington 99354, United States
| | - Xiaosong Li
- Department
of Chemistry, University of Washington, Seattle, Washington 98195, United States
| | - Hans Lischka
- Department
of Chemistry and Biochemistry, Texas Tech
University, Lubbock, Texas 79409, United States
| | - Karl T. Mueller
- Physical
and Computational Sciences Directorate, Pacific Northwest National Laboratory, Richland, Washington 99354, United States
| | - Erdal Mutlu
- Advanced
Computing, Mathematics, and Data Division, Pacific Northwest National Laboratory, Richland, Washington 99354, United States
| | - Anders M. N. Niklasson
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Mark R. Pederson
- Department
of Physics, The University of Texas at El
Paso, El Paso, Texas 79968, United States
| | - Bo Peng
- Physical
Sciences Division, Pacific Northwest National
Laboratory, Richland, Washington 99354, United States
| | - Ron Shepard
- Chemical
Sciences and Engineering Division, Argonne
National Laboratory, Lemont, Illinois 60439, United States
| | - Edward F. Valeev
- Department
of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, United States
| | | | - Bess Vlaisavljevich
- Department
of Chemistry, University of South Dakota, Vermillion, South Dakota 57069, United States
| | - Theresa L. Windus
- Department
of Chemistry, Iowa State University and
Ames Laboratory, Ames, Iowa 50011, United States
| | - Sotiris S. Xantheas
- Department
of Chemistry, University of Washington, Seattle, Washington 98195, United States
- Advanced
Computing, Mathematics and Data Division, Pacific Northwest National Laboratory, Richland, Washington 99354, United States
| | - Xing Zhang
- Division
of Chemistry and Chemical Engineering, California
Institute of Technology, Pasadena, California 91125, United States
| | - Paul M. Zimmerman
- Department
of Chemistry, University of Michigan, Ann Arbor, Michigan 48109, United States
| |
Collapse
|
2
|
Williams-Young DB, Asadchev A, Popovici DT, Clark D, Waldrop J, Windus TL, Valeev EF, de Jong WA. Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory. J Chem Phys 2023; 158:234104. [PMID: 37326157 DOI: 10.1063/5.0151070] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2023] [Accepted: 05/26/2023] [Indexed: 06/17/2023] Open
Abstract
With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. In the present work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn-Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.
Collapse
Affiliation(s)
- David B Williams-Young
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | - Andrey Asadchev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| | - Doru Thom Popovici
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| | - David Clark
- NVIDIA Corporation, Santa Clara, California 95051, USA
| | - Jonathan Waldrop
- Chemical and Biological Sciences Division, Ames National Laboratory, Ames, Iowa 50011, USA
| | - Theresa L Windus
- Chemical and Biological Sciences Division, Ames National Laboratory, Ames, Iowa 50011, USA
- Department of Chemistry, Iowa State University, Ames, Iowa 50011, USA
| | - Edward F Valeev
- Department of Chemistry, Virginia Tech, Blacksburg, Virginia 24061, USA
| | - Wibe A de Jong
- Applied Mathematics and Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, California 94720, USA
| |
Collapse
|
3
|
Galvez Vallejo JL, Barca GM, Gordon MS. High-performance GPU-accelerated evaluation of electron repulsion integrals. Mol Phys 2022. [DOI: 10.1080/00268976.2022.2112987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/15/2022]
Affiliation(s)
| | - Giuseppe M.J. Barca
- Department of Computer Science, Australian National University, Canberra, Australia
| | - Mark S. Gordon
- Department of Chemistry, Iowa State University, Ames, United States
| |
Collapse
|
4
|
Finkelstein J, Rubensson EH, Mniszewski SM, Negre CFA, Niklasson AMN. Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network. J Chem Theory Comput 2022; 18:4255-4268. [PMID: 35670603 DOI: 10.1021/acs.jctc.2c00274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
Time-independent quantum response calculations are performed using Tensor cores. This is achieved by mapping density matrix perturbation theory onto the computational structure of a deep neural network. The main computational cost of each deep layer is dominated by tensor contractions, i.e., dense matrix-matrix multiplications, in mixed-precision arithmetics, which achieves close to peak performance. Quantum response calculations are demonstrated and analyzed using self-consistent charge density-functional tight-binding theory as well as coupled-perturbed Hartree-Fock theory. For linear response calculations, a novel parameter-free convergence criterion is presented that is well-suited for numerically noisy low-precision floating point operations and we demonstrate a peak performance of almost 200 Tflops using the Tensor cores of two Nvidia A100 GPUs.
Collapse
Affiliation(s)
- Joshua Finkelstein
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545 United States
| | - Emanuel H Rubensson
- Division of Scientific Computing, Department of Information Technology, Uppsala University, Box 337, Uppsala SE-751 05, Sweden
| | - Susan M Mniszewski
- Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Christian F A Negre
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545 United States
| | - Anders M N Niklasson
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545 United States
| |
Collapse
|
5
|
Yoshida R, Lötstedt E, Yamanouchi K. Quantum computing of Hückel molecular orbitals of π-electron systems. J Chem Phys 2022; 156:184117. [PMID: 35568559 DOI: 10.1063/5.0086489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
In order to demonstrate an applicability of quantum computing to fundamental electronic structure problems of molecules, we describe the Hückel Hamiltonian matrix in terms of quantum gates and obtain the orbital energies of fundamental π-electron molecules (C2H4, C3H4, C4H4, C4H6, and C6H6) using a superconducting-qubit-type quantum computer (ibm_kawasaki) with a post-selection error mitigation method. We show that the orbital energies are obtained with sufficiently high accuracy and small uncertainties and that characteristic features of the electronic structure of the π-electron molecules can be extracted by quantum computing in a straightforward manner.
Collapse
Affiliation(s)
- Ryuhei Yoshida
- Department of Chemistry, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Erik Lötstedt
- Department of Chemistry, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| | - Kaoru Yamanouchi
- Department of Chemistry, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
| |
Collapse
|
6
|
Abstract
Approximating molecular wave functions involves heavy numerical effort; therefore, codes for such tasks are written completely or partially in efficient languages such as C, C++, and Fortran. While these tools are dominant throughout quantum chemistry packages, the efficient development of new methods is often hindered by the complexity associated with code development. In order to ameliorate this scenario, some software packages take a dual approach where a simpler, higher-level language, such as Python, substitutes the traditional ones wherever performance is not critical. Julia is a novel, dynamically typed, programming language that aims to solve this two-language problem. It gained attention because of its modern and intuitive design, while still being highly optimized to compete with "low-level" languages. Recently, some chemistry-related projects have emerged exploring the capabilities of Julia. Herein, we introduce the quantum chemistry package Fermi.jl, which contains the first implementations of post-Hartree-Fock methods written in Julia. Its design makes use of many Julia core features, including multiple dispatch, metaprogramming, and interactive usage. Fermi.jl is a modular package, where new methods and implementations can be easily added to the existing code. Furthermore, it is designed to maximize code reusability by relying on general functions with specialized methods for particular cases. The feasibility of the project is explored through evaluating the performance of popular ab initio methods. It is our hope that this project motivates the usage of Julia within the community and brings new contributions into Fermi.jl.
Collapse
Affiliation(s)
- Gustavo J R Aroeira
- Center for Computational Quantum Chemistry, University of Georgia, Athens, Georgia 30602, United States
| | - Matthew M Davis
- Center for Computational Quantum Chemistry, University of Georgia, Athens, Georgia 30602, United States
| | - Justin M Turney
- Center for Computational Quantum Chemistry, University of Georgia, Athens, Georgia 30602, United States
| | - Henry F Schaefer
- Center for Computational Quantum Chemistry, University of Georgia, Athens, Georgia 30602, United States
| |
Collapse
|
7
|
Finkelstein J, Smith JS, Mniszewski SM, Barros K, Negre CFA, Rubensson EH, Niklasson AMN. Quantum-Based Molecular Dynamics Simulations Using Tensor Cores. J Chem Theory Comput 2021; 17:6180-6192. [PMID: 34595916 DOI: 10.1021/acs.jctc.1c00726] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
Tensor cores, along with tensor processing units, represent a new form of hardware acceleration specifically designed for deep neural network calculations in artificial intelligence applications. Tensor cores provide extraordinary computational speed and energy efficiency but with the caveat that they were designed for tensor contractions (matrix-matrix multiplications) using only low-precision floating-point operations. Despite this perceived limitation, we demonstrate how tensor cores can be applied with high efficiency to the challenging and numerically sensitive problem of quantum-based Born-Oppenheimer molecular dynamics, which requires highly accurate electronic structure optimizations and conservative force evaluations. The interatomic forces are calculated on-the-fly from an electronic structure that is obtained from a generalized deep neural network, where the computational structure naturally takes advantage of the exceptional processing power of the tensor cores and allows for high performance in excess of 100 Tflops on a single Nvidia A100 GPU. Stable molecular dynamics trajectories are generated using the framework of extended Lagrangian Born-Oppenheimer molecular dynamics, which combines computational efficiency with long-term stability, even when using approximate charge relaxations and force evaluations that are limited in accuracy by the numerically noisy conditions caused by the low-precision tensor core floating-point operations. A canonical ensemble simulation scheme is also presented, where the additional numerical noise in the calculated forces is absorbed into a Langevin-like dynamics.
Collapse
Affiliation(s)
- Joshua Finkelstein
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| | - Justin S Smith
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| | - Susan M Mniszewski
- Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| | - Kipton Barros
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| | - Christian F A Negre
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| | - Emanuel H Rubensson
- Division of Scientific Computing, Department of Information Technology, Uppsala University, Box 337, SE-751 05 Uppsala, Sweden
| | - Anders M N Niklasson
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, 87545 New Mexico, United States
| |
Collapse
|
8
|
Abstract
The Ghent Quantum Chemistry Package (GQCP) is an open-source electronic structure software package that aims to provide an intuitive and expressive software framework for electronic structure software development. Its high-level interfaces (accessible through C++ and Python) have been specifically designed to correspond to theoretical concepts, while retaining access to lower-level intermediates and allowing structural run-time modifications of quantum chemical solvers. GQCP focuses on providing quantum chemical method developers with the computational "building blocks" that allow them to flexibly develop proof of principle implementations for new methods and applications up to the level of two-component spinor bases.
Collapse
Affiliation(s)
- Laurent Lemmens
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| | - Xeno De Vriendt
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| | - Daria Tolstykh
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| | - Tobias Huysentruyt
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| | - Patrick Bultinck
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| | - Guillaume Acke
- Ghent Quantum Chemistry Group, Department of Chemistry, Ghent University, Krijgslaan 281 (S3), B-9000 Gent, Belgium
| |
Collapse
|
9
|
Abstract
The quantum mechanics/molecular mechanics (QM/MM) approach is an essential and well-established tool in computational chemistry that has been widely applied in a myriad of biomolecular problems in the literature. In this publication, we report the integration of the QUantum Interaction Computational Kernel (QUICK) program as an engine to perform electronic structure calculations in QM/MM simulations with AMBER. This integration is available through either a file-based interface (FBI) or an application programming interface (API). Since QUICK is an open-source GPU-accelerated code with multi-GPU parallelization, users can take advantage of "free of charge" GPU-acceleration in their QM/MM simulations. In this work, we discuss implementation details and give usage examples. We also investigate energy conservation in typical QM/MM simulations performed at the microcanonical ensemble. Finally, benchmark results for two representative systems in bulk water, the N-methylacetamide (NMA) molecule and the photoactive yellow protein (PYP), show the performance of QM/MM simulations with QUICK and AMBER using a varying number of CPU cores and GPUs. Our results highlight the acceleration obtained from a single or multiple GPUs; we observed speedups of up to 53× between a single GPU vs a single CPU core and of up to 2.6× when comparing four GPUs to a single GPU. Results also reveal speedups of up to 3.5× when the API is used instead of FBI.
Collapse
Affiliation(s)
- Vinícius Wilian D Cruzeiro
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California 92093, United States.,Department of Chemistry and Biochemistry, University of California San Diego, La Jolla, California 92093, United States
| | - Madushanka Manathunga
- Department of Chemistry, Department of Biochemistry and Molecular Biology, Institute of Cyber-Enabled Research, Michigan State University, East Lansing, Michigan 48824, United States
| | - Kenneth M Merz
- Department of Chemistry, Department of Biochemistry and Molecular Biology, Institute of Cyber-Enabled Research, Michigan State University, East Lansing, Michigan 48824, United States
| | - Andreas W Götz
- San Diego Supercomputer Center, University of California San Diego, La Jolla, California 92093, United States
| |
Collapse
|
10
|
Finkelstein J, Smith JS, Mniszewski SM, Barros K, Negre CFA, Rubensson EH, Niklasson AMN. Mixed Precision Fermi-Operator Expansion on Tensor Cores from a Machine Learning Perspective. J Chem Theory Comput 2021; 17:2256-2265. [PMID: 33797253 DOI: 10.1021/acs.jctc.1c00057] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
We present a second-order recursive Fermi-operator expansion scheme using mixed precision floating point operations to perform electronic structure calculations using tensor core units. A performance of over 100 teraFLOPs is achieved for half-precision floating point operations on Nvidia's A100 tensor core units. The second-order recursive Fermi-operator scheme is formulated in terms of a generalized, differentiable deep neural network structure, which solves the quantum mechanical electronic structure problem. We demonstrate how this network can be accelerated by optimizing the weight and bias values to substantially reduce the number of layers required for convergence. We also show how this machine learning approach can be used to optimize the coefficients of the recursive Fermi-operator expansion to accurately represent the fractional occupation numbers of the electronic states at finite temperatures.
Collapse
Affiliation(s)
- Joshua Finkelstein
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Justin S Smith
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Susan M Mniszewski
- Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Kipton Barros
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Christian F A Negre
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| | - Emanuel H Rubensson
- Division of Scientific Computing, Department of Information Technology, Uppsala University, P.O. Box 337, Uppsala SE-751 05, Sweden
| | - Anders M N Niklasson
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, New Mexico 87545, United States
| |
Collapse
|