Overview
I am interested in developing better (faster, more accurate, more general) algorithms for leveraging deep neural networks to solve scientific simulation problems. I am particularly interested in using deep neural networks to solve challenging problems that cause traditional solvers to fail. The best examples of such problems come from strongly correlated quantum systems, which is where I focus the majority of my effort. However, I have also dabbled in physics-informed neural networks.
On the algorithmic side, questions that interest me include:
- What are the right optimization algorithms for training neural networks to solve scientific simulation problems?
- How can we design optimization algorithms that are both highly scalable and robust to ill-conditioning? (a promising direction that I have been exploring is to use ideas from randomized linear algebra and particularly, sketch-and-project methods)
- What kinds of network architectures are suitable for various scientific problems?
On the applications side, I’m excited about the potential impact of this technology on:
- Basic energy science
- Clean energy technologies like batteries, artificial photosynthesis, clean nitrogen, and carbon capture
- Quantum computers, quantum sensors, and other quantum technologies
Specifics
My current research focuses on the use of neural networks to represent ground-state wavefunctions for small but strongly correlated atoms and molecules. The pitch here is that neural network wavefunctions can allow us to find nearly exact ground-state wavefunctions for very challenging problems where other algorithms based on chemical heuristics fail. The challenge is that neural network training is very expensive and very stochastic, making it hard to attain the high accuracies that are required for making useful chemical predictions. There is a great deal of exciting recent and ongoing work aiming to develop better network architectures, training algorithms, and software which I believe will ultimately enable neural network wavefunctions to make a large impact in the field of quantum chemistry.
Within the field of neural network wavefunctions, my focus is on developing better training algorithms. Specifically, along with my advisor and several collaborators, I am working to exploit a previously unappreciated connection between neural network optimization and randomized numerical linear algebra. This connection has enabled us to develop the SPRING algorithm, which has shown promising results for neural network wavefunctions and is also be applicable to other scientific domains. Our more recent works have provided a better theoretical understanding of SPRING, and our ongoing work aims to use this understanding to develop new and even more powerful optimizers based on similar ideas.
References on neural network wavefunctions
Review article:
- J. Hermann et al, Ab initio quantum chemistry with neural-network wavefunctions, Nature Reviews Chemistry 2023
Trailblazers of the field:
- G. Carleo and M. Troyer, Solving the Quantum Many Body Problem with Artificial Neural Networks, Science 2017
- J. Hermann, Z. Schätzle, F. Noé, Deep-neural-network solution of the electronic Schrödinger equation, Nature Chemistry 2020
- D. Pfau, J. Spencer, A. Matthews, W. Foulkes, Ab initio solution of the many-electron Schrödinger equation with deep neural networks, Physical Review Research 2020
A few selected exciting developments:
- I. Glehn, J. Spencer, D. Pfau, A self-attention ansatz for ab-initio quantum chemistry, ICLR 2023
- R. Li et al, A computational framework for neural network-based variational Monte Carlo with Forward Laplacian, Nature Machine Intelligence 2024
- A. Chen, M. Heyl, Empowering deep neural quantum states through efficient optimization, Nature Physics 2024
- G. Goldshlager, N. Abrahamsen, L. Lin, A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions, Journal of Computational Physics 2024
Software: