1. Neural Networks for Quantum Simulation
Brian Barch

2. Problem: Quantum Simulation
Want to find motion of atoms in a system
Done by calculating energy, then finding force
Classically easy, but not for quantum wavefunctions
Want to learn a function from atomic position
to system energy
Regression problem
Poliovirus, used in 108 atom simulation of infection

3. Hydrogen Dataset
Configurations of hydrogen atoms and associated energy
produced by iterative simulation methods
8 sets, 3000 samples each
Each sample: 54 H atom positions in (x, y, z) and total energy
Preprocessing:
Previously: Calculated inter-atomic distances and angles, and from there symmetry function values. Normalized results
Currently: Just inter-atomic distances and angles. Not normalized.

4. Base Model: Atomic NN structure
Describe environment of each atom with symmetry functions
Atomic NN uses this vector to predict energy for that atom
Atomic NNs share weights
atom positions
total energy
preprocess
Separate NNs with shared weights

5. New Model: Atomic NN structure
Reformulated atomic NN structure as convolution
Used 1D convolutional layers, with atoms as the input dimension and symmetry function values as the channels
Rather than preprocessing, first layer calculates symmetry function values from distances and angles
Second is batch normalization layer
Allows application to arbitrary numbers of atoms and GPU CNN optimization
Con: much slower on complicated symmetry functions

6. New Model: Atomic NNs Ê Preprocess NN + Sum of atomic energies
angles ѳ + Ê D
distances
Sum of atomic energies
Sym. Func. Layer
Batch normalization
Atoms
Conv layers

7. Base Model: Symmetry Functions
Represent atomic environment in a form invariant to changes that don’t affect total energy (i.e. shifts, rotation, translation)
Currently manually selected and used to preprocess data
- NNs are trained on saved sets of symmetry functions

8. New Model: Symmetry Functions
Calculated per batch by first layer
- Made possible by the miracle of Theano tensor broadcasting
Slower than preprocessing, but… allows training of sym. func. parameters
- Don’t need to rely on hand-picking values anymore
- Still need to hand pick initial values for now
- Used Keras’ built in analytic gradient tools
Once good values are found, can use them for preprocessing

9. New Model: Results
Wasn’t trying to optimize parameters as much as apply a new NN structure to this problem
Angle-based symmetry functions required triple sum terms
- too expensive to compute on the fly for my computer
- haven’t had a chance to run on cluster yet
Current results use a subset of symmetry functions, so aren’t comparable to previous results
That being said...

10. Results: trainable vs static parameters
Trainable symmetry function parameters had a purely beneficial effect on accuracy
Reduced MSE by ~10%, faster training, less overfitting
Static sym funcs
Trainable sym funcs

11. Results continued
Improved accuracy more for worse accuracies – e.g. for worse hyperparameters
parameters often remained near initial value
- likely because other weights optimized for those parameters
Other times there was significant deviation, i.e. multiple parameters converging to nearly the same value
Future work will focus on avoiding such redundancies
- tried l1 regularization but it was insufficient

12. Lessons and Challenges
From checkpoint 1: Visualizing can be challenging but is very important
New Lessons:
Implementation efficient surprisingly important, because it allows testing on the fly during development
Linux is much easier to run code in than Windows 10 is
New Challenges:
Interpreting whether trained sym func parameters are meaningful
Making angle based sym funcs efficient enough to run