Large-Scale Density Functional Calculations James E. Raynolds, College of Nanoscale Science and Engineering Lenore R. Mullin, College of Computing and Information

Overview Using computers to carry out “numerical experiments” in Materials Science, Chemistry and Physics Quantum Mechanical equations solved for a system of atoms in a representative unit cell Measurable properties obtained from “first-principles” –mechanical, thermodynamic, electronic –optical, magnetic, transport

Example: Transport in molecular wire Benzene Phenolate/Benzenediazonium + V

Peierls Distortion Pi stacked pair dimerized pair metal insulator mechanical relaxation

Frontier Problems Non-equilibrium spin-transport in metals and semiconductors (Spintronics) Transport and coupled mechanical / electronic interactions in molecules (metal - insulator transition due to mechanical relaxation) Industrial applications: Modeling Chemical Vapor Deposition (CVD) processes atom by atom Challenges: correlated motion of electrons Coupled electron-phonon interactions (electron - vibration coupling)

Density Functional Theory Density Functional Theory (DFT) is a “mean-field” solution to the many-electron problem. Each electron interacts with an effective average field produced by all of the other electrons Non-linear set of coupled differential equations

Density Functional Equations Looks linear but... depends on the charge density through: Example: Local density approximation

DFT solution approach Expand the wave-functions in a basis set: Matrix eigenvalue-eigenvector problem: Orthogonality: Iterative solution to “self-consistency” (i.e. output V(r) coincides with input)

Popular implementations Plane wave basis functions (Fourier Series): –Drawback: –Benefit: easy to code, sophisticated non-linear response calculations possible Localized “atomic-like” basis functions scaling - exponential distance decay for insulators - power law distance decay for - metals

Contrasting Implementations Abinit: –Very sophisticated array of calculated properties –Calculations become prohibitive for more than a few dozen atoms VASP (Vienna Ab-Initio Simulation Package) –Less sophisticated by much faster –few hundred atoms possible Siesta: (Spanish Initiative for Electronic Simulations with Thousands of Atoms) –O(N) scaling: fast but less sophisticated –few thousand atoms possible

Public Access Many codes are freely available: go to for a list of more than 20 Most codes still not user-friendly and take months to years to master

The Brick Wall!! All of these methods run out of steam very quickly in terms of run time and memory Calculations with scaling take days or weeks to run!! Even calculations with scaling run into memory bottlenecks Materials Science simulations require thousands of atoms for thousands of time steps

Key Algorithms For plane wave based codes: the Fast Fourier Transform –We have gained factor’s of 4 improvement in speed and storage using Conformal Computing –A number of new developments are being implemented for further increases Matrix diagonalization routines for very large matrices

Conformal Computing Density Functional Calculations are an ideal setting for Conformal Computing! In fact: any array (matrix) based computational setting is ripe for Conformal Computing Why? Conformal Computing eliminates temporary arrays and un-necessary loops!

Opportunities Current electronic band structures fairly fast (on the order of one hour):

Contrasting: electron-phonon Electron-phonon calculations: on the order of 1 day for small systems Superconductivity in “conventional” materials determined by the electron - phonon interaction Aluminum (1 atom) takes roughly 1 day of computing Imagine several dozen atoms with scaling

Electron-Phonon improvements Many quantities currently written to files then later combined The size and number of these files is becoming prohibitively expensive Opportunities for parallelization of integrals Opportunities to eliminate temporaries through the use of direct indexing

Grid Computing Even with highly optimized code (which is still a way off) there is always a need for more and more resources For example: electron-phonon calculations involve dozens of separate calculations that could be run on independent machines Grid computing allows many independent calculations to be run in parallel

Grid Computing: First Steps QMolDyn GAT: a template for submitting Density Functional Calculations over the grid Vision: QMolDyn will eventually have a variety of codes (modules) Presently: Siesta ( ) running on the grid, 8, 16, 32, 64, 128, 256, 512- atom systems

Summary / Conclusions There is a great demand for large-scale array (matrix) based calculations in materials science Quantum calculations are increasingly important for Materials Science, Chemistry and Physics Grid computing combined with Conformal Computing techniques is very promising