Presentation is loading. Please wait.

Presentation is loading. Please wait.

C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Solving Large-scale Eigenvalue Problems in SciDAC Applications Chao Yang Lawrence Berkeley National.

Similar presentations


Presentation on theme: "C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Solving Large-scale Eigenvalue Problems in SciDAC Applications Chao Yang Lawrence Berkeley National."— Presentation transcript:

1 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Solving Large-scale Eigenvalue Problems in SciDAC Applications Chao Yang Lawrence Berkeley National Laboratory June 27, 2005

2 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N People Involved  LBNL:  W. Gao, P. Husbands, X. S. Li, E. Ng, C. Yang (TOPS)  J. Meza, L. W. Wang, C. Yang (Nano-science)  SLAC:  L. Lee, K. Ko  Stanford:  G. Golub  UC-Davis  Z. Bai

3 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N SciDAC Applications  Accelerator Modeling  Nano-science

4 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Algorithms  Krylov Subspace Method  Alternatives  Optimization based approach  non-linear solver based approach  Multi-level Sub-structuring  Non-linear Eigenvalue Problems  Structure preserving methods  Optimization based method

5 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Krylov Subspace Method Widely used, relatively well understood (Polynomial approximation theory): Convergence of KSM:  Well separated, large eigenvalues converge rapidly  the starting vector

6 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Acceleration Techniques  Implicit Restart  Spectral transformation filter out unwanted spectral components from v 0 ARPACK

7 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Using KSM in accelerator modeling  the spectrum of the problem  Example: H60VG3 structure, linear element, N=30M, nnz=484M  1024 CPUs, 738GB  Ordering time: 4143s  Numerical Factorization: 133s  Total: 5068s for 12 eigenvalues  Software: PARPACK (implicit restart) + SuperLU, WSMP (spectral transformation)

8 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Limitations of the KSM  High degree polynomial needed for computing small clustered eigenvalues  many matrix vector multiplications  Spectral transformation can be expensive  memory limitation  scalability  Not easy to introduce a preconditioner  eigenvectors of P -1 A are different from eigenvectors of A

9 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Alternative algorithms  Optimization based approach  Minimizing Rayleigh Quotient  Minimizing Residual (Wood & Zunger 85, Jia 97)  Nonlinear equation solver based approach (Jacobi- Davidson)  Newton correction  Preconditioner  stopping criteria for the inner iteration (Notay 2002, Stathopoulos 2005) Allows us to solve problems with more than 90M DOF

10 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Multi-level Sub-structuring (for computing many eigenpairs)  Domain Decomposition concept  Multi-level extension of the Component Mode Synthesis (CMS) method (Bennighof 92)  Decomposition can be done algebraically (Lehoucq & Bennighof 2002)  Success story in structure engineering....  Error analysis  Extend to accelerator modeling

11 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Single-level Sub-structuring  Matrix Partition  Block elimination  Sub-structure calculation (mode selection)  Subspace assembling

12 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Mode Selection

13 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Implementation & cost  Cost:  Flops: more than a single sparse Cholesky factorization  Storage: Block Cholesky factor + Projected matrix + some other stuff  NO triangular solves (involving the original K and M), NO orthogonalization attractive when: 1) the problem is large enough 2) a large number of eigenvalues are needed

14 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N AMLS vs. Shift-invert Lanczos (SIL) DOF=65K, 3 levels of partition

15 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Cavity with External Coupling  Vector wave equation with waveguide boundary conditions can be modeled by a non-linear eigenvalue problem Open Cavity Waveguide BC With

16 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Quadratic Eigenvalue Problem  Consider only one mode propagating in the waveguides  Algorithms  Linearize then solve by KSM (does not preserve the structure of the problem)  Second Order Arnoldi Iteration (Bai & Su 2005) project the QEP into 2nd order Krylov Subspace

17 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Second-Order Krylov Space (Bai)

18 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N SOAR is faster and more accurate (than linearization)  Accelerating cavity model for international linear collider (ILC)  9-cell superconducting cavity coupled to one input coupler and two Higher-Order-Mode couplers.  NDOFs=3.2million, NCPUs=768, Memory=300GB  18 eigenpairs in 2634 seconds (linearization took more than 1 hour)

19 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Electronic Structure Calculation  wave function  n – real space grid size, e.g ~32000  k – number of occupied states, 1~10% of n  Charge density E kinetic = E ionic = E Hartree = E xc = E total (X) = E kinetic + E ionic + E Hartree + E xc

20 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Non-linear Eigenvalue Problem  Total energy minimization  KKT condition

21 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N The Self Consistent Field Iteration  Input: initial guess and  Output:  Major steps oFor i=1,2,…, until converged 1)Form 2)Compute k smallest eigpairs of

22 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Direct Constrained Minimization (DCM) For i=1,2,… until convergence 1.Form 2.Compute 3.If (i>1) then set 4.else set 5.Solve 6.If (i>1) then set 7.else set

23 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N DCM vs. SCF  Atomic system: SiH4  Discretization: spectral method with plane wave basis: n=32 3 in real space, N=2103 (# of basis functions) in frequency space  Number of occupied states: k = 4  PETOT version of SCF uses 10 PCG steps (inner iterations) per outer iteration  DCM: 3 inner iterations

24 C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Concluding Remarks  Krylov Subspace Method (with appropriate acceleration strategies) continues to play an important role in solving SciDAC eigenvalue problems  Steady progress has been made in alternative approaches that can make better use of preconditioners  Multi-level sub-structuring is promising for computing many eigenpairs  Significant progress made in solving QEP  Non-linear eigenvalue problems remain challenging


Download ppt "C O M P U T A T I O N A L R E S E A R C H D I V I S I O N Solving Large-scale Eigenvalue Problems in SciDAC Applications Chao Yang Lawrence Berkeley National."

Similar presentations


Ads by Google