1. Scientific Computing Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues * C. BekasY. Saad Comp. Science & Engineering Dept. University of Minnesota, Twin Cities * Work supported by NSF under grants NSF/ITR-0082094, NSF/ACI-0305120 and by the Minnesota Supercomputing Institute

2. AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues Introduction and Motivation C. Bekas: SC Seminar Target Problem Compute a large number of the smallest eigenvalues of large sparse matrices Numerous important applications, including: structural engineering computational materials science (electronic structure calculations) Signal/Image processing and Control Shift and Invert techniques can be very successful (Grimes et al 94, MSC.NASTRAN) … BUT quickly become impractical to use: For very large problem sizes (…N>10 6 ) a supercomputer is needed When we need to compute several hundreds or thousands of eigenvalues (deep in the spectrum) reorthogonalization costs dominate and become prohibitive!

3. AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues Introduction and Motivation Component Mode Synthesis (CMS) (Hurty ’60, Graig-Bampton ’68) Well known alternative. Used for many years in Structural Engineering. But it too suffers from limitations due to problem size… AMLS, (Bennighof, Lehoucq, Kaplan and collaborators)  Multilevel CMS method (solves the dimensionality problem)  Automatic computation of substructures (easy application)  Approximation: Truncated Congruence Transformation  Builds very large projection basis without reorthogonalization  Successful in computing thousands of eigenvalues in vibro-acoustic analysis (N>10 7 ) in a few hours on workstations (Kropp–Heiserer, 02) Accuracy issues  AMLS accuracy is adequate in Structural Eng. (in the order of the discretization error), but  higher accuracy is needed in other applications, (i.e. electronic structure calculations)  AMLS is an one shot approach: no (iterative) refinement is done C. Bekas: SC Seminar

4. AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues Recent Advances The success of AMLS in Struct. Eng. and its potential for other applications has sparked several new research initiatives. Some include: - Bekas and Saad Purely algebraic analysis of AMLS 1) Approximation to a nonlinear (Schur) eigenvalue problem 2) Approximation of the resolvent (A- I) -1 by a careful projection 3) Improvements: a) 2 nd order expansions, b) Krylov projections and combinations - Yang et al Algebraic substructuring. Careful selection of added eigenvectors for improved accuracy. - Elssel and Voss A priori error bounds for algebraic substructuring. C. Bekas: SC Seminar

5. AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues In this talk We will describe - Approximation mechanism behind AMLS. We will review our recent purely algebraic analysis of AMLS (Bekas – Saad, 04). - Iterative application of AMLS 1) Refinement of approximated eigenvalues 2) Multiple shifts in AMLS 3) Computation of eigenvalues in an interval - Numerical examples C. Bekas: SC Seminar

6. 11 22 Subdivide  into 2 subdomains:  1 and  2 Component Mode Synthesis: a model problem Y X Consider the model problem: on the unit square . We wish to compute smallest eigenvalues. Component Mode Synthesis Solve problem on each  i “Combine” partial solutions AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

7. CMS: Approximation procedure. Ignore coupling! Component Mode Synthesis: Approximation CMS: Approximation procedure (2) Solve B v =  v Remember that B is block diagonal Thus: we have to solve smaller decoupled eigenproblems Then, CMS approximates the coupling among the subdomains by the application of a carefully selected operator on the interface unknowns AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

8. Recent CMS method: AMLS Block Gaussian eliminator matrix:such that: Where S = C – E > B -1 E is the Schur Complement Equivalent Generalized Eigenproblem : U T AUu= U T U u AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

9. AMLS Approximation AMLS: Approximation procedure. Ignore coupling! Solve decoupled problemsForm basis FINAL APPROXIMATION: AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

10. AMLS: Multilevel application S1S1 E1E1 E1*E1* B2B2 B1B1 S2S2 S3S3 Scheme applied recursively. Resulting to thousands of subdomains. Successful in computing thousands smallest eigenvalues in vibro-acoustic analysis with problem size N>10 7 In the following we analyze the approximation mechanism of one step of AMLS, adopting a purely algebraic setting. This will naturally lead to improved versions of the method AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

11. AMLS: approximation to a nonlinear eigenproblem! AMLS: Approximation procedure. DO NOT ignore coupling this time Leads to:Substitute equation (1) in equation (2) to give equivalent non-linear problem: AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

12. AMLS: approximation to a nonlinear eigenproblem! Define: Resolvent: Basic Property We can show that: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

13. AMLS: approximation to a nonlinear eigenproblem! Neumann Series of the Resolvent: Truncate the series. Keep the first two terms only. Then… Approximate problem: Remember: AMLS: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

14. AMLS: The projection view-point Initial problem: We can show that if: Then, is eigenvalue of (1) and Respective eigenvector AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

15. AMLS: The projection view-point 1. AMLS solves approximately (truncation) 4. Remedy: augment the space of approximants by eigenvectors of B. Why? 2. AMLS, change of basis:3. Thus: AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

16. AMLS: The projection view-point We examine the difference: Expansion in terms of eigenvectors of B: Then, this difference is: Thus: the difference is large in eigenvectors of B with eigenvalues close to the smallest eigenvalues of A. AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

17. AMLS: The projection view-point Therefore: augmenting the space of approximants with ‘’smallest’’eigenvectors of B is well justified. Can we bound the error? Let:m B “smallest” eigenvectors of B X B : restriction to the B-part (upper part) of the space of approximants. Then: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

18. Improvements The algebraic framework we have described leads to improvements: Nonlinear Schur complement problem: Introduce an additional term of the truncated Neumann series Utilize corresponding 2 nd order projection Approximation of the resolvent (B- I) -1 : Approximate with Krylov subspaces on B -1 and Utilize corresponding projection Combinations of the above improvements lead to hybrid algorithms with enhanced stability and robustness C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

19. Augmenting with Krylov Subspaces We need to approximate Therefore it is natural to consider the Krylov subspace V k is an orthonormal block Krylov basis The columns of U S are eigenvectors of the nonlinear Schur complement problem or in general the block Krylov subspace many j C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

20. Second Order Approximation Neumann Series of the Resolvent: AMLS approximation Can we do better? 2 nd order approximation Quadratic Eigenvalue Problem: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

21. Second Order Approximation: Solving the QED Quadratic Eigenvalue Problem: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues Linearization leads to equivalent generalized eigenproblem:

22. Second Order Approximation: Projection view point Eigenvector of A: Better approx. by the QEP We can add more eigenvectors of B…or add “second order” vectors: ADD THE VECTORS Construct basis Solve C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

23. Second Order Approximation: Error bounds Augmenting the space of approximants with eigenvectors of B and “second order vectors” leads to quadratic error bounds compared to adding just eigenvectors Let:m B smallest eigenvectors of B X B : restriction to the B-part (upper part) of the space of approximants. Then: C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

24. Improvements The algebraic framework we have described leads to improvements: Our previous work Nonlinear Schur complement problem: Introduce an additional term of the truncated Neumann series Approximation of the resolvent (B- I) -1 : Approximate with Krylov subspaces on B -1 and Utilize corresponding projection Recent Work Framework for the iterative refinement of the AMLS approximations Utilization of many different shifts (A-  k I) -1 that can be used for… …computation of eigenvalues deep in the spectrum More robust method. Currently examining connections with shift- invert Lanczos (Grimes et al) and Rational Krylov (Ruhe) AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

25. Iterative Refinement (1/3) AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

26. OR WITH SOME REARRANGEMENT… Iterative Refinement (2/3) NON-LINEAR SCHUR COMPLEMENT AMLS APPROXIMATION R : AMLS REMAINDER AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

27. Iterative Refinement (3/3) AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

28. Using Multiple Shifts in AMLS (1/3) MATRIX: BCSSTK11 1 st shift: 3, close to eigenvalues in the interval [2.9,3] 2 nd shift: 69, close to eigenvalues in the interval [68,70] CAN WE COMBINE THESE TWO SHIFTS? AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

29. Using Multiple Shifts in AMLS (2/3) Smallest eigenvectors of Smallest eigenvectors of PROJECTED PROBLEM AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

30. Using Multiple Shifts in AMLS (3/3) AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

31. Numerical Experiments AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

32. Numerical Experiments: Standard v.s. Krylov Thus: Small /    favors the Krylov version Matrix: BCSSTK11(N=1473), Reorder using Nes. Disec., size of Schur complement N S =94 Bef. reord. /    After reord. /    C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

33. Numerical Experiments: Standard v.s. Krylov Number of Schur eigenvectors: 5 Number of added vectors:  AMLS: 30 eigenvectors of B (left) and 40 eigenvectors of B (right) Krylov: 30=5 x 6 (left) and 40=5 x 8 (right) REORDERED VERSION C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

34. Numerical Experiments: Standard v.s. Krylov Number of Schur eigenvectors: 5 Number of added vectors:  AMLS: 30 eigenvectors of B (left) and 40 eigenvectors of B (right) Krylov: 30=5 x 6 (left) and 40=5 x 8 (right) NOT REORDERED VERSION C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

35. Second order method Matrix: BCSSTK11 Approximate S( ) u= u with a Quadratic eigenvalue problem instead of the original generalized eigenvalue problem of AMLS C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

36. Second order method Matrix: BCSSTK11 Compare 2 nd order AMLS (20 added vectors) v.s. standard AMLS with increasing number of added eigenvectors of B (m B =20,40,60,100) 2 nd order: 10 Schur vectors: 20 added Krylov vectors. 30 vectors in total AMLS: 10 Schur vectors: 20, 40, 60 and 100 added eigenvectors of B C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

37. Combine AMLS with Krylov AMLS Matrix: 5-point stencil discretization of the Laplacian Use only the 2 smallest Schur vectors. Then, for larger eigenvalues the Krylov version can fail. REMEDY: Augment subspace with eigenvectors of B too! V B : eigenvectors of B V K : Krylov basis U S : Schur eigenvectors C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

38. Combine AMLS with Krylov AMLS Matrix: 5-point stencil discretization of the Laplacian Use only 2 smallest Schur vectors. Then, for larger eigenvalues the Krylov version can fail. REMEDY: Augment subspace with eigenvectors of B too! V B : eigenvectors of B V K : Krylov subspace U S : Schur eigenvectors C. Bekas: SC Seminar AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues

39. Iterative Refinement Matrix: 5-point stencil discretization of the Laplacian Next shift  k is 1 st smallest approximate eigenvalue at step k-1 Next shift  k is 5 th smallest approximate eigenvalue at step k-1 AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

40. Multiple Shifts Dimension of each Q i : 40 Matrix: 5-point stencil discretization of the Laplacian Dimension of each Q i : 60 AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar

41. Conclusions AMLS is a promising alternative to Shift-Invert methods for very large problems In this work we have presented an analysis of AMLS based on a completely algebraic framework: AMLS is a nonlinear (spectral) Schur complement method that utilizes… Projection on carefully selected eigenspaces to approximate the solution Improvements Based on the algebraic framework we have proposed Krylov projection subspaces and second order approximations (and combinations) with significant improvements. Iterative Refinement Approximations can be iteratively refined, allowing for very good accuracy Many different shifts are combined and thus we can compute eigenvalues deep in the spectrum Current Work Investigate strategies to compute all eigenvalues in a (large) interval [a,b] AMLS, Spectral Schur Complements and Iterative Computation of Eigenvalues C. Bekas: SC Seminar