 U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 &

 U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 stevem@colorado.edu & john.ruge@colorado.edu http://amath.colorado.edu/faculty/stevem Office Hours MW 2-4 45 pm (except for class!) in ECOT 322

 U-Boulder2 of 322 APPM 6640 Syllabus Coursework –No exams –Homework exercises & computing assignments –Project & presentation--joint is OK –Can tailor some or all work & later lectures to your interests Philosophy –Big on fundamentals, principles, & simple ideas –Think about the scientific method in general –Understand by concrete examples & experience –Know the whole picture –Ask questions & interject comments –Make sure I explain what you need Text –A Multigrid Tutorial, 2 nd edition, 2 nd printing –Supplemental material as needed

 U-Boulder3 of 322 Sources MGNet Newsletter & software repository http://www.mgnet.org/ MathSciNet Many papers electronically available http://www.ams.org/mathscinet Copper Mountain Conference March 22-27, 2007 http://amath.colorado.edu/faculty/copper We hope to support your attendance there. Course web site: http://grandmaster.colorado.edu/appm6640/ These slides are there.

 U-Boulder4 of 322 Homework exercises 1: 3,4 2: 1-3, 8, 10, 13, 16 3: 2-6 4: 6, 7, 11 5: 1-3, 12, 13 6: 1, 2, 6 7: 1,2,7,10, 12, 15, 21 8: 2, 6, 8 9: 1, 3 10: 1, 3, 5, 8, 9 Due 1 week after the relevant chapter is covered.

 U-Boulder5 of 322 Computing assignments 2: 20 4: 13, 15 6: 6 by computer!!! 7: Convert your 2-D code developed for Chapter 4 (standard coarsening & point relaxation) to solve (7.14). Verify the code using  = 1. Test its performance using  = 1/2,1/5, 1/10, 2, 5, 10. Interpret your results. Due 2 weeks after the relevant chapter is covered. Start your assignments early: multigrid algorithms can be tricky to code! Use whatever computing language/platform you like.

 U-Boulder6 of 322 Project & presentation Project definition is intentionally vague. Think about what has captured your attention. See us, but only after you’ve thought about it. Computing &/or theory &/or application &/or exploration. I’d like to see a presentation (10 minutes) plus a short (<5 pages) writeup about what you did. Presentation: Prepare! Speak to your peers, not me. Check course web site for suggestions. Due at the end of the semester.

 U-Boulder7 of 322 Successful Scientific Inquiry Attitude –You can do it! Be positive. –But is it really right? Be critical. –Don’t hope or guess. THINK! –Control your emotions! Expect ups & downs. Method –Start simply. Reduce issue to the simplest possible case. –Take tiny steps, but keep the big picture in mind. –Study concrete examples. –Look for analogies. Can A be done in any way like how B was done? Creativity –What do you really want? What end are you really aiming for? –What do you really need? What you’re trying may be sufficient to do what you want, but would an easier-to-prove weaker result do instead? Intelligence –It doesn’t hurt to try to be “smart” too.

 U-Boulder8 of 322 A Multigrid Tutorial 2 nd Edition, 2 nd Printing By William L. Briggs CU-Denver Van Emden Henson LLNL Steve McCormick CU-Boulder  THANKS!

 U-Boulder9 of 322 Outline by chapter 1. Model Problems 2. Basic Iterative Methods Convergence tests Analysis 3. Elements of Multigrid Relaxation Coarsening 4. Implementation Complexity Diagnostics 5. Some Theory Spectral vs. algebraic 6. Nonlinear Problems Full approximation scheme 7. Selected Applications Neumann boundaries Anisotropic problems Variable meshes Variable coefficients 8. Algebraic Multigrid (AMG) Matrix coarsening 9. Multilevel Adaptive Methods FAC 10. Finite Elements Variational methodology

 U-Boulder10 of 322 Suggested reading CHECK THE MG LIBRARY & MGNET REPOSITORY A. Brandt, “Multi-level Adaptive Solutions to Boundary Value Problems,” Math Comp., 31, 1977, pp 333-390. A. Brandt, “Multigrid techniques: 1984 guide with applications to computational fluid dynamics,” GMD, 1984. W. Hackbusch, “Multi-Grid Methods & Applications,” Springer, 1985. W. Hackbusch & U. Trottenberg, “Multigrid Methods”, Springer-Verlag, 1982. S. McCormick, ed., “Multigrid Methods,” SIAM Frontiers in Applied Math. III, 1987. U. Trottenberg, C. Oosterlee, & A. Schüller, “Multigrid,” Academic Press, 2000. P. Wesseling, “An Introduction to Multigrid Methods,” Wylie, 1992.

 U-Boulder11 of 322 Multilevel methods have been developed for... PDEs, CFD, porous media, elasticity, electromagnetics. Purely algebraic problems, with no physical grid; for example, network & geodetic survey problems. Image reconstruction & tomography. Optimization (e.g., the traveling salesman & long transportation problems). Statistical mechanics, Ising spin models. Quantum chromo dynamics. Quadrature & generalized FFTs. Integral equations.

 U-Boulder12 of 322 Everyone uses multilevel methods Multigrid, multilevel, multiscale, multiphysics, … Use local “governing rules” on the finest level to resolve the state of the system at these detailed scales, but--recognizing that these “rules” have broader implications that are hard to determine there--use coarser levels to resolve larger scales. Continual feedback is essential because improving one scale impacts other scales. Common uses Sight, art, politics, thinking (scientific research), cooking, team sports, …

 U-Boulder13 of 322 1. Model problems 1-D boundary value problem: Grid: Let & for.           x xfxuxu 0 1 0 uu  0 10 xff ii  This discretizes the variables, but what about the equations?

 U-Boulder14 of 322 Approximate u ’’(x) via Taylor series Approximate 2 nd derivative using Taylor series: 00 Summing & solving: +

 U-Boulder15 of 322 Approximate equation via finite differences Approximate the BVP by a finite difference scheme:    fv h vvv ii iii  2 11 2 vv 0   N+1 0 i = 1,2,…,N           x xfxuxu 0 1 0 uu  0 10

 U-Boulder16 of 322 Discrete model problem Letting & we obtain the matrix equation Av = f, where A is N x N, symmetric, positive definite, & v   vvv 2 1 T N f  fff 21N ) T

 U-Boulder17 of 322 Stencil notation A = [-1 2 -1] dropping h -2 &  for convenience 2

 U-Boulder18 of 322 Basic solution methods Direct –Gaussian elimination –Factorization –Fast Poisson solvers (FFT-based, reduction-based, …) Iterative –Richardson, Jacobi, Gauss-Seidel, … –Steepest Descent, Conjugate Gradients, … –Incomplete Factorization,... Notes: –This simple 1-D problem can be solved efficiently in many ways. Pretend it can’t & that it’s very hard, because it shares many characteristics with some very hard problems. If we keep things as simple as possible by studying this model, we’ve got a chance to really understand what’s going on. –But, to keep our feet on the ground, let’s go to 2-D anyway…

 U-Boulder19 of 322 2-D model problem Consider the problem Consider the grid 0  L+1i 0  M+1j u = 0 when x = 0, x = 1, y = 0, or y = 1   

 U-Boulder20 of 322 Discretizing the 2-D problem Let &. Again, using 2 nd - order finite differences to approximate & we arrive at the approximate equation for the unknown, for i =1,2, …, L & j =1,2, …, M: Order the unknowns (& also the vector f ) lexicographically by y-lines: yxff jiji 

 U-Boulder21 of 322 Resulting linear system We obtain a block-tridiagonal system Av = f : where I x is the h x -2 times the identity matrix &

 U-Boulder22 of 322 Stencils preferred for grid issues Stencils are much better for showing the grid picture: Stencils show local relationships--grid point interactions. again dropping h -2 &  4

 U-Boulder23 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:

 U-Boulder24 of 322 2. Basic iterative methods Consider the NxN matrix equation Au = f & let v be an approximation to u. Two important measures: –The Error: with norms –The Residual: with vue  vAfr  What does ||r|| measure??? & & Why have both r & e ???

 U-Boulder25 of 322 Residual correction What does this do for us??? r = f - Av = Au - Av = A(u - v) = Ae Since e = u - v, we can write Au = f as A(v + e) = f which means that Ae = f - Av  r. Residual Equation: Residual Correction:

 U-Boulder26 of 322 Relaxation Consider the 1-D model problem Jacobi (simultaneous displacement): Solve the i th equation for holding all other variables fixed:   ≤  uuNifhuuu N+1 iiii  0 2 11 012 Nif h vvv i dlo i dlo i wen i       1 2 1 2 11 ≤ ≤ ≤

 U-Boulder27 of 322 Jacobi in matrix form Let A = D - L - U, where D is diagonal & -L & -U are the strictly lower & upper triangular parts of A. Then Au = f becomes Let. –R J is called the error propagation or iteration matrix. Then the iteration is  J = D -1 (D-A) = I-D -1 A

 U-Boulder28 of 322 Error propagation matrix & the error From the derivation, the iteration is subtracting, or hence, Error propagation!  J = I-D -1 A

 U-Boulder29 of 322 A picture R J = D -1 (L + U) = [ 0 ] so Jacobi is an error averaging process: e i (new)  (e i-1 (old) + e i+1 (old) )/2 1212 1212

 U-Boulder30 of 322 But…

 U-Boulder31 of 322 Another matrix look at Jacobi v (new)  D -1 (L + U) v (old) + D -1 f (L + U = D-A) = (I - D -1 A) v (old) + D -1 f v (new) = v (old) - D -1 (Av (old) - f) = v (old) + D -1 r Exact: u = u - D -1 (Au - f) Subtracting: e (new) = e (old) - D -1 Ae (old) Exact: u = u - A -1 (Au - f) = A -1 f General form: u = u - B (Au - f) with B ~ A -1 Damped Jacobi: u = u -  D -1 (Au - f) with 0<  <1/ ρ( D -1 A) Gauss-Seidel: u = u - (D - L) -1 (Au - f) Note that R  = I -  D -1 A is a polynomial in A when D = I. We exploit this simplicity (symmetry, etc.) in what follows. !!!This special property doesn’t usually hold in practice!!!

 U-Boulder32 of 322 Weighted Jacobi safer (0<  <1) changes Consider the iteration Letting A = D-L-U, the matrix form is Note that It is easy to see that if e (approx) = u - v (approx), then

 U-Boulder33 of 322 Gauss-Seidel (1-D) Solve equation i for u i & update immediately. Equivalently: set each component of r to zero in turn. Component form: for set Matrix form: Let Then iterate: Error propagation: Ni   21  G = (D-L) -1 (D-L-A) = I- (D-L) -1 A

 U-Boulder34 of 322 Red-black Gauss-Seidel Update the EVEN points: Update the ODD points: 2-D: +1+1

 U-Boulder35 of 322 Test? Au = f Need to know how we’re doing!!! What f ? What v ? Au = 0 v = rand

 U-Boulder36 of 322 Numerical experiments Solve, Use Fourier modes as initial iterates, with N = 63: component mode 00.10.20.30.40.50.60.70.80.91 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 sin(k π x), x =i/N+1

 U-Boulder37 of 322 Convergence factors differ for different error components Error, ||e|| , in weighted (  = 2/3 ) Jacobi on Au = 0 for 100 iterations using initial guesses v 1, v 3, & v 6 020406080100120 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

 U-Boulder38 of 322 0102030405060708090100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Stalling convergence relaxation shoots itself in the foot Weighted (  = 2/3 ) Jacobi on 1-D problem. Initial guess: Error plotted against iteration number:

 U-Boulder39 of 322 Analysis of stationary linear iteration Let v (new) = Rv (old) + g. The exact solution is unchanged by the iteration: u = Ru + g. Subtracting: e (new) = Re (old). Let e (0) be the initial error & e (i) be the error after the i th iteration. After n iterations, we have e (n) = R n e (0). We can deal with 2 4, but ??? 4 4 What ife = 2 4 e ???

 U-Boulder40 of 322 Quick review of eigenvectors & eigenvalues The number is an eigenvalue of a matrix B & w ≠ 0 its associated eigenvector if Bw = w. The eigenvalues & eigenvectors are characteristics of a given matrix. Eigenvectors are linearly independent, & if there is a complete set of N distinct eigenvectors for an NxN matrix, then they form a basis: for any v, there exist unique scalars v k such that Propagation: v  v k k  1 N  w k B n v  n v k k  1 N  w k Why is an eigenvector useful???

 U-Boulder41 of 322 “Fundamental Theorem of Iteration” R is convergent (R n  0 as n   ) iff  (R) = max | | < 1. Thus, e (n) = R n e (0)  0 for any initial vector v (0) iff  (R) < 1.  (R)<1 assures convergence of R iteration.  (R) is the spectral convergence factor. But  is doesn’t tell you much by itself--it’s generally valid only asymptotically. It’s useful for the symmetric case in particular because it’s equal to  R  2, so we’ll use it here.

 U-Boulder42 of 322 Rayleigh quotient vs. spectral radius assume A is symmetric (w k orthonormal ) & nonnegative definite ( ≥ 0) RQ(v) ≤  (A):v =  v k w k RQ(v) sup v ≠ 0 RQ(v) =  (A): RQ(w N )   Aw N,w N   w N,w N    N w N,w N   w N,w N   N  (A)   Av,v   v,v    A  v k w k,  v k w k   v k w k,  v k w k     k v k w k,  v k w k    v k w k,  v k w k    k v k 2  v k 2 ≤ N =  (A)≥ 1

 U-Boulder43 of 322 Euclidean norm vs. spectral radius use RQ ||R|| 2 =   (R T R): ||R|| 2 2 = sup e≠0 ||Re|| 2 2 /||e|| 2 2 = sup e≠0 / = sup e≠0 / =  (R T R) note: ||Re|| 2 ≤ ||R|| 2 ·||e|| 2 ||A|| 2 =   (A 2 ) =  (A) for symmetric A!!!

 U-Boulder44 of 322  (R) vs. ||R|| 2  (R) = sup | (R)|.  (R) is norm independent.  (R) is asymptotic: could get large ||R|| 2 but  (R)<1  “convergence”. Next slide. ||R|| 2 =sup e≠0 ||Re|| 2 /||e|| 2 =   (R T R). ||R|| 2 depends on ||· || 2. ||R|| 2 bounds ||e (n + 1) || 2 /||e (n) || 2 : worst case! Probably pessimistic initially, but sharp sooner or later.  (R) = inf ||R|| taken over all matrix norms.  (R) = ||R|| 2 for symmetric R.

 U-Boulder45 of 322 Example: R = ( ), K large  (R) = 0 but ||R|| 2 = K !   Thus, 1 iteration with e (0) shows dramatic L 2 divergence! But R 2 = 0, so e (2) = Re (1) = R 2 e (0) = 0! Thus, 2 iterations with e (0) show complete convergence! On one hand, this is special ( =0, large K, 2x2), so this behavior would be more subtle & persistent in general. On the other, this behavior would vanish for symmetric R.

 U-Boulder46 of 322 Convergence factor & rate How many iterations are enough to guarantee reduction of the initial error by 10 -d ? So, we have Convergence factor = ||R|| or  (R). Convergence rate =or -log 10  (R)) 

 U-Boulder47 of 322 Convergence analysis: Weighted Jacobi For our 1-D model, the eigenvectors of weighted Jacobi R  & the eigenvectors of A are the same! Why??? 1-D This is very special!!! Remember that A is without h 2 here!

 U-Boulder48 of 322 Eigenpairs of (scaled) A The eigenvectors of A are Fourier modes! [-1 2 -1]  N  4  1  2 h 2

 U-Boulder49 of 322 Eigenvectors of R  = eigenvectors of A Expand the initial error in terms of the eigenvectors: After n iterations: The k th error mode is reduced by k (R  ) each iteration.

 U-Boulder50 of 322 Relaxation suppresses eigenmodes unevenly Look carefully at. Note that if 0 <  ≤ 1, then for. For 0 <  ≤ 1,

 U-Boulder51 of 322 Low frequencies are “undamped” Notice that no value of will efficiently damp out long waves or low frequencies. What value of gives the best damping of short waves or high frequencies N/2≤k≤N? Choose such that

 U-Boulder52 of 322 Smoothing factor The smoothing factor is the largest magnitude of the iteration matrix eigenvalues corresponding to the oscillatory Fourier modes: smoothing factor = max | k (R)| for N/2≤k≤N. Why only the upper spectrum? For R  with  = 2/3, the smoothing factor is 1/3 : | N/2 |=| N |= 1/3 & | k |< 1/3 for N/2<k<N. But | k | ≈ 1 -  k 2  2 h 2 for long waves (k << N/2). “MG” spectral radius?

 U-Boulder53 of 322 Convergence of Jacobi on Au = 0 Jacobi on Au = 0 with N = 63. Number of iterations needed to reduce initial error ||e||  by 0.01. Initial guess : 0102030405060 0 10 20 30 40 50 60 70 80 90 100 0102030405060 0 10 20 30 40 50 60 70 80 90 100 Wavenumber, k Unweighted JacobiWeighted Jacobi

 U-Boulder54 of 322 Weighted Jacobi = smoother (error) Initial error: Error after 35 iteration sweeps: Many relaxation schemes are smoothers: oscillatory error modes are quickly eliminated, but smooth modes are slowly damped.

 U-Boulder55 of 322 Similar analysis for other smoothers Gauss-Seidel relaxation applied to the 3-point difference matrix A (1-D model problem): A little algebra & trigonometry shows that 0102030405060 0 0.2 0.4 0.6 0.8 1 k sin(3  x i ) What’s w k look like for large k ? 3 sin(3  x i ) i/2 0102030405060 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 i k

 U-Boulder56 of 322 Gauss-Seidel eigenvectors These a VERY different from Jacobi’s eigenvectors. It’s not clear how smoothness depends on k. You cannot expect G-S to quickly reduce Fourier modes. You can only expect G-S to produce smooth results!

 U-Boulder57 of 322 Gauss-Seidel convergence Au = 0 Eigenvectors of R G are not the same as those of A!!! Gauss-Seidel mixes the modes of A. 0102030405060 0 10 20 30 40 50 60 70 80 90 100 Gauss-Seidel on Au = 0, with N = 63. Number of iterations needed to reduce initial error ||e||  by 0.01. Initial guess (modes of A): Wavenumber, k OK, so G-S does reduce oscillatory Fourier modes. But you will see that this depends on ordering. In fact, red-black G-S does not reduce high Fourier modes.

 U-Boulder58 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10: 

 U-Boulder59 of 322 Many relaxation schemes have the smoothing property: oscillatory error modes are quickly eliminated, while smooth modes are often very slow to disappear. We’ll turn this adversity around: the idea is to use coarse grids to take advantage of smoothing. 3. Elements of multigrid How? 1 st observation toward multigrid +1+1 +1+1

 U-Boulder60 of 322 Reason #1 for coarse grids: Nested iteration Coarse grids can be used to compute an improved initial guess for the fine-grid relaxation. This is advantageous because: –Relaxation on the coarse-grid is much cheaper: half as many points in 1-D, one-fourth in 2-D, one-eighth in 3-D,… –Relaxation on the coarse grid has a marginally faster convergence factor (| 1 (R)| ≈ 1 -  2 h 2 ): instead of

 U-Boulder61 of 322 Idea! Nested iteration Relax on Au = f on  4h to obtain initial guess v 2h. Relax on Au = f on  2h to obtain initial guess v h. Relax on Au = f on  h to obtain … final solution??? What if the error still has large smooth components when we get to the fine grid  h ? What is A 2h u 2h = f 2h ? Analogous to A h u h = f h for now. How do we migrate between grids? Hang on…

 U-Boulder62 of 322 Reason #2 for coarse grids: Smooth error becomes more oscillatory A smooth function: can be represented by linear interpolation from a coarser grid: On the coarse grid, the smooth error appears to be relatively higher in frequency: in this example it is the 4-mode out of a possible 15 on the fine grid, ~1/4 the way up the spectrum. On the coarse grid, it is the 4-mode out of a possible 7, ~1/2 the way up the spectrum. Relaxation on 2h is (cheaper &) faster on this mode!!!

 U-Boulder63 of 322 For k=1,2,…(N-1)/2, the k th mode is preserved on the coarse grid. 0123456 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Also, note that on the coarse grid. k=4 mode, N=11 grid k=4 mode, N=5 grid What happens to the modes between (N+1)/2 & N ?

 U-Boulder64 of 322 For k > (N+1)/2, w k h is disguised on the coarse grid: aliasing!!! For k > (N+1)/2, the k th mode on the fine grid is aliased & appears as the (N +1 - k) th mode on the coarse grid: 0123456 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 k=9 mode, N=11 grid k=3 mode, N=5 grid

 U-Boulder65 of 322 1-D interpolation (prolongation) Values at points on the coarse grid map unchanged to the fine grid. Values at fine-grid points NOT on the coarse grid are the averages of their coarse-grid neighbors.

 U-Boulder66 of 322 1-D interpolation (prolongation) to migrate from coarse to fine grids Mapping from the coarse grid to the fine grid: Let, be defined on,. Then where for. for (including boundaries).

 U-Boulder67 of 322 1-D prolongation operator P P = is a linear operator:  (N-1)/2  N. N = 7: has full rank, so  (P ) = {0}. When is v 2h = 0? rank = max # linearly independent cols or rows

 U-Boulder68 of 322 “Give To” stencil for P ] 1/2 1 1/2 [ o x o x o x o 1/2 1 1/2

 U-Boulder69 of 322 How well could v 2h approximate u? Imagine that a coarse-grid approximation v 2h has been found. How well could it approximate the exact solution u ? If u is smooth, a coarse-grid interpolant v 2h might do very well.

 U-Boulder70 of 322 How well could v 2h approximate u? Imagine that a coarse-grid approximation v 2h has been found. How well could it approximate the exact solution u ? If u is oscillatory, a coarse-grid interpolant v 2h cannot work well.

 U-Boulder71 of 322 Moral If what we want to compute is smooth, a coarse-grid interpolant could do very well. If what we want to compute is oscillatory, a coarse-grid interpolant cannot do very well. What if u is not smooth? Can we make it so? Can we make something smooth? How about e ? Can we smooth it? How would we get e & use it to get u ? Ae = r & u  v + e ! Just because the coarse grid can approximate e well doesn’t mean we know how to do it! But we will soon! Thus, use nested iteration on residual equation to approximate the error after smoothing!!!

 U-Boulder72 of 322 2 nd observation toward multigrid The residual equation: Let v be an approximation to the solution of Au = f, where the residual r = f -Av. Then the error e = u - v satisfies Ae = r. After relaxing on Au = f on the fine grid, e will be smooth, so the coarse grid can approximate e well. This will be cheaper & e should be more oscillatory there, so relaxation will be more effective. Therefore, we go to a coarse grid & relax on the residual equation Ae = r. e = 0 !What’s a good initial guess on grid 2h? How do we get to grid 2h?Stay tuned…

 U-Boulder73 of 322 Idea! Coarse-grid correction 2-grid Relax on Au = f on  h to get an approximation v h. Compute r = f - Av h. Transfer Ae = r to  2h somehow & relax on it to obtain an approximation to the error, e 2h. Correct the approximation v h  v h + e 2h. This is the essence of multigrid. We need a way to transfer Ae = r to  2h.

 U-Boulder74 of 322 Assume we’ve relaxed so much that e is smooth. Ansatz: e = Pv 2h for some coarse-grid v 2h. How do we characterize e so we can hope to compute it? Ae = r  A P v 2h = r 7x7 7x3 3x1 = 7x1 Too many equations now & too few unknowns! How about just eliminating every other equation? How about multiplying both sides by some 3x7 matrix? P TP T P T A P v 2h = P T r 3x7 7x7 7x3 3x1 = 3x1 ) ) A 2h 3x3 A way to coarsen Ae = r

 U-Boulder75 of 322 Mapping from the fine grid to the coarse grid: Let, be defined on,. Then where. 1-D restriction by injection

 U-Boulder76 of 322 1-D restriction by full weighting Let, be defined on,. Then where.

 U-Boulder77 of 322 1-D restriction (full-weighting) R = is a linear operator:  N  (N-1)/2. N = 7:. has rank, so dim(  (R)). Look at the columns of R associated with grid 2h.

 U-Boulder78 of 322 Prolongation & restriction are often nicely related For the 1-D examples, linear interpolation & full weighting are So they’re related by the variational condition for c in .

 U-Boulder79 of 322 2-D prolongation We denote the operator by using a “give to” stencil ] [. Centered over a C-point, it shows what fraction of the C-point’s value contributes to a neighboring F-point.

 U-Boulder80 of 322 “Get From” interpolation 1/2 1/4 [1/2 1/2] 1/4 Centered over a fine-grid point.

 U-Boulder81 of 322 2-D restriction (full weighting) We denote the operator by using a “get from” stencil [ ]. Centered over a C-point, it shows what fraction of the value of the neighboring F-point contributes to the value at the C-point.

 U-Boulder82 of 322 Now we put all these ideas together Nested Iteration –effective on smooth solution (components). Relaxation –effective on oscillatory error (components). Residual Equation –characterizes the error. –enables nested iteration for smooth error (components)!!! Prolongation (variables) & Restriction (equations) –provides pathways between coarse & fine grids.

 U-Boulder83 of 322 2-grid coarse-grid correction 1) Relax  1 times on A h v h = f h on  h with arbitrary initial guess v h. If h = h coarsest, then go to 6. 2) Compute r h = f h - A h v h. 3) Compute r 2h = I h 2h r h. 4) “Solve” A 2h e 2h = r 2h on  2h. 5) Correct fine-grid solution v h  v h + I 2h e 2h. 6) Relax  2 times on A h v h = f h on  h. fvGCv h hh  21 h What is e 2h here?

 U-Boulder84 of 322 2-grid coarse-grid correction Relax on f vA h h h  vAf r h h h h  Compute Restrict Solve Correct e I ê h h h h  2 2 Interpolate ê vv hhh   Relax on A h v h = f h

 U-Boulder85 of 322 What is A 2h ? For this scheme to work, we must have A 2h, a coarse- grid operator. For the moment, we will simply assume that A 2h is “the coarse-grid version ” of the fine-grid operator A h. Later we’ll return to the question of constructing A 2h.

 U-Boulder86 of 322 How do we “solve” the coarse- grid residual equation? Recursion! ? A 2h e 2h = r 2h ?

 U-Boulder87 of 322 V-cycle (recursive form) 1) Relax times on with initial given. 2) If is the coarsest grid, go to 4; else: 3) Correct:. 4) Relax times on with initial guess. vAfIf 2 h hh2h2h h h    1   2

 U-Boulder88 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:  

 U-Boulder89 of 322 Storage cost:v h & f h on each level. In 1-D, a coarse grid has about half as many points as the fine grid. In 2-D, a coarse grid has about one-fourth as many points as the fine grid. In d-dimensions, a coarse grid has about 2 -d as many points as the fine grid. 21 22221 N N d d dMdddd        32 Total storage cost: less than 2, 4/3, & 8/7 the cost of storage on the fine grid for 1-D, 2-D, & 3-D problems, respectively. 4. Implementation

 U-Boulder90 of 322 Computational cost Let one Work Unit (WU) be the cost of one relaxation sweep on the fine grid. Ignore the cost of restriction & interpolation (typically about 20% of the total cost). Consider a V-cycle with 2 pre-coarse-grid correction sweep (    = 2) & 1 post-coarse-grid correction sweep (    = 1). Cost of a V-cycle (in WUs): Cost is about 2, 4/3, & 8/7 X 3 WUs per V-cycle in 1, 2, & 3 dimensions, respectively. 21 3 22221 3        d dMddd32

 U-Boulder91 of 322 Convergence analysis First, a heuristic argument: –The convergence factor for the oscillatory error modes (smoothing factor) is small & bounded uniformly in h. smoothing factor = max | k (R)| for N/2≤k≤N. –Multigrid focuses the relaxation process on attenuating the oscillatory components on each level.  The overall multigrid convergence factor is small & bounded uniformly in h ! smooth k=1 oscillatory k=N k=N/2 Relax on fine grid Relax 1st coarse grid Bounded uniformly in h ≠ independent of h.

 U-Boulder92 of 322 Revisiting the model problem The BVP: The finite difference scheme: Truncation error:   f h2h2 vvv i iii  11 2 vv 0   N+1 0 i = 1,2,…,N         x xfxu 0 1 0 uu  0 10   f h2h2 u(h)u(h) u(h)u(h) u(h)u(h) i ii i   11 2  O(h 2 ) u (h) = exact PDE solution vector

 U-Boulder93 of 322 Actual error Au (h) + O(h 2 ) = f  Au (h) = f + O(h 2 )  Au = f ( u = discrete sol’n)  A(u (h) - u) = O(h 2 )  AE = O(h 2 ). E = discretization error So||E|| = ||A -1 O(h 2 )|| ≤ ||A -1 ||·||O(h 2 )|| =  max (A -1 )·O(h 2 ) = O(h 2 )/  min (A) or ||E|| = O(h 2 ).

 U-Boulder94 of 322 Convergence analysis a little more precisely... Continuous problem: Au = f, u i = u(x i ) Discrete problem:A h u h = f h, v h ≈ u h Global/discretization error: E i = u(x i ) - u i h ||E || ≤ Kh p (p = 2 for model problem & proper norm) Algebraic error: e i h = u i h - v i h For tolerance , assume the aim is to find v h so that the total error, ||e || = ||u (h) - v h || ≤ , where u (h) = (u (x i )). Then this objective is assured as follows: ||e || ≤ ||u (h) - u h || + ||u h - v h || = ||E|| + ||e h || ≤ .

 U-Boulder95 of 322 We can satisfy the convergence objective by imposing two conditions 1) ||E|| ≤  /2. Achieve this condition by choosing an appropriately small grid spacing h: Kh p =  /2. 2) ||e h || ≤  /2. Achieve this condition by iterating until ||e h || ≤  /2  = Kh p on grid h; then we’ve converged to the level of discretization error. Once discretization error & algebraic error are in balance, then it would be better to go to grid h/2 than to iterate more! 

 U-Boulder96 of 322 Convergence to the level of discretization error Use an MV scheme with convergence factor  < 1 bounded uniformly in h (fixed  1 &  2 ). Assume a d-dimensional problem on an NxNx…xN grid with h = N -1. Initial error is ||e h || = ||u h - 0 || = ||u h || = O(1). Must reduce this to ||e h || = O(h p ) = O(N -p ). We can determine the number of V-cycles needed for this, if we can bound the convergence factor, .

 U-Boulder97 of 322 Work to converge to the level of discretization error Using  V-cycles with convergence factor  gives an overall convergence factor of  . We therefore have   = O(N -p ), or   = O(log N ). Since 1 V-cycle costs O(1) WUs & 1 WUs is O(N d ), then converging to the level of discretization error using the MV method cost O(N d log N ). This compares to fast direct methods (fast Poisson solvers). But multigrid can do even better…

 U-Boulder98 of 322 Numerical example Consider the 2-D model problem (with  = 0): - u xx - u yy = 2[(1 - 6x 2 )y 2 (1 - y 2 )+(1 - 6y 2 )x 2 (1 - x 2 )] in the unit square, with u = 0 Dirichlet boundary. The solution to this problem is u (x,y) = - (x 4 - x 2 )(y 4 - y 2 ). We examine effectiveness of MV cycling to solve this problem on (N+1)x(N+1) grids [(N-1) x (N-1) interior points] for N = 16, 32, 64, 128.

 U-Boulder99 of 322 Numerical results MV cycling Shown are the results of 15 V(2,1)-cycles. We display, after each cycle, residual norms, total error norms, & ratios of these norms to their values after the previous cycle. N = 16, 32, 64, 128. ||r h || h = h ||r h || 2 scaled residual error ||e || h = h ||u (h) - v h || 2 scaled discrete total error

 U-Boulder100 of 322 A warning about bounds Bounds like ||e n + 1 || ≤  ||e n || & ||u (h) - u h || = O(h) are only just that--bounds! If you see behavior that suggests that these bounds are sharp (e.g., halving h halves the discretization error), then great. If you don’t see this behavior, don’t assume things are wrong. Think about this: O(h 2 ) = O(h) but generally O(h) ≠ O(h 2 ) !!! (Any process that is O(h 2 ) is also O(h), but the converse isn’t necessarily true.)

 U-Boulder101 of 322 Reconsideration You want to approximate u h. A good iteration is the V-cycle. What’s a good way to start it? Can you do better than v h  0? Start on the coarse grid! Use nested iteration for the V-cycle!

 U-Boulder102 of 322 Look again at nested iteration Idea: It’s cheaper to solve a problem (fewer iterations) if the initial guess is good. How to get a good initial guess: –“Solve” the problem on the coarse grid first. –Interpolate the coarse solution to the fine grid. Now, let’s use the V-cycle as the solver on each grid level! This defines the Full Multigrid (FMG) cycle.

 U-Boulder103 of 322 Full multigrid (FMG) Initialize Solve on coarsest grid Interpolate initial guess Perform V-cycle Interpolate initial guess Perform V-cycle ffff Hhhh  42

 U-Boulder104 of 322 FMG-cycle Restriction Interpolation High-Order Interpolation?

 U-Boulder105 of 322 FMG-cycle (recursive form) 1) Initializef h, f 2h,…, f H. 2) If h = H, then go to 4 (where MV is a direct solve); else:v 2h < — FMG (f 2h ). 3) Set initial guess: v h < — I 2h v 2h. 4) Perform v h < — MV (v h, f h )  times. v h < — FMG (f h ),  h We use  = 1.

 U-Boulder106 of 322 FMG cycle cost One V(2,1)-cycle is performed per level, at a cost of 3/(1 - 2 -d ) WUs per grid (where the WU is for the size of the finest grid involved). The size for the WU for coarse-grid j is 2 -jd times the size for the WU for the fine grid (grid 0). Hence, the cost of the FMG(2,1) cycle in WUs is less than [3/(1 - 2 -d )](1 + 2 -d + 2 -2d +…) = 3/(1 - 2 -d ) 2. d = 1: 12 WUs; d = 2: 16/3 WUs; d = 3: 192/49 WUs.

 U-Boulder107 of 322 Has discretization error been reached by FMG? If discretization error is achieved, then ||e h || = O(h 2 ) & the error norms at the “solution” points in the cycle should form a Cauchy sequence: ||e h ||  0.25 ||e 2h || We need to be more careful…

 U-Boulder108 of 322 Comparing the right things Problem: We are thinking that u 2h approximates u h to order O(h 2 ), when all we really know is that u h approximates u (h) to order O(h 2 ) (any h). We know that u (2h) and u (h) are the “same”, right? So, if u 2h approximates u (2h) to order O(4h 2 ) and u h approximates u (h) to order O(h 2 ), shouldn’t u 2h approximate u h to order O(4h 2 )? How, exactly? When we interpolate u 2h to grid h, what errors does interpolation introduce? Sorting out these comparison is a bit technical. In other words, here comes the algebra…

 U-Boulder109 of 322 Interpolation stability how interpolation affects error Property: || P e 2h || ≤  || e 2h || Reasoning: || P e 2h || ≤ ||P|| || e 2h || = ||P T P|| 1/2 || e 2h ||  ||P T P|| 1/2 ≤  2 In practice,   1.

 U-Boulder110 of 322 Approximation property how the discrete solutions approximate each other Property: ||u h - P u 2h || ≤  Kh 2 Reasoning: |(u (h) - P u (2h) ) i | = |u(x i ) - (u(x i - 1 ) + u(x i + 1 ))/2| ≤ |u”(  )|h 2 / 2  ||u (h) - P u (2h) || ≤ cKh 2  ||u h - P u 2h || ≤ ||u h - u (h) || + ||u (h) - P u (2h) || + || P u (2h) - P u 2h || ≤ Kh 2 + cKh 2 + || P ||·||u (2h) - u 2h || ≤ Kh 2 + cKh 2 +  K(2h) 2 ≤  c  )Kh 2 In practice,   5.

 U-Boulder111 of 322 FMG accuracy ||e h || ≤ Kh 2 Assume: ||e 2h || ≤ K(2h) 2 induction hypothesis ||u h - P u 2h || ≤  Kh 2 approximation property (  ) || P w 2h || ≤  || w 2h || interpolation stability (  ) Triangle inequality: ||e h || = ||u h - P v 2h || ≤ ||u h - P u 2h || + || P (u 2h - v 2h )|| ≤  Kh 2 +  K(2h) 2 = (  + 4  )Kh 2  ||e h || ≤ “9”Kh 2 So we need only reduce ||e h || by “0.1”!!!

 U-Boulder112 of 322 Numerical example Consider again the 2-D model problem (with  = 0): - u xx - u yy = 2[(1 - 6x 2 )y 2 (1 - y 2 )+(1 - 6y 2 )x 2 (1 - x 2 )] inside the unit square, with u = 0 on the boundary. We examine the effectiveness of FMG cycling to solve the problem on (N+1)x(N+1) grids [(N-1)x(N-1) interior points] for N = 16, 32, 64, 128.

 U-Boulder113 of 322 FMG results 3 FMG cycles & comparison with MV cycle results ||e || h = h ||u (h) - v h || 2 scaled discrete total error 1.03e-04 2.58e-05 6.44e-06 1.61e-06

 U-Boulder114 of 322 Diagnostic tools for debugging the code, the method, the problem Finding mistakes in codes, algorithms, concepts, & the problem itself challenges our scientific abilities. This challenge is especially tough for multigrid: –Interactions between multilevel processes can be very subtle. –It’s often not easy to know how well multigrid should perform. Achi Brandt: –“The amount of computational work should be proportional to the amount of real physical changes in the computed solution.” –“Stalling numerical processes must be wrong.” The “computational culture” is best learned by lots of experience & interaction, but some discussion helps.

 U-Boulder115 of 322 Tool # 1: Be methodical Modularize your code. Test the algebraic solver first. Test the discretization next. Test the FMG solver last. Beware of boundaries, scales, & concepts. Ask whether the problem itself is well posed.

 U-Boulder116 of 322 Tool # 2: Start simply Start from something that already works if you can. Introduce complexities slowly & methodically, testing thoroughly along the way. Start with a very coarse fine grid (no oxymoron intended). Start with two levels if you can, using a direct solver or lots of cycles on coarse grids if nothing else. If you find trouble, your first job is to find the simplest case where that trouble is still evident!!!

 U-Boulder117 of 322 Tool # 3: Expose trouble Start simply, but don’t let niceties mask trouble : Set reaction/Helmholtz terms to zero. Take infinite or very big time steps. Don’t take 1-D too seriously, not even 2-D.

 U-Boulder118 of 322 Tool # 4: Test fixed point property Relaxation shouldn’t alter the exact solution of the linear system (up to machine precision). Create a right side: f = Au* with u* given. Make sure u* satisfies the right boundary conditions. Test relaxation starting with u* : Is r = 0, is it zero after relaxation, does u* change? Test coarse-grid correction starting with u* : Is the correction zero?

 U-Boulder119 of 322 Tool # 5: Test on Au = 0 The exact solution u* = 0 is known! Residual norm ||Au|| & error norm ||u|| are computable. Norms ||Au|| & ||u|| should eventually decrease steadily with a rate that might be predicted by mode analysis. Multigrid can converge so fast that early stalling suggests trouble when it’s just that all machine-representable numbers in a nonzero u* have already been computed! Computing r = f - Au & updating u shouldn’t have trouble with machine precision if you have u* = 0 & thus f = 0.

 U-Boulder120 of 322 Tool # 6: Zero out residual Using a normal test, try multiplying the residual by 0 before you go to the coarse grid. Check to see that the coarse-grid corrections are 0. Compare this test with a relaxation-only test--the results should be identical.

 U-Boulder121 of 322 Tool # 7: Print out residual norms Use the discrete L 2 norm: ||r|| h = (h d   r i 2 ) 1/2 = h d/2 ||r || 2 Output ||r|| h after each pre- & post-relaxation sweep. These norms should decline to zero steadily for each h. The norm after post-relaxation should be consistently smaller than after pre-relaxation--by the predictable convergence factor at least.

 U-Boulder122 of 322 A warning about residuals Residuals could mask large smooth errors: –If ||e||=O(h 2 ), then ||r||=O(h 2 ) for smooth e, but ||r||=O(1) for oscillatory e. ( N / 1 = O(h -2 )!!!) –This means that if we are controlling the error by monitoring ||r||, if we don’t know the nature of e, & if we don’t want to risk large e by requiring only ||r||=O(1), then we may have to work very hard to make ||r||=O(h 2 ). (We could in effect be trying to make ||e||=O(h 4 ) !!!) Multigrid tends to balance the errors, so ||r||=O(h 2 ) tends to mean ||e||=O(h 2 ).

 U-Boulder123 of 322 Tool # 8: Graph the error Run a test on a problem with known solution (Au = 0). Plot algebraic error before & after fine-grid relaxation. Is the error oscillatory after coarse-grid correction? Is the error much smoother after fine-grid relaxation? Are there any strange characteristics near boundaries, interfaces, or other special phenomena?

 U-Boulder124 of 322 Tool # 9: Test two-level cycling Replace the coarse-grid V-cycle recursive call with a direct solver if possible, or iterate many times with some method known to “work” (test ||r || to be sure it’s very small), or use many recursive V-cycle calls. This can be used to test performance between two coarser levels, especially if residual norm behavior identifies trouble on a particular level.

 U-Boulder125 of 322 Tool # 10: Beware of boundaries Boundaries usually require special treatment of the stencils, intergrid transfers, & sometimes relaxation. Special treatment often means special trouble, typically exposed in later cycles as it begins to infect the interior. Replace the boundary by periodic or Dirichlet conditions. Relax more at the boundary, perhaps using direct solvers. Make sure your coarse-grid approximation at the boundary is guided by good discretization at the fine-grid boundary.

 U-Boulder126 of 322 Tool # 11: Test for symmetry If your problem is symmetric or includes a symmetric case, test for it. Check symmetry of the fine-grid & coarse-grid matrices: are a ij & a ji relatively equal (to machine precision). Be especially watchful for asymmetries near boundaries.

 U-Boulder127 of 322 Tool # 12: Check for compatibility This is a bit ahead of schedule, but consider the problem -u ” = f with u ’(0) = u ’(1) = 0. It’s singular: If u = 1, then -u ” = 0 & u ’(0) = u ’(1) = 0. It’s is solvable iff f  Range(  xx ) =    xx ) = {1}  or f . First fix the grid h right side: f h  f h - ( / )1. Do this on coarse grids too: f 2h  f 2h - ( / )1. Uniqueness is also a worry: u h  u h - ( / )1.

 U-Boulder128 of 322 Tool # 13: Test for linearity Again, this is a bit ahead of schedule, but if you’re writing a nonlinear FAS code, it should agree with the linear code when you test it on a linear problem. Try it. Move gradually to the target nonlinear test problem by putting a parameter in front of the nonlinear term, then running tests as the parameter changes slowly from 0 to 1.

 U-Boulder129 of 322 Tool # 14: Use a known PDE solution Set up the source term (f = Au* in  ) & data (g = u* on  ). Do multigrid results compare qualitatively with sampled u* ? Monitor ||u* - u h || h. Test a case with no discretization error (let u* = 2 nd degree polynomial). The algebraic error should tend steadily to 0. Test a case with discretization error (let u* have a nonzero 3 rd derivative). The algebraic error should decrease rapidly at first, then stall at discretization error level. Check error behavior as you decrease h. Does it behave like O(h 2 ) (h halved  error halved) or whatever it should behave like?

 U-Boulder130 of 322 Tool # 15: Test FMG accuracy Make sure first that the algebraic solver converges as predicted, with uniformly bounded convergence factors. Test the discretization using Tool # 14. Compare FMG total error to discretization error for various h. You might need to tune the FMG process here (play with the number of cycles & relaxation sweeps).

 U-Boulder131 of 322 Computing assignments Document: norms/weights, V( 1, 2 ), errors, labels (table, graph) Use various scenarios: Ax = 0, Ax = f, varying N & i & , Jacobi/Gauss-Seidel Thoroughly test: don’t stop until you get what you expect. compare with known solution, text, others. study discretization & algebraic errors. report on “asymptotic” factors. Be kind to the reader: code = zzz… tables = + tables & graphs = ++ tables & graphs & discussion (clear, concise) = +++ Discuss, discuss, discuss: what do you see & think? what did you learn? by hand ok 

 U-Boulder132 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:   

 U-Boulder133 of 322 What is A 2h ? Recall the 2-grid coarse-grid correction scheme: –1) Relax on A h u h = f h on  h to get v h. –2) Compute f 2h = I h 2h (f h - A h v h ). –4) Solve A 2h u 2h = f 2h on  2h. –5) Correct fine-grid solution v h  v h + I 2h u 2h. Assume that e h  Range(I 2h ). Then the residual equation can be written r h = A h e h = A h I 2h u 2h for some u 2h   2h. This characterizes u 2h, but with too many equations. How does A h act on Range(I 2h )? h h h h 5. Some theory

 U-Boulder134 of 322 How does A h act on Range(I 2h )? Thus, the odd rows of A h I 2h are zero (1-D only) & r 2i + 1 = 0. So we keep the even rows of A h I 2h for the residual equations on  2h. We do this by applying restriction, either injection or full weighting: h h h We use full weighting from now on unless otherwise stated..

 U-Boulder135 of 322 Building A 2h : The Galerkin condition The residual equation on the coarse grid is A 2h = RAP How do we know RAP is symmetric? We thus identify the coarse-grid operator as If P T =  R (so that R T = (1/  )  P ), then (RAP) T = P T A T R T =  (1/  )  RAP = RAP.

 U-Boulder136 of 322  Why is this the i th row of A 2h ? Computing the i th row of A 2h Compute, where. 2 (2h) 2  1 2  1 2

 U-Boulder137 of 322 The i th row of A 2h is, which is the  2h version of A h. Note that IF relaxation on  h leaves only error in the range of interpolation, then solving A 2h u 2h = f 2h determines the error exactly! This is generally not feasible, but this logic leads to a very plausible representation for A 2h. The i th row of A 2h looks a lot like the i th row of A h !

 U-Boulder138 of 322 Variational properties of coarsening The definition for A 2h that resulted from the foregoing line of reasoning is useful for both theoretical & practical reasons. Together with the commonly used relationship between restriction & prolongation, we have the variational properties: Galerkin Condition for c in .

 U-Boulder139 of 322 Properties of restriction in a little more detail… Full Weighting: or  N  (N-1)/2 N = 7: has rank & null space  ( ) with dim.

 U-Boulder140 of 322 Spectral properties of restriction How does act on the eigenvectors of ? Consider, 1  k  N, 0  j  N. A little algebra & trigonometry shows that for 1  k  (N-1)/2.

 U-Boulder141 of 322 Spectral properties (cont’d) i.e., [k th mode on ] = c k [ k th mode on ] : N = 7, k = 2 : N = 3, k = 2

 U-Boulder142 of 322 Spectral properties (cont’d) Let k’ = N + 1 - k for 1 ≤ k ≤ (N - 1)/2, so that (N - 3)/2 ≤ k’ ≤ N. A little algebra & trigonometry shows that

 U-Boulder143 of 322 Spectral properties (cont’d) i.e., [(N + 1 - k) th mode on ] = -s k [k th mode on ] : N = 7, k = 6 : N = 3, k = 2

 U-Boulder144 of 322 Spectral properties (cont’d) Summarizing: Complementary modes: }{ 

 U-Boulder145 of 322 Null space of restriction Observe that  (I h 2h ) = span(A h ê i h ), where i is odd & ê i h is the i th unit vector. Let  i = A h ê i h. While the  i looks oscillatory, it generally contains all Fourier modes of A h : All the Fourier modes of A h are needed to represent the null space of restriction! 

 U-Boulder146 of 322 Properties of interpolation Interpolation: or :  (N-1)/2  N N = 7: has full rank & null space {0}.

 U-Boulder147 of 322 Spectral properties of interpolation How does act on the eigenvectors of ? Consider, 1  k  (N-1)/2, 0  j  (N+1)/2. A bit of work shows that the modes of are NOT “preserved” by, but that the space is “preserved”:

 U-Boulder148 of 322 Spectral properties of interpolation Interpolation of smooth modes excites oscillatory modes on. Note that if, then is 2 nd -order interpolation. I 2 h h wswc h kk h kk  w k 2h

 U-Boulder149 of 322 Range of interpolation The range of is the span of the columns of. Let be the i th column of. All the Fourier modes of are needed to represent Range( ).  i  k h kk k h i  N  1 bwb0

 U-Boulder150 of 322 Use all the facts to analyze the coarse-grid correction scheme relax only before correction 1) Relax once on :. 2) Compute & restrict residual. 3) Solve residual equation. 4) Correct fine-grid solution. The entire process appears as The exact solution satisfies vAfIf 22 h hhh h h   vRv hh  + B f h AfIAIRvv hhh h hh h hh       2 1 2 2  vR h + B f h   + B f h AfIAIuRu hhh h hh h hh      2 1 2 2  uR h  

 U-Boulder151 of 322 CG error propagation Subtracting the previous two expressions, we get How does CG act on the modes of ? Assume e h consists of the modes & for &. We know how act on &. eRAIAIIe h hh h hh h h    2 1 2 2 

 U-Boulder152 of 322 CG error propagation For now, assume no relaxation. Then is invariant under CG: where

 U-Boulder153 of 322 CG error propagation for k <<N Consider the case k << N (extremely smooth & oscillatory modes): Hence, CG eliminates the smooth modes but does not damp the oscillatory modes of the error!

 U-Boulder154 of 322 CG with relaxation Next, include one relaxation sweep. Assume that the relaxation R preserves the modes of A h (although this is often unnecessary). Let k denote the eigenvalue of R associated with w k. For k < < N/2: Small! wswsw kkkkkkk   wcwcw kkkkkkk  

 U-Boulder155 of 322 Crucial observation Between relaxation & coarse-grid correction, both smooth & oscillatory components of the error are effectively damped. This is the “spectral” picture of how multigrid works. We examine now another viewpoint, the “algebraic” picture of multigrid.

 U-Boulder156 of 322 Recall the variational properties All the analysis that follows assumes that the variational properties hold: Galerkin Condition for c in .

 U-Boulder157 of 322 Fundamental Theorem of Linear Algebra If x  N (B), then, for any y  R (B T ), we have y = B T z and, hence, = = = 0. So, x  R (B T )  and, thus, N (B) ⊂ R (B T ) . But if x  R (B T ) , then x  B T z for any z, so 0 = =. Picking z = Bx shows x  N (B), so N (B) ⊃ R (B T )   N (B) = R (B T ) 

 U-Boulder158 of 322 Algebraic interpretation of CG Consider the subspaces that make up  h &  2h. From now on, ‘R( )’ refers to the Range of a linear operator & ‘N( )’ to its Null Space. From the Fundamental Theorem of Linear Algebra: or But we really care about N (I h 2h A h ) !?!? IRIN  =  T h h h h 22 IRIN  =  h h h h2 2  

 U-Boulder159 of 322 Subspace decomposition of  h If u h  N (I h 2h A h ), then, for any u 2h, we have 0 =  I h 2h A h u h, u 2h  =  A h u h, I 2h u 2h , so, where means. Moreover, any can be written as, where &. Hence, we get the “energy-orthogonal” decomposition. h “energy” inner product

 U-Boulder160 of 322 Characteristics of the subspaces Since for some, we associate with the smooth components of. But, generally has all Fourier modes in it. Recall the basis vectors for : Similarly, we associate with oscillatory components of, although generally has all Fourier modes in it as well. Recall that is spanned by, so is spanned by the unit vectors for odd i, which “look” oscillatory. e T h i         0010 00

 U-Boulder161 of 322 Algebraic analysis CG Recall that (without relaxation) First note that if, then. This follows since for some & therefore It follows that N(CG)=R( ), that is, the null space of CG is the range of interpolation. by Galerkin property I 2h h.. What does this imply?

 U-Boulder162 of 322 More algebraic analysis of CG Next, note that if, then Thus, CG is the identity on.  What does this imply?

 U-Boulder163 of 322 How does the algebraic picture fit with the spectral view? We may view in two ways: = that is, or Low frequency modes 1  k  N/2 High frequency modes N/2 < k < N Are these “orthogonal” decompositions?.

 U-Boulder164 of 322 Actually, each view is just part of the picture The operations we have examined work on different spaces! While is mostly oscillatory, it isn’t, & while is mostly smooth, it isn’t. Relaxation eliminates error from. Coarse-grid correction eliminates error from.

 U-Boulder165 of 322 How it actually works (cartoon)

 U-Boulder166 of 322 Relaxation eliminates H, but increases the error in.

 U-Boulder167 of 322 CG eliminates error in, but increases error in H

 U-Boulder168 of 322 CG eliminates error in, but increases error in H Why is this working well?

 U-Boulder169 of 322 What if L points away from ?

 U-Boulder170 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:    

 U-Boulder171 of 322 Some reflection What do you want out of this course? Do you seem to be getting there? Can we get there more directly? Start thinking about your project!

 U-Boulder172 of 322 6. Nonlinear problems How should we approach the nonlinear system A(u) = f & can we use MG to solve it? A fundamental relation we’ve relied on is the linear residual equation: Au - Av = f - Av ⇒ Ae = r. We can’t rely on this now since a nonlinear A(u) generally means A(u) - A(v) ≠ A(e). HANG ON !!!

 U-Boulder173 of 322 The nonlinear residual equation We still base our development around the residual equation, now the nonlinear residual equation: A(u) = f ⇒ A(u) - A(v) = f - A(v) ⇒ A(u) - A(v) = r How can we use this equation as the basis for a solution method?

 U-Boulder174 of 322 Newton’s method for scalar F:  Expand F in a Taylor series about x : F(x + s) = F(x) + sF’ (x) + s 2 F” (  ). Ex: (x + s)e (x + s) - 1 = x e x - 1 + s (1 + x) e x + h.o.t. Dropping higher-order terms (h.o.t.), if x + s is a solution, 0 = F(x) + sF’ (x) ⇒ s = -F(x)/F’ (x). We thus arrive at Newton’s method: x ← x - F(x)/F’ (x) Best known & most important nonlinear problem solver! We wish to solve F(x) = 0. Ex: F(x) = x e x - 1, F’ (x) = (1 + x) e x. exponent, not error

 U-Boulder175 of 322 Newton’s method for systems Taylor series about v : A(v + e) = A(v) + J(v)e + h.o.t. Ex: - u ”(x) + u(x) e u(x) = f may be discretized as error, not exponent

 U-Boulder176 of 322 Newton’s method for general systems The system A(u) = f in vector form is Expanding A(v + e) in a Taylor series about v : A(v + e) = A(v) + J(v)e + h.o.t.

 U-Boulder177 of 322 Newton for systems (cont’d) J(v) is the Jacobian If u = v + e is a solution, f = A(v) + J(v)e + h.o.t., so e ≈ [J(v)] -1 (f - A(v)). This leads to the iteration v  v + [J(v)] -1 (f - A(v))

 U-Boulder178 of 322 Newton’s via the residual equation The nonlinear residual equation is A(v + e) - A(v) = r. Expanding A(v + e) in a two-term Taylor series about v & ignoring h.o.t.: A(v) + J(v) ê - A(v) = r or J(v) ê = r. Newton’s method is thus: v  v + [J(v)] -1 r, r = f - A(v) ê ≈ e

 U-Boulder179 of 322 How does multigrid fit in? One obvious method is to use multigrid to solve J(v) ê = r at each iteration step. This method is called Newton-MG & can be very effective. However, we might want to use multigrid ideas to treat the nonlinearity directly. To do that, we need to specialize multigrid components (relaxation & coarsening) for the nonlinear case.

 U-Boulder180 of 322 What is nonlinear relaxation? Several of the common relaxation schemes have nonlinear counterparts. For A(u) = f, we describe nonlinear Gauss-Seidel: For each i = 1, 2, …, N : Change the value of v i so that the i th equation is satisfied, that is, so that (A(v)) i = f i. Equivalently: For each i = 1, 2, …, N : Find s ∈ ℜ such that (A(v + s  i )) i = f i, where  i is the i th canonical unit basis vector.

 U-Boulder181 of 322 How is nonlinear Gauss-Seidel done? Each (A(v)) i = f i is a nonlinear scalar equation for v i. We can use the scalar Newton’s method to solve! Example: - u ”(x) + u(x) e u(x) = f may be discretized so that (A(v)) i = f i is given by Newton iteration for v i is given by fev vv h i v i h vvv ii      i iii  2 2 11 2 2 v i  1 i v e   fv h vvv i v i iii  2 11 2 i e

 U-Boulder182 of 322 How do we do coarsening for nonlinear multigrid? Recall the nonlinear residual equation A(v + e) - A(v) = r. In multigrid, we obtain an approximate solution v h on the fine grid, then solve the residual equation on the coarse grid. The residual equation on  2h appears as A 2h (v 2h + e 2h ) - A 2h (v 2h ) = r 2h. v 2h ? A 2h ?

 U-Boulder183 of 322 Look at the coarse residual equation We must evaluate the quantities on  2h in A 2h (v 2h + e 2h ) - A 2h (v 2h ) = r 2h. Given v h, a fine-grid approximation, we restrict the residual to the coarse grid:. For v 2h, we restrict v h by. vAfIvIAevIA 222 2 22 h hhh h h h h h hh h h h  Thus, vAfIvIAevIA 222 2 22 h hhh h h h h h hh h h h 

 U-Boulder184 of 322 We’ve obtained a coarse-grid equation of the form A 2h (u 2h ) = f 2h Consider the coarse-grid equation: We solve for & obtain We then apply the correction: f 2h all quantities are known coarse-grid unknown eIvv h h h hh  ← 2 2

 U-Boulder185 of 322 Full approximation scheme (FAS) 2-grid form Perform nonlinear relaxation on to obtain an approximation. Restrict the approximation & its residual:. Solve the coarse-grid equation:. Extract 2h approximation to h error:. Interpolate & correct:. eIvv h h h hh   2 2

 U-Boulder186 of 322 A few observations about FAS If A is a linear operator, then FAS reduces directly to the linear two-grid correction scheme: An exact solution to the fine-grid problem is a fixed point of the FAS iteration: 0 0

 U-Boulder187 of 322 A few more observations about FAS The FAS coarse-grid equation can be written as where is the so-called tau correction term & is the original 2h source term, provided you choose it that way:. In general, since, the solution to the FAS coarse-grid equation is not the same as the solution to the original coarse-grid problem. The tau correction is as a way to alter the coarse- grid equation to enhance its approximation properties. f=I 2hh h f 2h f hh 2h2h

 U-Boulder188 of 322 Still more observations about FAS A true multilevel FAS process is recursive, using FAS to solve the nonlinear  2h problem using  4h. Hence, FAS is generally employed in a V- or W- cycling scheme.

 U-Boulder189 of 322 Even more observations about FAS For linear problems, we use FMG to obtain a good initial guess on the fine grid. Convergence of nonlinear iterations depends critically on having a good initial guess. When FMG is used for nonlinear problems, the interpolant is generally accurate enough to be in the basin of attraction of the fine-grid solver. Thus, whether FAS, Newton, or Newton-multigrid is used on each level, one FMG cycle should provide a solution accurate to the level of discretization, unless the nonlinearity is extremely strong.

 U-Boulder190 of 322 Intergrid transfers for FAS Generally speaking, the standard operators (linear interpolation, full weighting) work effectively in FAS schemes. For strongly nonlinear problems or for the coarse- grid approximation that is to become a fine-grid initial guess, higher-order interpolation (e.g., cubic interpolation) may be beneficial.

 U-Boulder191 of 322 What is A 2h (u 2h ) in FAS? As in the linear case, there are two basic possibilities: 1. A 2h (u 2h ) is determined by discretizing the nonlinear operator, A(u), in the same fashion as was employed to obtain A h (u h ), except that the coarser mesh spacing is used. 2. A 2h (u 2h ) is determined from the Galerkin condition where the action of the Galerkin product can be captured in an implementable formula. A 2h  u 2h  I h 2h A h  u 2h  I 2h h The first method is usually easier & more common.

 U-Boulder192 of 322 Nonlinear problems: An example Consider on the unit square, [0,1] x [0,1], with homogeneous Dirichlet boundary conditions & a regular h = 1/128 Cartesian grid. Suppose the exact solution is.  u  x  y    u  x  y  e u  x  y   f  x  y 

 U-Boulder193 of 322 Discretization of the nonlinear example The operator can be written (sloppily) as Relaxation (nonlinear Gauss-Seidel) is given by e v vv v h ji h h ji h ji  ji    f ji   vA h h    4 2 1 ji  h e

 U-Boulder194 of 322 FAS & Newton’s method on FAS V(2,1)-cycles until ||r|| < 10 -10. Newton’s Method with exact inner solves until ||r|| < 10 -10. N = 127  u  x  y    u  x  y  e u  x  y   f  x  y 

 U-Boulder195 of 322 Newton, Newton-MG, & FAS on Newton uses exact solves, Newton-MG is inexact Newton with a fixed number of inner V(2,1)-cycles for the Jacobian problem, overall stopping criterion ||r|| < 10 -10. N = 127,  = 10  u  x  y    u  x  y  e u  x  y   f  x  y 

 U-Boulder196 of 322 Compare FMG-FAS & FMG-Newton-MG We will do one FMG cycle using a single FAS V(2,1) -cycle as the “solver” at each new level. We then follow that with as many FAS V(2,1)-cycles as is needed to obtain ||r|| < 10 -10. Next, we will do one FMG cycle using a Newton-MG step at each new level (with one linear V(2,1)-cycle as the Jacobian “solver.”) We then follow that with as many Newton-multigrid steps as is needed to obtain ||r|| < 10 -10.  u  x  y    u  x  y  e u  x  y   f  x  y 

 U-Boulder197 of 322 Compare FMG-FAS & FMG-Newton-MG  u  x  y    u  x  y  e u  x  y   f  x  y  N = 127,  = 10

 U-Boulder198 of 322 Remembering coarse-grid correction Au = f Relax (damped Jacobi) to smooth e = u - v : v  v - ω D -1 (Av - f). Form the residual equation Ae = r : Ae = A(u - v) = f - Av = r. Use premise that smooth error  e = I 2h e 2h : AI 2h e 2h = r. Use transpose I 2h = (I 2h ) T to reduce equations: I 2h AI 2h e 2h = I 2h r. h h h h h h h I2hI2h  fewer unknowns fewer equations h A2hA2h

 U-Boulder199 of 322 Motivating FAS for nonlinear A A(v + e) = f  A 2h (v 2h + e 2h ) = “f 2h ” What overriding principle can we find to get from h to 2h ? With known v, how do we discretize a PDE of the form A(v + e) = f ? Example:  uu’ - u’’ = f   (v + e) (v + e)’ - (v + e)’’ = f. Expand to get an equation in e of form g + ae + be’ + …:  vv’ - v’’ +  (v’e + ve’ + ee’ ) - e’’ = f. A(v)A(v) ??

 U-Boulder200 of 322 Differential residual equation  (v’e + ve’ + ee’ ) - e’’ = f - A(v)  r(v) Right side (analogous to injection): r(v)  r j h = r(x j ). Coefficients: v  v j h = v(x j ), v’  (v’) j h = (v(x j+1 ) - v(x j-1 ))/(2h). Unknowns: e  e j h, e’  (e’) j h = (e j+1 - e j-1 )/(2h), e’’  (e’’) j h = (e j+1 - 2e j + e j-1 )/h 2. hh hhh

 U-Boulder201 of 322 Leading to FAS…  uu’-u’’ = f  A(v+e) = f   (v’e+ve’+ee’ )-e’’ = f-A(v) Fine-grid residual equation (at h point 2j): Coarse-grid residual equation (at 2h point j): FAS   v 2j h e 2j  1 h  e 2j  1 h 2h  e 2j  1 h  2e 2j h  e 2j  1 h h 2   e 2j h v 2j  1 h  v 2j  1 h 2h   e 2j h e 2j  1 h  e 2j  1 h 2h  r 2j h  e j  1 2h  2e j 2h  e j  1 2h (2h) 2   v 2j h e j  1 2h  e j  1 2h 4h   e j 2h v  2 h  v  2 h 4h   e j 2h e j  1 2h  e j  1 2h 4h  I h 2h r h  j vAfIvIAevIA 222 2 22 h hhh h h h h h hh h h h  

 U-Boulder202 of 322 Example: Newton-MG vs. FAS PDE - u ”(x) + u(x) e u(x) = f(x). Discretization:.   f e v h vvv j v j jjj  2 11 2 j (er, ODE):

 U-Boulder203 of 322 One Newton-MG step Step 0. Given v, form the grid h linear correction equation: Initialize the Newton correction approximation: e = 0. Step 1: Relax on the grid h linear equation. Step 2: Solve the grid 2h error correction equation: Step 3: Correct the grid h Newton correction: Step 4: Stop if you’ve “solved” the linear equation well enough for Newton correction e & set v  v + e. Else, leave v alone & return to Step 1.  e j  1  2e j  e j  1 h 2  (1  v j )e v j e j  r j  f j   v j  1  2v j  v j  1 h 2  v j e v j    e j  1 2h  2e j 2h  e j  1 2h (2h) 2  (1  v 2j2j )e v 2j2j e j 2h  r 2j  (  e 2j  1  2e 2j  e 2j  1 h 2  (1  v 2j )e v 2j e 2j ) e  e  I 2h h e 2h

 U-Boulder204 of 322 One FAS step Step 0. Given v, form the grid h nonlinear equation: Step 1: Relax on the grid h nonlinear equation to improve v. Step 2: Solve the grid 2h FAS correction equation: Step 3: Correct the grid h approximation v : v  v  I 2h h e 2h   fev h vvv j v j jjj  2 11 2 j  e j  1 2h  2e j 2h  e j  1 2h (2h) 2  v 2j h  e j 2h  e v 2j h  e j 2h  v 2j h e v 2j h  I h 2h r h  j

 U-Boulder205 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:     

 U-Boulder206 of 322 7a. Neumann boundary conditions Consider the 1-D problem - u” (x) = f(x), 0 < x < 1, u’ (0) = u’ (1) = 0. We discretize on the interval [0,1] with h = 1/(N+1) grid spacing & nodes x j = jh, j = 0,1,2, …, N+1. We extend the interval with two ghost points: 7. Selected applications 0 x 1 N+2 0 1 j-1 j j+1 N N+1

 U-Boulder207 of 322 Central differences at boundary We use differences as before, but now also for the derivative in the Neumann condition: This yields the system N+2 0 1 j-1 j j+1 N N+1

 U-Boulder208 of 322 Eliminating the ghost points Use the boundary conditions to eliminate, Eliminating the ghost points in the j = 0 & j = N + 1 equations gives the (N+2)x(N+2) system of equations:

 U-Boulder209 of 322 Write the system in matrix form We can write A h u h = f h, where Note that A h is (N+2)x(N+2) & nonsymmetric, & the system involves unknowns & at the boundaries.

 U-Boulder210 of 322 We must consider compatibility The problem - u” (x) = f(x), for 0 < x < 1, with u’ (0) = u’ (1) = 0, is not well-posed! If u(x) is a solution, then so is u(x) + constant. We cannot be certain a solution exists. If one does, it must satisfy This integral compatibility condition is necessary! If f(x) doesn’t satisfy it, there is no solution!

 U-Boulder211 of 322 The well-posed system The compatibility condition is necessary for a solution to exist. In general, it is also sufficient:  ∂ 2 / ∂ x 2 is a well-behaved operator on the space of functions u(x) that have zero mean. Thus, we may conclude that if f(x) satisfies the compatibility condition, then the problem is well-posed: - u” (x) = f(x), 0 < x < 1, u’ (0) = u’ (1) = 0, ∫ 0 1 u (x)dx = 0. The last says: of all possible solutions u(x) + constant, we choose the one with zero mean.

 U-Boulder212 of 322 The discrete problem is not well posed Since all row sums of A h are zero, then 1 h ∈ N(A h ). It’s easy to see that dim(N(A h )) = 1, so N(A h ) = span{1 h }. By the Fundamental Theorem of Linear Algebra, A h u h = f h has a solution if & only if f h ∈ N((A h ) T ) . It is easy to show that N((A h ) T ) = c(1/2, 1, 1, …, 1, 1/2) T. Thus, A h u h = f h has a solution if & only if f h  c(1/2, 1, 1, …, 1, 1/2) T. That is,

 U-Boulder213 of 322 We have two issues to consider A h u h = f h Solvability: A solution exists iff f h ∈ N((A h ) T ) . Uniqueness: If u h is a solution, then so is u h + v h for any v h ∈ N(A h ). Note that if A h = (A h ) T, then N((A h ) T ) = N(A h ) & solvability & uniqueness can be handled together. This is easily done. Multiply the first & last equations by 1/2, giving

 U-Boulder214 of 322 The new system is symmetric We have the symmetric system : Solvability is guaranteed by ensuring that is orthogonal to the constant vector 1 h : o o

 U-Boulder215 of 322 One-sided differences at boundary a similar result No ghost points: This yields the system 0 1 j-1 j j+1 N N+1 uu u  ≈  ´ h 0 1  uu u   ´ h 1 NN  1 ≈ uu 1    0 h 1 ≤≤ Nj u u N+1    0 h 2 2

 U-Boulder216 of 322 The well-posed discrete system The (N+3)x(N+2) system is: or, more simply (choose the zero mean solution) for one-sided scheme 0 0

 U-Boulder217 of 322 Multigrid for the Neumann problem We must have the interval endpoints on all grids Relaxation is performed at all points, including endpoints: We add a global Gram-Schmidt step after relaxation on each level to enforce the zero-mean condition:

 U-Boulder218 of 322 Interpolation must include the endpoints We use linear interpolation:

 U-Boulder219 of 322 Restriction also treats the endpoints For restriction, we use, yielding the values 42 fff 10 2 0 hhh  11 fff N hh N h N  1 2 1  2 1 4 1 424 ffff h jj hh j h j12212 2  111 

 U-Boulder220 of 322 The coarse-grid operator We compute the coarse-grid operator using the Galerkin condition.       22 2 02 2 22 2 02 2 02 2 0 h h h h h h h h h h h h h h 4 1 4 1 2 1 0 2 1 0 2 1 1 01 hh IAI hh IA I j 2h = 0 1

 U-Boulder221 of 322 Coarse-grid solvability Assuming satisfies, the solvability condition, we can show that theoretically the coarse- grid problem is also solvable. To be certain numerical round-off does not perturb solvability, we incorporate a Gram-Schmidt-like step each time a new right-hand side is generated for the coarse grid: f h f 2h f ff 2 22 2 2 22 h hh h h hh    1 11 1.

 U-Boulder222 of 322 Neumann problem: An example Consider the problem which has as a solution for any c (c = -1/12 gives the zero mean solution). V(2,1) cycles grid sizeaveragenumber N conv. factorof cycles 316.30E-110.0799.70E-059 631.90E-110.0892.40E-0510 1272.60E-110.0935.90E-0610 2553.70E-110.0961.50E-0610 5115.70E-110.1003.70E-0710 10278.60E-110.1049.20E-0810 20472.10E-110.1122.30E-0810 40955.20E-110.1225.70E-0910,,,

 U-Boulder223 of 322 7. Selected applications 7b. Anisotropic problems All problems considered thus far have had -h -2 as the off-diagonal entries. We consider two situations when the matrix has two different constants on the off-diagonals. These situations arise when –the (2-d) differential equation has constant but different coefficients for the derivatives in the coordinate directions –the discretization has constant but different mesh spacing in the different coordinate directions

 U-Boulder224 of 322 We consider two types of anisotropy Different coefficients on the derivatives - u xx -  u yy = f discretized on a uniform grid with spacing h. Different mesh spacings:

 U-Boulder225 of 322 Both problems lead to the same stencil

 U-Boulder226 of 322 Why standard multigrid fails Note that A h has weak connections in the y-direction. MG convergence factors degrade as  gets small, with poor performance already at  = 0.1. Consider the limiting case   0: Collection of disconnected 1-D problems! Point relaxation will smooth oscillatory errors in the x-direction (strong connections), but with no connections in the y-direction, the errors in that direction will generally be random; point relaxation provides no smoothing in the y-direction. 2

 U-Boulder227 of 322 We analyze weighted Jacobi The eigenvalues of the weighted Jacobi iteration matrix for this problem are

 U-Boulder228 of 322 Two strategies for anisotropy Semicoarsening: Because we expect MG-like convergence for the 1-D problems along lines of constant y, we could coarsen the grid in the x-direction, but not in the y-direction. Line relaxation: Because the the equations are strongly coupled in the x-direction, we could solve simultaneously for entire lines of unknowns in the x-direction (along lines of constant y).

 U-Boulder229 of 322 Semicoarsening with point relaxation Point relaxation on smoothes in the x-direction. Coarsen by removing every other y-line. We do not coarsen along the remaining y-lines. Semicoarsening is not as “fast” as full coarsening. The number of points on  2h is about half the number of points on  2h, instead of the usual one-fourth.

 U-Boulder230 of 322 Interpolation with semicoarsening We interpolate in the 1-D way along each line of constant y. The formulas for interpolating the correction from the coarse to the fine grid for the 2D model problem are

 U-Boulder231 of 322 Line relaxation with full coarsening The other approach to this problem is to do full coarsening, but to relax entire x-lines (constant y) of variables simultaneously. Instead of lexicographic ordering, it’s easier here to order by x-lines. Write A h in block form as where & h c   2.

 U-Boulder232 of 322 Line relaxation One sweep of line relaxation consists of solving a tridiagonal system for each line of constant y. The k th such system has the form, where is the k th subvector of with entries & the k th right-hand side subvector is Because D is tridiagonal, the k th system can be solved very efficiently.      vv h fg h kj h kj h kjj h k  11 2.

 U-Boulder233 of 322 Why line relaxation works The eigenvalues of the weighted block Jacobi iteration matrix are

 U-Boulder234 of 322 Semicoarsening & line relaxation We might not know the direction of weak coupling or it might vary. Suppose we want a method that can handle either or We could use semicoarsening in the x-direction to handle & line relaxation in the y-direction to take care of.

 U-Boulder235 of 322 Semicoarsening & line relaxation The original grid. Original grid viewed as a stack of “pencils.” Line relaxation is used to solve problem along each “pencil”. Coarsening is done by deleting every other pencil.

 U-Boulder236 of 322 An anisotropic example Consider - u xx -  u yy = f with u = 0 on the boundaries of the unit square, & stencil given by. Suppose that f (x,y) = 2(y - y 2 ) + 2  (x - x 2 ) so that the exact solution is u (x,y) = (y - y 2 )(x - x 2 ). Note: If  is small, then the x-direction dominates, while if  is large, then the y-direction dominates.

 U-Boulder237 of 322 What is smooth error? Consider  = 0.001 & suppose point Gauss-Seidel is applied to a random initial guess. The error after 50 sweeps appears as: 00.10.20.30.40.50.60.70.80.91 0 0.02 0.04 0.06 0.08 0.1 0.12 00.10.20.30.40.50.60.70.80.91 0 0.02 0.04 0.06 0.08 0.1 0.12 Error along line of constant x Error along line of constant y x y

 U-Boulder238 of 322 We experiment with 3 methods Standard V(2,1)-cycling, with point Gauss-Seidel relaxation, full coarsening, & linear interpolation Semicoarsening in the x-direction. Coarse & fine grids have the same number of points in the y- direction. 1-D full weighting & linear interpolation are used in the x-direction, with no y-coupling in the intergrid transfers Semicoarsening in the x-direction combined with line relaxation in the y-direction. 1-D full weighting & interpolation.

 U-Boulder239 of 322 With semicoarsening, the operator must change To account for unequal mesh spacing, the residual & relaxation operators must use a modified stencil Note that as grids become coarser, h x grows while h y remains constant..

 U-Boulder240 of 322 How do the 3 methods work for various values of  ? Asymptotic convergence factors: Note: semicoarsening in x works well for  <.001 but degrades noticeably even at  =.1. N = 15

 U-Boulder241 of 322 A semicoarsening subtlety Suppose  is small, so that semicoarsening in x is used. As we progress to coarser grids, h x -2 gets small but h y -2 remains constant. If, on some coarse grid, h x -2 becomes comparable to  h y -2, then the problem effectively becomes recoupled in the y-direction. Continued semicoarsening can produce artificial anisotropy, strong in the y-direction. When this occurs, it is best to stop semicoarsening & use full coarsening on any further coarse grids.

 U-Boulder242 of 322 Non-uniform grids are commonly used for domain or data irregularities or emerging solution features. Consider how we might approach the 1-D problem - u” (x) = f(x), 0 < x < 1, u (0) = u (1) = 0 posed on the following nonuniform grid: 7c. Variable meshes 7. Selected applications x = 0 x = 1 x 0 x j - 1 x j x j + 1 x N + 1

 U-Boulder243 of 322 We need some notation for the mesh spacing Let N be a positive integer. We define the spacing interval between x j & x j + 1 : h j + 1/2 ≡ x j + 1 - x j, j = 0, 1, …, N. h j + 1/2 x = 0 x = 1 x 0 x j - 1 x j x j + 1 x N + 1

 U-Boulder244 of 322 Building second divided differences u j-1 u j u j+1 u j-1/2 = (u j+1/2 - u j-1/2 )/h j ’’’’ h j = (h j-1/2 + h j+1/2 )/2 h j-1/2 h j+1/2 || u j-1/2 = (u j - u j-1 )/h j-1/2 ' u j+1/2 = (u j+1 - u j )/h j+1/2 ' 

 U-Boulder245 of 322 We define the discrete differential operator Using second-order finite differences (& wading through a mess of algebra!), we obtain the discrete representation where Multiply by (h j-1/2 + h j+1/2 )/2 yields an SPD matrix.. &

 U-Boulder246 of 322 We modify standard multigrid to accommodate variable spacing We choose every other fine-grid point as a coarse-grid point: In [x 0, x 2 ], linear means v (x) = v 0 + (v 1 - v 0 ) (x - x 0 )/(x 2 - x 0 ). Now just plug in x = x 1 to get… hhh2h2h2h2h2h2h hh h +1

 U-Boulder247 of 322 Interpolation formula If, then, for 1 ≤ j ≤ (N - 1)/2, we have.,

 U-Boulder248 of 322 We use the variational properties to derive restriction & A 2h This produces a stencil on  2h that is similar, but not identical, to the fine-grid stencil. If the resulting system is scaled by (h j-1/2 + h j+1/2 ), then the Galerkin product is the same as the fine-grid stencil. For 2-D problems, this approach can be generalized readily to tensor-product grids. However, for general irregular grids, AMG is a better choice.

 U-Boulder249 of 322 A common difficulty is variable coefficients, exemplified in 1-D by - (a (x) u’ (x))’ = f(x), 0 < x < 1, u (0) = u (1) = 0, where a(x) is a positive function on [0,1]. We seek to develop a conservative, or self-adjoint, method for discretizing this problem. Assume we have available to us the values of a(x) (a j + 1/2 ≡ a (x j + 1/2 )) at midpoints of the uniform grid 7d. Variable coefficients 7. Selected applications +1

 U-Boulder250 of 322 Discretize using central differences We can use second-order differences to approximate the derivatives. To use a grid spacing of h, we evaluate a(x)u’(x) at points midway between the gridpoints: = Points used to evaluate (au’ )’ at x j +1

 U-Boulder251 of 322 Discretize using central differences (cont’d) To evaluate, we must sample a(x) at the point x j + 1/2 & use second-order differences: where Points used to evaluate (au’ )’ at x j Points used to evaluate u’ at x j + 1/2 +1.

 U-Boulder252 of 322 The basic stencil We combine the differences for u’ & for (au’ )’ to obtain the operator & the problem becomes.

 U-Boulder253 of 322 Coarsening the variable coefficient problem A reasonable approach is to use a standard multigrid algorithm with linear interpolation, full weighting, & the stencil where The same stencil is obtained by the Galerkin relation..

 U-Boulder254 of 322 Variable mesh vs. variable coefficients after scaling by  j = (h j-1/2 + h j+1/2 )/2 & h Variable mesh Variable coefficients Correspondence  a j  1/2 h  a j  1/2 h 1 h j  1/2 1 h j  1/2

 U-Boulder255 of 322 A variable coefficient example We use V(2,1) cycle, full weighting, linear interpolation. We use a(x) = 1 +  sin(k  x) & a(x) = 1 +  rand(x). N = 1024 a(x) = 1 +  sin(k  x) a(x) = 1 +  rand(x) 0.950.1910.6810.690.694 k=3k=25k=50k=100 00.085 0.250.0840.098 0.094 0.50.0930.1850.1940.196 0.750.1190.3740.3870.391 0.850.1420.4970.5110.514 

 U-Boulder256 of 322 Standard multigrid degrades if a(x) is highly variable MG for variable coefficients is equivalent to MG (with simple averaging) for Poisson’s equation on a variable mesh. But simple averaging won’t accurately represent smooth components if is close to but far from. -(au’ )’ = f -u’’ = f

 U-Boulder257 of 322 One remedy is to apply operator-induced interpolation Assume that relaxation does not change smooth error, so the residual is approximately zero. Applying at yields Solving for :

 U-Boulder258 of 322 Thus, operator-induced interpolation is As usual, the restriction & coarse-grid operators are defined by the Galerkin relations: The case a(x) ≡ 1 reduces to standard interpolation:,..,

 U-Boulder259 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:      

 U-Boulder260 of 322 8. Algebraic multigrid (AMG) l Automatically defines coarse “grid”. l AMG has two distinct phases: — setup phase: define MG components. — solution phase: perform MG cycles. l The AMG approach differs from geometric MG from the start: — fix relaxation (point Gauss-Seidel). — choose coarse “grids” & prolongation, P, so that error not reduced by relaxation is in range(P). (Geometric MG fixes coarse grids & P, then chooses suitable operators & smoothers.) l AMG uses the Galerkin principle so that coarse-grid correction eliminates error in range(P). unstructured grids, variable coefficients,… Assume A is SPD unless otherwise stated.

 U-Boulder261 of 322 Solve Phase Standard MG processes: V-cycle, W-cycle, FMG, FAS, … AMG has two phases Setup Phase –Select coarse “grids,” –Define interpolation, –Define restriction & coarse-grid operators,,. l Only the selection of coarse grids does not parallelize well using existing techniques!

 U-Boulder262 of 322 AMG fundamental concept: smooth error = “small” residuals/energy Error propagation via weighted Jacobi smoothing: e k + 1 = (I -  D - 1 A) e k. Error that is slow to converge satisfies (I -  D - 1 A) e ≈ e ⇒  D - 1 A e ≈ 0 A little more precisely, assuming that  D ≈ I, then slow-to-converge error has relatively small energy: ⇒ r ≈ 0

 U-Boulder263 of 322 Operator-induced interpolation To define e i in terms of e j and e k, we must eliminate e f. Erase it?! e f = e i ⇒ (a ii + a if )e i + a ij e j + a ik e k = 0 i k j f coarse point fine point i another F-point pretend that we have a graph of A & that we’ve identified coarse points r i = 0 ⇒ a ii e i + a ij e j + a ik e k + a if e f = 0. ? ⇒ F-points determined by C-points!

 U-Boulder264 of 322 AMG uses strong connection to determine MG components Smoothing assumption: r ≈ 0 or ≈ 0. We say that i is strongly connected to j if For M-matrices, some algebra leads to So smooth error is more or less constant along strong connections..

 U-Boulder265 of 322 Some useful set definitions The set of strong connections of a variable, that is, the variables upon whose values the value of depends, is defined as The set of points strongly connected to a variable is denoted. The set of coarse-grid variables is denoted C. The set of fine-grid variables is denoted F. The set of interpolatory coarse-grid variables used to interpolate the value of the fine-grid variable is denoted..

 U-Boulder266 of 322 Choosing the coarse grid Two Criteria –(C1) For each, each point should either be in or should be strongly connected to at least one point in. –(C2) should be a maximal subset with the property that no two -points are strongly connected to each other. Satisfying (C1) & (C2) is sometimes impossible. We use (C2) as a guide while enforcing (C1).

 U-Boulder267 of 322 Selecting the coarse-grid points C-point selected (point with largest “value”) Neighbors of C-point become F- points Next C-point selected (after updating “values”) F-points selected, etc.

 U-Boulder268 of 322 Examples: Laplacian operator 5-pt FD, 9-pt FE (quads), & 9-pt FE (stretched quads) 5-pt FD 9-pt FE (quads) 9-pt FE (stretched quads)

 U-Boulder269 of 322 Prolongation is based on smooth error, strong connections (from M-matrices) Prolongation : Smooth error is given by: i C C C F F F

 U-Boulder270 of 322 Prolongation is based on smooth error, strong connections (from M-matrices) The definition of smooth error, Sets: Strongly connected -pts. Weakly connected points. i C C C F F F gives Strong CStrong FWeak pts.

 U-Boulder271 of 322 Finally, the prolongation weights are defined In the smooth-error relation, use e j = e i for weak connections. For strong F-points, use yielding the prolongation weights:

 U-Boulder272 of 322 AMG setup costs a bad rap Many geometric MG methods need to compute prolongation & coarse-grid operators. The only additional expense in the AMG setup phase is the coarse-grid selection algorithm. AMG setup phase is only 10-25% more expensive than in geometric MG & may be considerably less than that!

 U-Boulder273 of 322 AMG performance: Sometimes a success story AMG performs extremely well on the model problem (Poisson’s equation, regular grid)- optimal convergence factor (e.g., 0.14) & scalability w.r.t. problem size. AMG appears to be both scalable & efficient on diffusion problems on unstructured grids (e.g., 0.1-0.3). AMG handles anisotropic diffusion coefficients on irregular grids reasonably well. AMG handles anisotropic operators on structured & unstructured grids relatively well (e.g., 0.35).

 U-Boulder274 of 322 How does it perform (vol I)? regular grids, plain, old, vanilla problems, unit square, N = 64, Dirichlet boundaries The Laplace operator: Anisotropic 5-Point Laplacian: Epsilon 0.0010.010.10.512101001000 Convergence/cycle 0.0840.0930.0580.0690.0540.0790.0870.0930.083

 U-Boulder275 of 322 How does it perform (vol II)? structured meshes, rectangular domains 5-point Laplacian on regular rectangular grids Convergence factor (y-axis) plotted against number of nodes (x-axis)

 U-Boulder276 of 322 How does it perform (vol III)? unstructured meshes, rectangular domains Laplacian on random unstructured grids (regular triangulations, 15-20% nodes randomly collapsed into neighboring nodes) Convergence factor (y-axis) plotted against number of nodes (x-axis)

 U-Boulder277 of 322 How does it perform (vol IV)? isotropic diffusion, structured/unstructured grids on structured, unstructured grids Problems used: “a” means parameter c = 10, “b” means c = 1000 N=16642 N=66049 N=13755 N=54518 Structured Structured Unstruct. Unstruct.

 U-Boulder278 of 322 How does it perform (vol V)? Laplacian operator, unstructured grids

 U-Boulder279 of 322 AMG for systems How can we do AMG on systems? Naïve approach: “Block” AMG (block Gauss-Seidel, using scalar AMG to “solve” at each cycle) Great Idea! Except that it often doesn’t work! Block AMG doesn’t account for strong inter-variable coupling.

 U-Boulder280 of 322 AMG for systems: A solution To solve the system problem, allow interaction between the unknowns at all levels: &. This is called the “unknown-based” approach. 2-D biharmonic, Dirichlet boundaries, unit square, uniform quadrilateral mesh: Mesh spacing 0.1250.06250.031350.015625 Convergence factor 0.220.350.420.44

 U-Boulder281 of 322 Adaptive AMG (  AMG) to broaden applicability adaptive interpolation to determine sense of smoothness + adaptive C-point choice to determine good coarse point

 U-Boulder282 of 322 Standard interpolation Standard AMG collapses stencils by assuming smooth error is locally constant (Poisson “sense of smoothness”): Strong CStrong FWeak pts.  e i j e *** smooth e ≈ c *** (constant)

 U-Boulder283 of 322 Isn’t standard interpolation OK? Isn’t smooth e ≈ c (constant) always? Suppose someone tried to make A “nice” for relaxation by scaling the diagonal so it’s the identity: A ← D -1/2 AD -1/2, D = diag(a ii ). Relaxation still gives small residuals: Ae ≈ 0. But: Ae = D -1/2 AD -1/2 e ≈ 0 ⇒ AD -1/2 e ≈ 0 ⇒ e ≈ D 1/2 c. So “smooth” here means e i ≈ c a ii. This could vary a lot!

 U-Boulder284 of 322 Pre-relaxation What if we found a smooth error x that’s far from c ? If we found that x j = 1.6x i for j  D i w, say, then we could just make the replacement e j  1.6e i. If, say, x j = 0.2x k for j  D i s & all k  C i, then we could set e j  (   a jk 0.2e k ) / (   a jk ). jCijCi jCijCi How do we obtain a representative smooth error x  ? Relax on Ax = 0 !!!

 U-Boulder285 of 322 Adaptive C-point choice AMG uses the ansatz that residuals are 0 at F-points. What if we had r f = 0 exactly? Choosing (ideal interpolation) makes MG exact:

 U-Boulder286 of 322 Compatible relaxation (CR) How do we make the residuals zero at F-points? Relax on A ff x f = -A fc x c ! To ensure that this works so that r f → 0 fast, we want A ff to be well conditioned. Thus, we can assess whether we have good C-points by “CR”: A ff x f = 0. Quick CR convergence means that the F-point residuals will be small after relaxation & that there is a P (the ideal one) that gives good MG convergence. Is there a good local P ? Maybe, since CR is local & fast. Hopefully, we can use pre-relaxation to compute a good P.

 U-Boulder287 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:       

 U-Boulder288 of 322 9. Multilevel adaptive methods fast adaptive composite grid method (FAC) two-spike problem Local enhancement to resolve special regions of activity or interest.

 U-Boulder289 of 322 Model 1-D problem - u”(x) = f(x),0 < x < 1 u(0) = u(1) = 0 01

 U-Boulder290 of 322 Local refinement suppose f(x) has a spike at x = 3/4 but is smooth elsewhere O fine grid ( = interface) coarse grid x y = f(x)y = f(x) Unneeded resolution & accuracy.

 U-Boulder291 of 322 Strategy Recognize that there’s little value of having the fine grid in the smooth region, [0, 1/2]. Start with uniform grid & standard MG, then: –first eliminate relaxation in [0, 1/2]; and –then eliminate intergrid transfers & residual calculations. Then interpret this process via the composite grid (= 2h-points in [0, 1/2] + h-points in [1/2, 1]). We’ll try absurdly hard to eliminate work @ x = 1/2, but we have in mind multi-dimensions & smaller patches.

 U-Boulder292 of 322 Local-Relaxation/Global-Correction MG ⊙ ◦ ◦ ◦    ⊙ ⊙ × ×  ×  ⊙ Consider the usual global grids. ⊙ ◦ ◦ ◦    ⊙ ⊙ × ×  ×  ⊙ Relax only in the local region. x = 1/2 becomes a boundary point. ⊙ ◦ ◦ ◦    ⊙ ⊙ × ×  ×  ⊙ Now correct via coarse grid as usual. Compute & transfer residuals. To save all of the work of computing & transferring residuals & corrections in regions where they don’t change, we need some messy ALGEBRA! We start by using local relaxation & eliminating unnecessary residual transfers. Trust me on the more messy stuff… Transfer & add correction. → → WARNING: You won’t get zero residuals on all of grid h. ← ← Remember that the concept is right here, but we need to make it efficient--without changing the results!!!

 U-Boulder293 of 322 Eliminate relaxation Relax on v h on the local fine grid (x 5 h, x 6 h, x 7 h ). Compute r h = f h - A h v h (@x 3 h -x 7 h ) & transfer to 2h: f 2 2h  (r 3 h + 2r 4 h + r 5 h )/4 & f 3 2h  (r 5 h + 2r 6 h + r 7 h )/4. Compute an approximation, v 2h, to the solution of the 2h residual equation, A 2h u 2h = f 2h. Update the residual at x 1 2h for later cycles: f 1 2h  f 1 2h - (- v 0 2h + 2 v 1 2h - v 2 2h )/(2h) 2. Correct: v h  v h + I 2h v 2h. notation v h = (v i h ) h v h changes only hereso r h changes only on 2h Once & forall: it changes only on 2h. ↩ Initialize v h = 0 & f 1 2h  (f 1 h + 2f 2 h + f 3 h )/4. cycle Why save v h outside of local region?

 U-Boulder294 of 322 Eliminate more Initialize v h = 0, w 1 2h = 0, & f 1 2h  (f 1 h + 2f 2 h + f 3 h )/4. Relax on v h on the local fine grid (x 5 h, x 6 h, x 7 h ). Compute r h = f h - A h v h (@x 2 h -x 7 h ) & transfer to 2h: f 2 2h  (r 3 h + 2r 4 h + r 5 h )/4 & f 3 2h  (r 5 h + 2r 6 h + r 7 h )/4. Compute an approximation, v 2h, to the solution of the 2h residual equation, A 2h u 2h = f 2h. Update the residual at x 1 2h for later cycles: f 1 2h  f 1 2h - (- v 0 2h + 2 v 1 2h - v 2 2h )/(2h) 2. Accumulate the 2h approximation: w 1 2h  w 1 2h + v 1 2h. Correct: v h  v h + I 2h v 2h (@x 3 h -x 7 h ). Store v h only @ x 3 h -x 7 h & save v 2 h on 2h (call it w 1 2h ). h f 2 2h involves r 3 h & r 4 h, so we “need” v 2 h -v 7 h. cycle

 U-Boulder295 of 322 Eliminate the rest r 3 h doesn’t change on h. Compute change in r 3 h on 2h. Initialize v h = 0, w 1 2h = 0, & f 1 2h  (f 1 h + 2f 2 h + f 3 h )/4. Relax on v h on the local fine grid (x 5 h, x 6 h, x 7 h ). Compute the right sides for 2h: f 2 2h  g 2 2h - (- w 1 2h + 2 v 4 h - v 6 h )/(2h) 2 & f 3 2h  (r 5 h + 2r 6 h + r 7 h )/4. Compute an approximation, v 2h, to the solution of the 2h residual equation, A 2h u 2h = f 2h. Update the residual at x 1 2h for later cycles: f 1 2h  f 1 2h - (- v 0 2h + 2 v 1 2h - v 2 2h )/(2h) 2. Accumulate the 2h approximation: w 1 2h  w 1 2h + v 1 2h. Correct: v h  v h + I 2h v 2h (@x 4 h -x 7 h ). 2h residual @ x 2 2h = g 2 2h - (- w 1 2h + 2 v 4 h - v 6 h )/(2h) 2, where g 2 2h = (f 3 h + 2f 4 h + f 5 h )/4. (Messy algebra!) h cycle

 U-Boulder296 of 322 Interpretation composite grid What equation in v h are we solving? (- v 0 c + 2 v 1 c - v 2 c )/(2h) 2 = f 1 c  (f 1 h + 2 f 2 h + f 3 h )/4 (- v c i-1 + 2 v i c - v c i+1 )/h = f i c  f h i+2 i = 3,4,5 (- v 1 c + 3v 2 c - 2v 3 c )/(2h) 2 = f 2 c  (f 3 h + 2 f 4 h )/4 } usual @interface → → The key is to treat the interface correctly. ← ←

 U-Boulder297 of 322 Issues Adaptivity Error estimates Norms (proper scaling) Multiple dimensions –Slave points –More complicated stencils Data structures Parallel algorithms (AFAC) Time-space ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊ ◊

 U-Boulder298 of 322 Two-spike example (Laplacian) Global grid h & one double-patch refined level h/2. V(1,0)-cycles, Gauss-Seidel. Asymptotic convergence of the solver. Scaled L 2 discretization error estimate.

 U-Boulder299 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:        

 U-Boulder300 of 322 10. Finite elements FE, a variational discretization methodology FE methodology FD methodology method Other methods: finite volume, collocation, spectral, … FD: differences at nodes & truncation error. FE: weak form on discrete functions & approximation property

 U-Boulder301 of 322 Discretization What do you see? → → FIRST KEY POINT ← ← FE sees grid points (nodes or dofs) only as characterizations of continuum functions.

 U-Boulder302 of 322 Localize What choice do you have? → → SECOND KEY POINT ← ← FE functions are localized to ensure a sparse matrix.

 U-Boulder303 of 322 uhuh Continuous piecewise linear functions How do you take 2 nd derivatives of u h ? → → THIRD KEY POINT ← ← FE integrates away 2 nd derivatives. ??? - u” = f ???

 U-Boulder304 of 322 Weak form! h u     The ⇒ can actually be reversed to ⇐ if u is smooth enough. So the weak & strong forms are “equivalent”. Need basis for space of admissible u h & v.

 U-Boulder305 of 322 Representation a basis of “hat” functions  (i) is continuous piecewise linear & 0 at all nodes except node i where it’s 1. h h (i)(i) Any continuous piecewise linear function can be represented by u h =  i  u i  (i). h h i Now we’re back to using node values. ↵

 U-Boulder306 of 322 Weak form So the matrix is   

 U-Boulder307 of 322 FE basics Self-adjoint positive definite ⇒ weak functional. Model problem    2 : Lu  - u xx - u yy = f in  u = 0 on  Sobolev spaces ( wider than classical spaces! ): L 2 (  ) = {u : ∫  u 2 d  < ∞} H 0 1 (  ) = {u : u, u x, u y  L 2 (  ), u|  = 0} (u, v) = ∫  u v d  L is self-adjoint positive definite ( more later ): (Lu, v) = (  u,  v) = ∫   u x v x +u y v y )d  = (u, Lv) (Lu, u) > 0 if u ≠ 0 Duality: Solving Lu = f is equivalent to minimizing the weak functional F(u) = “(Lu, u)”/2 - (f, u) = (  u,  v)/2 - (f, u) Short story: 1 st derivative test  F(u) = Lu - f = 0 2 nd derivative test F”(u) = L > 0 We use (Lu, v) for simplicity, but really we mean (  u,  v). This is formal: Lu is not defined on all of H 0 1 (  ).

 U-Boulder308 of 322 Long story Using symmetry & linearity of L & bilinearity of the inner product: F(u + v) = (L(u + v), u + v)/2 - (f, u + v) = (Lu, u)/2 + (Lu, v) + (Lv, v)/2 - (f, u) - (f, v) = F(u) + (Lu, v) - (f, v) + (Lv, v)/2 = F(u) + (Lu - f, v) + (Lv, v)/2. The last term, (Lv, v)/2, is positive for v ≠ 0, but it can be neglected for very small v. Thus, F(u + v) ≥ F(u )  v  H 0 1 (  )  (Lu, v) = (f, v)  v  H 0 1 (  )  “Lu = f”.

 U-Boulder309 of 322 FE constructs Assume that  is the unit square. Consider an nxn grid of square “cells”. Continuous piecewise bilinear elements: H h  H 0 1 (  ). Each u h in H h is determined by its node values. This is how we’ll represent them! Within each square: u h = axy + bx + cy + d & u h is linear in a coordinate direction. If u h on one side of an element matches u h on the other at the nodes, then it matches on the common edge. So u h is continuous.

 U-Boulder310 of 322 Continuity u h = axy + bx + cy + d y = y 0 = constant u h = (ay 0 + b)x + (cy 0 + d) ←

 U-Boulder311 of 322 FE discretization Minimize F(u h ) = (Lu h, u h )/2 - (f, u h ) over u h  H h. Equilavent to solving (Lu h, v h ) = (f, v h )  v h  H h. Basis:  (ij) is the element of H h that equals 1 @ node ij & 0 elsewhere. Expansion: u h (x,y) =  ij u ij  (ij) (x,y). Problem: What is Lu h  - u xx - u yy ??? hh h hh

 U-Boulder312 of 322 Weak form The Gauss Divergence Theorem & homogeneous boundary conditions yield (Lu, v) = (- u xx - u yy, v) = (-  u, v) = (  u,  v)  Note: (  u,  v) = ∫  (u x v x + u y v y )d  So the problem becomes (  u h,  v h ) = (f, v h )  v h  H h or ∫  (u x v x + u y v y )d  = ∫  fv h d  v h  H h. hhhh

 U-Boulder313 of 322 Towards the matrix equation for nodal values (  u h,  v h ) = (f, v h )  v h  H h Using u h (x,y) =  ij u ij  (ij) (x,y) & choosing v h =  (kl) :  ij u ij (  (ij),  (kl) ) = (f,  (kl) )  k, l. Matrix terms (  (ij),  (kl) ) are 0 when |i - k| or |j - l| > 1. We compute (  (ij),  (ij) ) = 8/3 & (  (ij),  (i ±1 j ±1) ) = -1/3. Assume f is bilinear: (f,  (kl) ) = ∫  f  (kl) d  ≈ h 2 f(x kl  hh h hhhh hh h hh h h h actually, B h f h (mass matrix) h u h = (u ij ) & f h = (h 2 f(x kl  )

 U-Boulder314 of 322 The matrix equation A h u h = f h where u h = (u ij ) & f h = (h 2 f(x kl  ) & the matrix is given by the stencil. h stiffness matrix

 U-Boulder315 of 322 Some matrix properties Symmetric! ij “reaches” to i±1j±1 as i±1j±1 to ij. Singular? A h 1 = 0 ?! Depends on boundaries! Dirichlet boundary: Positive definite! Diagonally dominant (strictly so @ boundaries)..

 U-Boulder316 of 322 Abstract FE relaxation focus on functions, not nodal vectors Relaxation involves “local” changes: u h  u h - s  h for some scalar s &  h =  (ij). But how do we pick s ??? Use FE principle of minimizing F(u h - s  h ) over s : F(u h - s  h ) = (L(u h - s  h ), u h - s  h )/2 - (f, u h - s  h ) = F(u h ) - s (Lu h - f,  h ) + (s 2 /2) (L  h,  h ). Let s be the root of the derivative of this quadratic polynomial w.r.t. s : s = (Lu h - f,  h )/(L  h,  h ) = r ij /L ij,ij, so F(u h - s  h ) - F(u h ) = - (Lu h - f,  h ) 2 /(2(L  h,  h )) = - r ij /(2L ij,ij ). h This is just Gauss-Seidel! 2 h h

 U-Boulder317 of 322 Abstract FE coarsening again with focus on functions Coarsening involves a “global” change: u h  u h + w 2h for some coarse-grid function w 2h. But how do we pick w 2h ??? Use FE principle of minimizing F(u h + w 2h ) over w 2h : F(u h + w 2h ) = (L(u h + w 2h ), u h + w 2h )/2 - (f, u h + w 2h ) = F(u h ) + (Lu h - f, w 2h ) + (Lw 2h, w 2h )/2. Let w 2h be thr root of the gradient of this quadratic functional w.r.t. w 2h. This is tricky because you need to write the gradient as a function in the subspace H 2h. We go instead from abstract functions to nodal vectors…

 U-Boulder318 of 322 Concrete FE coarsening: I 2h now with focus on nodal vectors Adding nodal representations of v h & v 2h : v 2h (x,y) =  ij v 2h ij  2h (ij) (x,y) 2h nodal values =  ij v h ij  h (ij) (x,y) ??? h nodal values We should be able to do this because v 2h  H 2h  H h. Cell “2i+,2j+”: (sum over 2h indices  even h indices) v h 2i 2j = v 2h ij v h 2i+1 2j = (v 2h ij + v 2h i+1 j )/2 v h 2i 2j+1 = (v 2h ij + v 2h ij+1 )/2 v h 2i+1 2j+1 = (v 2h ij + v 2h i+1 j + v 2h ij+1 + v 2h i+1 j+1 )/4 h (sum over h indices)

 U-Boulder319 of 322 Concrete FE coarsening (cont’d) A h u h = f h Solving this matrix equation is equivalent to minimizing F h (v h )  (A h v h, v h )/2 - (f h, v h ) (parens here mean Euclidean norm) over v h  H h. So how do we now correct v h ??? We minimize F h (v h + I 2h v 2h ) over v 2h  H 2h : F h (v h + I 2h v 2h ) = (A h (v h + I 2h v 2h ), v h + I 2h v 2h )/2 - (f h, v h + I 2h v 2h ) = F h (v h ) + (A 2h v 2h, v 2h )/2 - (f 2h, v 2h ) h hhh h F 2h (v 2h ).  F h (v h ) + variational conditions f 2h = I 2h (f h - A h v h ) A 2h = I 2h A h I 2h hThT hhThT

 U-Boulder320 of 322 Outline  Model Problems Basic Iterative Methods –Convergence tests –Analysis Elements of Multigrid –Relaxation –Coarsening Implementation –Complexity –Diagnostics Some Theory –Spectral vs. algebraic Nonlinear Problems –Full approximation scheme Selected Applications –Neumann boundaries –Anisotropic problems –Variable meshes –Variable coefficients Algebraic Multigrid (AMG) –Matrix coarsening Multilevel Adaptive Methods –FAC Finite Elements –Variational methodology Chapters 1-5: Chapters 6-10:         

 U-Boulder321 of 322 Multigrid rules! We conclude with a few observations: oWe have barely scratched the surface of the myriad ways that multigrid has been, & can be, employed. oWith diligence & care, multigrid can be made to handle many types of complications in a robust, efficient manner. oFurther extensions to multigrid methodology are being sought by many people working on many different problems.

 U-Boulder322 of 322 Multigrid/multilevel/multiscale an important methodology Multigrid has proved successful on a wide variety of problems, especially elliptic PDEs, but has also found application in parabolic & hyperbolic PDEs, integral equations, evolution problems, geodesic problems, … Multigrid can be optimal, often O(N ). Multigrid can be robust in a practical sense. Multigrid is of great interest because it is one of the very few scalable algorithms, & it can be parallelized readily & efficiently!

 U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 &

Similar presentations

Presentation on theme: " U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 &"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

 U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 &

Similar presentations

Presentation on theme: " U-Boulder1 of 322 APPM 6640 Multigrid Methods Steve McCormick & John Ruge 303-492-0662 &"— Presentation transcript:

Similar presentations

About project

Feedback