9 Nov. 200015-859B - Introduction to Scientific Computing1 Sparse Systems and Iterative Methods Paul Heckbert Computer Science Department Carnegie Mellon.

Slides:



Advertisements
Similar presentations
1 Numerical Solvers for BVPs By Dong Xu State Key Lab of CAD&CG, ZJU.
Advertisements

Linear Systems of Equations
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Parallel Programming in C with MPI and OpenMP Michael J. Quinn.
Rayan Alsemmeri Amseena Mansoor. LINEAR SYSTEMS Jacobi method is used to solve linear systems of the form Ax=b, where A is the square and invertible.
Iterative Methods and QR Factorization Lecture 5 Alessandra Nardi Thanks to Prof. Jacob White, Suvranu De, Deepak Ramaswamy, Michal Rewienski, and Karen.
Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system.
Modern iterative methods For basic iterative methods, converge linearly Modern iterative methods, converge faster –Krylov subspace method Steepest descent.
Special Matrices and Gauss-Siedel
Sparse Matrix Algorithms CS 524 – High-Performance Computing.
1 Systems of Linear Equations Iterative Methods. 2 B. Iterative Methods 1.Jacobi method and Gauss Seidel 2.Relaxation method for iterative methods.
Iterative methods for solving matrix equations 1. Jacobi 2. Gauss-Seidel 3. Successive overrelaxation (SOR) Idea behind iterative methods: Convert Ax=b.
1 Systems of Linear Equations Iterative Methods. 2 B. Direct Methods 1.Jacobi method and Gauss Seidel 2.Relaxation method for iterative methods.
Partial Differential Equations
Special Matrices and Gauss-Siedel
ECIV 520 Structural Analysis II Review of Matrix Algebra.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Parallel Programming in C with MPI and OpenMP Michael J. Quinn.
CS240A: Conjugate Gradients and the Model Problem.
ECIV 301 Programming & Graphics Numerical Methods for Engineers REVIEW II.
Monica Garika Chandana Guduru. METHODS TO SOLVE LINEAR SYSTEMS Direct methods Gaussian elimination method LU method for factorization Simplex method of.
CSE 245: Computer Aided Circuit Simulation and Verification
Partial differential equations Function depends on two or more independent variables This is a very simple one - there are many more complicated ones.
Iterative Methods for Solving Linear Systems of Equations ( part of the course given for the 2 nd grade at BGU, ME )
Exercise where Discretize the problem as usual on square grid of points (including boundaries). Define g and f such that the solution to the differential.
9 1 Performance Optimization. 9 2 Basic Optimization Algorithm p k - Search Direction  k - Learning Rate or.
ECE 530 – Analysis Techniques for Large-Scale Electrical Systems Prof. Hao Zhu Dept. of Electrical and Computer Engineering University of Illinois at Urbana-Champaign.
Numerical Analysis 1 EE, NCKU Tien-Hao Chang (Darby Chang)
Conjugate gradients, sparse matrix-vector multiplication, graphs, and meshes Thanks to Aydin Buluc, Umit Catalyurek, Alan Edelman, and Kathy Yelick for.
Solving Scalar Linear Systems Iterative approach Lecture 15 MA/CS 471 Fall 2003.
Iterative Methods for Solving Linear Systems Leo Magallon & Morgan Ulloa.
Elliptic Partial Differential Equations - Introduction
Partial Differential Equations Introduction –Adam Zornes, Deng Li Discretization Methods –Chunfang Chen, Danny Thorne, Adam Zornes.
1 Iterative Solution Methods Starts with an initial approximation for the solution vector (x 0 ) At each iteration updates the x vector by using the sytem.
Scientific Computing Partial Differential Equations Poisson Equation.
Linear Systems Iterative Solutions CSE 541 Roger Crawfis.
1 Incorporating Iterative Refinement with Sparse Cholesky April 2007 Doron Pearl.
Lecture 7 - Systems of Equations CVEN 302 June 17, 2002.
Elliptic PDEs and the Finite Difference Method
CSE 245: Computer Aided Circuit Simulation and Verification Matrix Computations: Iterative Methods I Chung-Kuan Cheng.
Parallel Solution of the Poisson Problem Using MPI
CS240A: Conjugate Gradients and the Model Problem.
Linear Systems – Iterative methods
Case Study in Computational Science & Engineering - Lecture 5 1 Iterative Solution of Linear Systems Jacobi Method while not converged do { }
CS 484. Iterative Methods n Gaussian elimination is considered to be a direct method to solve a system. n An indirect method produces a sequence of values.
Introduction to Scientific Computing II Overview Michael Bader.
Discretization for PDEs Chunfang Chen,Danny Thorne Adam Zornes, Deng Li CS 521 Feb., 9,2006.
Partial Derivatives Example: Find If solution: Partial Derivatives Example: Find If solution: gradient grad(u) = gradient.
Part 3 Chapter 12 Iterative Methods
Partial Differential Equations Introduction – Deng Li Discretization Methods –Chunfang Chen, Danny Thorne, Adam Zornes CS521 Feb.,7, 2006.
Krylov-Subspace Methods - I Lecture 6 Alessandra Nardi Thanks to Prof. Jacob White, Deepak Ramaswamy, Michal Rewienski, and Karen Veroy.
Programming Massively Parallel Graphics Multiprocessors using CUDA Final Project Amirhassan Asgari Kamiabad
Network Systems Lab. Korea Advanced Institute of Science and Technology No.1 Maximum Norms & Nonnegative Matrices  Weighted maximum norm e.g.) x1x1 x2x2.
Programming assignment # 3 Numerical Methods for PDEs Spring 2007 Jim E. Jones.
Engineering Analysis ENG 3420 Fall 2009 Dan C. Marinescu Office: HEC 439 B Office hours: Tu-Th 11:00-12:00.
Solving Systems of Linear Equations: Iterative Methods
Solving Linear Systems Ax=b
CSE 245: Computer Aided Circuit Simulation and Verification
Paul Heckbert Computer Science Department Carnegie Mellon University
CSE 245: Computer Aided Circuit Simulation and Verification
CSCE569 Parallel Computing
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Numerical Analysis Lecture14.
Introduction to Scientific Computing II
Introduction to Scientific Computing II
Partial Differential Equations
Numerical Linear Algebra
Solving Linear Systems: Iterative Methods and Sparse Systems
Introduction to Scientific Computing II
Linear Algebra Lecture 16.
Ax = b Methods for Solution of the System of Equations:
Presentation transcript:

9 Nov B - Introduction to Scientific Computing1 Sparse Systems and Iterative Methods Paul Heckbert Computer Science Department Carnegie Mellon University

9 Nov B - Introduction to Scientific Computing2 PDE’s and Sparse Systems A system of equations is sparse when there are few nonzero coefficients, e.g. O(n) nonzeros in an nxn matrix. Partial Differential Equations generally yield sparse systems of equations. Integral Equations generally yield dense (non-sparse) systems of equations. Sparse systems come from other sources besides PDE’s.

9 Nov B - Introduction to Scientific Computing3 Example: PDE Yielding Sparse System Laplace’s Equation in 2-D:  2 u = u xx +u yy = 0 –domain is unit square [0,1] 2 –value of function u(x,y) specified on boundary –solve for u(x,y) in interior

9 Nov B - Introduction to Scientific Computing4 Sparse Matrix Storage Brute force: store nxn array, O(n 2 ) memory –but most of that is zeros – wasted space (and time)! Better: use data structure that stores only the nonzeros col val bit integer indices: 2, 5, 6, 7,10 32 bit floats: 1, 1,-4, 1, 1 Memory requirements, if kn nonzeros: –brute force: 4n 2 bytes, sparse data struc: 6kn bytes

9 Nov B - Introduction to Scientific Computing5 An Iterative Method: Gauss-Seidel System of equations Ax=b Solve ith equation for x i : Pseudocode: until x stops changing for i = 1 to n x[i]  (b[i]-sum{j  i}{a[i,j]*x[j]})/a[i,i] modified x values are fed back in immediately converges if A is symmetric positive definite

9 Nov B - Introduction to Scientific Computing6 Variations on Gauss-Seidel Jacobi’s Method: –Like Gauss-Seidel except two copies of x vector are kept, “old” and “new” –No feedback until a sweep through n rows is complete –Half as fast as Gauss-Seidel, stricter convergence requirements Successive Overrelaxation (SOR) –extrapolate between old x vector and new Gauss-Seidel x vector, typically by a factor  between 1 and 2. –Faster than Gauss-Seidel.

9 Nov B - Introduction to Scientific Computing7 Conjugate Gradient Method Generally for symmetric positive definite, only. Convert linear system Ax=b into optimization problem: minimize x T Ax-x T b –a parabolic bowl Like gradient descent –but search in conjugate directions –not in gradient direction, to avoid zigzag problem Big help when bowl is elongated (cond(A) large)

9 Nov B - Introduction to Scientific Computing8 Conjugate Directions

9 Nov B - Introduction to Scientific Computing9 Convergence of Conjugate Gradient Method If K = cond(A) = max (A)/ min (A) then conjugate gradient method converges linearly with coefficient (sqrt(K)-1)/(sqrt(K)+1) worst case. often does much better: without roundoff error, if A has m distinct eigenvalues, converges in m iterations!