Sparse Matrices in Matlab John R. Gilbert Xerox Palo Alto Research Center with Cleve Moler (MathWorks) and Rob Schreiber (HP Labs)

Slides:

Advertisements

Similar presentations

Eigen Decomposition and Singular Value Decomposition

Advertisements

Chapter 28 – Part II Matrix Operations. Gaussian elimination Gaussian elimination LU factorization LU factorization Gaussian elimination with partial.

Scientific Computing QR Factorization Part 2 – Algorithm to Find Eigenvalues.

Lecture 3 Sparse Direct Method: Combinatorics Xiaoye Sherry Li Lawrence Berkeley National Laboratory, USA crd-legacy.lbl.gov/~xiaoye/G2S3/

CS 240A: Solving Ax = b in parallel Dense A: Gaussian elimination with partial pivoting (LU) Same flavor as matrix * matrix, but more complicated Sparse.

Least Squares example There are 3 mountains u,y,z that from one site have been measured as 2474 ft., 3882 ft., and 4834 ft.. But from u, y looks 1422 ft.

MATH 685/ CSI 700/ OR 682 Lecture Notes

Solving Linear Systems (Numerical Recipes, Chap 2)

Lecture 13 - Eigen-analysis CVEN 302 July 1, 2002.

The QR iteration for eigenvalues. ... The intention of the algorithm is to perform a sequence of similarity transformations on a real matrix so that the.

SOLVING SYSTEMS OF LINEAR EQUATIONS. Overview A matrix consists of a rectangular array of elements represented by a single symbol (example: [A]). An individual.

Solution of linear system of equations

1cs542g-term Notes  Assignment 1 is out (questions?)

1cs542g-term Notes  Assignment 1 is out (due October 5)  Matrix storage: usually column-major.

Part 3 Chapter 10 LU Factorization PowerPoints organized by Dr. Michael R. Gustafson II, Duke University All images copyright © The McGraw-Hill Companies,

3D Geometry for Computer Graphics

1cs542g-term Sparse matrix data structure  Typically either Compressed Sparse Row (CSR) or Compressed Sparse Column (CSC) Informally “ia-ja” format.

CS 290H: Sparse Matrix Algorithms

Symmetric Weighted Matching for Indefinite Systems Iain Duff, RAL and CERFACS John Gilbert, MIT and UC Santa Barbara June 21, 2002.

Sparse Matrices and Combinatorial Algorithms in Star-P Status and Plans April 8, 2005.

Graph Algorithms in Numerical Linear Algebra: Past, Present, and Future John R. Gilbert MIT and UC Santa Barbara September 28, 2002.

Sparse Matrix Methods Day 1: Overview Day 2: Direct methods Nonsymmetric systems Graph theoretic tools Sparse LU with partial pivoting Supernodal factorization.

Sparse Matrix Methods Day 1: Overview Matlab and examples Data structures Ax=b Sparse matrices and graphs Fill-reducing matrix permutations Matching and.

3D Geometry for Computer Graphics

3D Geometry for Computer Graphics

Ordinary least squares regression (OLS)

1 Systems of Linear Equations (Optional) Special Matrices.

CS240A: Conjugate Gradients and the Model Problem.

Linear Least Squares QR Factorization. Systems of linear equations Problem to solve: M x = b Given M x = b : Is there a solution? Is the solution unique?

The Evolution of a Sparse Partial Pivoting Algorithm John R. Gilbert with: Tim Davis, Jim Demmel, Stan Eisenstat, Laura Grigori, Stefan Larimore, Sherry.

CS 290H Lecture 17 Dulmage-Mendelsohn Theory

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

CS 290H Lecture 12 Column intersection graphs, Ordering for sparsity in LU with partial pivoting Read “Computing the block triangular form of a sparse.

8.1 Vector spaces A set of vector is said to form a linear vector space V Chapter 8 Matrices and vector spaces.

Advanced Computer Graphics Spring 2014 K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology.

Solving Scale Linear Systems (Example system continued) Lecture 14 MA/CS 471 Fall 2003.

Symbolic sparse Gaussian elimination: A = LU

The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.

By: David McQuilling and Jesus Caban Numerical Linear Algebra.

CS 290H Lecture 5 Elimination trees Read GLN section 6.6 (next time I’ll assign 6.5 and 6.7) Homework 1 due Thursday 14 Oct by 3pm turnin file1.

CS 219: Sparse Matrix Algorithms

MATH 685/ CSI 700/ OR 682 Lecture Notes Lecture 4. Least squares.

Lecture 4 Sparse Factorization: Data-flow Organization

CS240A: Conjugate Gradients and the Model Problem.

CS240A: Computation on Graphs. Graphs and Sparse Matrices Sparse matrix is a representation.

Direct Methods for Sparse Linear Systems Lecture 4 Alessandra Nardi Thanks to Prof. Jacob White, Suvranu De, Deepak Ramaswamy, Michal Rewienski, and Karen.

Department of Electronic Engineering, Tsinghua University Nano-scale Integrated Circuit and System Lab. Performance Analysis of Parallel Sparse LU Factorization.

Administrivia: October 5, 2009 Homework 1 due Wednesday Reading in Davis: Skim section 6.1 (the fill bounds will make more sense next week) Read section.

Linear Algebra Libraries: BLAS, LAPACK, ScaLAPACK, PLASMA, MAGMA

Graphs A graphs is an abstract representation of a set of objects, called vertices or nodes, where some pairs of the objects are connected by links, called.

CS 290H Administrivia: May 14, 2008 Course project progress reports due next Wed 21 May. Reading in Saad (second edition): Sections

CS 290H Lecture 15 GESP concluded Final presentations for survey projects next Tue and Thu 20-minute talk with at least 5 min for questions and discussion.

Consider Preconditioning – Basic Principles Basic Idea: is to use Krylov subspace method (CG, GMRES, MINRES …) on a modified system such as The matrix.

CS 290H Lecture 9 Left-looking LU with partial pivoting Read “A supernodal approach to sparse partial pivoting” (course reader #4), sections 1 through.

Symmetric-pattern multifrontal factorization T(A) G(A)

Conjugate gradient iteration One matrix-vector multiplication per iteration Two vector dot products per iteration Four n-vectors of working storage x 0.

The Landscape of Sparse Ax=b Solvers Direct A = LU Iterative y’ = Ay Non- symmetric Symmetric positive definite More RobustLess Storage More Robust More.

Parallel Direct Methods for Sparse Linear Systems

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

Introduction The central problems of Linear Algebra are to study the properties of matrices and to investigate the solutions of systems of linear equations.

CS 290N / 219: Sparse Matrix Algorithms

CS 290H Administrivia: April 16, 2008

The Landscape of Sparse Ax=b Solvers

RECORD. RECORD COLLABORATE: Discuss: Is the statement below correct? Try a 2x2 example.

~ Least Squares example

Part 3 Chapter 10 LU Factorization

~ Least Squares example

Read GLN sections 6.1 through 6.4.

Nonsymmetric Gaussian elimination

Presentation transcript:

Sparse Matrices in Matlab John R. Gilbert Xerox Palo Alto Research Center with Cleve Moler (MathWorks) and Rob Schreiber (HP Labs)

Design principles All operations should give the same results for sparse and full matrices (almost all) Sparse matrices are never created automatically, but once created they propagate Performance is important -- but usability, simplicity, completeness, and robustness are more important Storage for a sparse matrix should be O(nonzeros) Time for a sparse operation should be O(flops) (as nearly as possible)

Two storage classes for matrices Full: 2-dimensional array of real or complex numbers matrix uses (nrows * ncols) memory Sparse: compressed column storage matrix uses about (2*nonzeros + nrows) memory

Matrix arithmetic and indexing A+B, A*B, A.*B, A’, ~A, A|B,... A(2:n, 2:n) = A(2:n, 2:n) + u * v’ A = [B C ; D E];

Matrix factorizations Cholesky: R = chol(A); simple left-looking column algorithm Nonsymmetric LU: [L,U,P] = lu(A); left-looking “GPMOD”, depth-first search, symmetric pruning Orthogonal: [Q,R] = qr(A); George-Heath algorithm: row-wise Givens rotations

Fill-reducing matrix permutations Nonsymmetric approximate minimum degree: p = colamd(A); column permutation: lu(A(:,p)) often sparser than lu(A) also for qr factorization [Davis, Larimore, G, Ng] Symmetric approximate minimum degree: p = symamd(A); symmetric permutation: chol(A(p,p)) often sparser than chol(A) Reverse Cuthill-McKee p = symrcm(A); A(p,p) often has smaller bandwidth than A similar to Sparspak RCM

Matching and block triangular form Dulmage-Mendelsohn decomposition: Bipartite matching followed by strongly connected components Square, full rank A: [p, q, r] = dmperm(A); A(p,q) has nonzero diagonal and is in block upper triangular form also, strongly connected components of a directed graph Arbitrary A: [p, q, r, s] = dmperm(A); maximum-size matching in a bipartite graph minimum-size vertex cover in a bipartite graph decomposition into strong Hall blocks

Solving linear equations x = A \ b; Is A square? no => use QR to solve least squares problem Is A triangular or permuted triangular? yes => sparse triangular solve Is A symmetric with positive diagonal elements? yes => attempt Cholesky after symmetric minimum degree Otherwise => use LU on A(:, colamd(A))

Extensions (due to various people) Matlab code (“m-files”) Iterative linear solvers Sparse optimization toolbox... C or Fortran code with Matlab interfaces (“mex-files”) Sparse eigenvalues (based on ARPACK) Graph partitioning (based on METIS or Chaco) Supernodal sparse linear solver (based on SuperLU)...