Linear Systems What is the Matrix?

Outline Announcements: –Homework III: due Wed. by 5, by Office Hours: Today & tomorrow, 11-1 –Ideas for Friday? Linear Systems Basics Matlab and Linear Algebra

Ecology of Linear Systems Linear Systems are found in every habitat: –simple algebra –solutions to ODEs & PDEs –statistics (especially, least squares) If you can formulate your problem as linear system, it will be easy to solve on a computer

“Standard Linear System” Simplest linear system: finding the equation of a line: –y=m*x + b The goal is to find m and b from observations of (x,y) pairs 2 points form a line, so we need two observations (x1, y1) & (x2,y2)

The 8th Grade Solution Solve for b in the first equation: Substitute this for b in 2nd equation & solve for m: Put this into the equation for b above

The Sophomoric Solution Write the equations as a matrix problem Perform Gaussian Elimination on Matrix:

Comparing methods Gaussian Elimination is a simpler algorithm –Easily generalizes to systems with more unknowns Gaussian Elimination is the starting point of much of numerical linear algebra

A Closer Look at GE (optional) For Am=y –GE reduces A to an upper triangular matrix –Perform “back substitution” using modified y –Modified y is equivalent to Ly L is a lower triangular matrix A=L*U

Other Algorithms for Solving Linear Systems GE aka LU decomposition -- any A Cholesky Factorization -- symmetric, positive definite A Iterative solvers (conjugate gradients, GMRES)

Linear Systems in Matlab Linear systems are easy in Matlab –To solve Ax=b, type x=A\b –To solve x’A’=b’, type x’=b’/A’ (transposed)

Matrix multiplication (*) is easy, fast Matrix “division” (\) is hard and computationally intensive –In general, performs GE with partial pivoting –But, \ is smart & looks closely at A for opportunities to speed up If A is LT, just does back substitution If A is over-determined, A\b is the least-squares solution More About \

Factorization Can explicitly factor A using LU: –[L,U]=lu(A) –useful if you have to solve A\bmany times (different b each time) To solve LUx=b: first solve Ly=b, then solve Ux=y In Matlab: y=L\b; x=U\y; Other factorizations: chol, svd

What about A -1 ? Matlab can compute A -1 using inv(A), but … –inv(A) is slower than lu(A) –There are numerical problems with inv(A) Rarely needed, use lu(A) or another factorization