Linear Inverse Problems A MATLAB Tutorial Presented by Johnny Samuels

What do we want to do? We want to develop a method to determine the best fit to a set of data: e.g.

The Plan Review pertinent linear algebra topics Forward/inverse problem for linear systems Discuss well-posedness Formulate a least squares solution for an overdetermined system

Linear Algebra Review Represent m linear equations with n variables: A = m x n matrix, y = n x 1 vector, b = m x 1 vector If A = m x n and B = n x p then AB = m x p (number of columns of A = number of rows of B)

Linear Algebra Review: Example Solve system using MATLAB’s backslash operator “\”: A = [1 -1 3 1;3 -3 1 0;1 1 0 -2]; b=[2;-1;3]; y = A\b

Linear Algebra Review: What does it mean? Graphical Representation:

Linear Algebra Review: Square Matrices A = square matrix if A has n rows and n columns The n x n identity matrix = If there exists s.t. then A is invertible

Linear Algebra Review Square Matrices cont. If then Compute (by hand) and verify (with MATLAB’s “*” command) the inverse of

Linear Algebra Review: One last thing… The transpose of a matrix A with entries Aij is defined as Aji and is denoted as AT – that is, the columns of AT are the rows of A Ex: implies Use MATLAB’s “ ‘ “ to compute transpose

Forward Problem: An Introduction We will work with the linear system Ay = b where (for now) A = n x n matrix, y = n x 1 vector, b = n x 1 vector The forward problem consists of finding b given a particular y

Forward Problem: Example g = 2y : Forward problem consists of finding g for a given y If y = 2 then g = 4 What if and ? What is the forward problem for vibrating beam?

Inverse Problem For the vibrating beam, we are given data (done in lab) and we must determine m, c and k. In the case of linear system Ay=b, we are provided with A and b, but must determine y

Inverse Problem: Example g = 2y : Inverse problem consists of finding y for a given g If g = 10 then Use A\b to determine y

Well-posedness The solution technique produces the correct answer when Ay=b is well-posed Ay=b is well-posed when Existence – For each b there exists an y such that Ay=b Uniqueness – Stability – is continuous Ay=b is ill-posed if it is not well-posed

Well-posedness: Example In command window type y=well_posed_ex(4,0) y is the solution to K = condition number; the closer K is to 1 the more trusted the solution is

Ill-posedness: Example In command window type y=ill_posed_ex(4,0) y is the solution to where Examine error of y=ill_posed_ex(8,0) Error is present because H is ill-conditioned

What is an ill conditioned system? A system is ill conditioned if some small perturbation in the system causes a relatively large change in the exact solution Ill-conditioned system:

Ill-Conditioned System: Example II

What is the effect of noisy data? Data from vibrating beam will be corrupted by noise (e.g. measurement error) Compare: y=well_posed_ex(4,0) and z=well_posed(4,.1) y=well_posed_ex(10,0) and z=well_posed(10,.2) y=ill_posed_ex(4,0) and z=ill_posed(4,.1) y=ill_posed_ex(10,0) and z=ill_posed(10,.2) How to deal with error? Stay tuned for next talk

Are we done? What if A is not a square matrix? In this case does not exist Focus on an overdetermined system (i.e. A is m x n where m > n) Usually there exists no exact solution to Ay=b when A is overdetermined In vibrating beam example, # of data points will be much larger than # of parameters to solve (i.e. m > n)

Overdetermined System: Example Minimize

Obtaining the Normal Equations We want to minimize : is minimized when y solves provides the least squares solution

Least Squares: Example Approximate the spring constant k for Hooke’s Law: l is measured lengths of spring, E is equilibrium position, and F is the resisting force least_squares_ex.m determines the least squares solution to the above equation for a given data set

What did we learn? Harmonic oscillator is a nonlinear system, so the normal equations are not directly applicable Many numerical methods approximate the nonlinear system with a linear system, and then apply the types of results we obtained here