1. Lecture 2 Linear Inverse Problems and Introduction to Least Squares

2. Surprise Quiz! Lemma (you can assume) Given the Lemma, prove by induction the following inequality: Once you have proven the aforesaid inequality, derive the Cauchy Schwartz Inequality from it.

3. Solution

4. How many solutions? Case 1 Case 2 Case 3

5. How many solutions? Case 1 Infinite Case 2 One Case 3 None

6. Under-determined System Matrix-Vector Notation: The system is ‘Fat’ YΨ = θ

7. Perfectly Determined System The system is ‘Square’ YΨ = θ

8. Over-Determined System The system is ‘Tall’ YΨ = θ

9. Curve Fitting in Noisy Observation

10. Least Squares: Line The observation is ‘noisy’ Fit a line that minimizes the sum of least squared error, i.e. Like any other minimization, set gradient = 0

11. Least Squares: Line contd.

13. We have the solution:

14. Homework Find the least squares fit for a second degree polynomial of the form,

15. General Form The least squares problem is expressed as:

16. Solution

17. Taking the gradient

18. Solution Taking the gradient The ‘Normal Equations’

19. Properties of Least Squares θ LS is a linear function of Y Pseudo Inverse: This is also the left inverse of a ‘tall’ matrix Ψ, i.e.