1. Systems of Linear Equations (see Appendix A.6, Trucco & Verri) CS485/685 Computer Vision Prof. George Bebis

2. Systems of linear equations An arbitrary system of m linear equations in n unknowns can be written as: or where: Example (m=n=2):

3. Solutions of Ax=b (m = n) Characterize the solutions of Ax=b using conditions on the rank of A and A|b (i.e., augmented matrix).

4. Solutions of Ax=b (m=n) (1)The system has one solution if : rank(A|b) = rank(A) = n Solution: i.e., b be expressed as a linear combination of the columns of A:

5. Solutions of Ax=b (m=n) The following statements are equivalent: (a) rank(A|b) = rank(A) = n (b) A is invertible (c) (d) b has a unique expansion in the column space of A

6. Solving Ax=b using SVD Assuming that A=UDV T, then UDV T x=b or DV T x=U T b Setting V T x=z and U T b=d, we have Dz=d (1) Compute z=D -1 d (i.e., assume no zeroes in the diagonal) (2) Compute solution x =Vz Ax=b

7. Solutions of Ax=b (m=n) (2) The system has no solution if rank(A|b) > rank(A) b cannot be expressed as a linear combination of the columns of A e.g., using substitution leads to the contradiction 16=9

8. Solutions of Ax=b (m=n) (3) The system has infinitely many solutions if rank(A|b) = rank(A) < n - Less equations than unknowns (i.e, free variables). - b can be expressed as a linear combination of the columns of A in more than one ways.

9. Homogeneous system: Ax=0 (m = n) If b=0, then Ax=0 is called homogeneous. (1) Has the trivial solution x=0 iff rank(A) = n (i.e., A is invertible) (2) Has a non-trivial solution iff rank(A) < n (i.e., A is singular)

10. Over/Under determined Systems m>n m<n

11. Solving Ax=b (m > n) Consider the over-determined system of linear equations: Let r be the residual vector for some x: The vector x* which yields the smallest possible residual is called a least-squares solution:

12. Solving Ax=b (m > n) (cont’d) Although a least-squares solution always exist, it might not be unique! The least-squares solution x with the smallest norm ||x|| is unique and it is given by:

13. Solving Ax=b (m > n) - Example :

14. Computing A + using SVD If A T A is ill-conditioned (or singular), we can use SVD to obtain a least squares solution as follows: (where t is a small threshold) where:

15. Homogeneous systems The minimum-norm solution is x=0; need to modify the meaning of a least-squares solution by imposing the constraint: This is a "constrained" optimization problem:

16. Homogeneous systems (cont’d) The solution for homogeneous systems is not always unique. Special case: Solution: (v n is the last column of V; the one corresponding to the smallest σ)

17. Homogeneous systems (cont’d) General case: with (v n-k+1, …,v n are the last columns of V ; correspond to the smallest σ’s) Solution: