1. Dan Witzner Hansen Email: witzner@itu.dk

3.  Groups?  Improvements – what is missing?

4.  The goal is to be able to solve linear equations  Continue with linear algebra  Linear mappings  Basis vectors & independence  Solving linear equations & Determinants  Inverse & Least squares  SVD  Lot’s of stuff. Don’t despair – you will be greatly rewarded in the future

6.  A linear equation is an equation of the form, a n x n + a n-1 x n-1 +... + a 1 x 1 = b.

7.  A system of linear equations is simply a set of linear equations. i.e. a 1,1 x 1 + a 1,2 x 2 +... + a 1,n x n = b 1 a 2,1 x 1 + a 2,2 x 2 +... + a 2,n x n = b 2... a m,1 x 1 + a m,2 x 2 +... + a m,n x n = b m

8. Compact notation A x = b

9. Affine mapping

10.  Species 1: eats 5 units of A and 3 of B.  Species 2: eats 2 units of A and 4 of B.  Everyday a total of 900 units of A and 960 units of B are eaten. How many animals of each species are there? Species

14.  Subspaces  Independent vectors  Basis vectors / Orthonomal basis

15. A subspace is a vector space contained in another vector space

16. Can it happen that y=0 if x is nonzero? If y is non-zero for all non-zero x, then the column vectors of A are said to be linear independent. These vectors form a set of basis vectors Orthonormal basis when the vectors are unit size and orthogonal.

17. Change of basis

19. A = [1 4 2;2 8 6; 3 12 4]; [X,Y,Z] = meshgrid(-10:10,-10:10,-10:10); x = [X(:),Y(:),Z(:)]’; p = A*x; plot3(p(1,:),p(2,:),p(3,:),'rx')

22. 4x = =

24.  A solution to a system of equations is simply an assignment of values to the variables that satisfies (is a solution to) all of the equations in the system.  If a system of equations has at least one solution, we say it is consistent.  If a system does not have any solutions we say that it is inconsistent.

25. Solution

26. Which solution(s)? Can we always do this? How many solutions are there?

27.  For A (2x2 matrix)  When det A ≠0 a unique solution exists (nonsingular)  When det A =0 the matrix is singular (lines same slope) and are therefore the columns are linear dependent  Coincident (infinitely many solutions)  Parallel (no solutions)  Determinant can be used when solving linear equations (Cramers’ rule), but not useful in practice >>det(A)

28.  What to do when the dimension and the number of data points is large?  How many data points are needed to solve for the unknown parameters in x?

29.  Modify Ax=b to Ux=c with same solution x  Solve Ux=c with back substitution  Use Elementary operations  Swap two equations (rows and right side)  Modify one equation by subtracting a multiple of one of the other equations (rows)

30. 1. Modify equation 2,..n so that x1 is eliminated 2. Modify the (now modified) equations 3,..n so that x2 is eliminated,,,, 3. Modify equation n so that is eliminated 4. If the matrix is regular then all diagonal elements are non zero an we can therefore use back substitution

32.  Solve simple linear equation  Matrix inverse:  A (unique) inverse exist if det(A) ≠ 0 (NxN matrices)  Matlab: >>invA =inv(A)

33.  If m = n ( A is a square matrix & Det(A)!=0), then we can obtain the solution by simple inversion (:  If m > n, then the system is over-constrained and A is not invertible  If n>m then under constrained.

34. Solve Ax=b (notice multiply from right):

35.  Don’t use for solving the linear system. It is mostly meant for notational convenience.  It is faster and more accurate (numerically) to write (solve) x=A\b than inv(A)*b:

36.  Diagonal matrices  Orthogonal matrices

37.  A 2-D point x = (x, y) is on a line with slope m and intercept b if and only if y = mx + b  Equivalently,  So the line defined by two points x 1, x 2 is the solution to the following system of equations:

38.  Suppose we have points (2, 1), (5, 2), (7, 3), and (8, 3)?????

39. With more than two points, there is no guarantee that they will all be on the same line courtesy of Vanderbilt U.

40. Objective: Find the vector Fx in the column range of F, which is closest to the right-hand side vector y. The residual r=y-Fx

41.  Symmetric:  Positive definite:  Positive semi definite:

42.  Is this symmetric?  Is this positive semi definite? If the columns in F are linearly independent, then y=Fx is not 0 for all non-zero x and it follows that

43. Solution: Use the pseudoinverse A + = (A T A) -1 A T to obtain least- squares solution x = A + b

44. Solution: Use the pseudoinverse A + = (A T A) -1 A T to obtain least- squares solution x = A + b

45. courtesy of Vanderbilt U. Solution: Use the pseudoinverse A + = (A T A) -1 A T to obtain least- squares solution x = A + b

46.  Suppose we have points (2, 1), (5, 2), (7, 3), and (8, 3)  Then???? and x = A + b = (0.3571, 0.2857)T

47. (2, 1), (5, 2), (7, 3), and (8, 3)

48.  Suppose we want to solve A x = 0  There is a trivial solution x = 0, but we don’t want this. For what other values of x is A x close to 0 ?  This is satisfied by computing the singular value decomposition (SVD) A = UDV T (a non-negative diagonal matrix between two orthogonal matrices) and taking x as the last column of V  In Matlab [U, D, V] = svd(A)

51.  When the columns of A =UDV are independent then all  Tells how close to singular A is.  Inverse and pseudoinverse  The columns of U corresponding to nonzeros singular values span the range of A, the columns of V corresponding to zero singular values the nullspace.

52. A 2-D homogeneous point x = (x, y, 1) T is on the line l = (a, b, c) T only when ax + by + c = 0 We can write this equation with a dot product: x.l = 0, and hence the following system is implied for multiple points x 1, x 2,..., x n :

53. Again we have 4 points, but now in homogeneous form: (2, 1, 1), (5, 2, 1), (7, 3, 1), and (8, 3, 1)  The system of equations is:  Taking the SVD of A, we get: compare to x = (0.3571, 0.2857) T

54.  So what about outliers Other metrics such as other norms More about this later