1. INDR 262 INTRODUCTION TO OPTIMIZATION METHODS LINEAR ALGEBRA INDR 262 Metin Türkay 1

2. Matrices m and n are positive integers Order of matrix: mxn The number in the i th row and j th column of A is called the ij th element of A and is written a ij. INDR 262 Metin Türkay 2

3. Example INDR 262 Metin Türkay 3

4. Equal Matrices Two matrices A and B are equal if and only if A and B are of the same order and for all i and j, a ij =b ij. If A=B, then x=1, y=2, w=3, z=4 INDR 262 Metin Türkay 4

5. Vectors A list of n real numbers, say (a 1, a 2, …, a n ) is called an n- dimensional vector. An n-dimensional vector also maybe displayed as a 1 by n matrix. INDR 262 Metin Türkay 5

6. Scalar Product of Two Vectors The scalar product of vectors u = [u 1 u 2 … u n ] and is u 1 v 1 +u 2 v 2 +…+u n v n INDR 262 Metin Türkay 6

7. Example uv = 1 x 2+2 x 1+3 x 2 = 10 INDR 262 Metin Türkay 7

8. Notes If u = [1 2 3] and, then uv is not defined because the vectors are of different dimensions. Two vectors are perpendicular to each other if and only if their scalar product is equal to 0. E.g., u = [1 -1] and. INDR 262 Metin Türkay 8

9. Scalar Multiple of a Matrix Given any matrix A and any scalar c, the scalar multiple of matrix A, cA, is obtained from the matrix A by multiplying each element of A by c. INDR 262 Metin Türkay 9

10. Addition of two Matrices Let A=[a ij ] and B=[b ij ] be two matrices with the same order (say m x n). Then, the matrix C=A+B is defined to be the m x n matrix whose ij th element is a ij +b ij. INDR 262 Metin Türkay 10

11. The Transpose of a Matrix Given any m x n matrix the transpose of A (written A T ) is the n x m matrix INDR 262 Metin Türkay 11

12. Example For any matrix A, (A T ) T =A. INDR 262 Metin Türkay 12

13. Matrix Multiplication Given two matrices A and B, the matrix product of A and B is defined if and only if the number of columns in A is equal to the number of rows in B. The matrix product C=AB is determined as follows: c ij = scalar product of (row i of A and column j of B) INDR 262 Metin Türkay 13

14. Properties of Matrix Multiplication 1.Matrix multiplication is associative, i.e., A(BC)=(AB)C. 2.Matrix multiplication is distributive, i.e., A(B+C)=AB+AC. INDR 262 Metin Türkay 14

15. Example INDR 262 Metin Türkay 15

16. Matrices and Systems of Linear Equations Consider a system of linear equations given by x 1, x 2, …, x n are referred to as variables a ij ’s and b i ’s are constants A set of equations like above is called a linear system of m equations in n variables INDR 262 Metin Türkay 16

17. Solution A solution to a linear system of m equations in n unknowns is a set of values for the unknowns that satisfy each of the systems m equations. INDR 262 Metin Türkay 17

18. Example x 1 + 2x 2 = 5 2x 1 - x 2 = 0 Solution Not a solution INDR 262 Metin Türkay 18

19. Matrix Representation of Systems of Linear Equations Ax = b Augmented matrix INDR 262 Metin Türkay 19

20. The Gauss-Jordan Method for Solving Systems of Linear Equations Gauss-Jordan method is used to find solution(s) to systems of linear equations. A system of linear equations must satisfy one of the following cases: Case 1: The system has no solution Case 2: The system has a unique solution Case 3: The system has an infinite number of solutions INDR 262 Metin Türkay 20

21. Elementary Row Operations An ero transforms a given matrix A into a new matrix A’ via one of the following operations. Type 1 ero: A’ is obtained by multiplying any row of A by a nonzero scalar. Type 2 ero: Begin by multiplying any row of A (say, row i) by a nonzero scalar c. For some j  i, let row j of A’ = c(row i of A) + row j of A, and let the other rows of A’ be the same as the rows of A. Type 3 ero: Interchange any two rows of A. INDR 262 Metin Türkay 21

22. Facts If the matrix A’ is obtained from A via an ero, A’ and A are equivalent. If the augmented matrix [A’  b’] is obtained from [A  b] via an ero, the systems Ax=b and A’x=b’ are equivalent. Any sequence of ero’s performed on the augmented matrix [A  b] corresponding to the system Ax=b will yield an equivalent linear system. INDR 262 Metin Türkay 22

23. Gauss-Jordan Method The Gauss-Jordan method solves a linear system of equations by utilizing ero’s in a systematic fashion. Step 1To solve Ax=b, write down the augmented matrix [A  b]. Step 2At any stage, define a current row, current column, and a current entry. Begin with row 1 as the current row, column 1 as the current column, and a 11 as the current entry. INDR 262 Metin Türkay 23

24. Gauss-Jordan Method aIf a 11 (the current entry) is nonzero, use ero’s to transform column 1 (the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3. bIf a 11 (the current entry) equals 0, then do a Type 3 ero involving the current row and any row that contains a nonzero entry in the current column. Use ero’s to transform column 1 (the current column) to [1 0 … 0]T. Then, obtain the new current row, column, and entry by moving down one row and one column to the right, and go to Step 3. cIf there are no nonzero numbers in the first column, obtain a new current column and entry by moving one column to the right. Then go to Step 3. INDR 262 Metin Türkay 24

25. Gauss-Jordan Method Step 3 aIf the new current entry is nonzero, use ero’s to transform it to 1 and the rest of the current column’s entries to 0. When finished, obtain a new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3. bIf the current entry is 0, do a Type 3 ero with the current row and any row that contains a nonzero entry in the current column. Then, use ero’s to transform column entry to 1 and the rest of the current column’s entries to 0. When finished, obtain the new current row, column, and entry. If this is impossible, stop. Otherwise, repeat Step 3. cIf the current column has no nonzero numbers below the current row, obtain a new current column and entry and repeat Step 3. If it is impossible, stop. INDR 262 Metin Türkay 25

26. Gauss-Jordan Method Step 4 Write down the system of equations A’x=b’ that corresponds to the matrix [A’  b’] obtained when Step 3 is completed. Then, A’x=b’ will have the same set of solutions as Ax=b. The Gauss–Jordan method converts the augmented matrix [A  b] into [A’  b’] such that INDR 262 Metin Türkay 26

27. Example Solve the following system of linear equations using the Gauss-Jordan method. 2x 1 + 2x 2 + x 3 = 9 2x 1 - x 2 + 2x 3 = 6 x 1 - x 2 + 2x 3 = 5 INDR 262 Metin Türkay 27

28. Solution INDR 262 Metin Türkay 28

29. Solution INDR 262 Metin Türkay 29

30. Solution x 1 = 1x 2 = 2x 3 = 3 INDR 262 Metin Türkay 30

31. Analysis of the Solutions to Systems of Linear Equations For any linear system, a variable that appears with a coefficient of 1 in a single equation and a coefficient of 0 in all other equations is called a basic variable. Any variable that is not a basic variable is called a nonbasic variable. xset of all of the variables in the system Ax=b x B set of all of the basic variables in the system Ax=b x N set of all of the nonbasic variables in the system Ax=b x = x B  x N INDR 262 Metin Türkay 31

32. Analysis of the Solutions to Systems of Linear Equations The solution to A’x=b’ can be categorized in one of the three cases: Case 1: A’x=b’ has at least one row of the form [0 0 … 0  c] and c  0. Then, Ax=b has no solution. Case 2: When case 1 does not apply and x N = , then Ax=b has a unique solution. Case 3: When case 1 does not apply and x N , then Ax=b has infinite number of solutions. INDR 262 Metin Türkay 32

33. Example Case 1 does not apply since there are no rows of the form [0 0 … 0  c] and c  0. Case 2 does not apply since, x B ={x 1, x 2, x 3 } x N ={x 4, x 5 } There are infinite number of solutions. INDR 262 Metin Türkay 33

34. Example Assign arbitrary values to the variables in x N ; x 4 =c, x 5 =k. Write down the equations in [A’  b’], x 1 + c + k = 3  x 1 = 3 - c – k x 2 + 2c = 2  x 2 = 2 - 2c x 3 + k = 1  x 3 = 1 - k It is easy to see that there are infinite number of values of c and k that will satisfy this system of equations. INDR 262 Metin Türkay 34

35. Linear Combination A linear combination of the vectors in V is any vector of the form c 1 v 1 +c 2 v 2 +…+c k v k, where c 1, c 2, …, c k are arbitrary scalars. Example:V={[1,2], [2,1]} 2v 1 -v 2 = 2([1 2]) – [2 1] = [0 3] 0v 1 -3v 2 = 0([1 2]) – 3([2 1]) = [6 3] INDR 262 Metin Türkay 35

36. Linear Independence & Linear Dependence A set V of m-dimensional vectors is linearly independent if the only linear combination of vectors in V that equals 0 is the trivial linear combination. A set V of m-dimensional vectors is linearly dependent if there is a nontrivial linear combination of the vectors in V that adds up to 0. INDR 262 Metin Türkay 36

37. Example 1 V={[1,0], [0,1]} Try to find a linear combination of vectors in V that yields 0. c 1 ([1 0]) + c 2 ([0 1]) = [0 0] In order to satisfy this, [c 1 c 2 ] = [0 0]  c 1 =c 2 =0 The only linear combination of vectors in V that yields 0 is the trivial linear combination. Therefore, V is a linearly independent set of vectors. INDR 262 Metin Türkay 37

38. Example 2 V={[1,2], [2,4]} Try to find a linear combination of vectors in V that yields 0. c 1 ([1 2]) + c 2 ([2 4]) = [0 0] [c 1 2c 1 ] + [2c 2 4c 2 ] = [0 0] c 1 + 2c 2 = 0  c 1 = -2c 2 2c 1 + 4c 2 = 0  2c 1 = -4c 2 So, c 1 = 2c 2 = -1 is one of the possible solutions. There exists a nontrivial linear combination of vectors in V that yields 0. Therefore, V is a linearly dependent set of vectors. INDR 262 Metin Türkay 38

39. The Rank of a Matrix Let A be any mxn matrix, and denote the rows of A by r 1, r 2, …, r m. Also define R={ r 1, r 2, …, r m }. The rank of A is the number of vectors in the largest linearly independent subset of R. If for a matrix A with m rows, rank A=m; then the matrix is a collection of linearly independent set of vectors. If rank A<m; then the matrix contains a linearly dependent set of vectors. INDR 262 Metin Türkay 39

40. Examples Rank A = 3 Rank B = 2 INDR 262 Metin Türkay 40

41. The Inverse of a Matrix A single linear equation in a single variable can be solved by multiplying both sides of the equation by multiplicative inverse of the variable coefficient. Example: 4x=3  4 -1 (4x) = (4 -1 )3  x=3/4 We can generalize this approach to square systems of linear equations (i.e., number of equations = number of unknowns). INDR 262 Metin Türkay 41

42. Square and Identity Matrix A square matrix is any matrix that has an equal number of rows and columns. The diagonal elements of a square matrix are those elements a ij such that i=j. A square matrix for which all diagonal elements are equal to 1 and all non-diagonal elements are equal to 0 is called an identity matrix. INDR 262 Metin Türkay 42

43. Inverse of A For a given mxm matrix A, the mxm matrix B is the inverse of A if BA = AB = I m AA -1 =I INDR 262 Metin Türkay 43

44. Finding the Inverse of a Matrix with the Gauss-Jordan Method Step 1Write down the m x 2m matrix [A  I m ]. Step 2Use ero’s to transform [A  I m ] into [I m  B]. This will only be possible if rank A=m; in this case, B=A -1. If rank A<m, then A has no inverse. INDR 262 Metin Türkay 44

45. Example INDR 262 Metin Türkay 45

46. Using Matrix Inverse to Solve Linear Systems of Equations Given a linear system of equations, Ax=b Multiply both sides by A -1 AA -1 x = A -1 b  x = A -1 b INDR 262 Metin Türkay 46

47. Example 2x 1 + 5x 2 = 7 x 1 + 3x 2 = 4 INDR 262 Metin Türkay 47

48. Determinant of a Matrix Any square matrix A has a number called the determinant of A (shown by set A or  A  ). INDR 262 Metin Türkay 48

49. Minor of A If A is an m x m matrix; then for any values of i and j, the ij th minor of A (A ij ) is the (m-1) x (m-1) submatrix of A obtained by deleting row i and column j of A. INDR 262 Metin Türkay 49

50. Determinant of a Matrix Let A be any mxm matrix, det A = (-1) i+1 a i1 (det A i1 ) + (-1) i+2 a i2 (det A i2 ) + … +(- 1) i+m a im (det A im ) INDR 262 Metin Türkay 50

51. Example 1 det A = (-1) 1+1 1 det A 11 + (-1) 1+2 0 det A 12 + (-1) i+3 1 det A i3 INDR 262 Metin Türkay 51

52. Example 2 det A = -48 INDR 262 Metin Türkay 52