1. ENGG2013 Unit 5 Linear Combination & Linear Independence Jan, 2011.

2. Last time How to multiply a matrix and a vector Different ways to write down a system of linear equations – Vector equation – Matrix-vector product – Augmented matrix kshumENGG20132 Column vectors

3. Review: matrix notation In ENGG2013, we use capital bold letter for matrix. The first subscript is the row index, the second subscript is the column index. The number in the i-th row and the j-th column is called the (i,j)-entry. – c ij is the (i,j)-entry in C. kshumENGG20133 m  n

4. Matrix-vector multiplication kshumENGG20134

5. Today When is A x = b solvable? – Given A, under what condition does a solution exist for all b? For example, the nutrition problem: find a combination of food A, B, C and D in order to satisfy the nutrition requirement exactly. kshumENGG20135 Food AFood BFood CFood DRequirement Protein98335 Carbohydrate1511145 Vitamin A0.020.0030.010.0060.01 Vitamin C0.01 0.0050.050.01 Different people have different requirements Can we solve A x = b for fixed A and various b?

6. Today Basic concepts in linear algebra – Linear combination – Linear independence – Span kshumENGG20136

7. Three cases: 0, 1,  kshumENGG20137 A x = b No solutionUnique solution Infinitely many solutions How to determine? m equations n variables

8. GEOMETRY FOR LINEAR SYSTEM TWO EQUATIONS kshumENGG20138

9. Scaling kshumENGG20139 y x 1 1 y x c c c is any real number

10. Representing a straight line by vector kshumENGG201310 y x c c y x y=x Any point on the line y=x can be written as

11. Adding one more vector kshumENGG201311 y x y=x y x y=x+1

12. We can add another vector and get the same result kshumENGG201312 y x y=x+1 y x =

13. The whole plane kshumENGG201313 y x Scanner

14. Question 1 kshumENGG201314 Can you find c and d such that 1 2 3 4 5 6 7 2 435 ?

15. Question 2 kshumENGG201315 Can you find c and d such that 1 2 3 4 5 6 7 2 435

16. Question 3 kshumENGG201316 Can you find c, d, and e such that 1 2 3 4 5 6 7 2 435

17. GEOMETRY FOR LINEAR SYSTEM THREE EQUATIONS kshumENGG201317

18. From line to plane to space kshumENGG201318 x y z Scalar multiples of x z Any point in the 3-D space can be written as x y z Any point in the x-y plane can be written as

19. Question 4 kshumENGG201319 x y z The three red arrows all lie in the x-y plane Can you find a, b, and c, such that ?

20. Question 5 kshumENGG201320 x y z The three red arrows all lie in the shaded plane. Can you scale up (or down) the three red arrows such that the resulting vector sum is equal to the blue vector?

21. Question 6 kshumENGG201321 x y z The three red arrows all lie in a straight line. Can you find x, y and z such that ?

22. Question 7 kshumENGG201322 x y z The three red arrows and the blue arrow are all on the same line. Can you find x, y and z such that ?

23. ALGEBRA FOR LINEAR EQUATIONS kshumENGG201323

24. Review on notation A vector is a list of numbers. The set of all vectors with two components is called. is a short-hand notation for saying that – v is a vector with two components – The two components in v are real numbers. kshumENGG201324

25. The set of all vectors with three components is called. is a short-hand notation for saying that – v is a vector with three components – The three components in v are real numbers. kshumENGG201325

26. The set of all vectors with n components is called. We use a zero in boldface, 0, to represent the all-zero vector kshumENGG201326

27. Definition: Linear Combination Given vectors v 1, v 2, …, v i in, and i real number c 1, c 2, …, c i, the vector w obtained by w = c 1 v 1 + c 2 v 2 + …+ c i v i is called a linear combination of v 1, v 2, …, v i. Examples of linear combination of v 1 and v 2 : kshumENGG201327

28. Picture Linear combinations of two vectors u and v. kshumENGG201328 u v 2u+2v u–2v –v –v 0 3u 2u+0.5v

29. Definition: Span Given r vectors v 1, v 2, …, v r, the set of all linear combinations of v 1, v 2, …, v r called the span of v 1, v 2, …, v r, We use the notation span(v 1, v 2, …, v r ) for the span of span of v 1, v 2, …, v r. We also say that span(v 1, v 2, …, v r ) is spanned by, or generated by v 1, v 2, …, v r. span(v 1, v 2, …, v r ) is the collections of all vectors which can be written as c 1 v 1 + c 2 v 2 + … + c 2 v r for some scalars c 1, c 2, …, c r. kshumENGG201329

30. Span of u and v Linear combinations of this two vectors u and v form the whole plane kshumENGG201330 u v 2u+2v u–2v –v –v 0 3u

31. Span of a single vector u kshumENGG201331 x y z consists of the points on a straight line which passes through the origin. u

32. Span of two vectors in 3D kshumENGG201332 x y z u v is a plane through the origin.

33. Example kshumENGG201333 1 2 3 4 5 6 7 2 435 is a linear combination of and, because We therefore say that

34. Mathematical language kshumENGG201334 President Obama is not a Chinese. Ordinary languageMathematical language Let C be the set of all Chinese people. President Obama

35. Example kshumENGG201335 x y z

36. A fundamental fact Let – A be an m  n matrix – b be an m  1 vector Let the columns of A be v 1, v 2,…, v n. The followings are logically equivalent: kshumENGG201336 We can find a vector x such that 1 2 3 “Logically equivalent” means if one of them is true, then all of them is true if one of them is false, then all of them is false.

37. Theorem 1 With notation as in previous slide, if the span of be v 1, v 2,…, v n contains all vectors in then the linear system Ax = b has at least one solution. In other words, if every vector in can be written as a linear combination of v 1, v 2,…, v n, then Ax = b is solvable for any choice of b. kshumENGG201337 “Solvable” means there is one solution or more than one solutions.

38. Example kshumENGG201338 1 2 3 4 5 6 7 2 435 Since and span the whole plane, the linear system is solvable for any b 1 and b 2.

39. Example kshumENGG201339 x y z The three red arrows all lie in the x-y plane x y z (Infinitely many solutions)

40. Example kshumENGG201340 1 2 3 4 5 6 7 2 435 because is not a linear combination of

41. Example kshumENGG201341 1 2 3 4 5 6 7 2 435 has infinitely many solutions.

42. Infinitely many solutions kshumENGG201342 x y z x y Notice that is a scalar multiple of is a linear combination of and There is one common feature in the examples with infinitely many solutions The common feature is that one of the vector is a linear combination of the others.

43. Definition: Linear dependence Vectors v 1, v 2, …, v r are said to be linear dependent if we can find r real number c 1, c 2, …, c r, not all of them equal to zero, such that 0 = c 1 v 1 + c 2 v 2 + …+ c r v r Otherwise, are v 1, v 2, …, v r are said to be linear independent. In other words, v 1, v 2, …, v r are be linear independent if, the only choice of c 1, c 2, …, c r, such that 0 = c 1 v 1 + c 2 v 2 + …+ c r v r is c 1 = c 2 = …= c r =0. kshumENGG201343

44. Example of linear independent vectors kshumENGG201344

45. Example of linear dependent vectors kshumENGG201345

46. Example of linear independent vectors kshumENGG201346

47. Example and are linear dependent, because, and are linear dependent because kshumENGG201347

48. Picture kshumENGG201348 x y z The three vectors lie on the same plane, namely, the x-y plane.

49. Theorem 2 Let – A be an m  n matrix – b be an m  1 vector Let the columns of A be v 1, v 2,…, v n. Theorem: If v 1, v 2,…, v n, are linear independent, then Ax = b has at most one solution. kshumENGG201349

50. Proof (by contradiction) Suppose that and are two different solutions to Ax=b, i.e., Therefore Move every term to the left But v 1, v 2,…, v n are linear independent by assumption. So, the only choice is This contradicts the fact that vector x and vector x’ are different. kshumENGG201350

51. Example and are linearly independent. has a unique solution for any choice of b 1 and b 2. kshumENGG201351 In fact, x must equal b 1, and y must equal b 2 /3 in this example.

52. Example is solvable by Theorem 1, because the blue vector lies on the plane spanned by the two red vectors. The solution is unique because and are linearly independent. kshumENGG201352 x z y

53. Summary kshumENGG201353 At most one solution At least one solution Ax=b m equations n variables Every vector in is a linear combination of the columns in A. Columns of A are linearly independent Unique solution The columns of A contain a lot of information about the nature of the solutions.

54. A kind of mirror symmetry kshumENGG201354 If the columns of A are linear independent, then I am pretty sure that there is one or no solution to Ax=b, no matter what b is. If any vector in can be written as a linear combination of the column vectors in A, then Ax=b must have one or more than one solutions.

55. Basis A set of vector in which are simultaneously – linearly independent, and – spanning the whole space is of particular importance, and is called a set of basis vectors. (We will talk about basis in more detail later.) kshumENGG201355