1. ME 1202: Linear Algebra & Ordinary Differential Equations (ODEs)
Dr. Faraz Junejo

2. Vector and Matrix Equations
System of linear equations may appear in a problem in three different forms:
As an ordinary system of the type studied earlier during this course,
As a single vector equation involving several vectors, or
As a matrix equation containing at least one matrix and several vectors.
In this lecture, it will be shown how these three types of equations are related and how to convert from one type to another.

3. Vectors in euclidean space Rn
The n-dimensional (real) euclidean space Rn is the collection of n-tuples* of numbers, called vectors i.e. we will use vectors to mean a list of numbers.
Vectors can be either presented horizontally,
x = (x1, x2, ..., xn)
or vertically.
*Tuples: An ordered list of elements. For example, (2, 7, 4, 1, 7) denotes a 5-tuple

4. Vectors in euclidean space (contd.)
A matrix will only one column is called column vector or simply a vector. Examples of vector with two entries are:
The set of all vectors with two enteries is denoted by R2 (read as “r-two”)
R2 stands for real numbers that appears as entries in the vectors, whereas exponent 2 indicates that each vector contains two entries.

5. Vectors in euclidean space (contd.)
Two vectors in are equal if and only if their corresponding entries are equal. For example following vectors are not equal:
In summary, we say that vectors in R2 are ordered pairs of real numbers.

6. Geometric description of euclidean space Rn
The 1-dimensional euclidean space
R1 = {(x): x is a real number}
can be identified with all real numbers R and be visualized as a straight line.
The 2-dimensional euclidean space
R2 = {(x1, x2): x1, x2 are real numbers}
can be visualized as the coordinate plane.

7. Geometric description of euclidean space Rn (contd.)
The 3-dimensional euclidean space
R3 = {(x1, x2, x3): x1, x2, x3 are real numbers}
can be visualized as the world we are living in.

8. Vectors of the same dimension can be added For example;
Vector Addition
Vectors of the same dimension can be added For example;
(x1, x2, ..., xn) + (y1, y2, ..., yn) = (x1 + y1, x2 + y2, ..., xn + yn)
For example, given two vectors in u and v in R2, their sum i.e. the vector u+v is obtained by adding corresponding entries of u and v.

9. Vector Multiplication
A number (scalar) can also be multiplied to vectors. For example;
c (x1, x2, ..., xn) = (cx1, cx2, ..., cxn).
For example, given two vectors in u and real number c, the scalar multiple of u by c is the vector cu obtained by multiplying each entry in u by c.

10. Examples Let u = (1, 2), v = (3, 4), w = (5, 6), And
x = (1, 0, -3), y = (2, -1, 5). Then
u + v = (1 + 3, 2 + 4) = (4, 6)
3u = (3×1, 3×2) = (3, 6)
u + v + w = ( , ) = (9, 12)
3u - 4v + w = (3×1 - 4×3 + 5, 3×2 - 4×4 + 6) = (-4, -4)
x + y = (1 + 2, 0 - 1, ) = (3, -1, 2)
-2x = (-2×1, -2×0, -2×(-3)) = (-2, 0, 6)
-2x + 3y = (-2×1 + 3×2, -2×0 + 3×(-1), -2×(-3) + 3×5) = (4, -3, 21)
Moreover, because the dimension of u, v, w is different from the dimension of x, y, expressions such as u + x, 2v + 3y are meaningless.

11. Vectors: Addition & Multiplication (Contd.)
Geometrically, the addition is described by parallelogram, and the scalar multiplication is the stretching/shrinking and sometimes reversing (for negative scalars) of vectors. For example;

12. Parallelogram Rule for Addition
Let us look at the addition of two vectors (red and blue in the picture) in R2. The green vector is constructed from the parallelogram.
The parallelogram tells us that the two right triangles (in gray shade) are identical.
In particular, the two pink segments of the x-axis have the same length.
This implies that the x-coordinate of the green vector is x1 + x2. By similar argument, the y-coordinate of the green vector is y1 + y2.

13. Linear Combination
The addition and scalar multiplication can also be combined to form linear combinations. For example;
x1v1 + x2v xpvp.
is called a linear combination of vectors v1, ….,vp using weights (or scalars) x1,…xp.

14. Linear Combination (Contd.)
If v1, ….,vp are vectors in Rn ,then the set of all possible linear combinations of v1, ….,vp is called the subset of Rn spanned (or generated) by {v1, ….,vp } and is denoted by span {v1, ….,vp }

15. Linear Combination (Contd.)
That is Span {v1, ….,vp } is the collections of all vectors that can be written in the form x1v1 + x2v xpvp, with x1,…xp scalars (i.e. weights).
For example, the following vectors are all in Span {v1, v2}

16. Linear combination: Example
This example shows the linear combinations of two (nonparallel) vectors. It suggests that all the linear combinations of the two vectors form a plane passing through the origin.

17. Example: 1
Show that b is in the plane spanned by v1 and v2, that is, find weights (scalars) x1, x2 such that
x1v1 + x2v2 = b (Eq:1)
Note: Eq:1 represents a system of linear equations as a single vector equation containing several vectors.
Solution: Using the definitions of scalar multiplication and vector addition to rewrite the vector equation v1 v2 b

18. Example: 1 (Contd.) Which is the same as And ---------- (Eq:2)
Now the vectors on left and right sides of Eq:2 are equal, if and only if their corresponding entries are both equal. That is, x1 & x2 makes the vector equation 1 true if and only if x1 & x2 satisfy the system
(Eq:3)

19. Example: 1 (Contd.)
Augmented matrix of the system
(Eq:3) can be written as
Now performing Gauss-Jordan elimination to obtain reduced echelon form
Therefore, solution of Equation (3) system is x1 = 3 ,x2 = 2. Hence, b is a linear combination of v1 and v2 ,with weights x1=3 & x2=2. That is

20. Notes: Example 1
Note that in Example 1, the original vectors v1, v2 and b are the column of the augmented matrix i.e. v1 v2 b
Lets write above matrix in terms of its column
(Eq:4)
From above discussion, it is clear how to write the augmented matrix immediately after the vector equation (1), without going though the intermediate steps of Example 1.

21. Notes: Example 1 (Contd.)
Thus a vector equation
x1v1 + x2v xnvn = b
has the same solution set as the linear system whose augmented matrix is
[v1 v2 ….. vn b] Eq:5
In particular, b is in span {v1, ……,vn} if and only if the linear system corresponding to Eq:5 is consistent.

22. Linear combination Application
A company manufactures two products, namely A & B.
For $1 worth of product A, company spends $0.45 on materials, $0.25 on labor, $0.15 on overhead.
For $1 worth of product B, company spends $0.40 on materials, $0.30 on labor, $0.15 on overhead.
Suppose the company wishes to manufacture x1 dollars worth of product A and x2 dollars worth of product B. Give a vector that describes the various costs company will incur.

23. Linear combination Application (Contd.)
Vectors a and b represents the “cost per dollar of income” for two products i.e. A and B respectively.
Now costs of manufacturing x1 dollars worth of product A is given by the vector x1a. Similarly, costs of manufacturing x2 dollars worth of product B is given by the vector x2b.
Hence, the total costs for both products are given by the vector x1b+ x2b

24. The Matrix Equation (Ax = b)
A fundamental idea in linear algebra is to view a linear combination of vectors as the product of a matrix and a vector.
If A is an mxn matrix, with columns v1,….,vn, and if x is in Rn, then the vector Ax is the linear combination of the columns of A using the corresponding entries in x as weights, i.e.
Note that Ax is defined if the number of columns of A equals the number of entries (rows) in x

25. The Matrix Equation (Ax = b)
In words, Equation 1 tells us that the product A x of a matrix A with a column matrix x is a linear combination of the column matrices of A with the coefficients coming from the matrix x . For example:

26. The Matrix Equation (Ax = b)
Even though the equation Ax = b involves two vectors as well as a matrix (A), it is referred as matrix equation to distinguish it from vector equation
Thus given a m x n matrix A, with columns v1,….,vn, and given b in Rm, the matrix equation
Ax = b
has the same solution set as a vector equation
x1v1 + x2v xnvn = b
which in turn has the same solution set as the linear system whose augmented matrix is
[v1 v2 ….. vn b]

27. Exercise
Write the following system of linear equations as (a) vector equation (b) Matrix equation
Now here.
Since Vector equation is : x1v1 + x2v xnvn = b

28. Exercise (Contd.)
Now lets consider three vectors on the left hand side of Eq:1 as the column of a matrix (A)
Now using the definition of a matrix times a vector, Eq: 1 can be rewritten in the form Ax = b

29. Linear Combination in Matrix & vector equation from
A x = b
Vector equation : x1v1 + x2 v2 + x3 v3 = b

30. Summary: Matrix Equation of a Linear System
Consider any system of m linear equations in n unknowns.
Since two matrices are equal if and only if their corresponding entries are equal.
The m×1 matrix on the left side of this equation can be written as a product to give Ax= b form.

31. Summary: Matrix Equation of a Linear System (Contd.)
If we designate these matrices by A ,x ,and b ,respectively, the original system of m equations in n unknowns has been replaced by the single matrix equation
The matrix A in this equation is called the coefficient matrix of the system. The augmented matrix for the system is obtained by adjoining b to A as the last column; thus the augmented matrix is

32. Linear independence Consider an n × n matrix
This matrix can be viewed as an ordered set of column vectors:
Where , , etc.

33. Linear independence: Definition
A set of vectors is linearly dependent, if and only if we can find a set of scalars (i.e. weights) , (not all of which are zero) such that
If such a set of scalars cannot be found i.e. the vector equation has only trivial solution (i.e. all scalars are zero), then set of vectors is said to be linearly independent.

34. Example: 1
Determine if the set {v1, v2, v3} is linearly independent
Solution
Since, a set of vectors is linearly independent, if and only if the vector equation of homogenous system has only trivial solution. Therefore, we must determine if there is a nontrivial solution for given homogenous system using row operations on the associated augmented matrix
-2R1+R2
1/3 R2
-3R1+R3
6R2+R3

35. Example: 1 (Contd.) From row echelon form matrix it can be seen that:
x1, x2 : basic variables
x3: free variable
Therefore, each non-zero value of x3 determines a nontrivial solution of giver linear system. Hence, v1,v2,v3 are linearly dependent (i.e. not linearly independent).
b. Find a linear dependence relation among v1,v2,v3
To find a linear dependence relation among v1,v2,v3 reducing the row echelon matrix to reduced row echelon form
-4R2+R1

36. Example: 1 (Contd.)
Now assigning any arbitrary value of x3, say x3=2 will yield:
x1, x2 : basic variables
x3: free variable
Substituting these values into vector equation will yield
Note this is one (out of infinitely many) possible linear dependence relations among v1, v2, v3.

37. Example: 2
Determine if the set {v1, v2, v3} is linearly independent
Solution
We must determine if there is a nontrivial solution for given homogenous system using row operations on the associated augmented matrix
1/3 R1
-1/6 R3
R2+R1
1/2 R2
-2R3+R2
-3R2+R3
-2R3+R1
Since, it can be seen from reduced echelon form that x1=x2=x3=0
i.e. Trivial solution only, implying given set of vectors {v1, v2, v3} is linearly independent.

38. Exercise: 1
Determine if the set {v1, v2, v3} is linearly independent
Solution
We must determine if there is a nontrivial solution for given homogenous system using row operations on the associated augmented matrix
R1  R3
-1/8 R2
6R1+R2
-1/8 R3
Since, it can be seen from reduced echelon form that x1=x2=x3=0
i.e. Trivial solution only, implying given set of vectors {v1, v2, v3} is linearly independent.

39. Exercise: 2
Determine if the set {v1, v2, v3} is linearly independent
Solution
Using row operations on the associated augmented matrix yields row echelon matrix given below
x1, x2 : basic variables
x3: free variable
Therefore, each non-zero value of x3 determines a nontrivial solution of giver linear system. Hence, v1,v2,v3 are linearly dependent (i.e. not linearly independent).

40. Exercise: 3
Determine if the set {v1, v2, v3} is linearly independent
Solution
Using row operations on the associated augmented matrix yields row echelon matrix given below
x1, x2 , x3 : basic variables
Since, it can be seen from row echelon form that this matrix can be row reduced to obtain x1=x2=x3=0 i.e. Trivial solution only, implying given set of vectors {v1, v2, v3} is linearly independent.

41. Linear independence of Matrix columns
Suppose that we begin with a matrix A={V1,…,Vn} instead of a set of vectors as in previous problems. The matrix equation Ax = 0 can be written as:
The columns of a matrix A are linearly independent if and
only if the equation Ax = 0 has only the trivial solution.

42. Example: 1
Determine if the columns (i.e. vectors) of matrix A are linearly independent
Solution
Using row operations on the associated augmented matrix of Ax=0 yields row echelon matrix given below
x1, x2 , x3 : basic variables
Since, it can be seen from row echelon form that this matrix can be row reduced to obtain x1=x2=x3=0 i.e. Trivial solution only, implying columns of A are linearly independent.

43. Exercise: 1
Determine if the columns (i.e. vectors) of matrix A are linearly independent
Solution
Using row operations on the associated augmented matrix of Ax=0 yields reduced row echelon matrix given below
x1, x2 , x3 : basic variables
x4: free variable
Therefore, each non-zero value of x4 determines a nontrivial solution of giver linear system. Hence, columns of A are linearly dependent (i.e. not linearly independent).

44. Exercise: 2
Determine if the columns (i.e. vectors) of matrix A are linearly independent
Solution
Using row operations on the associated augmented matrix of Ax=0 yields row echelon matrix given below
x1, x2 , x3 : basic variables
x4: free variable
Therefore, each non-zero value of x4 determines a nontrivial solution of giver linear system. Hence, columns of A are linearly dependent (i.e. not linearly independent).

45. Linear Transformation

46. Linear Transformation (Contd.)
When we see this notation (T: RnRm) we know that we’re going to be dealing with a function that takes elements from the set Rn (called the domain) and associates them with elements from the set Rm (called the codomain).

47. Linear Transformation (Contd.)
These kinds of functions are called transformations and we say that T maps Rn  into Rm. 
On an element basis we will also say that T maps the element u from Rn  to the element v from Rm

48. Linear Transformation (Contd.)x

49. Example

50. Example: 1 a. Find T(u), the image of u under the transformation T
The vector T(u)= Au is called the image of u under the action f T

51. Example: 1 (Contd.)
b . Find an x in R2 whose image under T is b
The vector T(x)= Ax is called the image of x under the action f T. So solving T(x)= b for x i.e. Solving Ax = b or
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields
Hence,
x1=1.5 &
x2=-0.5
The image of this x under the action of T is b.

52. Example: 1 (Contd.)
c . Determine if c is in the range of the transformation T
The vector c is in the range of T if c is the image of some x in R2, i.e. if c = T(x) for some x. This is another way of asking if the system Ax = c is consistent i.e. is any solution exists for x
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields
The third equation 0 = - 35 implies system is inconsistent. So, c is not in the range of T

53. Example: 2
Suppose that T: R5 -> R2, and T(x) = Ax for some matrix A and each x in R5. How many rows and column does A have?
Since, the column dimension of the “lead” matrix (A) must be equal to the row dimension of the “lag” (x) matrix , therefore, A must have 5 columns for Ax to be defined.
A must have 2 rows for the range of T to be in R2
5 = 5 A x
2  5
5  1 Ax
2  1

54. Exercise: 1
Let A be a 7 x 5 matrix. What must n and m be in order to define T: Rn -> Rm, by T(x) = Ax?
Since, the column dimension of the “lead” matrix (A) must be equal to the row dimension of the “lag” (x) matrix , therefore, x must have 5 columns for Ax to be defined i.e. n = 5.
Ax will have 7 rows as shown below, so m = 7
5 = 5 A x
7  5
5  1 Ax
7  1

55. Exercise:2
The vector T(u)= Au is called the image of u under the action f T
The vector T(v)= Av is called the image of v under the action f T

56. Exercise:3
The vector T(x)= Ax is called the image of x under the action f T. So solving T(x)= b for x i.e. Solving Ax = b or
Since, A has 3 columns, so x must contain 3 rows, for Ax to be defined

57. Exercise:3 (Contd.)
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields reduced echelon form
The image of this x under the action of T is b.
Furthermore, it is clear that given system has unique solution i.e. x is unique.

58. Exercise:4
The vector T(x)= Ax is called the image of x under the action f T. So solving T(x)= b for x i.e. Solving Ax = b or
Since, A has 3 columns, so x must contain 3 rows, for Ax to be defined

59. Exercise:4 (Contd.)
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields reduced echelon form
The image of this x under the action of T is b.
Furthermore, each non-zero value of x3 determines a nontrivial solution of giver linear system. Hence, given system does not have unique solution i.e. x is not unique.

60. Exercise:5
The vector T(x)= Ax is called the image of x under the action f T. Since, x is transformed into a zero vector, So solving T(x)= 0 for x i.e. Solving Ax = 0 or
Since, A has 4 columns, so x must contain 4 rows, for Ax to be defined

61. Exercise:5 (Contd.)
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields row echelon form
x3 and x4 are free variables, implying vector x represents a general solution for all x in R4 that are mapped into the zero vector.

62. Exercise:6
Is b in the range of linear transformation x  Ax
The vector b is in the range of T if b is the image of some x , i.e. if b = T(x) for some x.
This is another way of asking if the system Ax = b is consistent i.e. is any solution exists for x

63. Exercise:6 (Contd.)
Now using the matrix equation [v1 v2 …vn b] and row reducing the augmented matrix yields
The third equation 0 = 2 implies system is inconsistent. So, b is not in the range of linear transformation x  Ax