Download presentation

1
**Signal , Weight Vector Spaces and Linear Transformations**

2
**Notation Vectors in Ãn. Generalized Vectors. đą đą=đnxn+âĻ+đ1x+đ0**

2 n = đą đą=đnxn+âĻ+đ1x+đ0 đą= đ1,1 đ1,2 đ2,1 đ2,2

3
Vector Space 1. An operation called vector addition is defined such that if xÂ Ã X and y Ã X then x + y Ã X. 2. x + y = y + x 3. (x + y) + z = x + (y + z) 4. There is a unique vector 0Â Ã X, called the zero vector, such that x + 0 = x for all xÂ Ã X. 5. For each vector there is a unique vector in X, to be called (-x ), such that x + (-x ) = 0 .

4
Vector Space (Cont.) 6. An operation, called multiplication, is defined such that for all scalars a Ã F, and all vectors xÂ Ã X, a x Â Ã X. 7. For any xÂ Ã X , 1x = x (for scalar 1). 8. For any two scalars a Ã F and b Ã F, and any xÂ Ã X, aÂ (bx)Â =Â (a b) x . 9. (aÂ + b) xÂ =Â a x + b x . 10. a (x + y) Â =Â a x + a y

5
**Examples (Decision Boundaries)**

Is the p2, p3 plane a vector space?

6
**Examples (Decision Boundaries)**

Is the line p1 + 2p2 - 2 = 0 a vector space?

7
**Other Vector Spaces Polynomials of degree 2 or less. đļ[0,1]:**

Continuous functions in the interval [0,1]. đ=sinâĄ(đĄ) đ´=đâ2đĄ

8
**is a set of linearly independent vectors.**

Linear Independence If implies that each then is a set of linearly independent vectors.

9
**Therefore the vectors are independent.**

Example Let This can only be true if Therefore the vectors are independent.

10
**Problem P5.3(Gramian) i. đą1= 1 1 1 đą2= 1 0 1 đą3= 1 2 1**

= â =â2+0+2=0 G= (đą1,đą1) (đą1,đą2) (đą1,đą3) (đą2,đą1) (đą2,đą2) (đą2,đą3) (đą3,đą1) (đą3,đą2) (đą3,đą3) = = â =24â8â16=0

11
**Problem P5.3(Gramian)(cont.)**

đĸđĸđĸ. đą1= đą2= đą3= G= (đą1,đą1) (đą1,đą2) (đą1,đą3) (đą2,đą1) (đą2,đą2) (đą2,đą3) (đą3,đą1) (đą3,đą2) (đą3,đą3) = = â =48â18â30=0 2 đą1- đą2 âđą3=0 G= =4â 0

12
Gramian đ=[đą1,đą2,â¯,đąđ] G đą1,đą2,â¯,đąđ = đTÃđ = (đą1,đą1) (đą1,đą2) (đą2,đą1) (đą2,đą2) â¯ (đą1,đąđ) (đą2,đąđ) âŽ âą âŽ (đąđ,đą1) (đąđ,đą2) â¯ (đąđ,đąđ) G đą1,đą2,â¯,đąđ â 0 iff đą1,đą2,â¯,đąđ are linearly independent G đą1,đą2,â¯,đąđ =0 iff đą1,đą2,â¯,đąđ are linearly dependent

13
Spanning a Space A subset spans a space if every vector in the space can be written as a linear combination of the vectors in the subspace.

14
Basis Vectors A set of basis vectors for the space X is a set of vectors which spans X and is linearly independent. The dimension of a vector space, Dim(X), is equal to the number of vectors in the basis set. Let X be a finite dimensional vector space, then every basis set of X has the same number of elements.

15
**Example Polynomials of degree 2 or less. Basis A: Basis B:**

(Any three linearly independent vectors in the space will work.) How can you represent the vector x = 1+2t using both basis sets? đ= đŽ1+đŽ2 2 +đŽ2âđŽ1= 3đŽ2âđŽ1 2 đŽ1+đŽ2=2,đŽ2âđŽ1=2t

16
Inner Product / Norm A scalar function of vectors x and y can be defined as an inner product, (x , y ), provided the following are satisfied (for real inner products): (x , y ) = (y , x ) . (x , ay1+by2) = a( x , y1) + b( x , y2) . (x , x) â§ 0 , where equality holds iff x = 0 . A scalar function of a vector x is called a norm, ||x || provided the following are satisfied: ||x || â§0 . ||x || = 0 iff x = 0 . ||a x || = |a| ||x || for scalar a . ||x + y || â¤ ||x || + ||y || .

17
**Example Angle: Standard Euclidean Inner Product**

Standard Euclidean Norm ||x || = (x , x)1/2 ||x|| = (xTx)1/2 = x12 + x xn2 đļ[0,1] Inner Product(see Problem P5.6): đĨ, đĻ = 0 1 đĨ đĄ đĻ đĄ đđĄ cosq = (x , y) ||x || ||y || Angle:

18
**Problem P5.6 An inner product must satisfy the following properties.**

1. đĨ,đĻ =(đĻ,đĨ) đĨ,đĻ = 0 1 đĨ đĄ đĻ đĄ đđĄ = 0 1 đĻ đĄ đĨ đĄ đđĄ =(đĻ,đĨ) 2. đĨ,đđĻ1+đđĻ2 =đ đĨ,đĻ1 +đ(đĨ,đĻ2) đĨ,đđĻ1+đđĻ2 = 0 1 đĨ đĄ đđĻ1 đĄ +đđĻ2 đĄ đđĄ = 0 1 đĨ đĄ đđĻ1 đĄ đđĄ đĨ đĄ đđĻ2 đĄ đđĄ= đ đĨ,đĻ1 +đ(đĨ,đĻ2)

19
Problem P5.6(cont.) 3. đĨ,đĨ âĨ0,where equality holds iff x is the zero vector đĨ,đĨ = 0 1 đĨ đĄ đĨ đĄ đđĄ 0 1 đĨ đĄ 2đđĄ âĨ0 Equality holds here only if đĨ đĄ =0 for 0â¤đĄâ¤1 ,which is the zero vector

20
**Two vectors x , y ÃX are orthogonal if (x , y) = 0 .**

Orthogonality Two vectors x , y ÃX are orthogonal if (x , y) = 0 . Example Any vector in the p2,p3 plane is orthogonal to the weight vector.

21
**Gram-Schmidt Orthogonalization**

Independent Vectors Orthogonal Vectors đĻ1,đĻ2,âĻ,đĻđ đŖ1,đŖ2,âĻ,đŖđ Step 1: Set first orthogonal vector to first independent vector. đŖ1=đĻ1 Step 2: Subtract the portion of y2 that is in the direction of v1. đŖ2=đĻ2âđđŖ1 Where a is chosen so that v2 is orthogonal to v1: đŖ1,đŖ2 = đŖ1,đĻ2âđđŖ1 = đŖ1,đĻ2 âđ đŖ1,đŖ1 =0 đ= (đŖ1,đĻ2) (đŖ1,đŖ1)

22
**Gram-Schmidt (Cont.) Projection of y2 on v1: (đŖ1,đĻ2) (đŖ1,đŖ1) đŖ1**

Step k: Subtract the portion of yk that is in the direction of all previous vi . đŖđ=đĻđâ đ=1 đâ1 (đŖđ,đĻđ) (đŖđ,đŖđ) đŖđ

23
Example đ˛1= ,đ˛2=

24
Example Step 1īŧ đ¯1=đ˛1= 2 1

25
Example (Cont.) Step 2.

26
Vector Expansion If a vector space X has a basis set {v1, v2, ..., vn}, then any x ÃX has a unique vector expansion: đ= đ=1 đ đĨđđŖđ =đĨ1đŖ1+đĨ2đŖ2+âĻ+đĨđđŖđ If the basis vectors are orthogonal, and we take the inner product of vj and x : đŖj,đ = đŖj, đ=1 đ đĨiđŖi = đ=1 đ đĨi(đŖj,đŖi) =đĨj(đŖj,đŖj) Therefore the coefficients of the expansion can be computed: đĨj= (đŖj,đ) (đŖj,đŖj)

27
**Column of Numbers The vector expansion provides a meaning for**

writing a vector as a column of numbers. đ= đ=1 đ đĨđđŖđ =đĨ1đŖ1+đĨ2đŖ2+âĻ+đĨđđŖđ đĨ= đĨ1 đĨ2 âŽ đĨđ To interpret x, we need to know what basis was used for the expansion.

28
**Reciprocal Basis Vectors**

Definition of reciprocal basis vectors, ri: đđ,đŖđ = đâ đ = đ=đ where the basis vectors are {v1, v2, ..., vn}, and the reciprocal basis vectors are {r1, r2, ..., rn}. For vectors in Ãn we can use the following inner product: đđ,đŖ đ =đ đ đđŖđ åæ¸åēåē Therefore, the equations for the reciprocal basis vectors become:

29
**Vector Expansion đ=đĨ1đŖ1+đĨ2đŖ2+âĻ+đĨđđŖđ**

Take the inner product of the first reciprocal basis vector with the vector to be expanded: đ1,đ =đĨ1 đ1,đŖ1 +đĨ2 đ2,đŖ2 +âĻ+đĨđ đđ,đŖđ By definition of the reciprocal basis vectors: đ1,đŖ2 = đ1,đŖ3 =âĻ= đ1,đŖđ =0 đ1,đŖ1 =1 Therefore, the first coefficient in the expansion is: đĨ1=(đ1,đ) In general, we then have (even for nonorthogonal basis vectors): đĨđ=(đđ ,đ)

30
Example đŖ1đ = ,đŖ2đ = 1 2 Basis Vectors: đĨđ = Vector to Expand:

31
**Example (Cont.) Reciprocal Basis Vectors: Expansion Coefficients:**

đĢ1= â đĢ2= â Expansion Coefficients: Matrix Form: đą1đŖ=đĢ1đĄđąđ = â =â 1 2 đąđŖ=đđđąđ =đâ1đąs = â 1 3 â = â đą2đŖ=đĢ2đĄđąđ = â =1

32
**Example (Cont.) Ī=0đ 1+ 3 2 đ 2=â 1 2 đŖ1+1đŖ2**

The interpretation of the column of numbers depends on the basis set used for the expansion.

33
**Linear Transformation**

đŖ1 đŖ2 = đ 1 đ = đ 1 đ 2 đ đ 1 đ 2 =đâ1 v1 đŖ2 3 2 đ 2= đ 1 đ = đŖ1 đŖ2 đâ = đŖ1 đŖ â1 3 â = đŖ1 đŖ2 â = â 1 2 đŖ1+đŖ2 đąđŖ=đâ1đąđ . This operation, call a change of basis

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google