Download presentation

Presentation is loading. Please wait.

Published byGannon Millington Modified about 1 year ago

1
Environmental Data Analysis with MatLab Lecture 15: Factor Analysis

2
Lecture 01Using MatLab Lecture 02Looking At Data Lecture 03Probability and Measurement Error Lecture 04Multivariate Distributions Lecture 05Linear Models Lecture 06The Principle of Least Squares Lecture 07Prior Information Lecture 08Solving Generalized Least Squares Problems Lecture 09Fourier Series Lecture 10Complex Fourier Series Lecture 11Lessons Learned from the Fourier Transform Lecture 12Power Spectral Density Lecture 13Filter Theory Lecture 14Applications of Filters Lecture 15Factor Analysis Lecture 16Orthogonal functions Lecture 17Covariance and Autocorrelation Lecture 18Cross-correlation Lecture 19Smoothing, Correlation and Spectra Lecture 20Coherence; Tapering and Spectral Analysis Lecture 21Interpolation Lecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-Tests Lecture 24 Confidence Limits of Spectra, Bootstraps SYLLABUS

3
purpose of the lecture introduce Factor Analysis a method of detecting patterns in data

4
source A ocean sediment source B s4s4 s2s2 s3s3 s1s1 s5s5 example: sediment samples are a mix of several sources

5
e1e1 e2e2 e3e3 e4e4 e5e5 e1e1 e2e2 e3e3 e4e4 e5e5 s1s1 s2s2 ocean sediment what does the composition of the samples tell you about the composition of the sources?

6
another example Atlantic Rock Dataset chemical composition for several thousand rocks

7
Rocks are a mix of minerals, and … mineral 1 mineral 2 mineral 3 rock 1rock 2 rock 3 rock 4 rock 5 rock 6 rock 7 …minerals have a well-defined composition

8
Which simpler? rocks have a chemical composition or rocks contain minerals and minerals have chemical compositions

9
answer will depend on how many minerals are involved and how many elements are in each mineral

10
representing mixing with matrices

11
the sample matrix, S N samples by M elements e.g. sediment samples rock samples word element is used in the abstract sense and may not refer to actual chemical elements

12
the factor matrix, F P factors by M elements e.g. sediment sources minerals note that there are P factors a simplification if P

13
the loading matrix, C N samples by P factors specifies the mix of factors for each sample

14
summary samples contain factors factors contain elements

15
an important issue how many factors are needed to represent the samples? need at most P=M but is P < M ?

16
simple example using ternary diagrams

17
samples element element B

18
samples element element B line of samples implies only 2 factors, so P=2

19
factors samples element element B

20
A) B) factor, f’ 2 factor, f’ 1 factor, f 1 factor, f 2 data do not uniquely determine factors two bracketing factorsmost typical factor and deviation from it

21
mathematically S = CF = C’ F’ with F’ = M F and C’ = C M -1 where M is any P×P matrix with an inverse must rely on prior information to choose M

22
a method to determine the minimum number of factors, P and one possible set of factors

23
a digression, but an important one suppose that we have an N×N square matrix, M and we experiment with it by multiplying “input” vectors, v, by it to create “output” vectors, w w = Mv

24
surprisingly, the answer to the question when is the output parallel to the input ? tells us everything about the matrix

25
if w is parallel to v then w = λ v where λ is a proportionality factor the equation w = Mv is then λ v = Mv or (M - λ I)v=0

26
but if (M - λ I)v=0 then it would seem that v = (M - λ I) -1 0 = 0 which is not a very interesting solution w is parallel to v when v is zero

27
to make an interesting solution you must choose λ so that (M - λ I) -1 doesn’t exist which is equivalent to choosing λ so that det(M - λ I)=0

28
since a matrix with zero determinant has no inverse

29
in the 2×2 case … this is a quadratic equation in λ and so has two solutions λ 1 and λ 2

30
in the N×N case det(M - λ I)=0 is an N -order polynomial equation and so has N solutions λ 1, λ 2, … λ N each corresponds to a different v v (1), v (2), … v (N)

31
“eigenvalues” “eigenvectors”

32
N × N matrix, M w = Mv when is the output parallel to the input ? N different cases Mv (1) = λ 1 v (1) Mv (2) = λ 2 v (2) … Mv (N) = λ N v (N)

33
Mv (1) = λ 1 v (1) Mv (2) = λ 2 v (2) … Mv (N) = λ N v (N) simplify notation MV = V Λ

34
In the text its shown that if M is symmetric then all λ ’s are real v ’s are orthonormal v (i)T v (j) = 1 if i=j 0 if i ≠ j

35
In the text its shown that if M is symmetric then all λ ’s are real v ’s are orthonormal v (i)T v (j) = 1 if i=j 0 if i ≠ j implies V T V = VV T = I

36
MV = V Λ post-multiply by V T M = V Λ V T M can be constructed from V and Λ so when is the output parallel to the input ? tells you everything about M

37
now here’s what this has to do with factors

38
suppose S is square and symmetric then S = CF = V Λ V T

39
C F

40
C F S can be represented by M mutually-perpendicular factors, F

41
furthermore, suppose that only P eigvenvalues are nonzero the eigenvectors with zero eigenvalues can be thrown out of the equation

42
we can reduce the number of factors from M to P S = CF = V P Λ P V P T C F S can be represented by P mutually-perpendicular factors, F P

43
unfortunately … S is usually neither square nor symmetric so a patch in the methodology is needed

44
the trick … S T S is an M × M square matrix

45
suppose S T S has eigenvalues Λ P and eigenvectors V P

46
S T S written in terms of its eigenvalues and eigenvectors

47
write Λ P as product of its square roots

48
S T S written in terms of its eigenvalues and eigenvectors write Λ P as product of its square roots insert identity matrix, I

49
S T S written in terms of its eigenvalues and eigenvectors write Λ P as product of its square roots write I = U p T U p, with U p as yet unknown insert identity matrix, I

50
S T S written in terms of its eigenvalues and eigenvectors write Λ P as product of its square roots write I = U p T Up, with U p as yet unknown insert identity matrix, I group and write first group as transpose of transpose

51
S T S written in terms of its eigenvalues and eigenvectors write Λ P as product of its square roots write I = U p T Up, with U p as yet unknown insert identity matrix, I group and write first group as transpose of transpose compare

52
so

53
and so

54
and so called the “singular value decomposition” of S now the non-square, non-symmetric matrix, S, is represented as a mix of P mutually perpendicular factors called the “singular values”

55
the matrix of loadings, C. the matrix of factors, F since C depends on Σ, the samples contains more of the factors with large singular values than of the factors with the small singular values

56
in MatLab svd() computes all M factors (you must decide how many to use)

57
singular values, ii index, i singular values of the Atlantic Rock dataset (sorted into order of size)

58
singular values, ii index, i singular values of the Atlantic Rock dataset (sorted into order of size) discard, since close to zero

59
factors of the Atlantic Rock dataset

60
factor of the Atlantic Rock dataset factor 1 is the “typical factor”

61
factor of the Atlantic Rock dataset factor 2 as MgO increases, Al 2 O 3 and CaO decreases

62
factor of the Atlantic Rock dataset factor 3: as Al 2 O 3 increases, FeO and CaO increase

63
graphical representation of factors 2 through 5 f5f5 f2f2 f3f3 f4f4 SiO 2 TiO 2 Al 2 O 3 FeO total MgO CaO Na 2 O K2OK2O

64
C2C2 C3C3 C4C4 factor loadings C 2 through C 4 plotted in 3D factors 2 through 4 capture most of the variability of the rocks

65
Al Ti0 2 Al Si0 2 K20K20 Fe0 Mg0 Al A)B) C)D)

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google