Download presentation

Presentation is loading. Please wait.

Published byTerrence Waggoner Modified over 2 years ago

1
بنام خدا 1

2
An Introduction to multi-way analysis Mohsen Kompany-Zareh IASBS, Nov 1-3, 2010 2 Session one

3
3 The main source:

4
4 Kronecker product Khatri-Rao product Multi-way data Matricizing the data Interaction triad GG PARAFAC Panel performance Matricizing and subarray Rank Dimensionality vector Rank-deficiency in three-way arrays Tucker3 rotational freedom Unique solution Tucker2 model Tucker1 model

5
>> A=[2 3 4; 2 3 4] >> B=[3 4; 3 5] >> krnAB=[A(1,1)*B A(1,2)*B A(1,3)*B ; A(2,1)*B A(2,2)*B A(2,3)*B] krnAB = 6 8 9 12 12 16 6 10 9 15 12 20 6 8 9 12 12 16 6 10 9 15 12 20 >> kronecker product (A B) 5

6
>> A=[2 3 4; 2 3 4] >>B=[3 4; 3 5] >> p=kron(A,B) >>p= 6 8 9 12 12 16 6 10 9 15 12 20 6 8 9 12 12 16 6 10 9 15 12 20 >> 6 All columns in A see all columns in B. kronecker product

7
>> A=[2 3 4; 2 3 4] >>C=[3 4 5; 3 5 2] >>krnAC=[kron(A(:, ),C(:, ))... column 1 kron(A(:,1),C(:,2))... column 2 kron(A(:,1),C(:,3))..... kron(A(:,2),C(:,1))..... kron(A(:, ),C(:, ))..... kron(A(:,2),C(:,3))... kron(A(:,3),C(:,1))... kron(A(:,3),C(:,2))... kron(A(:, ),C(:, ))] column 9 krnAC = 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 >> Khatri-Rao Product kronecker product 7 1 1 22 3 3

8
>> A=[2 3 4; 2 3 4] >>C=[3 4 5; 3 5 2] krnAC = 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 6 8 10 9 12 15 12 16 20 6 10 4 9 15 6 12 20 8 kronecker product 8 vec(a1 b1) vec(a2 b2)vec(a3 b3) vec(a1 b2) vec(a1 b3) vec(a2 b3) vec(a2 b1) vec(a3 b1) vec(a3 b2) Interaction terms

9
>> A=[2 3 4; 2 3 4] >> B=[3 4 5; 3 5 2] khtrAB= 6 12 20 6 15 8 6 12 20 6 15 8 >> 9 No of columns in A should be the same as the number of columns in B. Khatri-Rao Product

10
10 Kronecker product Khatri-Rao product Multi-way data Matricizing the data Interaction triad GG PARAFAC Panel performance Matricizing and subarray Rank Dimensionality vector Rank-deficiency in three-way arrays Tucker3 rotational freedom Unique solution Tucker2 model Tucker1 model

11
(generalization of matrix algebra) A zero-order tensor: a scalar; a first-order tensor : a vector; a second-order tensor (a matrix) for a sample => 3 way data, for analysis a third-order tensor (three-way array) for a sample => 4 way data, for analysis a fourth-order tensor : a four-way array and so on. 11 Multi-way Data

12
12 One component, HPLC-DAD a1 b1

13
13 One component, HPLC-DAD, different concentrations (elution profile) Only the intensities are changed... These 9 matrices form a TRIAD, the simplest trilinear data

14
14 >> a1' 0.0033 0.0971 0.8131 1.9506 1.3406 0.2640 0.0149 >> b1' 0.0222 1.7650 0.4060 0.8826 0.0111 0.0000 >> c1' 1 2 3 4 5 6 7 8 9 10 11 12 A triad : X A cube of data 12x7x7 3 rd order data for one sample Obtained from Tensor product of 3 vectors a1 b1 c1

15
% A triad by outer product % X111=a1 b1 c1... for l=1:length(a1) for m=1:length(b1) for n=1:length(c1) disp([l m n]) Xtriad(l,m,n)=a1(l)*b1(m)*c1(n); end X=Xtriad;.... a1 b1 c1

16
Matricizing the data 16 X111= Unfold3D(X111, 1) (in three directions) The first chemical component

17
17...and for the 2 nd and the next chemical components: X111 = a1 b1 c1 X222 a2 b2 c2 X222 = a2 b2 c2 X333 = a3 b3 c3 Each component in a separate triad (no interaction) + + X = X111 + X222 + X333 Trilinear PARAFAC

18
18 X111 = a1 b1 c1 X222 a2 b2 c2 X222 = a2 b2 c2 2b2 X121 = a1 b2 c1 + + X = X111 + X222 + X121 Non Trilinear!! Tucker In the presence of Interaction : Interaction triad

19
19 How many interaction triads? For two components in three modes: X111 = a1 b1 c1 X112 = a1 b1 c2 X121 = a1 b2 c1 X122 = a1 b2 c2 X211 = a2 b1 c1 X212 = a2 b1 c2 X221 = a2 b2 c1 X222 = a2 b2 c2 G(111)= 2 G(112)= 0 G(121)= 1 G(122)= 0 G(211)= 0 G(212)= 0 G(221)= 0 G(222)=-3 6 possible interaction triads1 interaction triads G

20
A(11x2) G(2x2x2) C(3x2) B(100 2) G(111)= 2 G(222)=-3 G(121)= 1

21
21 For three components in three modes: (3 3 3) – 3 = 24 possible interactions

22
A(15x4) G(?x?x?) C(20x2) B(100 3) How many G elements?

23
23 % Tucker3 outer product G=rand(4,3,2); for p=1:size(G,1) for q=1:size(G,2) for r=1:size(G,3) for i=1:size(A,2) for k=1:size(C,2) for m=1:size(B,2) disp([p q r i j k]) Xtriad(l,m,n)=A(i,l)*B(j,m)*C(k,n)*G(i,j,k); end X=X+Xtriad; end One triad

24
24 What about Tucker4?

25
25

26
26 % PARAFAC outer product G=zeros(3,3,3); G(1,1,1)=1;G(2,2,2)=1;G(3,3,3)=1; for p=1:size(G,1) for q=1:size(G,2) for r=1:size(G,3) for i=1:size(A,2) for k=1:size(C,2) for m=1:size(B,2) disp([p q r i j k]) Xtriad(l,m,n)=A(i,l)*B(j,m)*C(k,n)*G(i,j,k); end X=X+Xtriad; end One triad

27
A(15x3) C(20x3) B(100 3) PARAFAC Simple interpretation

28
Monitoring panel performance within and between experiments by multi-way models Rosaria Romano and Mohsen Kompany-Zareh Copenhagen Univ, 2007

29
Organic Milk of high Quality Sensory studies 2007- University of Copenhagen - Spring experiment (May, week 21 & 22) - Autumn experiment (September, week 36 & 37) Two different experiments were conducted in 2007: The objective is to establish knowledge about production of high quality organic milk with a composition and flavour different from conventionally produced milk.

30
Spring experiment data Data description: 7 varieties of milk with respect to: - 2 cow races: Holstein-Fries (HF), Jersey (JE); - 7 farms: WB, EMC, UGJ, JP, HM, OA, KI. panel: - 9 assessors, 2 sessions (focus on the second!), 3 replicates for each session. 12 descriptors: odor (green), appearance (yellow), flavor (creamy, boiled-milk, sweet, bitter, metallic, sourness, stald-feed) after taste (astringent0, fatness, astringent20). measurement scale: continuous scale anchored at 0 and 15.

31
Parafac on the spring experiment(1) Model: Parafac with two components (27.9% ExpVar), on data averaged across the samples mode HF JE high reproducibility of the replicates in both groups; big variation in the JE group: - WB is the less yellow JE milk; - UGJ seems have something in common with HF group.

32
Parafac on the spring experiment(2) Model: Parafac with two components (27.9% ExpVar), on data averaged across the samples mode Best Reliability on Multi-way Assessment (Bro and Romano, 2008)

33
33 Kronecker product Khatri-Rao product Multi-way data Matricizing the data Interaction triad GG PARAFAC Panel performance Matricizing and subarray Rank Dimensionality vector Rank-deficiency in three-way arrays Tucker3 rotational freedom Unique solution Tucker2 model Tucker1 model

34
A has full rank (if and only if ) : r(A) = min(I,J). If r(A )= R, [Schott 1997] A = t 1 p 1 + ·· ·+t R p R R rank one matrices (t r p r, components). 34 Bases are not unique: rotational freedom intensity intensity (or scale) indeterminacy. sign indeterm sign indeterminacy. Rank

35
If X (I × J ) : generated with I × J random numbers =>probability of (X has less than full rank) =0.. => measured data sets in chemistry: always full rank (mathematical rank) <= measurment noise Ex: UV spectra (100 wavelengths) ; ten different samples, each: same absorbing species at different concentrations. X (10 ×100) if Lambert–Beer law holds : rank one. 35 mathem rank = ten + measurement errors => mathem rank = ten.

36
X = cs’ + E = X hat + E (model of X) vector c : concns, s : pure UV spectrum of the abs species E : noise part. systematic varNoise 1. systematic variation 2. Noise (undesirable) X hat pseudo-rank =Math rank (X hat ) = one X < math rank (X). ‘chemical rank’ : number of chemical sources of variation in data. 36

37
Rank deficiency pseudo-rank < chemical rank. ( linear relations in or restrictions on the data). Ex; X = c 1 s 1 + c 2 s 2 + c 3 s 3 + E, s 1 = s 2 (linear relation) => X = (c 1 + c 2 )s 1 + c 3 s 3 + E Chem rank (X)= 3 pseudo-rank (X)= 2, rank deficient 37

38
38

39
A randomly generated 2 × 2 × 2 array to have a rank lower than three : a positive probability [Kruskal 1989]. a probability of 0.79 of obtaining a rank two array a probability of 0.21 of obtaining a rank three. probability of obtaining rank one or lower is zero. generalized to : 2 × n × n arrays [Ten Berge 1991]. 39

40
2 × 2 × 2 array: the maximum rank: three typical rank: {2, 3}, (almost all individual rank: very hard to establish. Three way rank : important in second-order calibration and curve resolution. for degrees of freedom ?? for significance testing. 40

41
X(4 × 3 × 2) Boldfaces : in the foremost frontal slice 41 Matricizing and Sub-arrays Matricizing

42
42 sub-arrays

43
Row-rank, column-rank, tube-rank two-way X : rank(X) = rank(X’) column rank= row rank :not hold for three-way arrays. three-way array X(I × J × K) : matricized in three different ways (i) row-wise, giving X(J ×IK), a two-way array (ii) column-wise, giving X(I×JK), (iii) tube-wise, giving X(K×IJ). and three more with the same ranks,not mentioned ranks of the arrays X(J×IK),X(I×JK) and X(K×IJ), = (P, Q, R): dimensionality vector of X. 43 Dimensionality vector

44
44 P, Q and R: not necessarily equal. In contrast with two-way P = Q = r(X). dimensionality vector (P, Q, R) of a three-way array X with rank S Obeys certain inequalities [Kruskal 1989]: (i) P ≤ QR ; Q ≤ PR; R ≤ PQ (ii) max(P, Q, R) ≤ S ≤ min(PQ, QR, PR)

45
These arrays have rank 4, 3, and 2. Dimensionality vector is [4 3 2] P, Q and R can be unequal. 45 Three matricized forms:

46
Pseudo-rank, rank deficiency and chemical sources of variation pseudo-rank of three-way arrays: straight generalization of the two-way definit. X = X hat + E E : array of residuals. pseudo-rank of X = minimum # PARAFAC components necessary to exactly fit X hat. 46

47
Spectrophometric acid-base titration of mixtures of three weak mono-protic acids (or Flow injection analysis + pH gradient) HA2 H + + A2 - A3 HA3 H + + A3 - HA4 HA4 H + + A4 - six components models of separate titration of the three analytes (HA2, HA3, HA4), X HA2 = c a,2 s a,2 + c b,2 s b,2 + E HA2 X HA3 = c a,3 s a,3 + c b,3 s b,3 + E HA3 X HA4 = c a,4 s a,4 + c b,4 s b,4 + E HA4 10 samples, 15 titn points, and 20 wavel.s => X(10×15×20), 47 Rank-deficiency in three-way arrays

48
X = X hat + E c a,2 + c b,2 = α(c a,3 + c b,3 ) = β(c a,4 + c b,4 ) only four independently varying concn profiles. Pseudo-rank (X(IJ K)) = four. pseudo-rank (X(3 × JK)) =three. six different ultraviolet spectra form, pseudo-rank (X(6 × KI)) =six ==>> a Tucker3 (6,4,3) model is needed to fit X. 48

49
49 3 6 4 = 72 nonzero elements !! Inequality laws: (i) P ≤ QR ; Q ≤ PR; R ≤ PQ (ii)max(3, 6, 4) ≤ S ≤ min(PQ, QR, PR) 6 ≤ S ≤ 12

50
50 three-way rank of X is ≥ 6 (six PARAFAC components fit the data) Pseudo rank (S=6) is not less than chemical rank(6) => no three-way rank deficiency. rank deficiencies in one loading matrix of a three-way array are not the same as a three-way rank deficiency.

51
51 How it is possible to have a rank deficient three-way data?

52
52 Kronecker product Khatri-Rao product Multi-way data Matricizing the data Interaction triad GG PARAFAC Panel performance Matricizing and subarray Rank Dimensionality vector Rank-deficiency in three-way arrays Tucker3 rotational freedom Unique solution Tucker2 model Tucker1 model

53
Tucker component models Ledyard Tucker was one of the pioneers in multi-way analysis. He proposed a series of models nowadays called N-mode PCA or Tucker models [Tucker 1964- 1966] 53

54
54 TUCKER3 MODELS : nonzero off-diagonal elements in its core.

55
In Kronecker product notation the Tucker3 model 55

56
PROPERTIES OF THE TUCKER3 MODEL T A : arbitrary nonsingular matrix Such a transformation of the loading matrix A can be defined similarly for B and C, using T B and T C, respectively 56 Tucker3 rotational freedom

57
Tucker3 model has rotational freedom, But: it is not possible to rotate Tucker3 core-array to a superdiagonal form (and to obtain a PARAFAC model.! 57 The Tucker3 model : not give unique component matrices it has rotational freedom.

58
rotational freedom Orthogonal component matrices (at no cost in fit by defining proper matrices T A, T B and T C ) convenient : to make the component matrices orthogonal easy interpretation of the elements of the core- array and of the loadings by the loading plots 58

59
59 SS of elements of core-array amount of variation explained by combination of factors in different modes. variation in X: unexplained and explained by model Using a proper rotation all the variance of explained part can be gathered in core.

60
60 The rotational freedom of Tucker3 models can also be used to rotate the core-array to a simple structure as is also common in two-way analysis (will be explained).

61
Imposing the restrictions A’A = B’B = C’C = I : not sufficient for obtaining a unique solution To obtain uniqe estimates of parameters, 1. loading matrices should be orthogonal, 2. A should also contain eigenvectors of X(CC’ ⊗ BB’)X’ corresp. to decreasing eigenvalues of that same matrix; similar restrictions should be put on B and C [De Lathauwer 1997, Kroonenberg et al. 1989]. 61 Unique solution

62
62 Unique Tucker Simulated data: Two components, PARAFAC model

63
63 Unique Tucker3 component model P=Q=R=3 Only two significant elements in core

64
64 Not exactly unique!

65
65 Not exactly unique! But very similar

66
66 Kronecker product Khatri-Rao product Multi-way data Matricizing the data Interaction triad GG PARAFAC Panel performance Matricizing and subarray Rank Dimensionality vector Rank-deficiency in three-way arrays Tucker3 rotational freedom Unique solution Tucker2 model Tucker1 model

67
all three modes are reduced In tucker 3 67

68
68 Data reduction only in two dimensions... Tucker2 model

69
Tucker1 models : reduce only one of the modes. + X (and accordingly G) are matricized : 69 Tucker1 model

70
70 different models [Kiers 1991, Smilde 1997]. Threeway component models for X (I × J × K), A : the (I × P) component matrix (of first (reduced) mode, X(I×JK) : matricized X; A,B,C : component matrices; G : different matricized core-arrays ; I :superdiagonal array (ones on superdiagonal. (compon matrices, core-arrays and residual error arrays : differ for each model => PARAFAC model is a special case of Tucker3 model. PARAFAC: X (IxJK) = A G (RxRR) (C B)’ Tucker3: X (IxJK) = A G (PxQR) (C B)’ Tucker2: X (IxJK) = A G (PxQK) (I B)’ Tucker1: X (IxJK) = A G (PxJK) (I I)’

71
71 Thanks and See you in the next session...

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google