Slide 1 EE3J2 Data Mining EE3J2 Data Mining Lecture 9(b) Principal Components Analysis Martin Russell

Slide 2 EE3J2 Data Mining Objectives  To illustrate PCA through an example application  3D dance motion modelling

Slide 3 EE3J2 Data Mining Data  Analysis of dance sequence data  Body position represented as 90 dimensional vector  Dance sequence represented as a sequence of these vectors  MEng FYP 2004/5, Wan Ni Chong

Slide 4 EE3J2 Data Mining Data Capture (1)

Slide 5 EE3J2 Data Mining Data Capture (2)

Slide 6 EE3J2 Data Mining Data Capture (3)

Slide 7 EE3J2 Data Mining Calculating PCA  Step 1: Arrange data as a matrix –Rows correspond to individual data points –Number of columns = dimension of data (= 90) –Number of rows = number of data points = N

Slide 8 EE3J2 Data Mining Calculating PCA (step 2)  Compute the covariance matrix of the data  In MATLAB >>C = cov(X)  Alternatively (as in slides from last lecture): – calculate the mean vector m, –subtract m from each row of X to give Y –Then

Slide 9 EE3J2 Data Mining Calculating PCA (step 3)  Do an eigenvector decomposition of C, so that: C = UDU T  Where –U is a unitary (rotation) matrix –D is a diagonal matrix (in fact all elements of D will be real and non-negative)  In MATLAB type >>[U,D] = eig(C)

Slide 10 EE3J2 Data Mining Calculating PCA (step 4)  Each column of U is a principal vector  The corresponding eigenvalue indictates the variance of the data along that dimension  Large eigenvalues indicate significant components of the data  Small eigenvalues indicate that the variation along the corresponding eigenvectors may be noise

Slide 11 EE3J2 Data Mining Eigenvalues Insignificant Component s More Significant Components PCsData 1 st 2 nd 3 rd 90 th 1 st 90 th Eigenvalues

Slide 12 EE3J2 Data Mining Calculating PCA (step 6)  It may be advantageous to ignore dimensions which correspond to small eigenvalues and only consider the projection of the data onto the most significant eigenvectors  In this way the dimension of the data can be reduced

Slide 13 EE3J2 Data Mining Visualising PCA Original pattern (blue) U Eigenspace Set coordinates 11 – 90 to zero U -1 Reduced pattern (red)

Slide 14 EE3J2 Data Mining PCA Example  Original 90 dimensional data reduced to just 1 dimension

Slide 15 EE3J2 Data Mining PCA Example  Original 90 dimensional data reduced to 10 dimensions

Slide 16 EE3J2 Data Mining Summary  Example of PCA  Analysis of 90 dimensional 3D dance data  Analysis shows that PCA can reduce 90 dimensional representation to just 10 dimensions with minimal loss of accuracy