Presentation is loading. Please wait.

Presentation is loading. Please wait.

Factor Analysis and Principal Components Removing Redundancies and Finding Hidden Variables.

Similar presentations


Presentation on theme: "Factor Analysis and Principal Components Removing Redundancies and Finding Hidden Variables."— Presentation transcript:

1 Factor Analysis and Principal Components Removing Redundancies and Finding Hidden Variables

2 Two Goals Measurements are not independent of one another and we need a way to reduce the dimensionality and remove collinearity – Principal components Measurements affected by unobserved, latent factors – we want to estimate those factors – Factor analysis

3 Principal Components Qualities we are interested in studying can be measured indirectly Measurements have redundancy – e.g. multiple measurements reflect size Measurements reflect more than one property – e.g. size and shape

4 Steps Select variables – generally interval or ratio scale variables – dichotomies can also be used Analysis usually begins with a covariance or correlation matrix of the variables Principal components are extracted that reflect correlations between variables

5 Terminology Eigenvalues – a measure of the variance “explained” by a component Eigenvectors – dimensions that have been extracted from the correlation matrix – the principal components Communality – amount of variance for a variable “explained” by a subset of the components

6 Issues Need more cases than variables Sum of the eigenvalues = number of variables or number of cases – 1 whichever is smaller Principal components are often standardized to a variance of 1. Each component is independent

7 Results Eigenvalues for extracted components and proportion of variance “explained” Loadings (correlations) between variables and components Scores for the components for each case

8 Number of Components Principal components can be used simply to produce k independent components for k inter-related variables More commonly, the number of components extracted is limited to a smaller number, e.g. those with eigenvalues>1

9 Example Rcmdr Statistics | Dimensional analysis | Principal-components princomp() and prcomp() in R compute principal components – prcomp() is more stable Packages psych and ade4 have principal component functions

10 Handaxes Collection of 600 handaxes from Furze Platt, Maidenhead, England at the Royal Ontario Museum Seven dimensional measurements measure shape and size

11 >.PC <- princomp(~L+L1+T+T1+W+W1+W2, cor=TRUE, data=HandAxes) > unclass(loadings(.PC)) # component loadings Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 L L T T W W W Comp.6 Comp.7 L L T T W W W

12 >.PC$sd^2 # component variances Comp.1 Comp.2 Comp.3 Comp.4 Comp.5 Comp Comp > summary(.PC) # proportions of variance Importance of components: Comp.1 Comp.2 Comp.3 Comp.4 Standard deviation Proportion of Variance Cumulative Proportion Comp.5 Comp.6 Comp.7 Standard deviation Proportion of Variance Cumulative Proportion > biplot(.PC, cex=c(.5, 1)) > scatterplotMatrix(~PC1+PC2+PC3+PC4, reg.line=FALSE, + smooth=FALSE, spread=FALSE, span=0.5, diagonal = 'density', + data=HandAxes, pch=20)

13

14

15 Factor Analysis We are interested in studying something that cannot be directly observed We can, however, observe variables which are affected by the unobserved factors Correlations between observed variables are assumed to reflect the unobserved factors

16 Steps Select variables as with principal components Analysis usually begins with a correlation matrix of the variables Communality estimates defined Extract one or more factors Rotate factors for interpretability

17 Terminology Eigenvalues, Eigenvectors, and Communality Communality relates to common variance in the variable as opposed to the unique variance:

18 Issues Need more cases than variables Sum of the eigenvalues = number of variables or number of cases – 1 whichever is smaller Factors are often standardized to a variance of 1. Each factor is independent if no rotation or orthogonal rotation is used

19 Results Eigenvalues for extracted components and proportion of variance “explained” Loadings (correlations) between variables and factors Factor rotation results Factor scores for each case

20 Number of Factors Default choice is usually to select the factors with eigenvalues > 1 – these factors explain the equivalent variance of at least one original variable Scree plots can be used to select more or fewer factors

21 Rotation Rotation is used to make the factors more interpretable Rotation tries to create variables with very high or very low loadings Orthogonal rotation preserves the independence of the factors Oblique rotation produces correlated factors

22 Interpretation Interpretability is not a test that the factors are “real” Factors are interpreted using information about the variables that load highly on them Interpretations should be evaluated against other information

23 Example In Rcmdr use Statistics | Dimensional analysis | Factor Analysis factanal() or fa() in psych

24 >.FA <- factanal(~L+L1+T+T1+W+W1+W2, factors=2, + rotation="varimax", scores="regression", data=HandAxes) >.FA Call: factanal(x = ~L + L1 + T + T1 + W + W1 + W2, factors = 2, data = HandAxes, scores = "regression", rotation = "varimax") Uniquenesses: L L1 T T1 W W1 W Loadings: Factor1 Factor2 L L T T W W W

25 Factor1 Factor2 SS loadings Proportion Var Cumulative Var Test of the hypothesis that 2 factors are sufficient. The chi square statistic is on 8 degrees of freedom. The p-value is 2.5e-75 > scatterplot(F2~F1, reg.line=FALSE, smooth=FALSE, spread=FALSE, boxplots=FALSE, span=0.5, data=HandAxes)

26


Download ppt "Factor Analysis and Principal Components Removing Redundancies and Finding Hidden Variables."

Similar presentations


Ads by Google