Multivariate Analysis and Data Reduction

Multivariate Analysis Multivariate analysis tries to find patterns and relationships among multiple dependent variables. Situations where multiple dependent variables are common:  Test validation and development of scales.  Multi-measure paradigms (e.g., physiological psychology).  Complex sociological studies, including surveys and archival research.

Extending Correlation Correlation examines the type and extent of relationship between two variables:  Positive (direct), negative (inverse), no relation.  A correlation matrix shows relationships among multiple pairs of variables (average interrcorrelation can be calculated for the matrix). What if latent causes exist that affect many of the variables in a study?  What if several different causes have different impacts on different of the variables?

Factor Analysis Factors are underlying constructs (causes) for the variability in a set of multiple variables. Factor analysis applies matrix algebra to a correlation matrix in order to extract a set of common factors.  The relationship between each variable and the extracted factors is quantified by “factor loadings”.  Eigenvalues show how much of the variance is explained by each factor.

Goals of Factor Analysis The goal of factor analysis is to make sense of the overlap and relationships among multiple dependent variables (measures). The procedure seeks a more economical explanation of the observed behavior.  The number of important factors should be less than the number of variables input. It is up to the investigator to make sense out of the factors identified by the analysis.

Rotation Factor analysis is an iterative process. It begins by trying to find the factor that produces the highest correlations among a set of variables.  It then locates the factor that produces the second highest correlations, and so forth.  You decide how many factors are relevant. This original factor structure may be difficult to interpret, so factors are rotated to find a meaningful interpretation.

Interpretation of Results Orthogonal factors are at right angles to each other.  Orthogonal factors are assumed to be independent of each other.  Varimax rotation produces orthogonal rotations. Ideally, a variable should load high on one factor and low or not at all on other factors.  In reality, factors may be complex to interpret. The emergent structure depends on the input.

Two Different Rotations of the Same Intelligence Data Spearman’s G  The main factor that emerges when minimum residual factor analysis is applied to multiple measures of intelligence (e.g., on an IQ test). Gardner’s Multiple Intelligences  A factor structure consisting of the maximum orthogonal factors emerging from the same analysis of multiple measures of intelligence with Varimax rotation. Both are valid solutions for the same data.

Cluster Analysis A method of reducing multiple input variables (or cases/subjects) into larger groups based on similarity of scores. Cluster analysis is used when you want to group cases or variables but don’t already know which belong together. Similarity of features, not shared variance is the basis for clustering.

Interpretation of Cluster Analysis Results Two approaches – putting items together (agglomorative) or dividing large groups into smaller ones (divisive). The technique provides the solution but it must be interpreted by the investigator. The structure of the solution is determined by the number and type of input variables or cases (subjects).  Systematic or valid sampling is essential.

Discriminant Analysis Discriminant analysis uses the characteristics of variables to predict membership in defined groups.  Used when the group membership is already known.  The relationship between the groups and the variables is analyzed, resulting in factor loadings for discriminant functions. Input variables can be changed to test different models for predicting groups.

Multidimensional Scaling Similarities and differences among items are used to create a plot showing relationships among them.  Input is similarity judgments, distances, or correlation matrix for multiple variables.  Similarities are converted to distances for plotting. An iterative calculation figures out the plot that best retains the relative distances among all items. Dimensions can be inferred from the plot.

Important Measures of Model Fit Frequently, more than two dimensions are needed to accurately display relationships among cases.  Stress shows how well the locations of the items fit within the number of dimensions requested.  The more items, the greater the stress.  The more dimensions, the lower the stress. Higher dimensionality is difficult to interpret. The highest dimensions are noise (error).

Individual Differences Scaling Some methods take into account that different individuals emphasize different dimensions when making judgments about items. The relative weights for each dimension, for each subject, can be analyzed.