D/RS 1013 Discriminant Analysis

Discriminant Analysis Overview n multivariate extension of the one-way ANOVA n looks at differences between 2 or more groups n goal is to discriminate between groups n considers several predictor variables simultaneously

Discriminant - Overview n provides a way to describe differences between groups in simple terms. n removes the redundancy among large numbers of variables by combining into a smaller number of Discriminant functions n can classify cases to groups when their group membership is unknown

Overview (cont.) n tests the significance of differences between two or more groups n examines several predictor variables simultaneously n construct linear combination of these variables, forming a single composite variable called a discriminant function n basically MANOVA flipped upside down

Discriminant parallels with MANOVA and Regression n Discriminant works the other way, predicting group membership by some kind of scores n The discriminant function takes the form: n D = d 1 z 1 + d 2 z d p z p

Discriminant functions n D i = d 1 z 1 + d 2 z d p z p –where, D = scores on the discriminant function –d 1 - d p = discriminant function weighting coefficients for each of p predictor variables –z 1 - z p = standardized scores on the original p variables

Unstandardized Functions n even more like regression equation n D i = a + d 1 x 1 + d 2 x d p x p –a= the discriminant function constant –d 1 - d p = discriminant function weighting coefficients for each of p predictor variables –x 1 - x p = raw scores on the original p variables

Forming discriminant functions n discriminant function is formed to maximize the F value associated with the D n F = bg variance on D / wg variance on D n provides a function with the greatest discriminating power.

Functions beyond the first n first function is one of many combinations of the p original predictor variables. n # of useful functions is p (# of original variables) or k-1 (k=# of groups being considered), whichever is smaller. n later functions maximize the separation between groups and are orthogonal with the preceding functions.

First discriminant function (3 gps) Separates group 1 from groups 2 & 3

Second function (3 gps) Separates group 3 from groups 1 & 2

Both functions together Orthogonal = uncorrelated

Confusion Matrix n assign cases to groups based on their discriminant function scores n assignments compared with actual group memberships n confusion matrix gives both overall accuracy of classification and the relative frequencies of various types of misclassification

Confusion matrix: example  our proportion correct is ( )/100=.82  by chance alone we would end up with.50 correct  if we evenly divided our group assignments  between A & B half in each group correct by chance  can consider prior probabilities, if known

Cross Validation n hold back some of the data to test the model that emerges n gives good idea of the kind of predictive accuracy we can expect for another sample n small samples and several variables unlikely to replicate across samples

Classification Functions n weights and constants used to calculate scores for each case n as many scores as there are groups for each case n assign to group that the case has the highest classification function score for

Assumptions n assumes that all predictors follow a multivariate normal distribution n test is robust with respect to normality, in practice, lack of normality doesn't make much of a difference n especially with large n and moderate number of predictors