1. The Multiple Correlation Coefficient

2. has (p +1)-variate Normal distribution with mean vector and Covariance matrix We are interested if the variable y is independent of the vector Definition The multiple correlation coefficient is the maximum correlation between y and a linear combination of the components of

3. This vector has a bivariate Normal distribution with mean vector and Covariance matrix We are interested if the variable y is independent of the vector Derivation The multiple correlation coefficient is the maximum correlation between y and a linear combination of the components of

4. Thus we want to choose to maximize The multiple correlation coefficient is the maximum correlation between y and The correlation between y and Equivalently

5. Note:

6. The multiple correlation coefficient is independent of the value of k.

7. We are interested if the variable y is independent of the vector The sample Multiple correlation coefficient Then the sample Multiple correlation coefficient is

8. Testing for independence between y and The test statistic If independence is true then the test statistic F will have an F- distributions with 1 = p degrees of freedom in the numerator and 1 = n – p + 1 degrees of freedom in the denominator The test is to reject independence if:

9. Other tests for Independence The test statistic If independence is true then the test statistic t will have a t - distributions with = n –2 degrees of freedom. The test is to reject independence if: Test for zero correlation (Independence between a two variables)

10. The test statistic If H 0 is true the test statistic z will have approximately a Standard Normal distribution Test for non-zero correlation (H 0 :    We then reject H 0 if:

11. The test statistic If independence is true then the test statistic t will have a t - distributions with = n – p - 2 degrees of freedom. The test is to reject independence if: Test for zero partial correlation correlation (Conditional independence between a two variables given a set of p Independent variables) = the partial correlation between y i and y j given x 1, …, x p.

12. The test statistic If H 0 is true the test statistic z will have approximately a Standard Normal distribution Test for non-zero partial correlation We then reject H 0 if:

13. Canonical Correlation Analysis

14. The problem Quite often when one has collected data on several variables. The variables are grouped into two (or more) sets of variables and the researcher is interested in whether one set of variables is independent of the other set. In addition if it is found that the two sets of variates are dependent, it is then important to describe and understand the nature of this dependence. The appropriate statistical procedure in this case is called Canonical Correlation Analysis.

15. Canonical Correlation: An Example In the following study the researcher was interested in whether specific instructions on how to relax when taking tests and how to increase Motivation, would affect performance on standardized achievement tests Reading, Language and Mathematics

16. A group of 65 third- and fourth-grade students were rated after the instruction and immediately prior taking the Scholastic Achievement tests on: In addition data was collected on the three achievement tests how relaxed they were (X 1 ) and how motivated they were (X 2 ). Reading (Y 1 ), Language (Y 2 ) and Mathematics (Y 3 ). The data were tabulated on the next page

18. Definition: (Canonical variates and Canonical correlations) have p-variate Normal distribution with and Let be such that U 1 and V 1 have achieved the maximum correlation  1. and Then U 1 and V 1 are called the first pair of canonical variates and  1 is called the first canonical correlation coefficient.

19. derivation: ( 1 st pair of Canonical variates and Canonical correlation) has covariance matrixThus Now

20. derivation: ( 1 st pair of Canonical variates and Canonical correlation) has covariance matrixThus Now hence

21. Thus we want to choose is at a maximum so that is at a maximum or Let

22. Computing derivatives and

23. Thus This shows thatis an eigenvector of k is the largest eigenvalue of andis the eigenvector associated with the largest eigenvalue.

24. Also and

25. Summary: are found by finding, eigenvectors of the matrices associated with the largest eigenvalue (same for both matrices) The first pair of canonical variates The largest eigenvalue of the two matrices is the square of the first canonical correlation coefficient  1

26. The remaining canonical variates and canonical correlation coefficients are found by finding, so that 1. (U 2,V 2 ) are independent of (U 1,V 1 ). The second pair of canonical variates 2. The correlation between U 2 and V 2 is maximized The correlation,  2, between U 2 and V 2 is called the second canonical correlation coefficient.

27. are found by finding, so that 1. (U i,V i ) are independent of (U 1,V 1 ), …, (U i-1,V i-1 ). The i th pair of canonical variates 2. The correlation between U i and V i is maximized The correlation,  2, between U 2 and V 2 is called the second canonical correlation coefficient.

28. derivation: ( 2 nd pair of Canonical variates and Canonical correlation) has covariance matrix Now

29. and maximizing Is equivalent to maximizing subject to Using the Lagrange multiplier technique

30. Now and alsogives the restrictions

31. These equations can used to show that are eigenvectors of the matrices associated with the 2 nd largest eigenvalue (same for both matrices) The 2 nd largest eigenvalue of the two matrices is the square of the 2 nd canonical correlation coefficient  2

32. Coefficients for the i th pair of canonical variates, are eigenvectors of the matrices associated with the i th largest eigenvalue (same for both matrices) The i th largest eigenvalue of the two matrices is the square of the i th canonical correlation coefficient  i continuing

33. Example Variables relaxation Score (X 1 ) motivation score (X 2 ). Reading (Y 1 ), Language (Y 2 ) and Mathematics (Y 3 ).

34. Summary Statistics

35. Canonical Correlation statistics Statistics

36. continued

37. Summary U 1 = 0.197 Relax + 0.979 Mot V 1 = 0.504 Read + 0.900 Lang + 0.565 Math  1 =.592 U 2 = 0.980 Relax + 0.203 Mot V 2 = 0.391 Math - 0.361 Read - 0.354 Lang  2 =.159