Regression Analysis

1. To comprehend the nature of correlation analysis. 2. To understand bivariate regression analysis. 3. To become aware of the coefficient of determination

Bivariate Analysis Defined The degree of association between two variables Bivariate techniques Statistical methods of analyzing the relationship between two variables. Multivariate Techniques When more than two variables are involved Independent variable Affects the value of the dependent variable Dependent variable explained or caused by the independent variable Bivariate Analysis of Association

Types of Bivariate Procedures Two group t-tests chi-square analysis of cross-tabulation or contingency tables ANOVA (analysis of variance) for two groups Bivariate regression Pearson product moment correlation Bivariate Analysis of Association

Bivariate Regression Defined Analyzing the strength of the linear relationship between the dependent variable and the independent variable. Nature of the Relationship Plot in a scatter diagram Dependent variable Y is plotted on the vertical axis Independent variable X is plotted on the horizontal axis Nonlinear relationship Bivariate Regression

Y X A - Strong Positive Linear Relationship Types of Relationships Found in Scatter Diagrams Bivariate Regression Example Bivariate Regression

Y X B - Positive Linear Relationship Types of Relationships Found in Scatter Diagrams Bivariate Regression

Y X C - Perfect Negative Linear Relationship Types of Relationships Found in Scatter Diagrams Bivariate Regression

X D - Perfect Parabolic Relationship Types of Relationships Found in Scatter Diagrams Bivariate Regression

Y X E - Negative Curvilinear Relationship Types of Relationships Found in Scatter Diagrams Bivariate Regression

Y X F - No Relationship between X and Y Types of Relationships Found in Scatter Diagrams Bivariate Regression

Least Squares Estimation Procedure Results in a straight line that fits the actual observations better than any other line that could be fitted to the observations. where Y = dependent variable X = independent variable e = error b = estimated slope of the regression line a = estimated Y intercept Bivariate Regression Y = a + bX + e

Values for a and b can be calculated as follows:  X i Y i - nXY b =  X 2 i - n(X) 2 n = sample size a = Y - bX X = mean of value X Y = mean of value y Bivariate Regression

y= β 0 + β 1 + Є β 1 = S xy /S xx β 0 = y - β 1 x Bivariate Regression

Strength of Association: R 2 Coefficient of Determination, R 2 : The measure of the strength of the linear relationship between X and Y. The Regression Line Predicted values for Y, based on calculated values. Bivariate Regression

R 2 = explained variance total variance explained variance = total variance - unexplained variance R 2 = total variance - unexplained variance total variance = 1 - unexplained variance total variance Bivariate Regression

R 2 = 1 - unexplained variance total variance =1 -  (Y i - Y i ) 2 n I = 1  (Y i - Y) 2 n I = 1 Bivariate Regression Predicted response

Statistical Significance of Regression Results Total variation = Explained variation + Unexplained variation To become aware of the coefficient of determination, R 2. The total variation is a measure of variation of the observed Y values around their mean. It measures the variation of the Y values without any consideration of the X values. Bivariate Regression

Total variation: Sum of squares (SST) SST =  (Y i - Y) 2 n i = 1  Y i 2 n i = 1 =  Y i 2 n i = 1 n Bivariate Regression

Sum of squares due to regression (SSR) SSR =  (Y i - Y) 2 n i = 1  Y i n i = 1 = a  Y i n i = 1 n b  X i Y i n i = Bivariate Regression

Error sums of squares (SSE) SSE =  (Y i - Y) 2 n i = 1  Y 2 i n i = 1 = a  Y i n i = 1 b  X i Y i n i = 1 Bivariate Regression

Hypotheses Concerning the Overall Regression Null Hypothesis H o : There is no linear relationship between X and Y. Alternative Hypothesis H a : There is a linear relationship between X and Y. Bivariate Regression

Hypotheses about the Regression Coefficient Null Hypothesis H o : b = 0 Alternative Hypothesis H a : b  0 The appropriate test is the t-test. Bivariate Regression

0 X XiXi X (X, Y) a Y Total Variation Explained variation Y Unexplained variation Measures of Variation in a Regression Y i =a + bX i

Correlation for Metric Data - Pearson’s Product Moment Correlation Correlation analysis Analysis of the degree to which changes in one variable are associated with changes in another variable. Pearson’s product moment correlation Correlation analysis technique for use with metric data Correlation Analysis To become aware of the coefficient of determination, R 2.

R = + - R2R2 √ R can be computed directly from the data: R = n  XY - (  X) - (  Y) [n  X 2 - (  X) 2 ] [n  Y 2 -  Y) 2 ] √ To become aware of the coefficient of determination, R 2. Correlation Analysis

SUMMARY Bivariate Analysis of Association Bivariate Regression Correlation Analysis