Presentation is loading. Please wait.

Presentation is loading. Please wait.

Correlation and Regression Analysis

Similar presentations

Presentation on theme: "Correlation and Regression Analysis"— Presentation transcript:

1 Correlation and Regression Analysis
Dr. Mohammed Alahmed

2 GOALS Understand and interpret the terms dependent and independent variable. Calculate and interpret the coefficient of correlation, the coefficient of determination, and the standard error of estimate. Conduct a test of hypothesis to determine whether the coefficient of correlation in the population is zero. Calculate the least squares regression line. Predict the value of a dependent variable based on the value of at least one independent variable. Explain the impact of changes in an independent variable on the dependent variable. Dr. Mohammed Alahmed

3 Introduction Correlation and regression analysis are related in the sense that both deal with relationships among variables. For example, we may be interested in studying the relationship between blood pressure and age, height and weight…. The nature and strength of the relationship between variables may be examined by Correlation and Regression analysis. Dr. Mohammed Alahmed

4 Correlation Analysis The term “correlation” refers to a measure of the strength of association between two variables. Finding the relationship between two quantitative variables without being able to infer causal relationships Correlation is a statistical technique used to determine the degree to which two variables are related. Dr. Mohammed Alahmed

5 If the two variables increase or decrease together, they have a positive correlation.
If, increases in one variable are associated with decreases in the other, they have a negative correlation Dr. Mohammed Alahmed

6 Visualizing Correlation
A scatter plot (or scatter diagram) is used to show the relationship between two variables. Linear relationships implying straight line association are visualized with scatter plots Dr. Mohammed Alahmed

7 Linear Correlation Only!
Linear relationships Curvilinear relationships Y X Y X Dr. Mohammed Alahmed

8 Correlation Coefficient
The population correlation coefficient ρ (rho) measures the strength of the association between the variables. The sample (Pearson) correlation coefficient r is an estimate of ρ and is used to measure the strength of the linear relationship in the sample observations. Dr. Mohammed Alahmed

9 r is a statistic that quantifies a relation between two variables.
Can be either positive or negative Falls between and 1.00 Dr. Mohammed Alahmed

10 The value of the number (not the sign) indicates the strength of the relation.
The purpose is to measure the strength of a linear relationship between 2 variables. A correlation coefficient does not ensure “causation” (i.e. a change in X causes a change in Y) Dr. Mohammed Alahmed

11 Calculating the Correlation Coefficient
The sample (Pearson) correlation coefficient (r) is defined by where: r = Sample correlation coefficient n = Sample size x = Value of the independent variable y = Value of the dependent variable Dr. Mohammed Alahmed

12 Statistical Inference for Correlation Coefficients
Significance Test for Correlation Hypotheses H0: ρ = 0 (no correlation) H1: ρ ≠ 0 (correlation exists) Test statistic Dr. Mohammed Alahmed

13 Example A small study is conducted involving 17 infants to investigate the association between gestational age at birth, measured in weeks, and birth weight, measured in grams. Dr. Mohammed Alahmed

14 Dr. Mohammed Alahmed

15 Dr. Mohammed Alahmed

16 r H0: ρ = 0 H1: ρ ≠ 0 Dr. Mohammed Alahmed

17 Cautions about Correlation
Correlation is only a good statistic to use if the relationship is roughly linear. Correlation can not be used to measure non-linear relationships Always plot your data to make sure that the relationship is roughly linear! Dr. Mohammed Alahmed

18 Regression Analysis Regression analysis is used to:
Predict the value of a dependent variable based on the value of at least one independent variable. Explain the impact of changes in an independent variable on the dependent variable. Dependent variable: the variable we wish to explain. Independent variable: the variable used to explain the dependent variable. Dr. Mohammed Alahmed

19 Simple Linear Regression Model
Only one independent variable, X Relationship between X and Y is described by a linear function. Changes in Y are assumed to be caused by changes in X. Dr. Mohammed Alahmed

20 The formula for a simple linear regression
Linear component Population y intercept Population Slope Coefficient Random Error term, or residual Dependent Variable Independent Variable Random Error component The regression coefficients β0 and β1 are unknown and have to be estimated from the observed data (sample). Dr. Mohammed Alahmed

21 y x εi Slope = β1 Random Error for this x value β0 xi
Observed Value of y for xi Predicted Value of y for xi xi Slope = β1 εi β0 Dr. Mohammed Alahmed

22 Linear Regression Assumptions
The assumption of linearity The relationship between the dependent and independent variables is linear. The assumption of homoscedasticity The errors have the same variance The assumption of independence The errors are independent of each other The assumption of normality The errors are normally distributed Dr. Mohammed Alahmed

23 Estimated Regression Model
The sample regression line provides an estimate of the population regression line Estimate of the regression intercept Estimate of the regression slope Estimated (or predicted) y value Independent variable The individual random error terms ei have a mean of zero Dr. Mohammed Alahmed

24 Least Squares Method b0 and b1 are called the regression coefficients and obtained by finding the values of b0 and b1 that minimize the sum of the squared residuals Define a residual e as the difference between the observed y and fitted 𝑦 , that is, Residuals are interpreted as estimates of random errors e‘s Dr. Mohammed Alahmed

25 The Least Squares Equation
The formulas for b1 and b0 are: b0 is the estimated average value of y when the value of x is zero b1 is the estimated change in the average value of y as a result of a one-unit change in x The coefficients b0 and b1 will usually be found using computer software, such as SPSS. Dr. Mohammed Alahmed

26 Relationship between the Regression Coefficient (b1) and the Correlation Coefficient (r)
What is the relationship between the sample regression coefficient (b1) and the sample correlation coefficient (r)? Sx is the standard deviation of X and Sy the standard deviation of Y Dr. Mohammed Alahmed

27 Example Use the previous example assuming the birth weight is the dependent variable and gestational age as the independent variable. Fit a linear-regression line relating birth weight to gestational age using these data. Predict the birth weight of a baby from a women with gestational age weeks. Dr. Mohammed Alahmed

28 Dr. Mohammed Alahmed

29 b0 b1 Dr. Mohammed Alahmed

30 Coefficient of Determination, R2
The coefficient of determination is the portion of the total variation in the dependent variable that is explained by variation in the independent variable The coefficient of determination is also called R-squared and is denoted as R2 Dr. Mohammed Alahmed

31 R2 = Explained variation / Total variation
R2 is always (%) and between 0% and 100%: 0% indicates that the model explains none of the variability of the response data around its mean. 100% indicates that the model explains all the variability of the response data around its mean. In general, the higher the R-squared, the better the model fits your data. Dr. Mohammed Alahmed

32 r2 = 0.668 66.8 % of the variation in birth weight is explained by variation in gestational age in week Dr. Mohammed Alahmed

33 F- test for Simple Linear Regression
The criterion for goodness of fit is the ratio of the regression sum of squares to the residual sum of squares. A large ratio indicates a good fit, whereas a small ratio indicates a poor fit. In hypothesis-testing terms we want to test the hypothesis: H0: β = 0 vs. H1: β ≠ 0 Dr. Mohammed Alahmed

34 The P-value < 0.05. Therefore H0 is rejected, implying a significant linear relationship between birth weight and gestational age. Dr. Mohammed Alahmed

35 Checking the Regression Assumptions
There are two strategies for checking the regression assumptions: Examining the degree to which the variables satisfy the criteria, .e.g. normality and linearity, before the regression is computed by plotting relationships and computing diagnostic statistics. Studying plots of residuals and computing diagnostic statistics after the regression has been computed. Dr. Mohammed Alahmed

36 Check Linearity assumption:
A scatter plot (or scatter diagram) is used to show the relationship between two variables. Dr. Mohammed Alahmed

37 Check Independence assumption:
Error terms associated with individual observations should be independent of each other. Rule of thumb: Random samples ensure independence. scatterplot of residuals and predicted value should show no trends Dr. Mohammed Alahmed

38 Check Equal Variance Assumption (Homoscedasticity):
Variability of error terms should be the same (constant) for all values of each predictor. Check 1: Scatterplot of residuals against the predicted value shows consistent spread. Check 2: Boxplot of y against each predictor of x should show consistent spread. Dr. Mohammed Alahmed

39 Check Normality Assumption:
Check normality of residuals and individual variables and identify outliers of variables using normal probability plot Run normality tests. All or almost all of them should have P-value > 0.05 Dr. Mohammed Alahmed

40 Construct normal probability plot (qq_plot) of residuals
Plot histogram of residuals. A bell-shaped curve centered around zero should be displayed. Construct normal probability plot (qq_plot) of residuals Dr. Mohammed Alahmed

Download ppt "Correlation and Regression Analysis"

Similar presentations

Ads by Google