Presentation on theme: "9. SIMPLE LINEAR REGESSION AND CORRELATION"— Presentation transcript:
1 9. SIMPLE LINEAR REGESSION AND CORRELATION 9.1 Regression and Correlation9.2 Regression Model9.3 Probabilistic Models9.4 Fitting The Model: The Least Square Approach9.5 The Least-Square Lines9.6 The Least-Squares Assumption9.7 Model Assumptions of Simple Regression9.8 Assessing the utility of the model-making inference about the slope9.9 The Coefficient of Correlation9.10 Calculating r29.11 Correlation Model9.12 Correlation Coefficient9.13 The Coefficient of Determination9.14 Using The Model for Estimation and Prediction
2 9.1 Regression and Correlation Regression: Helpful in ascertaining the probable form of the relationship between variables, and predict or estimate the value corresponding to a given value of another variable.Correlation: Measuring the strength of the relationship between variables.
3 9.2 Regression Model Two variables, X and Y, are interest. Where, X = independent variableY = dependent variable = Random error component(beta zero) = y intercept of the line(beta one) = Slope of the line , the amount ofincrease or decrease in thedeterministic of y for every 1 unit of xincrease or decrease.
5 9.3 Probabilistic Models 9.3.1 General Form of Probabilistic Models y = Deterministic component + Random errorWhere y is the variable of interest. We always assume that the mean value of the random error equals 0. This is equivalent to assuming that the mean value of y, E(y), equals the deterministic component of the model; that is,E(y) = Deterministic component
6 9.3.2 A First-Order (Straight-line) Probabilistic Model Wherey = Dependent or response variable (variable to be modeled)x = Independent or predictor variable (variable used as a predictor of y)E(y) = = Deterministic component(epsilon) = Random error component(beta zero) = y-intercept of the line, that is, the point at which the line intercepts or cuts through the y-axis (see figure 9b below)(beta one) = Slope of the line, that is, the amount of increase (or decrease) in the deterministic component of y for every one-unit increase in x.
8 9.4 Fitting The Model: The Least Square Approach Table 9a. Reaction Time Versus Drug PercentageSubjectAmount of Drug x (%)Reaction Time y (seconds)12345
9 Figure 9c. 1) Scattergram 2) Visual straight line (for data in table above)(fitted for the data above)
10 9.5 The Least-Squares Lines The Least-Squares Line is result in obtaining the desired line which called method of lease-squares Where, y = value on the vertical axisx = value on the horizontal axis = point where the line crosses the vertical axis = shows the amount by which y changes for each unit change in x.
11 9.5.1 Definition of Least Square Line The least square line is one that has the following two properties:The sum of the errors (SE) equals 0The sum of squared errors (SSE) is smaller than that for any other straight-line model
12 9.5.2 Formulas for the Least Squares Estimation Where;
14 The total deviation:- measuring the vertical distance from line. The explained deviation:- shows how much the total deviation is reduced when the regression line is fitted to the points.Unexplained deviation:- shows the proportion of the total deviation accounted for by the introduction of the regression line.total deviationExplaineddeviationUnexplaineddeviation
15 Total sum of squares (SST):- to measure of the total variation in observed values of Y. Explained sum of squares (SSR) :- measures the amount of the total variability in the observes values of Y that is accounted for by the linear relationship between the observed values of X and Y.Unexplained sum of squares (SST):-measure the dispersion of the observed Y values about the regression line.
16 9.6 The Least-Squares Assumption Consider now a reasonable criterion for estimating and from data. The method of ordinary least squares (OLS) determines values of and (since these will be estimated from data, we will replace and with Latin letters a and b).
17 so that the sum of the squared vertical deviations residuals) between the data and the fitted line, Residuals = Data -Fit,is less than the sum of the squared vertical deviations from any other straight line that could be fitted through the data:Minimum of (Data - Fit)²
18 A "vertical deviation" is the vertical distance from an observed point to the line. Each deviation in the sample is squared and the least-squares line is defined to be the straight line that makes the sum of these squared deviations a minimum:Data = a + bX + Residuals.
19 Figure 1 (a) illustrates the regression relationship between two variables, Y and X. The arithmetic mean of the observed values of Y is denoted by . The vertical dashed lines represent the total deviations of each value y from the mean value .Part (b) in Figure 1 shows a linear least-squares regression line fitted to the observed points
20 Figure 9e: The total variation of Y and the least-squares regression between Y and X. (a) Total variation (b) Least-squares regression
21 The total variation can be expressed in terms of (1) the variation explained by the regression and (2) a residual portion called the unexplained variation.
22 Figure 2 (a) shows the explained variation, which is expressed by the vertical distance between any fitted (predicted) value and the mean or The circumflex (^) over the y is used to represent fitted values determined by a model. Thus, it is also customary to write a = and b = Figure 2 (b) shows the unexplained or residual variation-the vertical distance between the observed values and the pre-dicted values (y - )
23 Figure 9f. The explained and unexplained variation in least-squares regression. (a) Explained variation (b) Unexplained variation
24 9.7 Model Assumptions of Simple Regression The mean of the probability distribution of is 0. That is, the average of the values of over an infinitely long series of experiments is 0 for each setting of the independents variable x. This assumption implies that the mean value of y, E(y), for given value of x is
25 Assumption 2:The variance of the probability distribution of is constant for all settings of the independent variable x. For our straight-line model, this assumption means that the variance of is equal to a constant, say , for all values of x.Assumption 3:The probability distribution of is normal.Assumption 4:The values of associated with any two observed values of y are independent. That is, the value of associated with one value of y has no effect on the values of associated with other y values.
27 9. 8. Assessing the Utility of the Model: 9.8 Assessing the Utility of the Model: Making Inference About the SlopeA Test Of Model Utility: Simple Linear RegressionOne-Tailed Test Two-Tailed Test
28 Where are based on degrees of freedom Assumption: Refer the four assumption about
29 Figure 9h. Rejection region and calculated t value for testing versus
30 9.8.2 A Confidence Interval for the Simple Linear Regression Slope Where the estimated standard error of is calculated byAnd is based on (n-2) degrees of freedom.Assumption: Refer the four assumption about
31 9.9 The Coefficient of Correlation Definition:The Pearson product moment coefficient of correlation, r, is a measure of a strength of the linear relationship between two variables x and y. It is computed (for a sample of n measurements on x and y) as follows:
32 Figure 9i. Value of r and their implication 1) Positive r : y increases as x increases
33 2) r near zero: little or no relationship between y and x
35 4) r = 1: a perfect positive relationship between y and x
36 5) r = -1: a perfect negative relationship between y and x
37 6) r near 0: little or no relationship between y and x
38 9.10 Calculating r2Where:r = The sample correlation
39 Fig 9j. r2 as a measure of closeness of fit of the sample regression line to the sample observation
40 Correlation ModelWe have what is called Correlation Model, when Y and X are random variable.Involving two variables implies a co-relationship between them.One variable as dependent and another one as independent.
41 9.12 The Correlation Coefficient ( ) Measures the strength of the linear relationship between X and Y.May assumed any value between –1 and +1.If = 1, there is perfect direct linear.If = -1, indicates perfect inverse linear correlation.
42 9.13 The Coefficient of Determination Figure 9k. A comparison of the sum of squares of deviations for two models
45 9.13.1 Coefficient of Determination Definition It represents the proportion of the total sample variability around that is explained by the linear relationship between y and x. (In simple linear regression, it may also be computed as the square of the coefficient of correlation r.
46 9.14 Using The Model for Estimation and Prediction
47 9. 14. 1. Sampling Errors for the Estimator of the Mean Sampling Errors for the Estimator of the Mean of y and the Predictor of an Individual New Value of yThe standard deviation of the sampling distribution of the estimator of the mean of y at a specific value of x, say xp isWhere is the standard deviation of the random error . We refer to as the standard error of .
48 The standard deviation of the prediction error for the predictor of an individual new y value at a specific value of x isWhere is the standard deviation of the random error . We refer to as the standard error of prediction.
49 9.14.2 A Confidence Interval for the Mean Value of y at x = xp Where is based on (n-2) degrees of freedom.
50 9.14.3 A Prediction Interval for an Individual New Value of y at x = xp Where is based on (n-2) degrees of freedom.
51 Figure 9l. A 95% confidence interval for mean sales Figure 9l. A 95% confidence interval for mean sales and a prediction interval for drug concentration when x = 4
52 Figure 9m. Error of estimating the mean value of y for a given value of x
53 Figure 9n. Error of predicting a future value of y for a given value of x
54 Figure 9o. Confidence intervals for mean value Figure 9o. Confidence intervals for mean value and prediction intervals for new values