1. 9. SIMPLE LINEAR REGESSION AND CORRELATION
9.1 Regression and Correlation
9.2 Regression Model
9.3 Probabilistic Models
9.4 Fitting The Model: The Least Square Approach
9.5 The Least-Square Lines
9.6 The Least-Squares Assumption
9.7 Model Assumptions of Simple Regression
9.8 Assessing the utility of the model-making inference about the slope
9.9 The Coefficient of Correlation
9.10 Calculating r2
9.11 Correlation Model
9.12 Correlation Coefficient
9.13 The Coefficient of Determination
9.14 Using The Model for Estimation and Prediction

2. 9.1 Regression and Correlation
Regression: Helpful in ascertaining the probable form of the relationship between variables, and predict or estimate the value corresponding to a given value of another variable.
Correlation: Measuring the strength of the relationship between variables.

3. 9.2 Regression Model Two variables, X and Y, are interest.
Where, X = independent variable
Y = dependent variable
 = Random error component
(beta zero) = y intercept of the line
(beta one) = Slope of the line , the amount of
increase or decrease in the
deterministic of y for every 1 unit of x
increase or decrease.

4. Figure 9a. Regression Model

5. 9.3 Probabilistic Models 9.3.1 General Form of Probabilistic Models
y = Deterministic component + Random error
Where y is the variable of interest. We always assume that the mean value of the random error equals 0. This is equivalent to assuming that the mean value of y, E(y), equals the deterministic component of the model; that is,
E(y) = Deterministic component

6. 9.3.2 A First-Order (Straight-line) Probabilistic Model
Where
y = Dependent or response variable (variable to be modeled)
x = Independent or predictor variable (variable used as a predictor of y)
E(y) = = Deterministic component
(epsilon) = Random error component
(beta zero) = y-intercept of the line, that is, the point at which the line intercepts or cuts through the y-axis (see figure 9b below)
(beta one) = Slope of the line, that is, the amount of increase (or decrease) in the deterministic component of y for every one-unit increase in x.

7. Figure 9b. The straight-line model

8. 9.4 Fitting The Model: The Least Square Approach
Table 9a. Reaction Time Versus Drug Percentage
Subject
Amount of Drug x (%)
Reaction Time y (seconds) 1 2 3 4 5

9. Figure 9c. 1) Scattergram 2) Visual straight line
(for data in table above)
(fitted for the data above)

10. 9.5 The Least-Squares Lines
The Least-Squares Line is result in obtaining the desired line which called method of lease-squares Where, y = value on the vertical axis
x = value on the horizontal axis = point where the line crosses the vertical axis = shows the amount by which y changes for each unit change in x.

11. 9.5.1 Definition of Least Square Line
The least square line is one that has the following two properties:
The sum of the errors (SE) equals 0
The sum of squared errors (SSE) is smaller than that for any other straight-line model

12. 9.5.2 Formulas for the Least Squares Estimation
Where;

13. Figure 9d. Scatter Diagram

14. The total deviation:- measuring the vertical distance from line.
The explained deviation:- shows how much the total deviation is reduced when the regression line is fitted to the points.
Unexplained deviation:- shows the proportion of the total deviation accounted for by the introduction of the regression line.
total deviation
Explained
deviation
Unexplained
deviation

15. Total sum of squares (SST):- to measure of the total variation in observed values of Y.
Explained sum of squares (SSR) :- measures the amount of the total variability in the observes values of Y that is accounted for by the linear relationship between the observed values of X and Y.
Unexplained sum of squares (SST):-measure the dispersion of the observed Y values about the regression line.

16. 9.6 The Least-Squares Assumption
Consider now a reasonable criterion for estimating  and  from data. The method of ordinary least squares (OLS) determines values of  and  (since these will be estimated from data, we will replace  and  with Latin letters a and b).

17. so that the sum of the squared vertical deviations residuals) between the data and the fitted line,
Residuals = Data -Fit,
is less than the sum of the squared vertical deviations from any other straight line that could be fitted through the data:
Minimum of (Data - Fit)²

18. A "vertical deviation" is the vertical distance from an observed point to the line. Each deviation in the sample is squared and the least-squares line is defined to be the straight line that makes the sum of these squared deviations a minimum:
Data = a + bX + Residuals.

19. Figure 1 (a) illustrates the regression relationship between two variables, Y and X. The arithmetic mean of the observed values of Y is denoted by . The vertical dashed lines represent the total deviations of each value y from the mean value .
Part (b) in Figure 1 shows a linear least-squares regression line fitted to the observed points

20. Figure 9e: The total variation of Y and the least-squares regression between Y and X.
(a) Total variation (b) Least-squares regression

21. The total variation can be expressed in terms of (1) the variation explained by the regression and (2) a residual portion called the unexplained variation.

22. Figure 2 (a) shows the explained variation, which is expressed by the vertical distance between any fitted (predicted) value and the mean or The circumflex (^) over the y is used to represent fitted values determined by a model. Thus, it is also customary to write a = and b = Figure 2 (b) shows the unexplained or residual variation-the vertical distance between the observed values and the pre-dicted values (y - )

23. Figure 9f. The explained and unexplained variation in least-squares regression.
(a)  Explained variation (b) Unexplained variation

24. 9.7 Model Assumptions of Simple Regression
The mean of the probability distribution of is 0. That is, the average of the values of over an infinitely long series of experiments is 0 for each setting of the independents variable x. This assumption implies that the mean value of y, E(y), for given value of x is

25. Assumption 2:
The variance of the probability distribution of is constant for all settings of the independent variable x. For our straight-line model, this assumption means that the variance of is equal to a constant, say , for all values of x.
Assumption 3:
The probability distribution of is normal.
Assumption 4:
The values of associated with any two observed values of y are independent. That is, the value of associated with one value of y has no effect on the values of associated with other y values.

26. Figure 9g. The probability distribution of

27. 9. 8. Assessing the Utility of the Model:
9.8 Assessing the Utility of the Model: Making Inference About the Slope
A Test Of Model Utility: Simple Linear Regression
One-Tailed Test Two-Tailed Test

28. Where are based on degrees of freedom
Assumption: Refer the four assumption about

29. Figure 9h. Rejection region and calculated t value for testing versus

30. 9.8.2 A Confidence Interval for the Simple Linear Regression Slope
Where the estimated standard error of is calculated by
And is based on (n-2) degrees of freedom.
Assumption: Refer the four assumption about

31. 9.9 The Coefficient of Correlation
Definition:
The Pearson product moment coefficient of correlation, r, is a measure of a strength of the linear relationship between two variables x and y. It is computed (for a sample of n measurements on x and y) as follows:

32. Figure 9i. Value of r and their implication
1) Positive r : y increases as x increases

33. 2) r near zero: little or no relationship between y and x

34. 3) Negative r : y decreases as x increases

35. 4) r = 1: a perfect positive relationship between y and x

36. 5) r = -1: a perfect negative relationship between y and x

37. 6) r near 0: little or no relationship between y and x

38. 9.10 Calculating r2
Where:
r = The sample correlation

39. Fig 9j. r2 as a measure of closeness of fit of the sample regression line to the sample observation

40. Correlation Model
We have what is called Correlation Model, when Y and X are random variable.
Involving two variables implies a co-relationship between them.
One variable as dependent and another one as independent.

41. 9.12 The Correlation Coefficient ( )
Measures the strength of the linear relationship between X and Y.
May assumed any value between –1 and +1.
If = 1, there is perfect direct linear.
If = -1, indicates perfect inverse linear correlation.

42. 9.13 The Coefficient of Determination
Figure 9k. A comparison of the sum of squares of deviations for two models

43. b

44. c

45. 9.13.1 Coefficient of Determination Definition
It represents the proportion of the total sample variability around that is explained by the linear relationship between y and x. (In simple linear regression, it may also be computed as the square of the coefficient of correlation r.

46. 9.14 Using The Model for Estimation and Prediction

47. 9. 14. 1. Sampling Errors for the Estimator of the Mean
Sampling Errors for the Estimator of the Mean of y and the Predictor of an Individual New Value of y
The standard deviation of the sampling distribution of the estimator of the mean of y at a specific value of x, say xp is
Where is the standard deviation of the random error . We refer to as the standard error of .

48. The standard deviation of the prediction error for the predictor of an individual new y value at a specific value of x is
Where is the standard deviation of the random error . We refer to as the standard error of prediction.

49. 9.14.2 A Confidence Interval for the Mean Value of y at x = xp
Where is based on (n-2) degrees of freedom.

50. 9.14.3 A Prediction Interval for an Individual New Value of y at x = xp
Where is based on (n-2) degrees of freedom.

51. Figure 9l. A 95% confidence interval for mean sales
Figure 9l. A 95% confidence interval for mean sales and a prediction interval for drug concentration when x = 4

52. Figure 9m. Error of estimating the mean value of y for a given value of x

53. Figure 9n. Error of predicting a future value of y for a given value of x

54. Figure 9o. Confidence intervals for mean value
Figure 9o. Confidence intervals for mean value and prediction intervals for new values