1. Simple Linear Regression
Chapter 12
Simple Linear Regression

2. Objectives In this chapter, you learn:
Understand the concept of linear correlation and coefficient of determination
How to use regression analysis to predict the value of a dependent variable based on a value of an independent variable
To understand the meaning of the regression coefficients b0 and b1

3. We Discuss Two Measures of The Relationship Between Two Numerical Variables
Scatter plots allow you to visually examine the relationship between two numerical variables and now we will discuss two quantitative measures of such relationships.
The Covariance
The Coefficient of Correlation

4. The Covariance
DCOVA
The covariance measures the strength of the linear relationship between two numerical variables (X & Y)
The sample covariance:
Only concerned with the strength of the relationship
No causal effect is implied

5. Interpreting Covariance
DCOVA
Covariance between two variables:
cov(X,Y) > X and Y tend to move in the same direction
cov(X,Y) < X and Y tend to move in opposite directions
cov(X,Y) = X and Y are independent
The covariance has a major flaw:
It is not possible to determine the relative strength of the relationship from the size of the covariance

6. Coefficient of Correlation
DCOVA
Measures the relative strength of the linear relationship between two numerical variables
Sample coefficient of correlation:
where

7. Features of the Coefficient of Correlation
DCOVA
The population coefficient of correlation is referred as ρ.
The sample coefficient of correlation is referred to as r.
Either ρ or r have the following features:
Unit free
Range between –1 and 1
The closer to –1, the stronger the negative linear relationship
The closer to 1, the stronger the positive linear relationship
The closer to 0, the weaker the linear relationship

8. Scatter Plots of Sample Data with Various Coefficients of CorrelationY Y
DCOVA X X
r = -1
r = -.6 Y Y Y X X X
r = +1
r = +.3
r = 0

9. The Coefficient of Correlation Using Microsoft Excel Function
DCOVA

10. The Coefficient of Correlation Using Microsoft Excel Data Analysis Tool
DCOVA
Select Data
Choose Data Analysis
Choose Correlation & Click OK

11. The Coefficient of Correlation Using Microsoft Excel
DCOVA
Input data range and select appropriate options
Click OK to get output

12. Interpreting the Coefficient of Correlation Using Microsoft Excel
DCOVA
r = .733
There is a relatively strong positive linear relationship between test score #1 and test score #2.
Students who scored high on the first test tended to score high on second test.

13. Correlation vs. Regression
DCOVA
A scatter plot can be used to show the relationship between two variables
Correlation analysis is used to measure the strength of the association (linear relationship) between two variables
Correlation is only concerned with strength of the relationship
No causal effect is implied with correlation
Scatter plots were first presented in Ch. 2
Correlation was first presented in Ch. 3

14. Types of Relationships
DCOVA
Linear relationships
Curvilinear relationships Y Y X X Y Y X X

15. Types of Relationships
DCOVA
(continued)
Strong relationships
Weak relationships Y Y X X Y Y X X

16. Types of Relationships
DCOVA
(continued)
No relationship Y X Y X

17. Introduction to Regression Analysis
DCOVA
Regression analysis is used to:
Predict the value of a dependent variable based on the value of at least one independent variable
Explain the impact of changes in an independent variable on the dependent variable
Dependent variable: the variable we wish to
predict or explain
Independent variable: the variable used to predict or explain the dependent variable

18. Simple Linear Regression Model
DCOVA
Only one independent variable, X
Relationship between X and Y is described by a linear function
Changes in Y are assumed to be related to changes in X

19. Simple Linear Regression Model
DCOVA
Random Error term
Population Slope Coefficient
Population Y intercept
Independent Variable
Dependent Variable
Linear component
Random Error
component

20. Simple Linear Regression Model
DCOVA
(continued) Y
Observed Value of Y for Xi εi
Slope = β1
Predicted Value of Y for Xi
Random Error for this Xi value
Intercept = β0 Xi X

21. Simple Linear Regression Equation (Prediction Line)
DCOVA
The simple linear regression equation provides an estimate of the population regression line
Estimate of the regression intercept
Estimate of the regression slope
Estimated (or predicted) Y value for observation i
Value of X for observation i

22. The Least Squares Method
DCOVA
b0 and b1 are obtained by finding the values that minimize the sum of the squared differences between Y and Y : 

23. Regression Analysis – Least Squares Principle
The least squares principle is used to obtain a and b.
The equations to determine a and b are:

24. Interpretation of the Slope and the Intercept
DCOVA
b0 is the estimated mean value of Y when the value of X is zero
b1 is the estimated change in the mean value of Y as a result of a one-unit increase in X

25. Simple Linear Regression Example
DCOVA
A real estate agent wishes to examine the relationship between the selling price of a home and its size (measured in square feet)
A random sample of 10 houses is selected
Dependent variable (Y) = house price in $1000s
Independent variable (X) = square feet

26. Simple Linear Regression Example: Data
DCOVA
House Price in $1000s
(Y)
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255

27. Simple Linear Regression Example: Scatter Plot
DCOVA
House price model: Scatter Plot

28. Simple Linear Regression Example: Graphical Representation
DCOVA
House price model: Scatter Plot and Prediction Line
Slope =
Intercept =

29. Simple Linear Regression Example: Interpretation of bo
DCOVA
b0 is the estimated mean value of Y when the value of X is zero (if X = 0 is in the range of observed X values)
Because a house cannot have a square footage of 0, b0 has no practical application

30. Simple Linear Regression Example: Interpreting b1
DCOVA
b1 estimates the change in the mean value of Y as a result of a one-unit increase in X
Here, b1 = tells us that the mean value of a house increases by ($1000) = $109.77, on average, for each additional one square foot of size

31. Simple Linear Regression Example: Making Predictions
DCOVA
Predict the price for a house with 2000 square feet:
The predicted price for a house with 2000 square feet is ($1,000s) = $317,850

32. Simple Linear Regression Example: Making Predictions
DCOVA
When using a regression model for prediction, only predict within the relevant range of data
Relevant range for interpolation
Do not try to extrapolate beyond the range of observed X’s

33. Coefficient of Determination, r2
DCOVA
The coefficient of determination is the portion of the total variation in the dependent variable that is explained by variation in the independent variable
The coefficient of determination is also called r-square and is denoted as r2
note:

34. Examples of Approximate r2 Values
DCOVA Y
Perfect linear relationship between X and Y:
100% of the variation in Y is explained by variation in X X
r2 = 1 Y X
r2 = 1

35. Examples of Approximate r2 Values
DCOVA Y
0 < r2 < 1
Weaker linear relationships between X and Y:
Some but not all of the variation in Y is explained by variation in X X Y X

36. Examples of Approximate r2 Values
DCOVA
r2 = 0 Y
No linear relationship between X and Y:
The value of Y does not depend on X. (None of the variation in Y is explained by variation in X) X
r2 = 0

37. Chapter Summary In this chapter we discussed:
How to use regression analysis to predict the value of a dependent variable based on a value of an independent variable
To understand the meaning of the regression coefficients b0 and b1
To understand the concept of correlation coefficient and interpretation of coefficient of determination