1. Linear Regression - Topics
Basics of Linear Regression
Variation in Linear Regression
Linear Regression Analysis
Goodness of Fit
Standard Error terms for Linear Regression
Hypothesis testing

2. Regression - Types

3. Linear Regression
A statistical technique that uses a single, independent variable (X) to estimate a single dependent variable (Y).
Based on the equation for a line:
Y = b + mX

4. Linear Regression - ModelY
? (the actual value of Yi) Y X
b b + = Yi i e X Xi

5. Linear Regression - Model
Population
Regression Coefficients for a . . . ˆ
Y = b0 + b1Xi + e
Sample ˆ
Y = b0 + b1Xi

6. ANOVA CRD - Variation SSTR SST SSE SST = SSTR + SSE
SST is a measure of the total variation of observations. A measure of the differences in observations.
SSTR
Due to treatments.
SST
SSE
SST = SSTR + SSE
Random/unexplained.

7. Linear Regression - Variation
Ice Cream Example Y Y
= 2.53

8. Linear Regression - Variation
Ice Cream Example
Sample Regression Line
Regression Model

9. Linear Regression - Variation
SSR
Due to regression.
SST
SSE
SST = SSR + SSE
Random/unexplained.

10. Linear Regression - VariationY 
SSE =(Yi - Yi )2 _
SST = (Yi - Y)2 _ 
SSR = (Yi - Y)2 _ Y X Xi

11. Determining the Regression Line/Model
Use Excel (or any other popular statistical software)
Select Tools, Data Analysis, Regression
Provide the X range
Provide the Y range
Output the analysis to a new sheet
Manual Calculations

12. Determining the Regression Line/Model using Excel
Like in ANOVA, the df for Regression is the number of columns – 1.
Total df is always n-1. That leaves Error/Residual df at n-2.

13. Determining the Regression Line/Model Manual Calculations_ _  
SSE =(Yi - Yi )2
SSR = (Yi - Y)2
SST = (Yi - Y)2 _
SSx =(Xi - X )2
b1=SSxy/SSx
SSy =(Yi - Y)2 _ _ _
SSxy =(Xi - X )(Xi - Y ) _
b0 = Y – b1X
MSE = SSE / df
MSR = SSR / df
R2 = SSR/SST YX
SSE S
n-2 =
t-test = b1 / Sb1

14. Measures of Model Goodness
R2 – Coefficient of Determination
F-test > F-crit or p-value less than alpha
Standard Error
t-test

15. Hypothesis testing for
Testing to see if the linear relationship between X and Y is significant at the population level.
t-test
Follow the 5-step process
H0:
HA:
t-crit, alpha or alpha/2, n-2 df

16. Standard Error Terms in Linear Regression
Se (standard error of the estimate)
A measure of variation around the regression line
If the Se is small…
Standard deviation Of the Errors
Sb1 (standard error of the the sampling distribution of b1)
Standard deviation of the slopes
A measure of the variation of the slopes from different samples If the Sb1 is small…our b1 estimate is probably very accurate
Estimates of … b1 b1 b1

17. Linear Regression Example
Petfood, Estimate Sales based on Shelf Space
Two sets of samples, 12 observations each
Perform a Regression Analysis on both sets of data
Sample1
Sample2