Presentation is loading. Please wait.

Presentation is loading. Please wait.

Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13.

Similar presentations


Presentation on theme: "Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13."— Presentation transcript:

1 Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13

2 13-2 Simple Linear Regression 13.1The Simple Linear Regression Model and the Least Square Point Estimates 13.2Model Assumptions and the Standard Error 13.3Testing the Significance of Slope and y-Intercept 13.4Confidence and Prediction Intervals

3 13-3 Simple Linear Regression Continued 13.5Simple Coefficients of Determination and Correlation 13.6Testing the Significance of the Population Correlation Coefficient (Optional) 13.7An F Test for the Model 13.8Residual Analysis (Optional) 13.9Some Shortcut Formulas (Optional)

4 The Simple Linear Regression Model and the Least Squares Point Estimates The dependent (or response) variable is the variable we wish to understand or predict The independent (or predictor) variable is the variable we will use to understand or predict the dependent variable Regression analysis is a statistical technique that uses observed data to relate the dependent variable to one or more independent variables

5 13-5 Form of The Simple Linear Regression Model y = β 0 + β 1 x + ε  y = β 0 + β 1 x + ε is the mean value of the dependent variable y when the value of the independent variable is x β 0 is the y-intercept; the mean of y when x is 0 β 1 is the slope; the change in the mean of y per unit change in x ε is an error term that describes the effect on y of all factors other than x

6 13-6 Regression Terms β 0 and β 1 are called regression parameters β 0 is the y-intercept and β 1 is the slope We do not know the true values of these parameters So, we must use sample data to estimate them b 0 is the estimate of β 0 and b 1 is the estimate of β 1

7 13-7 The Simple Linear Regression Model Illustrated Figure 13.4

8 13-8 The Least Squares Point Estimates

9 13-9 Example 13.3: Fuel Consumption Case #1

10 13-10 Example 13.3: Fuel Consumption Case #2 (Slope b 1 )

11 13-11 Example 13.3: Fuel Consumption Case #3 (y-Intercept b 0 )

12 13-12 Example 13.3: Fuel Consumption Case #4 Prediction (x = 40) y ̂ = b 0 + b 1 x = ( )(28) y ̂ = MMcf of Gas

13 Model Assumptions and the Standard Error 1.Mean of Zero 2.Constant Variance Assumption 3.Normality Assumption 4.Independence Assumption

14 13-14 An Illustration of the Model Assumptions Figure 13.10

15 13-15 Sum of Squares

16 Testing the Significance of the Slope and y-Intercept A regression model is not likely to be useful unless there is a significant relationship between x and y To test significance, we use the null hypothesis: H 0 : β 1 = 0 Versus the alternative hypothesis: H a : β 1 ≠ 0

17 13-17 Testing the Significance of the Slope #2 AlternativeReject H 0 Ifp-Value H a : β 1 > 0t > t α Area under t distribution right of t H a : β 1 < 0t < –t α Area under t distribution left of t H a : β 1 ≠ 0|t| > t α/2 * Twice area under t distribution right of |t| * That is t > t α/2 or t < –t α/2

18 13-18 Testing the Significance of the Slope #3 Test Statistics 100(1-α)% Confidence Interval for β 1 [b 1 ± t  /2 S b1 ] t , t  /2 and p-values are based on n–2 degrees of freedom

19 Confidence and Prediction Intervals The point on the regression line corresponding to a particular value of x 0 of the independent variable x is y ̂ = b 0 + b 1 x 0 It is unlikely that this value will equal the mean value of y when x equals x 0 Therefore, we need to place bounds on how far the predicted value might be from the actual value We can do this by calculating a confidence interval mean for the value of y and a prediction interval for an individual value of y

20 13-20 A Confidence Interval and a Prediction Interval

21 13-21 Which to Use? The prediction interval is useful if it is important to predict an individual value of the dependent variable A confidence interval is useful if it is important to estimate the mean value The prediction interval will always be wider than the confidence interval

22 The Simple Coefficient of Determination and Correlation How useful is a particular regression model? One measure of usefulness is the simple coefficient of determination It is represented by the symbol r²

23 13-23 Calculating The Simple Coefficient of Determination 1.Total variation is given by the formula  (y i -y ̄)² 2.Explained variation is given by the formula  (y ̂ i -y ̄)² 3.Unexplained variation is given by the formula  (y i -y ̂)² 4.Total variation is the sum of explained and unexplained variation 5.r ² is the ratio of explained variation to total variation

24 13-24 The Simple Correlation Coefficient The simple correlation coefficient measures the strength of the linear relationship between y and x and is denoted by r Where b 1 is the slope of the least squares line

25 13-25 Different Values of the Correlation Coefficient Figure 13.21

26 Testing the Significance of the Population Correlation Coefficient (Optional) The simple correlation coefficient (r) measures the linear relationship between the observed values of x and y from the sample The population correlation coefficient (ρ) measures the linear relationship between all possible combinations of observed values of x and y r is an estimate of ρ

27 13-27 Testing ρ We can test to see if the correlation is significant using the hypotheses H 0 : ρ = 0 H a : ρ ≠ 0 The statistic is This test will give the same results as the test for significance on the slope coefficient b 1

28 An F Test for Model For simple regression, this is another way to test the null hypothesis H 0 : β 1 = 0 This is the only test we will use for multiple regression The F test tests the significance of the overall regression relationship between x and y

29 13-29 Mechanics of the F Test To test H 0 : β 1 = 0 versus Ha: β 1  0 at the  level of significance Test statistics based on F Reject H 0 if F(model) > F  or p-value <  F  is based on 1 numerator and n - 2 denominator degrees of freedom

30 13-30 Mechanics of the F Test Graphically Figure 13.22

31 Residual Analysis (Optional) Regression assumptions are checked by analyzing the regression residuals Residuals (e) are the difference between the observed value of y and the predicted value of y, e = y - ŷ If regression assumptions valid, the population of potential error terms will be normally distributed with a mean of zero and a variance σ² Different error terms will be statistically independent

32 13-32 Residual Analysis #2 Residuals should look like they were randomly and independently selected from normal population with μ=0 and variance σ² With real data, assumptions will not hold exactly Mild departures do not affect our ability to make statistical inferences We are looking for pronounced departures from the assumptions Only require residuals to approximately fit the description above

33 13-33 Residual Plots 1.Residuals versus independent variable 2.Residuals versus predicted y’s 3.Residuals in time order (if the response is a time series)

34 13-34 Constant Variance Assumption Figure 13.25

35 13-35 Assumption of Correct Functional Form If the relationship between x and y is something other than a linear one, the residual plot will often suggest a form more appropriate for the model For example, if there is a curved relationship between x and y, a plot of residuals will often show a curved relationship

36 13-36 Normality Assumption If the normality assumption holds, a histogram or stem-and-leaf display of residuals should look bell-shaped and symmetric Another way to check is a normal plot of residuals Order residuals from smallest to largest Plot e (i) on vertical axis against z (i) If the normality assumption holds, the plot should have a straight-line appearance

37 13-37 Independence Assumption Independence assumption is most likely to be violated when the data are time-series data If the data is not time series, then it can be reordered without affecting the data For time-series data, the time-ordered error terms can be autocorrelated Positive autocorrelation is when a positive error term in time period i tends to be followed by another positive value in i+k Negative autocorrelation is when a positive error term tends to be followed by a negative value

38 13-38 Independence Assumption Visually Positive Autocorrelation Figure (a)

39 13-39 Independence Assumption Visually Negative Autocorrelation Figure (b)

40 Some Shortcut Formulas


Download ppt "Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13."

Similar presentations


Ads by Google