Presentation is loading. Please wait.

Presentation is loading. Please wait.

Chapter 17 Linear regression is a procedure that identifies relationship between independent variables and a dependent variable. This relationship helps.

Similar presentations


Presentation on theme: "Chapter 17 Linear regression is a procedure that identifies relationship between independent variables and a dependent variable. This relationship helps."— Presentation transcript:

1 Chapter 17 Linear regression is a procedure that identifies relationship between independent variables and a dependent variable. This relationship helps reduce the unexplained variation of the dependent variable behavior, thus provide better predictions of its future values.

2 The Simple linear regression model
The model is:

3 The Simple linear regression model
The model is: We try to estimate the deterministic part of it by developing the line with the best fit. Best fit is defined as the minimum sum of squared errors. An error is the difference between the line value and the actual value for a given x.

4 The Simple linear regression model
The analysis yields a prediction equation of the form , where… b0 is an estimate of b0 b1 is an estimate of b1 is the predicted value of y for a given x.

5 Problem 1 Are the costs of welding machines breakdowns related to their age? From the data answer the following: Find the sample regression line What is the coefficient of determination. Interpret. Are machine-age and monthly repair costs linearly related? Is the fit good enough to use the model to predict the monthly repair costs of a 120 months old machine? Make the prediction.

6 Problem 1 Find the sample regression line
To calculate b0 and b1 by hand we can use: Instead we’ll use Excel to find the covariance of x and y the variance of x, and the means of x and y, then use the following formulae.

7 Problem 1 To calculate the covariance of x and y by Excel use the function covar(x,y) and multiply the result by n/(n-1). To calculate and use Data Analysis Plus Descriptive Statistics

8 Problem 1 From Excel we get:
Data Problem 1 From Excel we get: The Cov(age,cost)= Mean age (x) = = Mean cost (y) = = b1 = cov(x,y)/ = / = b0 = y-b1x = (113.35) = The regression line:

9 Problem 1 Coefficient of determination. Data In this case
56.59% of the variation in costs, are explained by the model (that is, by the age variation).

10 Data Problem 1 Is there a linear relationship between monthly costs and machine age? If b1¹ 0 there is a linear relationship between x and y. Thus, we test the coefficient b1 H0: b1= 0 H1: b1¹ 0

11 Problem 1 Data We use a t-statistic: In this case
Comment: Sb1 can be calculated by But in this course we’ll use the calculated value from the Excel Run. See the printout in the next slide. In this case t= [2.47-0]/.5106 = 4.837 The rejection region is t > ta/2 or t < -ta/2 (a two tail test) with n-2 degrees of freedom For a = .05, t.025,18 =2.10 Since 4.837>2.10, the null hypothesis is rejected at 5% significance level. There is linear relationship between Monthly cost and machine age.

12 Data Problem 1 Here we perform the t-test for b1 using the p-value (read from the printout) The p value < alpha

13 Data Problem 1 We need to forecast the expected cost for a 120 months old machine. The equation provides a point prediction: Cost = (120) = $ The prediction interval (use data analysis plus, select the “Prediction Interval” option, observe the results under “Prediction Interval”): Lower Confidence level = $318.12; Upper Confidence Level = What’s the prediction for the average monthly repair cost for all the machines 120 months old? To answer this question construct the confidence interval (notice, not the prediction interval!). Again, use Data analysis Plus select the “Prediction Interval” option, observe the results under “Interval Estimate of Expected value”.

14 Chapter 18 The multiple regression model allows more than one independent variable explain the values of the dependent variable. We assess the model as before using t test for linear relationships between the independent variables and the dependent variable (tested one at a time) F test for the over usefulness of the model Coefficient of determination for the fit.

15 Problem 2 When a company buys another company it is not unusual that some workers are terminated. A buyout contract between Laurier Comp and the Western Comp required that Laurier provides a severance package to Western workers fired, equivalent to packages offered to Laurier workers. It is suggested that severance is determined by three factors: Age, length of service, pay. Bill smith, a Western employee, is offered a 5 weeks severance pay when his employment is terminated. Based on the data provided by Laurier about severance offered to 50 of its employees in the past, answer the following questions:

16 The results and analysis appear in the Excel data file “Laurier”
Problem 2 - continued Determine the regression equation. Interpret the coefficients. Comment on how well the model fits the data. Do all the independent variables belong in the model? Does Laurier meet its obligation to Bill Smith? The results and analysis appear in the Excel data file “Laurier”

17 Longevity = b0+b1MotherAge+ b2FatherAge+b3GrandM+b4GrandF+e
Problem 3 A linear regression model for life longevity Insurance companies are interested in predicting life longevity of their customers. Data for 100 deceased male customers was collected, and a regression model run. The model studied was: Longevity = b0+b1MotherAge+ b2FatherAge+b3GrandM+b4GrandF+e

18 Problem 3 Coefficient of determination The equation

19 Problem 3 Overall usefulness: H0: all bi = 0 H1: At least one bi = 0
F Significance = p value = 4.86(10-27) Reject H0. The model is useful.

20 Problem 3 Mother’s age and father’s age at death
have strong linear relationships to an Individual’s age at death. Grandparents’ age at death are not good predictors of an individual’s age at death. The t-test for bi: H0: bi = 0 H1: bi = 0 t= (bi – bi)/sbi Rejection region: t>ta/2, n-k-1 or t<-ta/2, n-k-1

21 Qualitative Variables
Dummy variables help include qualitative data in a regression model. If qualitative data can be categorized by n categories, there are n-1 dummy variables needed to express all the categories. Dummy variables take on the values 0 or 1. Xi = 0 if the data point in question does not belong to category i Xi = 1 if the data point in question belongs to category i.

22 Problem 4 In problem 1 we studied the relationship between age of welding machines and breakdown costs. This study was expanded. It is now including also lathe machine and stamping machines. See Data file. Code for machine type: 1=Welding; 2=Lathe; 3=Stamping Answer the following: Develop a regression model Interpret the coefficient Can we conclude that welding machines cost more to repair than stamping machine. Predict the monthly cost to repair an 85 month old lathe machine

23 Problem 4 First we need to prepare the input data Original data

24 Problem 4 First we need to prepare the input data

25 Problem 4 Run the multiple regression

26 Problem 4 Run the multiple regression More Review questions
Cost= Age W L Repair cost increase on the average by $2.53 a month. The monthly repair cost for a welding machine is $11.75 lower than for a stamping machine of the same age. However, this result is not significant p value=.55). There is insufficient evidence in the sample to support the hypothesis that there is any difference between repair costs of welding machines and stamping machines. The monthly repair cost for a lathe machine is $ lower than for a stamping machine of the same age. This result is significant. Run the multiple regression Note the reference line (for the stamping machine): Cost= Age

27 Example 5 To predict the asking price of a used Chevrolet Camaro, the following data were collected on the car’s age and mileage. Data is stored in CAMARO1. Determine the regression equation and answer additional questions stated later. Solution Run the regression tool from Excel > Data analysis. Click to see the output next

28 The regression equation
The regression equation: Price = Age-72.31Mileage Be careful about the interpretation of the intercept (17499). Do not argue that this is the price of a used car with no mileage when its age is “zero”. Although such cars may exist (a car purchased and returned within a week with almost no mileage) might need to be re-sold as a used car. Yet, such values of Age and Mileage were not covered by the sample range!!. CAMARO1

29 The model usefulness Does the overall model contribute significantly to predicting the asking price of a used Chevrolet Camaro? Use .01 for the significance level Answer: Observe the Significance F. This is the p value for the F Test of the hypotheses H0: b1= b2 = 0 H1: At least one b ¹0. Since the p value is practically zero, it is smaller than alpha. The null hypothesis is rejected, and therefore at least one b ¹0. The variable associated with this b is linearly related to the price, and the model is useful, thus contributes to predicting the asking price. CAMARO1

30 Model’s fit How well does the model fit the data? Would you expect the predictions to be accurate with this model? Solution Observing the coefficient of determination (R2), 81% of the variation in car prices are explained by this model. This is quite high, and we can expect accurate predictions.

31 Predicting ‘y’ Predict the value of the asking price for a 5-years old car, with 70,000 miles on the odometer, with 95% confidence. Solution To obtain an interval estimate for the prediction of a single car asking price when Age=5, and Mileage=70, we look for the prediction interval. From Data Analysis Plus we have {$ , $ }. The general form of the interval is: , where D is determined from the data. Specifically: (5)-72.31(70)= So the interval is ± D, For the Data Analysis Plus procedure go to the worksheet “Prediction Interval” in “CAMARO1”.

32 Estimating the mean ‘y’
Predict the value of the mean asking price for all 5-years old cars, with 70,000 miles on the odometer, with 95% confidence. Solution To obtain an interval estimate for the mean asking price of all cars for which Age=5 and Mileage=70, we look for the confidence interval. From Data Analysis Plus we have {$ , $ } For details go to the worksheet “Prediction Interval” in “CAMARO1”.

33 Testing linear relationship
Are both variables (Age and Mileage each one in the presence of the other one), serve as good predictors of Asking Price? Test at alpha=.025. Solution Perform a t-test for the b coefficient of each variable. The hypotheses tested are: H0: bAge=0 vs. H1: bAge¹ 0 for which the p value is .002; H0: bMileage=0 vs. H1: bMileage¹ 0 for which the p value is In both cases the null hypothesis is rejected, therefore, both have linear relationship to the asking price at 2.5% significance level.

34 Problem 5-continued The previous model for the prediction of the asking price of used Chevrolet Camaro, is now extended by adding two new independent variables: car condition (Excellent, Average, Poor), and the type of the seller who sells the car (Dealer, Individual). The data for this case is stored in CAMARO2 (see next slide). Develop the linear regression model for this case and answer several questions formulated next. Solution The two new variables describe the values of qualitative data (the state of a car and the type of the seller). Thus, they are dummy variables, taking on the values ‘0’ and ‘1’.

35 Using dummy variables Solution – continued:
There are three possible car condition values, so we need two dummy variables. Let us select the variables ‘Average’ and ‘Poor’. In describing the two values of the car condition, these variables are used as follows: Average Poor An “Excellent condition” car 0 0 An “Average condition” car 1 0 A “Poor condition” car 0 1 In a similar manner we use one dummy variable to describe who sold the car. Let us define Dealer = 1 if the car was sold by a dealer. Dealer = 0 if sold by an individual. CAMARO2

36 The linear regression equation
The linear regression equation: Price= Age Mileage Avg Poor Dealer

37 Interpreting the coefficients bi
Interpret the coefficient estimates bi of each variable and test the strength of their predicting power. Solution bAge= In this model, For each additional year the asking price drops by $1132, keeping the rest of the variables unchanged. bMile= In this model, for each additional 1000 miles the asking price drops by $33.24, keeping the rest of the variables unchanged. bAvg = In this model, the asking price for a car whose condition is average is $ lower than the asking price for a car whose condition is excellent, keeping the rest of the variables unchanged. bPoor = In this model, the asking price for a car whose condition is poor is $ lower than the asking price for a car whose condition is excellent, keeping the rest of the variables unchanged. bDeal = In this model the asking price for a car sold by a dealer is $ higher than this sold by an individual, keeping the rest of the variables unchanged.

38 The role of the dummy variable coefficients
Let us compare the asking price equations of two cars, with the same age, mileage, and condition, one sold by a dealer, the other one by an individual: Price(Dealer)=b0+b1Age+b2Mileage+b3Avg.+b4Poor +b5(Dealer=1)= b0+b1Age+b2 Mileage+b3Avg.+b4Poor +b5 Price(Individual)=b0+b1Age+b2Mileage+b3Avg.+b4Poor +b5(Dealer=0)= b0+b1Age+b2Mileage+b3Avg.+b4Poor Conclusion: When the only difference between cars is the type of sellers who sell them, the base line equation was selected to be the Price(Individual) equation, and then b5 is the average difference in asking price between them.

39 The role of the dummy variable coefficients
Let us compare the asking price equations of three cars, that differ in their overall condition but have the same age, mileage, and are sold by the same type of a seller: Price(Excellent)=b0+b1Age+b2 Mileage+b3(Avg.=0)+b4(Poor=0) +b5(Dealer)= b0+b1Age+b2 Mileage+b5(Dealer) Price(Avg.)=b0+b1Age+b2Mileage+b3(Avg.=1)+b4(Poor=0) +b5(Dealer)= b0+b1Age+b2 Mileage+b5(Dealer) + b3 Price(Poor)=b0+b1Age+b2Mileage+b3(Avg.=0)+b4(Poor=1) +b5(Dealer)= b0+b1Age+b2 Mileage+b5(Dealer) + b4 Conclusion: When the only difference between cars is the car condition, the base line equation was selected to be the Price(Excellent) equation, and then b3 and b4 are the average differences in asking price between an “excellent condition” car and the other two cars.

40 Prediction power of independent variable (are there linear relationships?)
Testing the prediction power. Formulate the t-test for each b. Observing the p values we have: For bAge the p value= Age is a strong predictor For bMileage the p value=.17. Mileage is not a good predictor, not having linear relationship with price. For bAverage the p value= There is sufficient evidence to infer at 1% significance level that the asking price of a car whose condition is average is different from the asking price of a car whose condition is excellent. In fact, the argument is even stronger. Since the t-statistic is negative (-2.79), the rejection region is at the left hand tail of the distribution, so we have sufficient evidence to claim that bavarage<0. This means the asking price of an “Avg. Condition” car is on the average $2556 lower than the asking price of an “Excellent condition” car.

41 Prediction power of independent variable (are there linear relationships?)
Testing the prediction power - continued. For bPoor the p value = There is a very strong evidence to believe that the asking price for a “Poor Condition” car is different than the asking price for an “Excellent condition” car. Specifically, a “Poor condition” car is sold for $ less than an “Excellent condition” car. For bDealer the p value = .40. There is insufficient evidence to infer at 2.5% significant level that on the average the asking price for a car sold by a dealer is different than the asking price for a car sold by an individual.

42 Prediction power of independent variable (are there linear relationships?)
Predict the asking price of the following cars: 4 years old, miles, Average condition, sold by an individual. Price=17357 – (4) – (45) – (1) (0) The variable “Average” is equal to 1 when the car is in average conditions. The variable “Dealer” is equal to 0 when the car is sold by an individual.

43 Chapter 16 We test the hypotheses that a set of data belongs to certain distributions: The multinomial distribution The normal distribution We also study whether two variables are dependent or not. We apply a tool called a Chi-squared test

44 The multinomial experiment
The multinomial experiment is an extension of the binomial experiment. Characteristics There are n independent trials. Each trial can result in one of k possible outcomes. There is a probability of a type k success (pk) in each trial. We test whether the sample gathered support the hypothesis that p1, p2,…,pk are equal to specified values. The test is called: The goodness of fit test.

45 Problem 6 To determine whether a single die is balanced, or fair, the die was rolled 600 times. Is there sufficient evidence at 5% significance level to allow you to conclude that the die is not fair?

46 Problem 6 The hypothesis:
H0: p1 = p2 =…p6 = 1/6 H1: At least one p is not 1/6. Build a rejection Region: In our case: c2>c2a,5

47 Problem 6 We calculate c2 as follows:
In our case: e1=e2=…=e6=600(1/6)=100 From the file we have: f1=114; f2=92; f3=84; f4=101; f5=107; f6=103

48 Contingency table Here we test the relationship between two variables. Are they dependent? We build a contingency table and a Chi-Square statistic Variable/ Category 2 c columns Variable/ Category 1 r rows

49 Problem 7 Type of music vs. geographic location
A group of 30-years-old people is interviewed to determined whether the type of music is somehow related to the geographic location of their residence. From the data presented can we infer that music preference is affected by the geographic location? Use (a=.10). H0: Type of music and geographic location are independent. H1: Type of music and geographic location are dependent.

50 Problem 7 – contd. Rock R & B Country Classical Northeast 140 32 5 18 South 134 41 52 8 West 154 27 13 195 235 202 632 e11=(195)(428)/632=129.59; e12=(195)(100)/632=30.85 e23=(235)(65)/632=24.16; c2 = ( )2/ …+( )2/24.16+…=64.92 c2.10,(3-1)(4-1) = 10.64; >10.64. Reject the null hypothesis. Type of music and geographic location are not independent.

51 Problem 7 – contd. Using data analysis Plus

52 Goodness of Fit test for normality
Hypothesize on m and s (m=m0 and s=s0). Divide the Z interval into equal size sub-intervals. [i.e. (–2, – 1); (-1,0); (0,1); (1,2)] Determine the corresponding probabilities covered by each subinterval. [i.e. p1=P(Z<-2); p2=P(-2<Z<-1); …] Translate the Z scores to the associated X values. [i,.e. x1=m0+(-2)s0; x2=m0+(-1)s0; …] Find the actual frequency for each subinterval [i.e. f1 - for the interval below x1; f2 - for the interval (x1,x2); …] Calculate the expected frequency for each interval: e1 = np1; e2 = np2; … Build a Chi squared statistic and perform the test


Download ppt "Chapter 17 Linear regression is a procedure that identifies relationship between independent variables and a dependent variable. This relationship helps."

Similar presentations


Ads by Google