Presentation on theme: "Multiple Regression Models: Interactions and Indicator Variables"— Presentation transcript:
1 Multiple Regression Models: Interactions and Indicator Variables
2 Today’s Data SetA collector of antique grandfather clocks knows that the price received for the clocks increases linearly with the age of the clocks. Moreover, the collector hypothesizes that the auction price of the clocks will increase linearly as the number of bidders increases.(Let’s hypothesize a first order MLR model.)
3 First order model y = β0 + β1x1 + β2x2 + ε y = auction price x1= age of clock (years)x2 = number of bidders
4 The regression equation is… AuctionPrice = AgeOfClock BiddersAnalysis of VarianceSource DF SS MS F PRegressionResidual ErrorTotal
5 Tests of Individual β Parameters Predictor Coef SE Coef T PConstantAgeOfClockBidders
6 What if the relationship between E(y) and either of the independent variables depends on the other? In this case, the two independent variables interact, and we model this as a cross-product of the independent variables.
7 For our example Do age and the number of bidders interact? In other words, is the rate of increase of the auction price with age driven upward by a large number of bidders?In this case, as the number of bidders increases, the slope of the price versus age line increases.To facilitate investigation, number of bidders has been separated into:A: 0-6 biddersB: 7-10 biddersC: bidders
8 Are these slopes parallel, or do they change with the number of bidders?
9 Caution!Once an interaction has been deemed important in a model, all associated first-order terms should be kept in the model, regardless of the magnitude of their p-values.
10 Another Example: Graph and interpret the following findings Let’s say we want to study how hard students work on tests. We have some achievement-oriented students and some achievement-avoiders. We create two random halves in each sample, and give half of each sample a challenging test, the other an easy test. We measure how hard the students work on the test. The means of this study are:Achievement-oriented (n=100)Achievement –avoiders (n=100)Challenging test105Easy test
11 Conclusions E(y)= β0 + β1x1 + β2x2+ β3x1x2 The effect of test difficulty (x1) on effort (y) depends on a student’s achievement orientation (x2).Thus, the type of achievement orientation and test difficulty interact in their effect on effort.This is an example of a two-way interaction between achievement orientation and test difficulty.
12 Basic premises up to this point We have used continuous variables (we can assume that having a value of 2 on a variable means having twice as much of it as a 1.)We often work with categorical variables in which the different values have no real numerical relationship with each other (race, political affiliation, sex, marital status…)Democrat(1), Independent(2), Republican(3)Is a Republican three times as politically affiliated as a Democrat?How do we resolve this problem?
13 Dummy VariablesA dummy variable is a numerical variable used in regression analysis to represent subgroups of the sample in your study.Dummy variables have two values: 0, 1"Republican" variable: someone assigned a 1 on this variable is Republican and someone with an 0 is not.They act like 'switches' that turn various parameters on and off in an equation.
14 Creating Dummy Variables In Minitab, we can recode the categorical variable into a set of dummy variables, each of which has two levels.In the regression model, we will use all but one of the original levels.The level which is not included in the analysis is the category to which all other categories will be compared (base level.) You decide this.The coefficient on the variable in your regression will show the effect that being that variable has on your dependent variable.
15 Returning to the Clocks at Auction Data Set The collector of antique grandfather clocks knows that the price received for the clocks increases linearly with the age of the clocks and he hypothesized that the auction price of the clocks will increase linearly as the number of bidders increases. But let’s say he doesn’t have the exact number of bidders, only knows if there was a high number of bidders (we’ll say 9 and above) or a low number (below 9.)
16 Let’s Create a Dummy Variable in Minitab We’ll use the Bidders2Cat columnCalc Make Indicator VariablesIn top box, specify that you want to make indicator variables for Bidders2CatLet’s store results in C10 - C11Once you have created the variables, name columns 10 and 11 ManyBidders and FewBiddersWhich is which? Why?
17 Before we run the analysis… Let’s say we decide to include ManyBidders (FewBidders is the base level.)Because FewBidders is not included, we can determine if ManyBidders predicts a different Auction Price than FewBidders.If ManyBidders is significant in our regression, with a positive β coefficient, we conclude that ManyBidders has a significant effect on the price of the clocks at auction.
18 Thinking Through the Variables What is x1?Let’s hypothesize the model in plain English… just looking at high/low bidders.)What’s the Null Hypothesis?
19 Run the Analysis Results of the t-test? What would happen if we used ManyBidders as our base?
20 Let’s look at your Journal Application What does it mean to create a dummy variable and when is it appropriate to do this?What are all the terms in the original model?This researcher started with a complex model and simplified it. Which model was better? How can we know?
21 Nested ModelsTwo models are nested if both contain the same terms and one has at least one additional term.Example:The first (straight-line) model is nested within the second (curvilinear) model.The first model is the reduced model and the second is the full or complete model.
22 Which is better? How do we decide? In this example, we would test:To test, we compare the SSE for the reduced model (SSER) and the SSE for the complete model (SSEC).Which will be larger?At least one
23 Error is Always Greater for the Reduced Model SSER>SSECIs the drop in SSE from fitting the complete model ‘large enough’?We use an F-test to compare modelsHere, we test the null hypothesis that our curvature coefficients simultaneously equal zero.
24 Test statistic: F F= drop in SSE/number of βs being tested s2 for larger modelTable C4 (p. 766) gives you the critical value for Fdf for numerator (v1):Number of βs being testeddf for denominator (v2):Number of βs in the complete modelIf F ≥ The critical F value, reject H0. At least one of the additional terms contributes information about the response.
25 Conclusions?Parsimonious models are preferable to big models as long as both have similar predictive power.A model is parsimonious when it has a small number of predictorsIn the end, choice of model is subjective.
26 Question 3 (Journal)What type of error do we risk making by conducting multiple t-tests?Pages 184, 188