Multiple Regression II 4/11/12 Categorical explanatory variables Adjusted R 2 Not in book Professor Kari Lock Morgan Duke University

Regrade requests must be submitted in writing, with the original project, by Friday, 4/13/12, at 5pm If regraded, I will grade the entire project, and the grade may go up or down Project 1 Regrade Requests

Project 2 Proposal (due TODAY, 5pm) Project 2 Proposal Homework 9 (due Monday, 4/16) Homework 9 Project 2 Presentation (Thursday, 4/19) Project 2 Presentation Project 2 Paper (Wednesday, 4/25) Project 2 Paper To Do

Chapter 9 odd solutions available Chapter 9 odd solutions Chapter 9 Odd Solutions

Your group members are in the same lab (mostly) so this is a great time to meet with your group in person There is no “On Your Own” and I’ve tried to make the lab short, so you should have some time with your group to get started with your project If you want to work through the lab on your own before coming to class, you can have the whole lab time with your group to work on the project Come to lab tomorrow! Lab

US States We will build a model to predict the % of the state that voted Republican in the 2008 US presidential election, using the 50 states as cases Sample? Population? This can help us to understand how certain features of a state are associated with political beliefs

US States Response Variable: Our first explanatory variable is region of the country: Midwest, Northeast, South, or West

Categorical Variables For this to make any sense, each x value has to be a number. How do we include categorical variables in a regression setting?

Categorical Variables Take one categorical variable, and replace it with several “dummy” variables A dummy variable is 1 if the case falls into the category represented by the dummy variable, and 0 otherwise Create one dummy variable for each category of the categorical variable

Dummy Variables StateRegionSouthWestNortheastMidwest AlabamaSouth1000 AlaskaWest0100 ArkansasSouth1000 CaliforniaWest0100 ColoradoWest0100 ConnecticutNortheast0010 DelawareNortheast0010 FloridaSouth1000 GeorgiaSouth1000 HawaiiWest0100 ………………

Dummy Variables When using dummy variables, one has to be left out of the model The dummy variable left out is called the reference level When using region of the country (Northeast, South, Midwest, West) to predict % McCain vote, how many dummy variables will be included? a)Oneb) Twoc) Three d) Four

Dummy Variables Predicting % vote for McCain with one categorical variable: region of the country If “midwest” is the reference level:

Voting by Region Based on the output above, which region had the highest percent vote for McCain? a)Midwest b)Northeast c)South d)West

Voting by Region What is the predicted % Republican vote for a state in the northeast? a)–10.2% b)48.6% c)38.4% d)58.8%

Voting by Region What is the predicted % Republican vote for a state in the midwest? a)50% b)48.6% c)0% d)58.8%

Categorical Variables The p-value for each dummy variable tests for a significant difference between that category and the reference level For an overall p-value for the significance of the categorical variable with multiple categories, use a)z-test b)T-test c)Chi-square test d)ANOVA

Categorical Variables

Categorical Variables in R R automatically creates dummy variables for you if you include a categorical explanatory variable The first level alphabetically is usually the reference level If you want to change the reference level, see me

Categorical Variables Either all dummy variables associated with a categorical variable have to be included in the model, or none of them RegionW is not significant, but leaving it out would clump West with the reference level, Midwest, which does not make sense

Variables Let’s include some more explanatory variables! What helps to predict % voting Republican?

Categorical Variables Be careful not to include a categorical variable for which every case is it’s own category Example: using “State” as an explanatory variable would be silly, even though R 2 = 1! If you want to know how each state voted, it would make more sense to just look directly at McCainVote, rather than fitting a model and giving each state it’s own coefficient

Explanatory Variables Also, be careful not to include explanatory variables that are essentially just another form of the response variable For example, ObamaMcCain is “M” if the state went for McCain, and “O” if the state went for Obama This is certainly associated with the % of people in the state that voted for McCain, but tells us nothing interesting

Explanatory Variables Models should be created either to learn about relationships between explanatory variables and the response, or for prediction Make sure the explanatory variables you include in the model are not contradicting the point of the model

Visualization How would we visualize the association between region and % vote for McCain? a)Scatterplot b)Side-by-side boxplots c)Bar chart d)Pie chart e)Mosaic plot

Side-by-Side Boxplots

Test for Association How would we test for an association between region and % vote for McCain? a)t-test for difference in means b)test for a correlation c)ANOVA d)chi-square test e)test for a difference in proportions

ANOVA

Visualization All of the other potential explanatory variables are quantitative. How would we visualize the association between each of them and % vote for McCain? a)Scatterplot b)Side-by-side boxplots c)Bar chart d)Pie chart e)Mosaic plot

What do you see?

Test for Association How would we test the association between each of these variables and % vote for McCain? a)t-test for difference in means b)test for a correlation c)ANOVA d)chi-square test e)test for a difference in proportions

Test for Correlation

Regression Model

Physical Activity Given all the other variables in the model, states with a higher percentage of physically active citizens are more likely to vote (a) Republican (b) Democratic

West Region With only region as an explanatory variable, interpret the meaning of the negative coefficient of RegionW. With all the other explanatory variables included, interpret the meaning of the positive coefficient of RegionW. In this data set, states in the West voted less Republican than states in the Midwest. States in the West voted more Republican than would be expected based on the other variables in the model, as compared to states in the Midwest.

Goal of the Model? If the goal of the model is to see what and how each variable is associated with a state’s voting patterns, given all the other variables in the model, then we are done If the goal is to predict the % of the vote that will be Republican, say in the 2012 election, we want to prune out insignificant variables to improve the model

Over-fitting It is possible to over-fit a model: to include too many explanatory variables The fewer the coefficients being estimated, the better they will be estimated Usually, a good model has pruned out explanatory variables that are not helping

R2R2 Adding more explanatory variables will only make R 2 increase or stay the same Adding another explanatory variable can not make the model explain less, because the other variables are all still in the model Is the best model always the one with the highest proportion of variability explained, and so the highest R 2 ? (a) Yes(b) No

Adjusted R 2 Adjusted R 2 is like R 2, but takes into account the number of explanatory variables As the number of explanatory variables increases, adjusted R 2 gets smaller than R 2 One way to choose a model is to choose the model with the highest adjusted R 2

Adjusted R 2 You now know how to interpret all of these numbers!