1. Regression Analysis in Theory and Practice

2. DON’T WRITE THE FORMULAS AHEAD!!!

3. REGRESSION ANALYSIS Formula for simple regression Formula for simple regression where is the predicted value of Y on the regression line. Do you remember y=mx + b? Same thing!

4. The dependence of Y on X can be of two types: “deterministic” or “probabilistic”. The classic case of deterministic relationship is that between Fahrenheit and Celsius measure of temperature: F 0 = 32 + (9/5)C Where a, the intercept, is 32 0. So when C=0, degrees F=32, b beta, is the slope of the line, here (9/5) or 1.8. C is X, degrees Celsius.

5. So for every on degree of change in degrees C, Fahrenheit goes up by 1.8 degrees, starting at 32 degrees. So when C =0 F = 32 0 + (9/5)0 = 32 0 When C = 100 0 F = 32 + (9/5)100=212 0 Note: 1.8 = 9/5

6. Probabilistic Regression  Not perfectly predictive.  On average, we expect a certain amount of change in Y for a certain change in X

7. Regression Example  Judges are advised to give longer sentences to repeat offenders than to first- time offenders. Does it really happen?  Hypothesis: In comparing criminals, those who illustrate the characteristic of having been convicted before will receive longer prison sentences than those with no prior convictions.  We collect data for 10 convicted criminals

8. Data and Formula: X (convctn) y (sen len) y (sen len) 012 313 115 019 626 527 329 431 1040 848 Σx = 40 Σy = 260 X = 4 Y = 26 X – X Y – Y Y – Y-4-14-13 -3-11 -4-7 20 113 05 684 488

9. Continued: (X-X) * (Y-Y) (X-X) 2 5616 131 339 2816 04 11 -31 50 1436 2216 Σ = 300 Σ= 100 X = 4 Y = 26 b = 3 X – X Y – Y Y – Y-4-14-13 -3-11 -4-7 20 113 05 684 488

10. Now Calculate “A” a = 26 – (3) * 4 a = 26 – 12 a = 14 Y = 14 + 3*X

11. Interpret the Equation Y = 14 + 3*X Interpret 14 Interpret 3

12. Scatterplot

16. Multiple Regression - 1  The mathematics of how the computer calculates regression coefficients in multiple regression is very complicated. Fortunately, there is an intuitive process that generates the correct answers and is much easier to understand. Let’s see how the computer obtained the value of -.644 for the impact of senator conservatism on the degree to which a senator voted for tax changes primarily benefitting households at, or below, the median income.

17. Multiple Regression - 2  Our “main equation” is:  Y = a 1 + b 1 X 1 + b 2 X 2 + b 3 X 3 + e 1  Y = percentage support for tax changes benefitting households with incomes at, or below, the median  X 1 = senator conservatism  X 2 = senator party affiliation  X 3 = state median household income  Our goal is to estimate b 1

18. Multiple Regression - 3  X 1 = a 2 + b 4 X 2 + b 5 X 3 + e 2  In the above equation e 2 represents that portion of a senator’s conservatism than CANNOT be explained by either their party affiliation or the median family income in their state.

19. Multiple Regression - 4  Y = a 3 + b 6 X 2 + b 7 X 3 + e 3  In the above equation e 3 represents that portion of a senator’s degree of support for tax changes favorable to households with incomes at, or below, the median that CANNOT be explained by either their party affiliation or the median family income in their state.

20. Multiple Regression - 5  e 3 = a 4 + b 8 e 2 + e 4  In the above equation b 8 represents the impact of that portion of a senator’s conservatism that CANNOT be explained by party and state median income on the percentage of times the senator voted in favor of tax changes primarily benefitting households at, or below, the median income that CANNOT be explained by either their party affiliation or the median income in their state. Thus, b 8 in the above equation = b 1 in the “main equation” (i.e., -.644).

21. Maximum Likelihood Estimation