Regression Analysis in Theory and Practice

DON’T WRITE THE FORMULAS AHEAD!!!

REGRESSION ANALYSIS Formula for simple regression Formula for simple regression where is the predicted value of Y on the regression line. Do you remember y=mx + b? Same thing!

The dependence of Y on X can be of two types: “deterministic” or “probabilistic”. The classic case of deterministic relationship is that between Fahrenheit and Celsius measure of temperature: F 0 = 32 + (9/5)C Where a, the intercept, is So when C=0, degrees F=32, b beta, is the slope of the line, here (9/5) or 1.8. C is X, degrees Celsius.

So for every on degree of change in degrees C, Fahrenheit goes up by 1.8 degrees, starting at 32 degrees. So when C =0 F = (9/5)0 = 32 0 When C = F = 32 + (9/5)100=212 0 Note: 1.8 = 9/5

Probabilistic Regression  Not perfectly predictive.  On average, we expect a certain amount of change in Y for a certain change in X

Regression Example  Judges are advised to give longer sentences to repeat offenders than to first- time offenders. Does it really happen?  Hypothesis: In comparing criminals, those who illustrate the characteristic of having been convicted before will receive longer prison sentences than those with no prior convictions.  We collect data for 10 convicted criminals

Data and Formula: X (convctn) y (sen len) y (sen len) Σx = 40 Σy = 260 X = 4 Y = 26 X – X Y – Y Y – Y

Continued: (X-X) * (Y-Y) (X-X) Σ = 300 Σ= 100 X = 4 Y = 26 b = 3 X – X Y – Y Y – Y

Now Calculate “A” a = 26 – (3) * 4 a = 26 – 12 a = 14 Y = *X

Interpret the Equation Y = *X Interpret 14 Interpret 3

Scatterplot

Multiple Regression - 1  The mathematics of how the computer calculates regression coefficients in multiple regression is very complicated. Fortunately, there is an intuitive process that generates the correct answers and is much easier to understand. Let’s see how the computer obtained the value of for the impact of senator conservatism on the degree to which a senator voted for tax changes primarily benefitting households at, or below, the median income.

Multiple Regression - 2  Our “main equation” is:  Y = a 1 + b 1 X 1 + b 2 X 2 + b 3 X 3 + e 1  Y = percentage support for tax changes benefitting households with incomes at, or below, the median  X 1 = senator conservatism  X 2 = senator party affiliation  X 3 = state median household income  Our goal is to estimate b 1

Multiple Regression - 3  X 1 = a 2 + b 4 X 2 + b 5 X 3 + e 2  In the above equation e 2 represents that portion of a senator’s conservatism than CANNOT be explained by either their party affiliation or the median family income in their state.

Multiple Regression - 4  Y = a 3 + b 6 X 2 + b 7 X 3 + e 3  In the above equation e 3 represents that portion of a senator’s degree of support for tax changes favorable to households with incomes at, or below, the median that CANNOT be explained by either their party affiliation or the median family income in their state.

Multiple Regression - 5  e 3 = a 4 + b 8 e 2 + e 4  In the above equation b 8 represents the impact of that portion of a senator’s conservatism that CANNOT be explained by party and state median income on the percentage of times the senator voted in favor of tax changes primarily benefitting households at, or below, the median income that CANNOT be explained by either their party affiliation or the median income in their state. Thus, b 8 in the above equation = b 1 in the “main equation” (i.e., -.644).

Maximum Likelihood Estimation