Linear Functions 2 Sociology 5811 Lecture 18 Copyright © 2004 by Evan Schofer Do not copy or distribute without permission

Announcements Proposals due November 15 Today’s class: Linear functions as summaries; introduction to the linear regression model

Review: Scatterplots Question: Can you describe the association? Answer: Negative linear association

Review: Scatterplots Question: Can you describe the association? Answer: Non-linear positive association

Review: Scatterplots Question: Can you describe the association? Answer: No association

Review: Scatterplots No relationship is represented by a “cloud” of evenly distributed points Strong linear relationships are reflected by visible “diagonal lines” on the graph Non-linear (curved) relationships are reflected by various curved patterns –U-shaped, upside down U-shaped –S-shaped, J-shaped

Review: Linear Association The closer points fall to a single line, the higher the linear association –Measured by the correlation coefficient (r) Some Linear Assoc.Higher Linear Assoc.

Review: Linear Functions Formula: Y = a + bX –Is a linear formula. If you graphed X and Y for any chosen values of a and b, you’d get a straight line. It is a family of functions, like the normal curve –For any value of a and b, you get a particular line a is referred to as the “constant” or “intercept” b is referred to as the “slope” To graph a linear function: Pick values for X, compute corresponding values of Y –Then, connect dots to graph line

Linear Functions: Y = a + bX Y axis X axis The “constant” or “intercept” (a) –Determines where the line intersects the Y-axis –If a increases (decreases), the line moves up (down) Y= X Y= X Y= X

Linear Functions: Y = a + bX The slope (b) determines the steepness of the line Y axis X axis Y=3-1.5X Y=2+.2X Y=3+3X

Linear Functions: Slopes The slope (b) is the ratio of change in Y to change in X Y=3+3X The slope tells you how many points Y will increase for any single point increase in X Change in X =5 Change in Y=15 Slope: b = 15/5 = 3

Linear Functions as Summaries A linear function can be used to summarize the relationship between two variables: Change in X = 40,000 Change in Y = 2 Slope: b = 2 / 40,000 = pts/$ If you change units: b =.05 / $1K b =.5 pts/$10K b = 5 pts/$100K

Linear Functions as Summaries Slope and constant can be “eyeballed” to approximate a formula: Slope (b): b = 2 / 40,000 = pts/$ Constant (a) = Value where line hits Y axis a = 2 Happy = Income

Linear Functions as Summaries Linear functions can powerfully summarize data: –Formula: Happy = Income Gives a sense of how the two variables are related –Namely, people get a increase in happiness for every extra dollar of income (or 5 pts per $100K) Also lets you “predict” values. What if someone earns $150,000? –Happy = ($150,000) = 9.5 But be careful… You shouldn’t assume that a relationship remains linear indefinitely –Also, negative income or happiness make no sense…

Linear Functions as Summaries Come up with a linear function that summarizes this real data: years of education vs. job prestige It isn’t always easy! The line you choose depends on how much you “weight” these points.

Linear Functions as Summaries One estimate of the linear function The line meets the Y-axis at Y=5. Thus a = 5 Formula: Y = 5 + 3X The line increases to about 65 as X reaches 20. The increase is 60 in Y per 20 in X. Thus: b = 60/20 = 3

Linear Functions as Summaries Questions: How much additional job prestige do you get by going to college (an extra 4 years of education)? –Formula: Prestige = 5 + 3*Education Answer: About 12 points of job prestige Change in X is 4… Slope is 3. 3 x 4 = 12 points If X=12, Y=5+3*12 = 41; If X=16, Y=5+3*16 = 53 What is the interpretation of the constant? It is the predicted job prestige of someone with zero years of education… (Prestige = 5)

Linear Functions as Summaries What do you think happens to the relationship between education and job prestige when education exceeds 20? –Would it remain linear? –Or would the effect taper off? Answer: Some would argue that the returns from education diminish beyond a certain point.

Interpreting Linear Functions New Example: In a society, the relationship between education (years) and income (in 1000s of dollars per year) can be summarized by: – Income (in 1000’s) = (Education) Questions: What is the general range of salaries? –0 education = 10k, 20 yrs education = 70K What is the economic benefit of college? Would you encourage your child to attend school? –What if it were: Income = (Education) ?

Interpreting Linear Functions Example: Income (1000s) = (Education) Questions: How would the society be different if the constant was 0? Provide a possible social interpretation. How would the society be different if the constant was 30? How would the society be different if the slope was zero? If it was negative? How would the society be different of the slope was 8?

Linear Functions Many issues remain: 1. How to test for “independence” among two interval measures (like a chi-square test)? –In order to know if a linear relationship exists 2. How to calculate correlation coefficients (r) to measure linear association? 3. How to calculate the linear formula that best summarizes the relationship between two real variables (i.e., based on actual data)? 4. What kinds of hypothesis tests can be done?

Lines: Summaries and Prediction Recall: Lines can be used to summarize –dollars in 1000’s: Slope (b): b = 2 / 40 =.05 pts/K$ Constant (a) = Value where line hits Y axis a = 2 Happy = Income Change in X = 40 Change in Y = 2

Linear Functions as Prediction Linear functions can summarize the relationship between two variables: –Formula: Happy = Income (in 1,000s) Linear functions can also be used to “predict” (estimate) a case’s value of variable (Y i ) based on its value of another variable (X i ) –If you know the constant and slope “Y-hat” indicates an estimation function: b YX denotes the slope of Y with respect to X

Prediction with Linear Functions If X i (Income) = 60K, what is our estimate of Y i (Happiness)? Happy = Income Happiness-hat = (60) = 5 There is an case with Income =60K The prediction in imperfect… The case falls at X = 5.3 (above the line).

The Linear Regression Model To model real data, we must take into account that points will miss the line Similar to ANOVA, we refer to the deviation of points from the estimated value as “error” (e i ) In ANOVA the estimated value is: the group mean –i.e., the grand mean plus the group effect In regression the estimated value is derived from the formula Y = a + bX –Estimation is based on the value of X, slope, and constant (assumes linear relationship between X and Y)

The Linear Regression Model The value of any point (Y i ) can be modeled as: The value of Y for case (i) is made up of A constant (a) A sloping function of the case’s value on variable X (b YX ) An error term (e), the deviation from the line By adding error (e), an abstract mathematical function can be applied to real data points

The Linear Regression Model Visually: Y i = a + bX i + e i Y=2+.5X Constant (a) = 2 a = 2 bX = 3(.5) = 1.5 Case 7: X=3, Y=5 e = 1.5

Estimating Linear Equations Question: How do we choose the best line to describe our real data? –Previously, we just “eyeballed” it Answer: Look at the error If a given line formula misses points by a lot, the observed error will be large If the line is as close to all points as possible, observed error will be small Of course, even the best line has some error –Except when all data points are perfectly on a line

Estimating Linear Equations A poor estimation (big error) Y=1.5-1X

Estimating Linear Equations Better estimation (less error) Y=2+.5X

Estimating Linear Equations Look at the improvement (reduction) in error: High Error vs. Low Error

Estimating Linear Equations Idea: The “best” line is the one that has the least error (deviation from the line) Total deviation from the line can be expressed as: But, to make all deviation positive, we square it, producing the “sum of squares error”

Estimating Linear Equations Goal: Find values of constant (a) and slope (b) that produce the lowest squared error –The “least squares” regression line The formula for the slope (b) that yields the “least squares error” is: Where s 2 x is the variance of X And s YX is the covariance of Y and X.

Covariance Variance: Sum of deviation about Y-bar over N-1 Covariance (s YX ): Sum of deviation about Y-bar multiplied by deviation around X-bar:

Covariance Covariance: A measure of how much variance of a case in X is accompanied by variance in Y It measures whether deviation (from mean) in X tends to be accompanied by similar deviation in Y Or if cases with positive deviation in X have negative deviation in Y This is summed up for all cases in the data The covariance is one numerical measure that characterizes the extent of linear association As is the correlation coefficient (r).