1. LINEAR REGRESSION 1

2. Introduction Simple regression = relationship between two variables
Multiple regression = relationship between more variables
Dependent variable (Outcome, explained variable)
Independent variable (predictor, explanatory variable)
Dependent variable: cardinal (scale)
Independet variable(s): cardinal or binary (but see dummy variables later)

3. Introduction
Main goal: explain dependent variable by independent variable(s)
Assumption: relationship between dependent variable and independent variable(s) is linear (can be described by line – example)
Statistical solution: find the equation for relationship and describe it

4. SIMPLE LINEAR REGRESSION4

5. Simple lin. regression
Cardinal dependent variable and one cardinal independent variable
Assumption: relationship between dependent variable and independent variable is linear
Example in SPSS (chart and fit line): Graphs-Chart builder-Scatter/Dot (Add Fit Line at Total)
Reco: Every time before any regression computation use chart

6. Some details How to fit the „best“ line?
What is the meaning of regression equation?
Is my regression good enough?
How to improve my regression?

7. SR by picture

8. Ordinary least squares (OLS)
Find the „best“ line?
try to minimize sum of squares

9. Ordinary least squares (OLS)
residual = difference between real value of dep. var. and estimate by reg. line
b0 = intercept (intercection of regression line and Y axis, value of dep. var. if independent is zero)
b1 = regression coeff./slope (average increase/decrease for unit change in indep. var)

10. SR results
R – correlation between real values of dep. var. and estimates by reg. line
R2 – square of R, Reco: multiply by 100 and interpret in % as percentage of explained variance (measure strenght if relationship)
Regression coeff. and intercept
T-test (is the relationship expected in the population, can be generalized to the population?)
Example in SPSS

11. MULTIPLE LINEAR REGRESSION11

12. MR by picture

13. Multiple regression Data matrix (indep. variables X)
X1 X2 X3 X4 ETC.
YES M 1,2
NO F 4,3
NO F 2,3
NO M 3,8
YES M 2,6 .
ETC.

14. Multiple regression Vector y
135
112
187
189
ETC.

15. Multiple regression Vector β (regression coeffs.)
β0 intercept
β1 reg. coeff. for the 1st var
β2 reg. coeff. for the 2nd var
Β reg. coeff. for the 3rd var
ETC.

16. Multiple linear regression model
y = 0 + 1x1 + 2x pxp + 
Regression equation
E(y) = 0 + 1x1 + 2x pxp
Estimate of reg. eq. (based on sample)
y = b0 + b1x1 + b2x bpxp

17. Model in matrix form
Model:
y = βX + ε

18. Estimates by OLS What does it mean?  Excursus of vector algebra 
multiplying-matrices-by-matrices/v/multiplying-a-matrix-by-a-matrix

19. Multiple regression in SPSS
Result in SPSS: regression equation of line, plane or overplane, statistical test for model and coeffs. and regression diagnostics (see next week)
Menu in SPSS – basic options

20. Syntax in SPSS Syntax for stepwise and selected outputs
REGRESSION /MISSING LISTWISE
/STATISTICS COEFF OUTS R ANOVA
/CRITERIA=PIN(.05) POUT(.10)
/NOORIGIN
/DEPENDENT Y
/METHOD= STEPWISE X1 X2 X3.

21. SPSS outputs for regression
Example of multiple regression in SPSS
Interpretation of results: ANOVA table,T-tests, R, R2, R2Adj.
Regression coeffs. : interpretation (ceteris paribus principle)
Beta coeffs.: comparison of individual imnpact of variables (regression coeff. for standardized data)
What is standardization?

22. Regression in SPSS
Selection of variables - forward, backward, stepwise (principals)
Stepwise is very often used but not recommended by statisticians (Why?)
Predicted outcomes from regression
Residuals: meaning and saving

23. DUMMY VARIABLES 23

24. Dummy variables
Possibilty to use nominal and/or ordinal variables as independent variables in regression model by a set of dummy variables
Basic rule – number of dummy variables is computed: number of categories-1
„omitted category" is called referenece category – all other categories will be compared to this category (example in SPSS)
Reco: use category with the lowest level of dependent variable as reference (all coeffs will be positive and your interpretation would be simple)

25. Dummy – more info
Principle of dummy variables can be applied in other techniques (e.g. logistic regression)
Omitted last category is applied in special statistical techniques (loglinear models, logit models)
Some procedures in SPSS will create dummy by default (e.g. procedures for logistic regressions)

26. INTERACTIONS 26

27. Interactions Combine two or more variables into one new variable
Necessary to prepare in data
Why use interactions?
A) joint effect of two variables (synergy)
B) solve different relationships in groups
Example – two way interaction (two variables), one cardinal and one binary
Interactions by picture and practical application in SPSS