1. Applied Quantitative Analysis and Practices LECTURE#22 By Dr. Osman Sadiq Paracha

2. Previous Lecture Summary Application in SPSS for factor analysis stages Interpretation of factor matrix Validation of factor analysis Factor Scores

3. Simple Linear Regression

4. Correlation vs. Regression A scatter plot can be used to show the relationship between two variables Correlation analysis is used to measure the strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the relationship No causal effect is implied with correlation DCOVA

5. Types of Relationships Y X Y X Y Y X X Linear relationshipsCurvilinear relationships DCOVA

6. Types of Relationships Y X Y X Y Y X X Strong relationshipsWeak relationships (continued) DCOVA

7. Types of Relationships Y X Y X No relationship (continued) DCOVA

8. Introduction to Regression Analysis Regression analysis is used to: Predict the value of a dependent variable based on the value of at least one independent variable Explain the impact of changes in an independent variable on the dependent variable Dependent variable: the variable we wish to predict or explain Independent variable: the variable used to predict or explain the dependent variable DCOVA

9. Simple Linear Regression Model Only one independent variable, X Relationship between X and Y is described by a linear function Changes in Y are assumed to be related to changes in X DCOVA

10. Linear component Simple Linear Regression Model Population Y intercept Population Slope Coefficient Random Error term Dependent Variable Independent Variable Random Error component DCOVA

11. (continued) Random Error for this X i value Y X Observed Value of Y for X i Predicted Value of Y for X i XiXi Slope = β 1 Intercept = β 0 εiεi Simple Linear Regression Model DCOVA

12. The simple linear regression equation provides an estimate of the population regression line Simple Linear Regression Equation (Prediction Line) Estimate of the regression intercept Estimate of the regression slope Estimated (or predicted) Y value for observation i Value of X for observation i DCOVA

13. The Least Squares Method b 0 and b 1 are obtained by finding the values of that minimize the sum of the squared differences between Y and :

14. Finding the Least Squares Equation The coefficients b 0 and b 1, can be found through the below mentioned formula b 1 = b 0 =

15. b 0 is the estimated average value of Y when the value of X is zero b 1 is the estimated change in the average value of Y as a result of a one-unit increase in X Interpretation of the Slope and the Intercept

16. Simple Linear Regression Example A real estate agent wishes to examine the relationship between the selling price of a home and its size (measured in square feet) A random sample of 10 houses is selected Dependent variable (Y) = house price in $1000s Independent variable (X) = square feet

17. Simple Linear Regression Example: Data House Price in $1000s (Y) Square Feet (X) 2451400 3121600 2791700 3081875 1991100 2191550 4052350 3242450 3191425 2551700

18. Simple Linear Regression Example: Scatter Plot House price model: Scatter Plot

19. Simple Linear Regression Example Regression Statistics Multiple R0.76211 R Square0.58082 Adjusted R Square0.52842 Standard Error41.33032 Observations10 ANOVA dfSSMSFSignificance F Regression118934.9348 11.08480.01039 Residual813665.56521708.1957 Total932600.5000 CoefficientsStandard Errort StatP-valueLower 95%Upper 95% Intercept98.2483358.033481.692960.12892-35.57720232.07386 Square Feet0.109770.032973.329380.010390.033740.18580 The regression equation is:

20. Simple Linear Regression Example: Graphical Representation House price model: Scatter Plot and Prediction Line Slope = 0.10977 Intercept = 98.248

21. Simple Linear Regression Example: Interpretation of b o b 0 is the estimated average value of Y when the value of X is zero (if X = 0 is in the range of observed X values) Because a house cannot have a square footage of 0, b 0 has no practical application

22. Simple Linear Regression Example: Interpreting b 1 b 1 estimates the change in the average value of Y as a result of a one-unit increase in X Here, b 1 = 0.10977 tells us that the mean value of a house increases by.10977($1000) = $109.77, on average, for each additional one square foot of size

23. Predict the price for a house with 2000 square feet: The predicted price for a house with 2000 square feet is 317.85($1,000s) = $317,850 Simple Linear Regression Example: Making Predictions

24. When using a regression model for prediction, only predict within the relevant range of data Relevant range for interpolation Do not try to extrapolate beyond the range of observed X’s

25. Measures of Variation Total variation is made up of two parts: Total Sum of Squares Regression Sum of Squares Error Sum of Squares where: = Mean value of the dependent variable Y i = Observed value of the dependent variable = Predicted value of Y for the given X i value

26. SST = total sum of squares (Total Variation) Measures the variation of the Y i values around their mean Y SSR = regression sum of squares (Explained Variation) Variation attributable to the relationship between X and Y SSE = error sum of squares (Unexplained Variation) Variation in Y attributable to factors other than X (continued) Measures of Variation

27. Lecture Summary Simple Linear Regression Correlation Vs Regression Introduction to Simple Linear Regression Simple Linear Regression Model Least Square Method Interpretation of Model Measures of variation