Chapter 5: Introductory Linear Regression

INTRODUCTION TO LINEAR REGRESSION Regression – is a statistical procedure for establishing the relationship between 2 or more variables. This is done by fitting a linear equation to the observed data. The regression line is used by the researcher to see the trend and make prediction of values for the data. There are 2 types of relationship: – Simple ( 2 variables) – Multiple (more than 2 variables)

 Many problems in science and engineering involve exploring the relationship between two or more variables.  Two statistical techniques: (1) Regression Analysis (2) Computing the Correlation Coefficient (r).  Linear regression - study on the linear relationship between two or more variables.  This is done by formulate a linear equation to the observed data.  The linear equation is then used to predict values for the data.

 In simple linear regression only two variables are involved: i.X is the independent variable. ii. Y is dependent variable.  The correlation coefficient (r) tells us how strongly two variables are related.

Example 5.1: 1) A nutritionist studying weight loss programs might wants to find out if reducing intake of carbohydrate can help a person reduce weight. a)X is the carbohydrate intake (independent variable). b)Y is the weight (dependent variable). 2) An entrepreneur might want to know whether increasing the cost of packaging his new product will have an effect on the sales volume. a)X is cost b)Y is sales volume

SCATTER DIAGRAM A scatter plot is a graph or ordered pairs (x,y). The purpose of scatter plot – to describe the nature of the relationships between independent variable, X and dependent variable, Y in visual way. The independent variable, x is plotted on the horizontal axis and the dependent variable, y is plotted on the vertical axis.

n Positive Linear Relationship E(y)E(y) x Slope  1 is positive Regression line Intercept  0 SCATTER DIAGRAM

n Negative Linear Relationship E(y)E(y) x Slope  1 is negative Regression line Intercept  0 SCATTER DIAGRAM

n No Relationship E(y)E(y) x Slope  1 is 0 Regression line Intercept  0 SCATTER DIAGRAM

A linear regression can be develop by freehand plot of the data. Example 10.2: The given table contains values for 2 variables, X and Y. Plot the given data and make a freehand estimated regression line. GRAPHICAL METHOD FOR DETERMINING REGRESSION

5.1 SIMPLE LINEAR REGRESSION MODEL  Linear regression model is a model that expresses the linear relationship between two variables.  The simple linear regression model is written as: where ;

 The Least Square method is the method most commonly used for estimating the regression coefficients  The straight line fitted to the data set is the line: where is the estimated value of y for a given value of X. 5.2 INFERENCES ABOUT ESTIMATED PARAMETERS LEAST SQUARES METHOD

i)y-Intercept for the Estimated Regression Equation,

ii) Slope for the Estimated Regression Equation,

Math 1, x Math 2, y a)Develop an estimated linear regression model with “Math 1” as the independent variable and “Math 2” as the dependent variable. b)Predict the score a student would obtain “Math 2” if he scored 60 marks in “Math 1”. The data below represent scores obtained by ten students in subject Mathematics 1 and Mathematics 2. E XAMPLE 5.2: S TUDENTS S CORE I N MATHEMATICS

5.3 ADEQUACY OF THE MODEL COEFFICIENT OF DETERMINATION( R 2 ) The coefficient of determination is a measure of the variation of the dependent variable (Y) that is explained by the regression line and the independent variable (X). The symbol for the coefficient of determination is r 2 or R 2. Example : If r = 0.90, then r 2 =0.81. It means that 81% of the variation in the dependent variable (Y) is accounted for by the variations in the independent variable (X).

The rest of the variation, 0.19 or 19%, is unexplained and called the coefficient of non determination. Formula for the coefficient of non determination is 1- r 2

n The coefficient of determination is:

5.4 PEARSON PRODUCT MOMENT CORRELATION COEFFICIENT (r) Correlation measures the strength of a linear relationship between the two variables. Also known as Pearson’s product moment coefficient of correlation. The symbol for the sample coefficient of correlation is (r) Formula :

Properties of (r): Values of r close to 1 implies there is a strong positive linear relationship between x and y. Values of r close to -1 implies there is a strong negative linear relationship between x and y. Values of r close to 0 implies little or no linear relationship between x and y.

E XAMPLE 5.4: R EFER P REVIOUS E XAMPLE 5.2, Calculate the value of r and interpret its meaning. S OLUTION : Thus, there is a strong positive linear relationship between score obtain Math 1 (x) and Math 2 (y).

To test the existence of a linear relationship between two variables x and y, we proceed with testing the hypothesis. Test commonly used: 5.5TEST FOR LINEARITY OF REGRESSION t -Test F -Test

1. Determine the hypotheses. 2. Compute Critical Value/ level of significance. 3. Compute the test statistic. ( no linear relationship) (exist linear relationship) t-Test

4. Determine the Rejection Rule. Reject H 0 if : There is a significant relationship between variable X and Y. 5.Conclusion.

E XAMPLE 5.5: R EFER P REVIOUS E XAMPLE 5.3, Test to determine if their scores in Math 1 and Math 2 is related. Use α =0.05 S OLUTION : 1) 2) ( no linear r/ship) (exist linear r/ship)

3)

4)Rejection Rule: 5) Conclusion: Thus, we reject H 0. The score Math 1(x) has a linear relationship to the score in Math 2(y).

F Test 1.Determine the hypothesis 2.Determine the rejection region 3.Compute the test statistics 4.Conclusion

1.Determine the hypothesis (NO RELATIONSHIP) (THERE IS RELATIONSHIP) 2.Compute Critical Value/ level of significance. 3.Compute the test statistics

2.Determine the rejection region We reject H 0 if p -value <  3.Conclusion If we reject H 0 there is a significant relationship between variable X and Y.

General form of ANOVA table: ANOVA Test 1) State the hypothesis 2) Select the distribution to use: F-distribution 3) Calculate the value of the test statistic: F 4) Determine rejection and non rejection regions: 5) Make a decision: Reject Ho/failed to reject H0 Source of Variation Degrees of Freedom(df) Sum of SquaresMean SquaresValue of the Test Statistic Regression1MSR=SSR/1 F=MSR MSE Errorn-2MSE=SSE/n-2 Totaln-1

Example The manufacturer of Cardio Glide exercise equipment wants to study the relationship between the number of months since the glide was purchased and the length of time the equipment was used last week. 1)Determine the regression equation. 2)At α=0.01, test whether there is a linear relationship between the variables

Solution (1): Regression equation:

Solution (2): 1)Hypothesis: 1)F-distribution table: 2)Test Statistic: F = MSR/MSE = or using p-value approach: significant value = )Rejection region: Since F statistic > F table (17.303> ), we reject H 0 or since p-value (0.003>0.01 )we reject H 0 5)Thus, there is a linear relationship between the number of months and length of time the equipment was used.

EXERCISE 5.1: The owner of a small factory that produces working gloves is concerned about the high cost of air conditioning in the summer. Keeping the higher temperature in the factory may lower productivity. During summer, he conducted an experiment with temperature settings from 68 to 81 degrees Fahrenheit and measures each day’s productivity which produced the following table: (a)Find the regression model. (b)Predict the number of pairs of gloves produced if x = 74. (c)Compute the Pearson correlation coefficient. What you can say about the relationship of the two variables? (d)Can you conclude that the temperature is linearly related to the number of pairs of gloves produced? Use α=0.05. Temperature Number of Pairs of gloves (in hundreds)

EXERCISE 5.2 : An agricultural scientist planted alfalfa on several plots of land, identical except for the soil pH. Following are the dry matter yields (in pounds per acre) for each plot. pHYield

a)Compute the estimated regression line for predicting yield from pH. b)If the pH is increased by 0.1, by how much would you predict the yield to increase or decrease? c)For what pH would you predict a yield of 1500 pounds per acre? d)Calculate coefficient correlation, and interpret the results.