Linear Regression When looking for a linear relationship between two sets of data we can plot what is known as a scatter diagram. x y Looking at the graph we can see that there is some positive correlation.

x y It is possible to draw a line called a regression line. There are two types y on x and x on y. First lets consider y on x regression line. y on x The y on x line, draws the regression line by keeping the sum of the squares of the vertical distance to a minimum. Note: The equation of the line is called “The Equation of the Least Squares regressions Lines”

x y Now consider the x on y regression line. x on y The x on y line, draws the regression line by keeping the sum of the squares of the horizontal distance to a minimum.

x y y on x x on y Drawing both graphs on the same graph we have We should note that both graphs will pass through the means of both sets of data,.

It is possible to calculate the equations of the y on x and x on y regression lines. Important formulae y on x regression line is of the form and can be calculated by using the formula. Where is called the covariance and links the x and y data. is the variance of the x data

x on y regression line is of the form and can be calculated by using the formula. Where is called the covariance and links the x and y data. is the variance of the y data

Example In the table below are the results of ten students in both their Mathematics and Physics examinations. The teacher thinks there might be a relationship between the two. His hypothesis is “a student who has Mathematical ability also has ability in Physics.” Mathematics Mark /100 (x)Physics Mark /100 (y)

Drawing a scatter graph xy

Now calculating the regression lines

For regression line y on x which has form For regression line x on y which has form

Plotting both lines on the scatter diagram y on x, and for x on y, Note: For x on y line, remember to rearrange it into the following form before trying to plot y on x x on y

Correlation We need a way to determine if there is linear correlation or not. So we calculate what is known as the Product-Moment Correlation Coefficient (r). (covariance), (standard deviation of x) (standard deviation of y). We can see that the quantity r from the following five sets of data above tells us something about the degree of scatter of the two sets of data, if we are looking for a linear relationship.

x y Table 1 y on x x on y The product moment correlation coefficient In table 1 we notice that the two regressions lines (y on x and x on y) nearly coincide and that as the x-data increases the y-data decreases. The value of r is , which is close to –1. Here we have what is called strong negative linear correlation.

x y Table 2 y on x x on y The product moment correlation coefficient In table 2, the two regression lines are further apart although there is weak negative linear correlation. The value of r is and it is getting closer to 0.

Table 3 y on x x on y The product moment correlation coefficient x y In table 3, the two regression lines are virtually perpendicular and there is no linear correlation. The value of r is and it is very close to 0.

Table 4 y on x x on y The product moment correlation coefficient x y In table 4, the two regression lines are further apart but we notice that as the x-data increases the y-data increases. We say there is weak positive linear correlation. The value of r is and it is moving away from 0 and getting closer to 1.

x y Table 5 y on x x on y The product moment correlation coefficient In table 5, we notice that the two regressions lines (y on x and x on y) nearly coincide and that as the x-data increases the y-data increases. The value of r is 0.990, which is very close to 1. Here we have what is called strong positive linear correlation.

The value of r determines the degree of linear scatter of the two sets of data and - indicates that the data have perfect negative linear correlation, - indicates that the data has no linear correlation, - indicates that the data have perfect positive linear correlation. r is called Product-Moment Correlation Coefficient.

y on x x on y Returning to our example So we can conclude that as r is close to 1, that the results show that his hypothesis that “a student who has Mathematical ability also has ability in Physics’” might be true.