1. Correlation and regression

2. Introduction
Scientific rules and principles are often expressed mathematically
There are two main approaches to finding a mathematical relationship between variables
Analytical
Based on theory
Empirical
Based on observation and experience

3. The straight line (1) Most graphs based on numerical data are curves.
The straight line is a special case
Data is often manipulated to yield straight line graphs as the straight line is relatively easy to analyse

4. The Straight line (2) Straight line equation y = mx + c slope = m
m = Dy/Dx
Intercept = c

5. Correlation & Regression
These are statistical processes which;
Suggest the existence of a relationship
Determine the best equation to fit the data
Correlation is a measure of the strength of a relationship between two variables
Regression is the process of determining that relationship

6. Correlation and Regression
The next few slides illustrate correlation and regression

7. No Correlation

8. Positive correlation

9. Negative correlation

10. Curvilinear correlation

11. Correlation coefficient
A statistical measure of the strength of a relationship between two variables.
Pearson’ product-moment correlation coefficient, r
Spearman’s rank correlation coefficient, r
All these take a value in the range -1.0 to + 1.0
r or r = +1.0 represents a perfect positive correlation
r or r = -1.0 represents a perfect negative correlation
r or r = 0.0 represents a no correlation
values of r or r are associated with a probability of there being a relationship.

12. Linear regression
Is the process of trying to fit the best straight line to a set of data.
The usual method is based on minimising the squares of the errors between the data and the predicted line
For this reason, it is called “the method of least squares”

13. Linear regression - assumptions
The error in the independent (x) variable is negligible relative to the error in the dependant (y) variable
The errors are normally, independently and identically distributed with mean 0 and constant variance - NIID(0,s2)

14. Linear regression model
For a set of data, (x,y), there is an equation that best fits the data of the form
Y = a + bx + e
x is the independent variable or the predictor
y is the measured dependant or predicted variable
Y is the calculated dependant or predicted variable
e is the error term and accounts for that part of Y not “explained” by x.
For any individual data point, i, the difference between the observed and predicted value of y is called the residual, ri
i.e. ri = yi – Yi = yi - (a + bxi)
The residuals provide a measure of the error term

15. Regression analysis (1)
Check the correlation coefficient
Null Hypothesis
H0: There is no correlation between x & y
H1: There is a correlation between x & y
Decision rule
reject H0 if |r|  critical value at a = 0.05
If you cannot reject H0 then proceed no further, otherwise carry out a full regression

16. Regression analysis (2)
Regression analysis can be carried out using either Excel or Minitab. Excel will need the analysis ToolPak add-in installed.
The output from both Minitab and Excel will give the following information
The regression equation ( in the form y = a + bx)
Probabilities that a  0 and b  0
The coefficient of determination, R2
Analysis of variance
In addition you will need to produce at least one of
Residuals vs. fitted values
Residuals vs. x-values
Residuals vs. y values

17. Interpreting output
Regression equation:- this is the equation that best fits the data and provides the predicted values of y
Analysis of variance:- Determines the proportion of the variation in x & y that can be accounted for by the regression equation and what proportion is accounted for by the error term. The p-value arising out of this tells us how well the regression equation fits the data.
The proportion of the variation in the data accounted for by the regression equation is called the coefficient of determination, R2 and is equal to the square of the correlation coefficient

18. Output plots
The output plots are used to check the assumptions about the errors
The normal probability plot should show the residuals lying on a straight line.
The residual plots should have no obvious pattern and should not show the residuals increasing or decreasing with increase in the fitted or measured values.

19. Non linear relationships
Many functions can be manipulated mathematically to yield a straight line equation.
Some examples are given in the next few slides

20. Linearisation (2)

21. Linearisation (3)

22. Functions involving logs (1)
Some functions can be linearised by taking logs
These are
y = A xn
and y = A ekx

23. Functions involving logs (2)
For y = Axn, taking logs gives
log y = log a + n log x
A graph of log y vs. log x gives a straight line, slope n and intercept log A.
To find A you must take antilogs (= 10x)

24. Functions involving logs (3)
For y = Aekx, we must use natural logs
ln y = ln A + kx
This gives a straight line slope k and intercept ln A
To find A we must take antilogs (= ex)

25. Polynomials These are functions of general formula
y = a + bx + cx2 + dx3 + …
They cannot be linearised
Techniques for fitting polynomials exist
Both Excel and Minitab provide for fitting polynomials to data