Correlation Assume you have two measurements, x and y, on a set of objects, and would like to know if x and y are related. If they are directly related, when x is high, y tends to be high, and when x is low, y tends to be low. If they are indirectly related, when x is high, y tends to be low, and when x is low, y tends to be high. This suggests summing the products of the z- scores.

xy case Scatter plot of y vs. x

x (zscore) y (zscore) case mean xy xzyzxzyz Correlation coefficient (Pearson’s) r

Properties of r r ranges from -1.0 to +1.0 r = 1 means perfect linear relationship r = -1 means perfect linear relationship with negative slope r = 0 means no correlation

r = r = Example scatterplots r = -.24r =.41 r = 0 r = -.66r =.94

Correlation and causation “Correlation does not imply causation” More precisely, x correlated with y does not imply x causes y, because correlation could be a type I error y could cause x z could cause both x and y

Uncorrelated does not mean independent x y x is highly predictive of y, but r = 0

Significance test for r The aim is to test the null hypothesis that the population correlation ρ (rho) is 0. The larger n, the less likely a given r will happen under the null hypothesis. From r and n, we can compute a p-value From n and α, we can compute a critical r Numerical example

Regression Correlation suggests a linear model of y as a function of x A linear model is defined by ŷ = mx + b + e random error with mean 0equation for a line slopeinterceptpredicted y

x regression y residuals x e R 2 = , F = , p = Regression line: y = x

r vs. R 2 R 2 is actually the square of r. So why is it capitalized and squared in a regression? r ranges from -1 to 1. But in a regression, r cannot meaningfully be negative, because it is the correlation between y and ŷ. Since ŷ is the best estimate of y, this correlation is automatically positive. The capitalization and squaring reflects this situation. It is squared to

Interpretation of R2 R 2 can be interpreted as the proportion of the variance accounted for R 2 = 1 - SS error SS total SS reg SS total = regression line mean R 2 is high when the unexplained (residual) variance is small relative to the total amount of variance

Simpson’s paradox Size of animal length of ears Negatively correlated Or positively correlated? Rabbits Humans Whales Adding a variable can change the sign of the correlation

Effect size Beyond computing significance, we often need an estimate of the magnitude of an effect. There are two basic ways of expressing this: - Normalized mean difference - Proportion of variance accounted for

The normalized difference between means Cohen’s d expresses how the difference between two means relative to the spread in the data.

Proportion of variance accounted for R 2 can be interpreted as the proportion of all the variance in the data that is predicted by a regression model η 2 (eta squared) can be interpreted as the proportion of all variance in a factorial design that is accounted for by main effects and interactions

Power Power is the probability of finding an effect of a given size in your experiment, i.e. The probability of rejecting the null hypothesis if the null hypothesis is actually false.

Outliers An outlier is a measurement that is so discrepant from others that it seems “suspicious.” If p(x suspicious |distribution) is low enough, we “reject the null hypothesis” that x suspicious came from the same distribution as the others, and remove it. A common rule of thumb is z > 2.5 (or 2 or 3), BUT... But also consider transforms that avoid outliers in the first place, like 1/x. Removed data is best NOT REPLACED. But if it must be replaced, do so “conservatively,” i.e. in a manner biased towards the null hypothesis.

Chi squared Assume that mutually K exclusive outcomes are predicted to occur E 1,E 2,...,E K, times...but are actually observed to occur N 1,N 2,...,N K times respectively... A chi-square test allows us to evaluate the null hypothesis that the proportions were as expected, with deviations “by chance.”

Performing a chi-squared test For each outcome, compute Sum them up over all outcomes Then, under the null hypothesis, this total will be distributed as a χ 2 distribution with n-1 degrees of freedom.

The Bayesian perspective Conventional statistics is based on a frequentist definition of probability, which insists that hypotheses do not have “probabilities.” → All we can do is “reject” H, or not reject it. Bayesian inference is based on a subjectivist definition of probability, which considers p(H) to be the “degree of belief” in hypothesis H, simply expressing our uncertainty about H in light of the data. → Instead of accepting or rejecting, we seek p(H|E).

Cartoon 1: Fisher Fisher: Given the sampling distribution of the null p(E|H 0 ), consider the likelihood of the null hypothesis, integrated out to the tail. If this probability is low, this tends to contradict the null hypothesis. In fact, if it is lower than.05, we informally “reject” the null. 0 p(E|H0)p(E|H0) E probability density

Cartoon 2: Neyman & Pearson N&P: There are really two hypotheses, the null H 0 and some alternative H 1. Our main goal is to avoid a Type I error. So set this probability at α, which determines our criterion for rejecting the null. Note though that there is also a possibility of making a Type II error, a hit, or a correct rejection. Compute power and set sample size to control the probability of a Type II error. 0 p(E|H0)p(E|H0) Expected effect size p(E|H1)p(E|H1) μ1μ1 E probability density

Cartoon 3: Bayes (/Laplace/Jeffreys) What we really want is to evaluate how strongly our data favors either hypothesis, not just make an accept/reject decision. For each H, the degree of belief in it, conditioned on the data, is p(H|E). So to evaluate the relative strength of the H 1 and H 0, consider the posterior ratio This expresses how strongly the data and priors favor H 1 relative to H 0, taking into account everything we know about the situation. Degree of belief in H 1 Degree of belief in H 0 =

Decomposing the posterior ratio posterior ratio = prior ratio × likelihood ratio If you want to be “unbiased”, set the prior ratio to 1, sometimes called an “uninformative prior.” Then your posterior belief about H 0 and H 1 depends entirely on the likelihood ratio, aka “Bayes factor.”

Visualizing the Likelihood Ratio 0 Expected effect size μ1μ1 = p(E|H0)p(E|H0)p(E|H1)p(E|H1) height of green bar at E height of red bar at E E probability density

Interpretation of likelihood ratios LR = 1 means the evidence was neutral about which hypothesis was correct. LR > 1 means the evidence favors the hypothesis. Jeffreys (1939) suggested rules of thumb, e.g. LR > 3 means “substantial” evidence in favor of H 1, LR >10 means “strong,” evidence etc. LR < 1 means the evidence actually favored the null hypothesis.

LRs vs. p-values Likelihood ratios and p- values are not at all the same thing. But in practice, they are related. Dixon (1998)