1. The Triangle of Statistical Inference: Likelihoood Data Scientific Model Probability Model Inference

2. An example... The Data: x i = measurements of DBH on 50 trees y i = measurements of crown radius on those trees The Scientific Model: y i =  x i +  (linear relationship, with 2 parameters (  and an error term (  ) (the residuals)) The Probability Model:  is normally distributed, with E[  ] and variance estimated from the observed variance of the residuals...

3. So what is likelihood –and what is it good for? 1.Probability based (“inverse probability”). “mathematical quantity that appears to be appropriate for measuring our order of preference among different possible populations but does not in fact obey the laws of probability” --RA. Fischer 2.Foundation of theory of statistics. 3.Enables comparison of alternate models.

4. So what is likelihood –and what is it good for?  Scientific hypotheses cannot be treated as outcomes of trials (probabilities) because we will never have the full set of possible outcomes.  However, we can calculate the probability of obtaining the results, given our model (scientific hypothesis (P(data|model).  Likelihood is proportional to this probability.

5. Likelihood is proportional to probability P(data|hypothesis (  ))  L (hyp|data) P(data|hypothesis (  )) = k L (  | data) In plain English: “The likelihood (L) of the set of parameters (  ) (in the scientific model), given the data (x), is proportional to the probability of observing the data, given the parameters...” {and this probability is something we can calculate, using the appropriate underlying probability model (i.e. a PDF)}

6. Parameter values can specify your hypotheses P(data i |θ) = k L (θ |data) Parameter is fixed, data variable. What is the prob. of observing the data if our model and parameters are correct? Parameter is variable, data fixed. What is the likelihood of the parameter given the data?

7. General Likelihood Function L (θ|x) = cg(x|θ ) Likelihood function Data (x i ) Parameters in probability model Probability density function or discrete density function c is a constant, and thus, unimportant in comparison of alternate hypotheses or models as long as the data remain constant.

8. General Likelihood Function L (θ|x) = g(x i |θ ) Likelihood function Data (x i ) Parameters in probability model Probability density function or discrete density function The parameters of the pdf are determined by the data and by the value of the parameters in scientific model!!

9. Likelihood Axiom “Within the framework of a statistical model, a set of data supports one statistical hypothesis better than other if the likelihood of the first hypothesis, on the data, exceeds the likelihood of the second hypothesis”. (Edwards 1972, p.)

10. How to derive a likelihood function: Binomial The most likely parameter value is 10/50 = 0.20 Probability Density Function Likelihood Event  10 trees die out of a population of 50 Question: What is the mortality rate (p)?

11. Likelihood Profile: Binomial -2E-12 0 2E-12 4E-12 6E-12 8E-12 1E-11 1.2E-11 1.4E-11 1.6E-11 00.20.40.60.81 Value of estimated parameter (p) likelihood The model (parameter p) is defined by the data!!

12. An example: Can we predict tree fecundity as a function of tree size? The Data: x i = measurements of DBH on 50 trees y i = counts of seeds produced by trees The Scientific Model: y i = DBH  +  exponential relationship, with 1 parameter (  and an error term (  ) The Probability Model: Data follow a Poisson distribution, with E[x] and variance = λ Data Scientific Model Probability Model Inference

13. Iterative process 1.Pick a value for the parameter in your scientific model,  Recall scientific model is y i = DBH  2.For each data point, calculate the expected (predicted) value for that value of  3.Calculate the probability of observing what you observed given that parameter value and your probability model. 4.Multiply the probabilities of individual observations. 5.Go back to 1 until you find maximum likelihood estimate for parameter  Data Scientific Model (hypothesis) Probability Model Inference Data Scientific Model (hypothesis) Probability Model Inference

14. Likelihood Poisson Process E[x]= λ

15. First pass… Model y i = DBH    Predicted = 0.0617 Observed = 2 Poisson random Variable with E[x 1 ]=0.0617 0 1 2 Do for n observations…… E[x]= λ

16. Pick a new value of beta... Poisson random Variable with E[x 1 ]=0.498 0 1 2 3 4 Do for n observations…… Model y i = DBH   Predicted = 0.498 Observed = 2

17. Probability and Likelihood 1.Multiplying probabilities is not convenient from a computational point of view. 2.We take the log of the probabilities and we maximize that number. 3.This gives us the Maximum Likelihood Estimate of the parameter.

18. Likelihood Profile ML estimate Beta Model y i = DBH 

19. Model comparison Data Scientific Model (hypothesis) Probability Model Inference Data Scientific Model (hypothesis) Probability Model Inference The Data: x i = measurements of DBH on 50 trees y i = counts of seed produced by trees The Scientific Models: y i = DBH  +  exponential relationship, with 1 parameter (   OR  y i =  DBH  +  linear relationship with 1 parameter (  The Probability Model: Data follow a Poisson distribution, with E[x] and variance = λ

20. Model comparison Data Scientific Model (hypothesis) Probability Model Inference Data Scientific Model (hypothesis) Probability Model Inference The Data: x i = measurements of DBH on 50 trees y i = counts of seed produced by trees The Scientific Models: y i = DBH  +  exponential relationship, with 1 parameter (  The Probability Model: Data follow a Poisson distribution, with E[x] and variance = λ OR Data follow a negative binomial distribution with E[x]=m and clumping parameter k. (Variance is defined by m and k (estimated).

21. FIRST PRINCIPLES 1.Proportions  Binomial 2.Several categories  Multinomial 3.Count events  Poisson, Neg. binomial 4.Continuous data, additive processes  Normal 5.Quantities from multiplicative probabilities  Lognormal, Gamma. EMPIRICAL 1.Examine residuals. 2.Tests different probability distributions for model errors. Determination of appropriate likelihood function Probability models can be thought of as competing hypotheses in exactly the same way that different parameter values (structural models) are competing hypotheses.

22. Likelihood functions: An aside about logarithms Taking the logarithm in base a of a number is the inverse of raising that number to the power a. Example: log 10 1000= 3 Basic Log Operations

23. Poisson Likelihood Function Likelihood Discrete Density Function

24. Negative Binomial Distribution Likelihood Function Likelihood Discrete Density Function k is an estimated parameter!!

25. Normal Distribution Likelihood Function Prob. Density Function Likelihood E[x] = μ Variance = δ 2

26. Lognormal Distribution Likelihood Function Likelihood Prob. Density Function

27. Gamma Distribution Likelihood Function Prob. Density Function

28. Exponential Distribution Likelihood Function Prob. Density Function Likelihood

29. Some references A.W.F. Edwards. 1972. Likelihood. Cambridge University Press. Feller, W. 1968. An introduction to probability theory and its application. Wiley & Sons.