Download presentation

Presentation is loading. Please wait.

1
Basics of Statistical Estimation

2
Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability is unknown Given iid data, estimate using an estimator with good properties: low bias, low variance, consistent (e.g., maximum likelihood estimate)

3
Maximum Likelihood Principle Choose the parameters that maximize the probability of the observed data

4
Maximum Likelihood Estimation (Number of heads is binomial distribution)

5
Computing the ML Estimate Use log-likelihood Differentiate with respect to parameter(s) Equate to zero and solve Solution:

6
Sufficient Statistics (#h,#t) are sufficient statistics

7
Bayesian Estimation tails heads True probability is unknown Bayesian probability density for p()p() 01

8
Use of Bayes’ Theorem prior likelihood posterior

9
Example: Application to Observation of Single “Heads" p( |heads) 01 p()p() 01 p(heads| )= 01 priorlikelihoodposterior

10
Probability of Heads on Next Toss

11
MAP Estimation Approximation: –Instead of averaging over all parameter values –Consider only the most probable value (i.e., value with highest posterior probability) Usually a very good approximation, and much simpler MAP value ≠ Expected value MAP → ML for infinite data (as long as prior ≠ 0 everywhere)

12
Prior Distributions for Direct assessment Parametric distributions –Conjugate distributions (for convenience) –Mixtures of conjugate distributions

13
Conjugate Family of Distributions Beta distribution: Resulting posterior distribution:

14
Estimates Compared Prior prediction: Posterior prediction: MAP estimate: ML estimate:

15
Intuition The hyperparameters h and t can be thought of as imaginary counts from our prior experience, starting from "pure ignorance" Equivalent sample size = h + t The larger the equivalent sample size, the more confident we are about the true probability

16
Beta Distributions Beta(3, 2 )Beta(1, 1 )Beta(19, 39 )Beta(0.5, 0.5 )

17
Assessment of a Beta Distribution Method 1: Equivalent sample - assess h and t - assess h + t and h /( h + t ) Method 2: Imagined future samples

18
Generalization to m Outcomes (Multinomial Distribution) Dirichlet distribution: Properties:

19
Other Distributions Likelihoods from the exponential family Binomial Multinomial Poisson Gamma Normal

Similar presentations

© 2019 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google