Exercise on the concept of Likelihood

Slides:



Advertisements
Similar presentations
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 6 Point Estimation.
Advertisements

Point Estimation Notes of STAT 6205 by Dr. Fan.
Parameter and Structure Learning Dhruv Batra, Recitation 10/02/2008.
Chapter 7 Title and Outline 1 7 Sampling Distributions and Point Estimation of Parameters 7-1 Point Estimation 7-2 Sampling Distributions and the Central.
Sampling: Final and Initial Sample Size Determination
Estimation  Samples are collected to estimate characteristics of the population of particular interest. Parameter – numerical characteristic of the population.
SOLVED EXAMPLES.
Parameter Estimation: Maximum Likelihood Estimation Chapter 3 (Duda et al.) – Sections CS479/679 Pattern Recognition Dr. George Bebis.
Estimation A major purpose of statistics is to estimate some characteristics of a population. Take a sample from the population under study and Compute.
Descriptive statistics Experiment  Data  Sample Statistics Sample mean Sample variance Normalize sample variance by N-1 Standard deviation goes as square-root.
Chapter 6 Introduction to Sampling Distributions
AGC DSP AGC DSP Professor A G Constantinides© Estimation Theory We seek to determine from a set of data, a set of parameters such that their values would.
Fall 2006 – Fundamentals of Business Statistics 1 Chapter 6 Introduction to Sampling Distributions.
Estimation of parameters. Maximum likelihood What has happened was most likely.
Basics of Statistical Estimation. Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability  is unknown.
LARGE SAMPLE TESTS ON PROPORTIONS
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
Statistical inference form observational data Parameter estimation: Method of moments Use the data you have to calculate first and second moment To fit.
Maximum Likelihood We have studied the OLS estimator. It only applies under certain assumptions In particular,  ~ N(0, 2 ) But what if the sampling distribution.
8 Statistical Intervals for a Single Sample CHAPTER OUTLINE
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.
Today Today: Chapter 9 Assignment: 9.2, 9.4, 9.42 (Geo(p)=“geometric distribution”), 9-R9(a,b) Recommended Questions: 9.1, 9.8, 9.20, 9.23, 9.25.
. Maximum Likelihood (ML) Parameter Estimation with applications to reconstructing phylogenetic trees Comput. Genomics, lecture 6b Presentation taken from.
Thanks to Nir Friedman, HU
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.
The Method of Likelihood Hal Whitehead BIOL4062/5062.
Likelihood probability of observing the data given a model with certain parameters Maximum Likelihood Estimation (MLE) –find the parameter combination.
Chapter 5 Sampling Distribution of the Proportion
.. . Maximum Likelihood (ML) Parameter Estimation with applications to inferring phylogenetic trees Comput. Genomics, lecture 6a Presentation taken from.
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 8-1 Confidence Interval Estimation.
Random Sampling, Point Estimation and Maximum Likelihood.
Chapter 7 Point Estimation
Empirical Research Methods in Computer Science Lecture 7 November 30, 2005 Noah Smith.
Forecasting Choices. Types of Variable Variable Quantitative Qualitative Continuous Discrete (counting) Ordinal Nominal.
© Department of Statistics 2012 STATS 330 Lecture 20: Slide 1 Stats 330: Lecture 20.
Methodology Solving problems with known distributions 1.
Chapter 7 Point Estimation of Parameters. Learning Objectives Explain the general concepts of estimating Explain important properties of point estimators.
The final exam solutions. Part I, #1, Central limit theorem Let X1,X2, …, Xn be a sequence of i.i.d. random variables each having mean μ and variance.
Discrepancy between Data and Fit. Introduction What is Deviance? Deviance for Binary Responses and Proportions Deviance as measure of the goodness of.
5. Maximum Likelihood –II Prof. Yuille. Stat 231. Fall 2004.
Week 41 How to find estimators? There are two main methods for finding estimators: 1) Method of moments. 2) The method of Maximum likelihood. Sometimes.
Point Estimation of Parameters and Sampling Distributions Outlines:  Sampling Distributions and the central limit theorem  Point estimation  Methods.
South Dakota School of Mines & Technology Introduction to Probability & Statistics Industrial Engineering.
Maximum Likelihood Estimation
Statistical Estimation Vasileios Hatzivassiloglou University of Texas at Dallas.
M.Sc. in Economics Econometrics Module I Topic 4: Maximum Likelihood Estimation Carol Newman.
CLASSICAL NORMAL LINEAR REGRESSION MODEL (CNLRM )
Lecture 3: MLE, Bayes Learning, and Maximum Entropy
The Uniform Prior and the Laplace Correction Supplemental Material not on exam.
CS 2750: Machine Learning Linear Models for Classification Prof. Adriana Kovashka University of Pittsburgh February 15, 2016.
Hidden Markov Models. A Hidden Markov Model consists of 1.A sequence of states {X t |t  T } = {X 1, X 2,..., X T }, and 2.A sequence of observations.
Presentation : “ Maximum Likelihood Estimation” Presented By : Jesu Kiran Spurgen Date :
Probability and Likelihood. Likelihood needed for many of ADMB’s features Standard deviation Variance-covariance matrix Profile likelihood Bayesian MCMC.
Stats 242.3(02) Statistical Theory and Methodology.
Virtual University of Pakistan
CS479/679 Pattern Recognition Dr. George Bebis
Statistical Estimation
Estimation Maximum Likelihood Estimates Industrial Engineering
Worst-Case Execution Time
POINT ESTIMATOR OF PARAMETERS
Estimation Maximum Likelihood Estimates Industrial Engineering
Stat Lab 9.
CS 594: Empirical Methods in HCC Introduction to Bayesian Analysis
BN Semantics 3 – Now it’s personal! Parameter Learning 1
Maximum Likelihood We have studied the OLS estimator. It only applies under certain assumptions In particular,  ~ N(0, 2 ) But what if the sampling distribution.
Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.
Statistical Model A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced.
Applied Statistics and Probability for Engineers
Maximum Likelihood Estimation (MLE)
Presentation transcript:

Exercise on the concept of Likelihood “There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy.” - William Shakespeare, Hamlet (1.5.167-8)

probability (probabilità) likelihood (verosimiglianza) known unknown PARAMETERS DATA unknown known likelihood (verosimiglianza) A model is never true, a model can be useful. A model is a mathematical interpretation of observed phenomena.

BINOMIAL DISTRIBUTION p(x) = (nx) πx (1-π)n-x known π, x probability can be computed known x, π likelihood can be computed Number of SONS on 4 deliveries π = 0.514 x = 3 p(0) = (40) 0.5140 (1-0.514)4-0 = 0.0558 p(3) = (43) 0.53 (1-0.5)4-3 = 0.25 p(1) = (41) 0.5141 (1-0.514)4-1 = p(2) = (42) 0.5142 (1-0.514)4-2 = p(3) = (43) 0.753 (1-0.75)4-3 = 0.422 p(3) = (43) 0.5143 (1-0.514)4-3 = p(4) = (44) 0.5140 (1-0.514)4-0 = p(3) = (43) 0.93 (1-0.9)4-3 = 0.292

p(x) = (---------) πx (1-π)n-x BINOMIAL DISTRIBUTION x!(n-x)! Probability to have a male = 0.514 according to the WHO π = 0.514 p(0) = (40) 0.5140 (1-0.514)4-0 = 0.0558 p(1) = (41) 0.5141 (1-0.514)4-1 = p(2) = (42) 0.5142 (1-0.514)4-2 = p(3) = (43) 0.5143 (1-0.514)4-3 = p(4) = (44) 0.5140 (1-0.514)4-0 =

p(x) = (---------) πx (1-π)n-x BINOMIAL DISTRIBUTION x!(n-x)! NUMBER OF MALES ON 4 DELIVERIES = 3 π=0.5 p(3) = (43) 0.53 (1-0.5)4-3 =0.25 π=0.55 p(3) = (43) 0.553 (1-0.55)4-3 = π=0.6 p(3) = (43) 0.63 (1-0.6)4-3 = π=0.65 p(3) = (43) 0.653 (1-0.65)4-3 = π=0.7 p(3) = (43) 0.73 (1-0.7)4-3 = π=0.75 p(3) = (43) 0.753 (1-0.75)4-3 =0.422 π=0.8 p(3) = (43) 0.83 (1-0.8)4-3 = π=0.85 p(3) = (43) 0.853 (1-0.85)4-3 = π=0.9 p(3) = (43) 0.93 (1-0.9)4-3 =0.292 π=0.95 p(3) = (43) 0.953 (1-0.95)4-3 =

p(x) = (---------) πx (1-π)n-x BINOMIAL DISTRIBUTION x!(n-x)! NUMBER OF MALES ON 4 DELIVERIES = 3 π = 0.514 p(0) = (40) 0.5140 (1-0.514)4-0 = 0.0558 p(1) = (41) 0.5141 (1-0.514)4-1 = 0.2360 p(2) = (42) 0.5142 (1-0.514)4-2 = 0.3744 p(3) = (43) 0.5143 (1-0.514)4-3 = 0.2640 p(4) = (44) 0.5140 (1-0.514)4-0 = 0.0698

Observed data: 3 male newborns out of 4 deliveries parameter

^ ^ Observed data: 3 male newborns out of 4 deliveries Maximum likelihood parameter Max likelihood = 0.422  = 0.75 ^ Maximum Likelihood Estimate (MLE),  (Stima di massima verosimiglianza) ^

^ ^ Observed data: 30 male newborns out of 40 deliveries Maximum likelihood parameter Max likelihood = 0.144  = 0.75 ^ ^ Maximum Likelihood Estimate (MLE),  (Stima di massima verosimiglianza)

(Profilo della verosimiglianza) Likelihood profile (Profilo della verosimiglianza) Observed data: 3 male newborns out of 4 deliveries Parameter (π) Observed data: 30 male newborns out of 40 deliveries Parameter (π)

less useful in Medical Statistics PROBABILITY p(x/π) easy to understand less useful in Medical Statistics Σ probabilities = 1 LIKELIHOOD l(π/x) difficult to understand more useful in Medical Statistics Σ likelihood > 1 (indeed every likelihood is computed using a different probability distribution)

DEVIANCE Saturated model Model under study n parameters = n data