Probability and Likelihood. Likelihood needed for many of ADMB’s features Standard deviation Variance-covariance matrix Profile likelihood Bayesian MCMC.

Slides:



Advertisements
Similar presentations
Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
Advertisements

Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 6 Point Estimation.
1 Parametric Empirical Bayes Methods for Microarrays 3/7/2011 Copyright © 2011 Dan Nettleton.
Sampling: Final and Initial Sample Size Determination
Estimation  Samples are collected to estimate characteristics of the population of particular interest. Parameter – numerical characteristic of the population.
SOLVED EXAMPLES.
Probability Distribution
1 Point and Interval Estimates Examples with z and t distributions Single sample; two samples Result: Sums (and differences) of normally distributed RV.
Review of Basic Probability and Statistics
Descriptive statistics Experiment  Data  Sample Statistics Sample mean Sample variance Normalize sample variance by N-1 Standard deviation goes as square-root.
Statistical Inference Chapter 12/13. COMP 5340/6340 Statistical Inference2 Statistical Inference Given a sample of observations from a population, the.
458 Fitting models to data – II (The Basics of Maximum Likelihood Estimation) Fish 458, Lecture 9.
Estimation of parameters. Maximum likelihood What has happened was most likely.
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.
Today Today: Chapter 9 Assignment: 9.2, 9.4, 9.42 (Geo(p)=“geometric distribution”), 9-R9(a,b) Recommended Questions: 9.1, 9.8, 9.20, 9.23, 9.25.
Estimating a Population Proportion
Copyright © Cengage Learning. All rights reserved. 6 Point Estimation.
Maximum likelihood (ML)
Modern Navigation Thomas Herring
Standard error of estimate & Confidence interval.
1 Bayesian methods for parameter estimation and data assimilation with crop models Part 2: Likelihood function and prior distribution David Makowski and.
Likelihood probability of observing the data given a model with certain parameters Maximum Likelihood Estimation (MLE) –find the parameter combination.
AP Statistics Chapter 9 Notes.
HW adjustment Q70 solve only parts a, b, and d. Q76 is moved to the next homework and added-Q2 is moved to the next homework as well.
Chapter 14 Monte Carlo Simulation Introduction Find several parameters Parameter follow the specific probability distribution Generate parameter.
Bayesian inference review Objective –estimate unknown parameter  based on observations y. Result is given by probability distribution. Bayesian inference.
Estimating parameters in a statistical model Likelihood and Maximum likelihood estimation Bayesian point estimates Maximum a posteriori point.
Multinomial Distribution
Estimating a Population Proportion
Generalized Linear Models All the regression models treated so far have common structure. This structure can be split up into two parts: The random part:
R Introduction and Training Patrick Gurian, Drexel University CAMRA 1st QMRA Summer Institute August 7, 2006.
ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: Conjugate Priors Multinomial Gaussian MAP Variance Estimation Example.
Maximum Likelihood - "Frequentist" inference x 1,x 2,....,x n ~ iid N( ,  2 ) Joint pdf for the whole random sample Maximum likelihood estimates.
Chapter 7 Sampling and Sampling Distributions ©. Simple Random Sample simple random sample Suppose that we want to select a sample of n objects from a.
: Chapter 3: Maximum-Likelihood and Baysian Parameter Estimation 1 Montri Karnjanadecha ac.th/~montri.
1 Topic 5 - Joint distributions and the CLT Joint distributions –Calculation of probabilities, mean and variance –Expectations of functions based on joint.
Statistics. A two-dimensional random variable with a uniform distribution.
1 Standard error Estimated standard error,s,. 2 Example 1 While measuring the thermal conductivity of Armco iron, using a temperature of 100F and a power.
Point Estimation of Parameters and Sampling Distributions Outlines:  Sampling Distributions and the central limit theorem  Point estimation  Methods.
Statistical Estimation Vasileios Hatzivassiloglou University of Texas at Dallas.
M.Sc. in Economics Econometrics Module I Topic 4: Maximum Likelihood Estimation Carol Newman.
Bernoulli Trials, Geometric and Binomial Probability models.
Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability.
Review of statistical modeling and probability theory Alan Moses ML4bio.
A Joint Distribution of Discrete Random Variables X Marginal Y Distribution Of Y Marginal Distribution.
Bayesian Approach Jake Blanchard Fall Introduction This is a methodology for combining observed data with expert judgment Treats all parameters.
Statistical NLP: Lecture 4 Mathematical Foundations I: Probability Theory (Ch2)
Ch 2. Probability Distributions (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, Summarized by Joo-kyung Kim Biointelligence Laboratory,
Anders Nielsen Technical University of Denmark, DTU-Aqua Mark Maunder Inter-American Tropical Tuna Commission An Introduction.
Estimating Uncertainty. Estimating Uncertainty in ADMB Model parameters and derived quantities Normal approximation Profile likelihood Bayesian MCMC Bootstrap.
Introduction We consider the data of ~1800 phenotype measurements Each mouse has a given probability distribution of descending from one of 8 possible.
Bayesian Estimation and Confidence Intervals Lecture XXII.
Crash course in probability theory and statistics – part 2 Machine Learning, Wed Apr 16, 2008.
Data Modeling Patrice Koehl Department of Biological Sciences
Chapter Six Normal Curves and Sampling Probability Distributions
Maximum Likelihood Estimation
Maximum Likelihood Find the parameters of a model that best fit the data… Forms the foundation of Bayesian inference Slide 1.
Probability & Statistics Probability Theory Mathematical Probability Models Event Relationships Distributions of Random Variables Continuous Random.
More about Posterior Distributions
Confidence Intervals for a Population Mean, Standard Deviation Known
Covered only ML estimator
EC 331 The Theory of and applications of Maximum Likelihood Method
Statistical NLP: Lecture 4
Parametric Empirical Bayes Methods for Microarrays
LECTURE 07: BAYESIAN ESTIMATION
Parametric Methods Berlin Chen, 2005 References:
Multivariate Methods Berlin Chen, 2005 References:
Bernoulli Trials Two Possible Outcomes Trials are independent.
Presentation transcript:

Probability and Likelihood

Likelihood needed for many of ADMB’s features Standard deviation Variance-covariance matrix Profile likelihood Bayesian MCMC Random effects See Hilborn and Mangel 1997 for a simple introduction See Pawitan 2001 for a comprehensive description

Probability distributions Probability of an event given a probability distribution Probability distribution defined by its form and the values of its parameters

Use of probability distributions Gambling, working out what is the best bet in a game of cards

What we desire The probability of a parameter given the information (data) we have (observed)

Likelihood: compare the probability of the observed data under different values of the parameter The outcome 3 is more probable if the true parameter value is 0.6

Likelihood: a numerical quantity to express the order of preference of values of the parameter MLE

Normal distribution maximum likelihood (one data point) Likelihood -ln(Likelihood) -ln(Likelihood) without constants -ln(Likelihood) without constants, σ known

Joint likelihood: Combining multiple data sets Share the parameter values for each data set Estimate the parameters while maximizing the combined likelihood (assuming independence) Think: Bernoulli → Binomial But, with the possibility of combining different likelihood functions

Using Likelihoods PARAMETER_SECTION. init_number sigma. PROCEDURE_SECTION pred_y=a+b*x; f=nobs*log(sigma) +0.5*sum(square((pred_y-y)/sigma));

.pin file #a 4 #b 2 #init_number sigma 1.5

Standard deviation file (*.std) index name value std dev 1 a e e-01 2 b e e-01 3 sigma e e-01

Correlation Matrix (*.COR) index name value std dev a4.0782e e b e e sigma e e