Maximum Likelihood. The likelihood function is the simultaneous density of the observation, as a function of the model parameters. L(  ) = Pr(Data| 

Slides:



Advertisements
Similar presentations
Modeling of Data. Basic Bayes theorem Bayes theorem relates the conditional probabilities of two events A, and B: A might be a hypothesis and B might.
Advertisements

Basics of Statistical Estimation
Bioinformatics Phylogenetic analysis and sequence alignment The concept of evolutionary tree Types of phylogenetic trees Measurements of genetic distances.
Likelihood Ratio, Wald, and Lagrange Multiplier (Score) Tests
EM Algorithm Jur van den Berg.
Fundamentals of Data Analysis Lecture 12 Methods of parametric estimation.
Maximum Likelihood Estimation Navneet Goyal BITS, Pilani.
Probabilistic Modeling of Molecular Evolution Using Excel, AgentSheets, and R Jeff Krause (Shodor)
SOLVED EXAMPLES.
Solving Equations = 4x – 5(6x – 10) -132 = 4x – 30x = -26x = -26x 7 = x.
CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 10: The Bayesian way to fit models Geoffrey Hinton.
Likelihood methods Given a particular model of evolution, we can estimate phylogenies using maximum likelihood.
Phylogenetic Trees Lecture 4
Maximum likelihood Conditional distribution and likelihood Maximum likelihood estimations Information in the data and likelihood Observed and Fisher’s.
Basics of Statistical Estimation. Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability  is unknown.
A gentle introduction to Gaussian distribution. Review Random variable Coin flip experiment X = 0X = 1 X: Random variable.
Phylogenetic reconstruction
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference (Sec. )
Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.
CENTER FOR BIOLOGICAL SEQUENCE ANALYSIS Probabilistic modeling and molecular phylogeny Anders Gorm Pedersen Molecular Evolution Group Center for Biological.
CENTER FOR BIOLOGICAL SEQUENCE ANALYSIS Probabilistic modeling and molecular phylogeny Anders Gorm Pedersen Molecular Evolution Group Center for Biological.
CISC667, F05, Lec16, Liao1 CISC 667 Intro to Bioinformatics (Fall 2005) Phylogenetic Trees (III) Probabilistic methods.
Copyright © Cengage Learning. All rights reserved. 6 Point Estimation.
CENTER FOR BIOLOGICAL SEQUENCE ANALYSIS Probabilistic modeling and molecular phylogeny Anders Gorm Pedersen Molecular Evolution Group Center for Biological.
Maximum Likelihood Molecular Evolution. Maximum Likelihood The likelihood function is the simultaneous density of the observation, as a function of the.
CSE 221: Probabilistic Analysis of Computer Systems Topics covered: Statistical inference.
. Phylogenetic Trees Lecture 13 This class consists of parts of Prof Joe Felsenstein’s lectures 4 and 5 taken from:
Likelihood probability of observing the data given a model with certain parameters Maximum Likelihood Estimation (MLE) –find the parameter combination.
ECE 8443 – Pattern Recognition LECTURE 06: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Bias in ML Estimates Bayesian Estimation Example Resources:
Random Sampling, Point Estimation and Maximum Likelihood.
Logistic Regression Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata September 1, 2014.
Recitation on EM slides taken from:
CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.
Lecture 10 – Models of DNA Sequence Evolution Correct for multiple substitutions in calculating pairwise genetic distances. Derive transformation probabilities.
Statistical stuff: models, methods, and performance issues CS 394C September 16, 2013.
Lecture 12: Linkage Analysis V Date: 10/03/02  Least squares  An EM algorithm  Simulated distribution  Marker coverage and density.
CY3A2 System identification1 Maximum Likelihood Estimation: Maximum Likelihood is an ancient concept in estimation theory. Suppose that e is a discrete.
Maximum Likelihood Given competing explanations for a particular observation, which explanation should we choose? Maximum likelihood methodologies suggest.
Estimation. The Model Probability The Model for N Items — 1 The vector probability takes this form if we assume independence.
Week 41 How to find estimators? There are two main methods for finding estimators: 1) Method of moments. 2) The method of Maximum likelihood. Sometimes.
Maximum Likelihood Estimation
Statistical Estimation Vasileios Hatzivassiloglou University of Texas at Dallas.
Distribution Function properties Let us consider the experiment of tossing the coin 3 times and observing the number of heads The probabilities and the.
M.Sc. in Economics Econometrics Module I Topic 4: Maximum Likelihood Estimation Carol Newman.
CENTER FOR BIOLOGICAL SEQUENCE ANALYSIS Probabilistic modeling and molecular phylogeny Anders Gorm Pedersen Molecular Evolution Group Center for Biological.
Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability.
Bioinf.cs.auckland.ac.nz Juin 2008 Uncorrelated and Autocorrelated relaxed phylogenetics Michaël Defoin-Platel and Alexei Drummond.
Statistical stuff: models, methods, and performance issues CS 394C September 3, 2009.
Building Phylogenies Maximum Likelihood. Methods Distance-based Parsimony Maximum likelihood.
Maximum Likelihood Estimate Jyh-Shing Roger Jang ( 張智星 ) CSIE Dept, National Taiwan University.
Unit 6 Probability & Simulation: the Study of randomness Simulation Probability Models General Probability Rules.
Information Bottleneck versus Maximum Likelihood Felix Polyakov.
Building Phylogenies. Phylogenetic (evolutionary) trees Human Gorilla Chimp Gibbon Orangutan Describe evolutionary relationships between species Cannot.
Conditional Expectation
Fundamentals of Data Analysis Lecture 11 Methods of parametric estimation.
Chi Square Chi square is employed to test the difference between an actual sample and another hypothetical or previously established distribution such.
Data Modeling Patrice Koehl Department of Biological Sciences
Copyright © Cengage Learning. All rights reserved.
Maximum Likelihood Estimate
Probability Theory and Parameter Estimation I
CS 2750: Machine Learning Density Estimation
Ch3: Model Building through Regression
Classification of unlabeled data:
Packet #15 Exponential and Logarithmic Equations
Recitation 5 2/4/09 ML in Phylogeny
Implicit Differentiation
Linear regression Fitting a straight line to observations.
5.1 Introduction to Curve Fitting why do we fit data to a function?
A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.
A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models Jeff A. Bilmes International.
Presentation transcript:

Maximum Likelihood

The likelihood function is the simultaneous density of the observation, as a function of the model parameters. L(  ) = Pr(Data|  ) If the observations are independent, we can decompose the term into

An example Consider the estimation of heads probability of a coin tossed n times Heads probability p Data = HHTTHTHHTTT L(p) = Pr(D|p) = pp(1-p)(1-p)p(1-p)pp(1- p)(1-p)(1-p) = p 5 (1-p) 6

L(p) = p 5 (1-p) 6 = 5/11

Maximum Likelihood Take the derivative of L with respect to p: Equate it to zero and solve: p = 5/11 ^

Log Likelihood For computational reasons, we maximise the logarithm lnL = 5 lnp + 6 ln(1-p) with derivative p = 5/11 ^

A tree

Tree likelihood: Assumptions 1.Evolution in different sites is independent. 2.Evolution in different lineages is independent.

Pr(A,C,C,C,G,x,y,z,w|T) = Pr(x) Pr(y|x,t 6 ) Pr(A|y,t 1 ) Pr(C|y,t 2 ) Pr(z|x,t 8 ) Pr(C|z,t 3 ) Pr(w|z,t 7 ) Pr(C|w,t 4 ) Pr(G|w,t 5 )

Using models Observed differences Actual changes AG CT Example: Jukes-Cantor, if i=j, if i≠j p t : proportion of different nucleotides

30 nucleotides from  -globin genes of two primates on a one-edge tree * * Gorilla GAAGTCCTTGAGAAATAAACTGCACACTGG Orangutan GGACTCCTTGAGAAATAAACTGCACACTGG There are two differences and 28 similarities tt lnL  t= lnL=