Download presentation
Presentation is loading. Please wait.
1
ETHEM ALPAYDIN © The MIT Press, 2010 alpaydin@boun.edu.tr http://www.cmpe.boun.edu.tr/~ethem/i2ml2e Lecture Slides for
3
Rationale Bayes’ Rule: Generative model: 3Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)
4
Estimating the Parameters of a Distribution: Discrete case x t i =1 if in instance t is in state i, probability of state i is q i Dirichlet prior, i are hyperparameters Sample likelihood Posterior Dirichlet is a conjugate prior With K=2, Dirichlet reduced to Beta 4 Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)
5
5 Estimating the Parameters of a Distribution: Continuous case p(x t )~N( , 2 ) Gaussian prior for , p( )~ N( , 2 ) Posterior is also Gaussian p( X)~ N ( N, N 2 ) where
6
6Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 6 Estimating the Parameters of a Function: Regression r=w T x+ where p( )~N(0,1/ ), and p(r t |x t,w, ) ~ N(w T x t, 1/ ) Log likelihood ML solution Gaussian conjugate prior: p(w)~N(0,1/ ) Posterior: p(w|X)~N( N N where
7
7Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)
8
8 8 8 Basis/Kernel Functions For new x’, the estimate r’ is calculated as Linear kernel For any other (x), we can write K(x’,x)= (x’) T (x) Dual representation
9
Kernel Functions 9Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0)
10
10 Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 10 Gaussian Processes Assume Gaussian prior p(w)~N(0,1/ ) y=Xw, where E[y]=0 and Cov(y)=K with K ij = (x i ) T x i K is the covariance function, here linear With basis function (x), K ij = ( (x i )) T (x i ) r~N N (0,C N ) where C N = (1/ )I+K With new x’ added as x N+1, r N+1 ~N N+1 (0,C N+1 ) where k = [K(x’,x t ) t ] T and c=K(x’,x’)+1/ . p(r’|x’,X,r)~N(k T C N-1 r,c-k T C N-1 k)
11
11
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.