Download presentation

Presentation is loading. Please wait.

Published bySavanah Blazier Modified over 2 years ago

1
INTRODUCTION TO MACHINE LEARNING Bayesian Estimation

2
Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 Estimating parameters of a model from the data Regression Classification Have some prior knowledge on possible parameter range Before looking at the data Distribution of the parameter

3
Generative Model Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 3

4
Bayes Rule Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 4

5
Multinomial variable Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 5 Sample of multinomial data taking one of K state Sample Likelihood Good way to specify prior distribution on state probabilities q

6
Dirichlet Distribution Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 6 Probability of each combination of state probabilities Parameters: approximate proportions of data in state q i

7
Posteriori Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 7 Likelihood Posteriori

8
Conjugate Prior Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 8 Posteriori and prior have the same form Sequential learning Instance by instance Calculate posteriori for the current item Make it prior for the next item

9
Continuous Variable Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 9 Instances are Gaussian Distributed with unknown parameters Conjugate prior

10
Continuous Variable Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 10 Posteriori Mean is weighted combination of sample mean and prior mean More samples, estimate is closer to m Little prior uncertainty=>closer to prior mean

11
Precision/Variance Prior Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 11 More convenient to work with precision Conjugate prior is a Gamma Distribution

12
Precision Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 12 Posteriori is a weighted sum of prior and sample statistics

13
Parameter Estimation Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 13 Used prior to refine distribution parameter estimates User prior to refine parameter of some function of the input Regression Classification discriminant

14
Regression Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 14

15
Regression Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 15 Maximum Likelihood Prediction Gaussian Prior

16
Prior on weights Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 16

17
Examples 17

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google