1. Learning: Parameter Estimation
Eran Segal
Weizmann Institute

2. Learning Introduction
So far, we assumed that the networks were given
Where do the networks come from?
Knowledge engineering with aid of experts
Automated construction of networks
Learn by examples or instances

3. Learning Introduction
Input: dataset of instances D={d[1],...d[m]}
Output: Bayesian network
Measures of success
How close is the learned network to the original distribution
Use distance measures between distributions
Often hard because we do not have the true underlying distribution
Instead, evaluate performance by how well the network predicts new unseen examples (“test data”)
Classification accuracy
How close is the structure of the network to the true one?
Use distance metric between structures
Hard because we do not know the true structure
Instead, ask whether independencies learned hold in test data

4. Prior Knowledge Prespecified structure Prespecified variables
Learn only CPDs
Prespecified variables
Learn network structure and CPDs
Hidden variables
Learn hidden variables, structure, and CPDs
Complete/incomplete data
Missing data
Unobserved variables

5. Learning Bayesian NetworksX1 X2
Inducer
Data
Prior information Y
P(Y|X1,X2) X1 X2 y0 y1
x10
x20 1
x21
0.2
0.8
x11
0.1
0.9
0.02
0.98

6. Known Structure, Complete Data
Goal: Parameter estimation
Data does not contain missing values X1 X2 X1 X2
Inducer
Initial network Y Y X1 X2 Y
x10
x21 y0
x11
x20 y1
P(Y|X1,X2) X1 X2 y0 y1
x10
x20 1
x21
0.2
0.8
x11
0.1
0.9
0.02
0.98
Input Data

7. Unknown Structure, Complete Data
Goal: Structure learning & parameter estimation
Data does not contain missing values X1 X2 X1 X2
Inducer
Initial network Y Y X1 X2 Y
x10
x21 y0
x11
x20 y1
P(Y|X1,X2) X1 X2 y0 y1
x10
x20 1
x21
0.2
0.8
x11
0.1
0.9
0.02
0.98
Input Data

8. Known Structure, Incomplete Data
Goal: Parameter estimation
Data contains missing values
Example: Naïve Bayes X1 X2 X1 X2
Inducer
Initial network Y Y X1 X2 Y ?
x21 y0
x11
x10
x20 y1
P(Y|X1,X2) X1 X2 y0 y1
x10
x20 1
x21
0.2
0.8
x11
0.1
0.9
0.02
0.98
Input Data

9. Unknown Structure, Incomplete Data
Goal: Structure learning & parameter estimation
Data contains missing values X1 X2 X1 X2
Inducer
Initial network Y Y X1 X2 Y ?
x21 y0
x11
x10
x20 y1
P(Y|X1,X2) X1 X2 y0 y1
x10
x20 1
x21
0.2
0.8
x11
0.1
0.9
0.02
0.98
Input Data

10. Parameter Estimation Input Goal: Learn CPD parameters
Network structure
Choice of parametric family for each CPD P(Xi|Pa(Xi))
Goal: Learn CPD parameters
Two main approaches
Maximum likelihood estimation
Bayesian approaches

11. Biased Coin Toss Example
Coin can land in two positions: Head or Tail
Estimation task
Given toss examples x[1],...x[m] estimate P(H)= and P(T)=1-
Assumption: i.i.d samples
Tosses are controlled by an (unknown) parameter 
Tosses are sampled from the same distribution
Tosses are independent of each other

12. Biased Coin Toss Example
Goal: find [0,1] that predicts the data well
“Predicts the data well” = likelihood of the data given 
Example: probability of sequence H,T,T,H,H
0.2
0.4
0.6
0.8 1 
L(D:)

13. Maximum Likelihood Estimator
Parameter  that maximizes L(D:)
In our example, =0.6 maximizes the sequence H,T,T,H,H
0.2
0.4
0.6
0.8 1 
L(D:)

14. Maximum Likelihood Estimator
General case
Observations: MH heads and MT tails
Find  maximizing likelihood
Equivalent to maximizing log-likelihood
Differentiating the log-likelihood and solving for  we get that the maximum likelihood parameter is:

15. Sufficient Statistics
For computing the parameter  of the coin toss example, we only needed MH and MT since
 MH and MT are sufficient statistics

16. Sufficient Statistics
A function s(D) is a sufficient statistics from instances to a vector in k if for any two datasets D and D’ and any  we have
Datasets
Statistics

17. Sufficient Statistics for Multinomial
A sufficient statistics for a dataset D over a variable Y with k values is the tuple of counts <M1,...Mk> such that Mi is the number of times that the Y=yi in D
Sufficient statistic
Define s(x[i]) as a tuple of dimension k
s(x[i])=(0,...0,1,0,...,0)
(1,...,i-1)
(i+1,...,k)

18. Sufficient Statistic for Gaussian
Gaussian distribution:
Rewrite as
 sufficient statistics for Gaussian: s(x[i])=<1,x,x2>

19. Maximum Likelihood Estimation
MLE Principle: Choose  that maximize L(D:)
Multinomial MLE:
Gaussian MLE:

20. MLE for Bayesian Networks
Parameters
x0, x1
y0|x0, y1|x0, y0|x1, y1|x1
Data instance tuple: <x[m],y[m]>
Likelihood X x0 x1
0.7
0.3 X Y Y X y0 y1 x0
0.95
0.05 x1
0.2
0.8
 Likelihood decomposes into two separate terms, one for each variable

21. MLE for Bayesian Networks
Terms further decompose by CPDs:
By sufficient statistics
where M[x0,y0] is the number of data instances in which X takes the value x0 and Y takes the value y0
MLE

22. MLE for Bayesian Networks
Likelihood for Bayesian network
 if Xi|Pa(Xi) are disjoint then MLE can be computed by maximizing each local likelihood separately

23. MLE for Table CPD BayesNets
Multinomial CPD
For each value xX we get an independent multinomial problem where the MLE is

24. MLE for Tree CPDs Assume tree CPD with known tree structure Y Z X
Terms for <y0,z0> and <y0,z1> can be combined Y y0 y1 Z
x0|y0,x1|y0 z0 z1
Optimization can be done by leaves
x0|y1,z1,x1|y1,z1
x0|y1,z1,x1|y1,z1

25. MLE for Tree CPD BayesNets
Tree CPD T, leaves l
For each value lLeaves(T) we get an independent multinomial problem where the MLE is

26. Limitations of MLE
Two teams play 10 times, and the first wins 7 of the 10 matches  Probability of first team winning = 0.7
A coin is tosses 10 times, and comes out ‘head’ 7 of the 10 tosses  Probability of head = 0.7
Would you place the same bet on the next game as you would on the next coin toss?
We need to incorporate prior knowledge
Prior knowledge should only be used as a guide

27. Bayesian Inference … Assumptions
Given a fixed  tosses are independent
If  is unknown tosses are not marginally independent – each toss tells us something about 
The following network captures our assumptions 
X[1]
X[M]
X[2] …

28. For a uniform prior, posterior is the normalized likelihood
Bayesian Inference
Joint probabilistic model
Posterior probability over   …
X[1]
X[2]
X[M]
Likelihood
Prior
For a uniform prior, posterior is the normalized likelihood
Normalizing factor

29. Bayesian Prediction Predict the data instance from the previous ones
Solve for uniform prior P()=1 and binomial variable

30. Example: Binomial Data
Prior: uniform for  in [0,1]
P() = 1
 P( |D) is proportional to the likelihood L(D:)
MLE for P(X=H) is 4/5 = 0.8
Bayesian prediction is 5/7
0.2
0.4
0.6
0.8 1
(MH,MT ) = (4,1)

31. Dirichlet Priors
A Dirichlet prior is specified by a set of (non-negative) hyperparameters 1,...k so that
 ~ Dirichlet(1,...k) if
where and
Intuitively, hyperparameters correspond to the number of imaginary examples we saw before starting the experiment

32. Dirichlet Priors – Example5
Dirichlet(1,1)
Dirichlet(2,2)
4.5
Dirichlet(0.5,0.5)
Dirichlet(5,5) 4
3.5 3
2.5 2
1.5 1
0.5
0.2
0.4
0.6
0.8 1

33. Dirichlet Priors
Dirichlet priors have the property that the posterior is also Dirichlet
Data counts are M1,...,Mk
Prior is Dir(1,...k)
 Posterior is Dir(1+M1,...k+Mk)
The hyperparameters 1,…,K can be thought of as “imaginary” counts from our prior experience
Equivalent sample size = 1+…+K
The larger the equivalent sample size the more confident we are in our prior

34. Effect of Priors
Prediction of P(X=H) after seeing data with MH=1/4MT as a function of the sample size
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
0.55 20 40 60 80
100
0.1
0.6
Different strength H + T
Fixed ratio H / T
Fixed strength H + T
Different ratio H / T

35. Effect of Priors (cont.)
In real data, Bayesian estimates are less sensitive to noise in the data
0.7
MLE
Dirichlet(.5,.5)
Dirichlet(1,1)
Dirichlet(5,5)
Dirichlet(10,10)
0.6
0.5
P(X = 1|D)
0.4
0.3
0.2 N
0.1 5 10 15 20 25 30 35 40 45 50 1
Toss Result N

36. General Formulation Joint distribution over D,
Posterior distribution over parameters
P(D) is the marginal likelihood of the data
As we saw, likelihood can be described compactly using sufficient statistics
We want conditions in which posterior is also compact

37. Conjugate Families
A family of priors P(:) is conjugate to a model P(|) if for any possible dataset D of i.i.d samples from P(|) and choice of hyperparameters  for the prior over , there are hyperparameters ’ that describe the posterior, i.e., P(:’) P(D|)P(:)
Posterior has the same parametric form as the prior
Dirichlet prior is a conjugate family for the multinomial likelihood
Conjugate families are useful since:
Many distributions can be represented with hyperparameters
They allow for sequential update within the same representation
In many cases we have closed-form solutions for prediction

38. Bayesian Estimation in BayesNets
Bayesian network for parameter estimation
Bayesian network X X …
X[1]
X[2]
X[M] … Y
Y[1]
Y[2]
Y[M]
Y|X
Instances are independent given the parameters
(x[m’],y[m’]) are d-separated from (x[m],y[m]) given 
Priors for individual variables are a priori independent
Global independence of parameters

39. Bayesian Estimation in BayesNets
Bayesian network for parameter estimation
Bayesian network X X …
X[1]
X[2]
X[M] … Y
Y[1]
Y[2]
Y[M]
Y|X
Posteriors of  are independent given complete data
Complete data d-separates parameters for different CPDs
As in MLE, we can solve each estimation problem separately

40. Bayesian Estimation in BayesNets
Bayesian network for parameter estimation
Bayesian network X X …
X[1]
X[2]
X[M] … Y
Y[1]
Y[2]
Y[M]
Y|X=0
Y|X=1
Posteriors of  are independent given complete data
Also holds for parameters within families
Note context specific independence between Y|X=0 and Y|X=1 when given both X and Y

41. Bayesian Estimation in BayesNets
Bayesian network for parameter estimation
Bayesian network X X …
X[1]
X[2]
X[M] … Y
Y[1]
Y[2]
Y[M]
Y|X=0
Y|X=1
Posteriors of  can be computed independently
For multinomial Xi|pai posterior is Dirichlet with parameters Xi=1|pai+M[Xi=1|pai],..., Xi=k|pai+M[Xi=k|pai]

42. Assessing Priors for BayesNets
We need the (xi,pai) for each node xi
We can use initial parameters 0 as prior information
Need also an equivalent sample size parameter M’
Then, we let (xi,pai) = M’  P(xi,pai|0)
This allows to update a network using new data
Example network for priors
P(X=0)=P(X=1)=0.5
P(Y=0)=P(Y=1)=0.5
M’=1
Note: (x0)=0.5 (x0,y0)=0.25 X Y

43. Case Study: ICU Alarm Network
The “Alarm” network
37 variables
Experiment
Sample instances
Learn parameters
MLE
Bayesian
PCWP CO
HRBP
HREKG
HRSAT
ERRCAUTER HR
HISTORY
CATECHOL
SAO2
EXPCO2
ARTCO2
VENTALV
VENTLUNG
VENITUBE
DISCONNECT
MINVOLSET
VENTMACH
KINKEDTUBE
INTUBATION
PULMEMBOLUS
PAP
SHUNT
ANAPHYLAXIS
MINOVL
PVSAT
FIO2
PRESS
INSUFFANESTH
TPR
LVFAILURE
ERRBLOWOUTPUT
STROEVOLUME
LVEDVOLUME
HYPOVOLEMIA
CVP BP

44. Case Study: ICU Alarm Network
MLE
1.4
Bayes w/ Uniform Prior, M'=5
Bayes w/ Uniform Prior, M'=10
1.2
Bayes w/ Uniform Prior, M'=20
Bayes w/ Uniform Prior, M'=50 1
0.8
KL Divergence
0.6
0.4
0.2
500
1000
1500
2000
2500
3000
3500
4000
4500
5000 M
MLE performs worst
Prior M’=5 provides best smoothing

45. Parameter Estimation Summary
Estimation relies on sufficient statistics
For multinomials these are of the form M[xi,pai]
Parameter estimation
Bayesian methods also require choice of priors
MLE and Bayesian are asymptotically equivalent
Both can be implemented in an online manner by accumulating sufficient statistics
MLE
Bayesian (Dirichlet)