1. Markov Random Fields & Conditional Random Fields
John Winn MSR Cambridge

2. Road map Markov Random Fields Conditional Random Fields What they are
Uses in vision/object recognition
Advantages
Difficulties
Conditional Random Fields
Further difficulties

3. Markov Random Fields X1 X2
12 X3
23 X4
234

4. Examples of use in vision
Grid-shaped MRFs for pixel labelling e.g. segmentation
MRFs (e.g. stars) over part positions for pictorial structures/constellation models.

5. Advantages Probabilistic model: Undirected model
Captures uncertainty
No ‘irreversible’ decisions
Iterative reasoning
Principled fusing of different cues
Undirected model
Allows ‘non-causal’ relationships (soft constraints)
Efficient algorithms: inference now practical for MRFs with millions variables – can be applied to raw pixels.

6. Maximum Likelihood Learning
Add partition function back in
Sufficient statistics of data
Expected model sufficient statistics

7. Difficulty I: Inference
Exact inference intractable except in a few cases e.g. small models
Must resort to approximate methods
Loopy belief propagation
MCMC sampling
Alpha expansion (MAP solution only)

8. Difficulty II: Learning
Gradient descent – vulnerable to local minima
Slow – must perform expensive inference at each iteration.
Can stop inference early…
Contrastive divergence
Piecewise training + variants
Need fast + accurate methods

9. Difficulty III: Large cliques
For images, we want to look at patches not pairs of pixels. Therefore would like to use large cliques.
Cost of inference (memory and CPU) typically exponential in clique size.
Example: Field of Experts, Black + Roth
Training: contrastive divergence over a week on a cluster of 50+ machines
Test: Gibbs sampling very slow?

10. Other MRF issues…
Local minima when performing inference in high-dimensional latent spaces
MRF models often require making inaccurate independence assumptions about the observations.

11. Conditional Random Fields
Lafferty et al., 2001
12
23 X1 X2 X3
234 I X4

12. Examples of use in vision
Grid-shaped CRFs for pixel labelling (e.g. segmentation), using boosted classifiers.

13. Difficulty IV: CRF Learning
Sufficient statistics of labels given the image
Expected sufficient statistics given the image

14. Difficulty V: Scarcity of labels
CRF is a conditional model – needs labels.
Labels are expensive + increasingly hard to define.
Labels are also inherently lower dimensional than the data and hence support learning fewer parameters than generative models.