Markov Random Fields & Conditional Random Fields John Winn MSR Cambridge

Road map Markov Random Fields Conditional Random Fields What they are Uses in vision/object recognition Advantages Difficulties Conditional Random Fields Further difficulties

Markov Random Fields X1 X2 12 X3 23 X4 234

Examples of use in vision Grid-shaped MRFs for pixel labelling e.g. segmentation MRFs (e.g. stars) over part positions for pictorial structures/constellation models.

Advantages Probabilistic model: Undirected model Captures uncertainty No ‘irreversible’ decisions Iterative reasoning Principled fusing of different cues Undirected model Allows ‘non-causal’ relationships (soft constraints) Efficient algorithms: inference now practical for MRFs with millions variables – can be applied to raw pixels.

Maximum Likelihood Learning Add partition function back in Sufficient statistics of data Expected model sufficient statistics

Difficulty I: Inference Exact inference intractable except in a few cases e.g. small models Must resort to approximate methods Loopy belief propagation MCMC sampling Alpha expansion (MAP solution only)

Difficulty II: Learning Gradient descent – vulnerable to local minima Slow – must perform expensive inference at each iteration. Can stop inference early… Contrastive divergence Piecewise training + variants Need fast + accurate methods

Difficulty III: Large cliques For images, we want to look at patches not pairs of pixels. Therefore would like to use large cliques. Cost of inference (memory and CPU) typically exponential in clique size. Example: Field of Experts, Black + Roth Training: contrastive divergence over a week on a cluster of 50+ machines Test: Gibbs sampling very slow?

Other MRF issues… Local minima when performing inference in high-dimensional latent spaces MRF models often require making inaccurate independence assumptions about the observations.

Conditional Random Fields Lafferty et al., 2001 12 23 X1 X2 X3 234 I X4

Examples of use in vision Grid-shaped CRFs for pixel labelling (e.g. segmentation), using boosted classifiers.

Difficulty IV: CRF Learning Sufficient statistics of labels given the image Expected sufficient statistics given the image

Difficulty V: Scarcity of labels CRF is a conditional model – needs labels. Labels are expensive + increasingly hard to define. Labels are also inherently lower dimensional than the data and hence support learning fewer parameters than generative models.