Daphne Koller Variable Elimination Graph-Based Perspective Probabilistic Graphical Models Inference

Daphne Koller Initial Graph C D I SG L J H D C D I SG L J H D

Daphne Koller Elimination as Graph Operation Eliminate: C C D I SG L J H D Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: D C D I SG L J H D Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: I C D I SG L J H Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: H C D I SG L J H Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: G C D I SG L J H Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: L,S C D I SG L J H Induced Markov network for the current set of factors

Daphne Koller Elimination as Graph Operation Eliminate: L,S C D I SG L J H Induced Markov network for the current set of factors

Daphne Koller Induced Graph The induced graph I ,  over factors  and ordering  : – Undirected graph – X i and X j are connected if they appeared in the same factor in a run of the VE algorithm using  as the ordering C D I SG L J H D

Daphne Koller Cliques in the Induced Graph Theorem: Every factor produced during VE is a clique in the induced graph C D I SG L J H D

Daphne Koller Cliques in the Induced Graph Theorem: Every (maximal) clique in the induced graph is a factor produced during VE C D I SG L J H D

Daphne Koller Cliques in the Induced Graph Theorem: Every (maximal) clique in the induced graph is a factor produced during VE C D I SG L J H D

Daphne Koller Induced Width The width of an induced graph is the number of nodes in the largest clique in the graph minus 1 Minimal induced width of a graph K is min  (width(I K,  )) Provides a lower bound on best performance of VE to a model factorizing over K

Daphne Koller Finding Elimination Orderings Theorem: For a graph H, determining whether there exists an elimination ordering for H with induced width  K is NP-complete Note: This NP-hardness result is distinct from the NP-hardness result of inference – Even given the optimal ordering, inference may still be exponential

Daphne Koller Finding Elimination Orderings Greedy search using heuristic cost function – At each point, eliminate node with smallest cost Possible cost functions: – min-neighbors: # neighbors in current graph – min-weight: weight (# values) of factor formed – min-fill: number of new fill edges – weighted min-fill: total weight of new fill edges (edge weight = product of weights of the 2 nodes)

Daphne Koller Finding Elimination Orderings Theorem: The induced graph is triangulated – No loops of length > 3 without a “bridge” Can find elimination ordering by finding a low-width triangulation of original graph H  BD C A

Daphne Koller Robot Localization & Mapping Square Root SAM, F. Dellaert and M. Kaess, IJRR, 2006

Daphne Koller Robot Localization & Mapping x z x z 2 2 x z... t t x robot pose sensor observation L1L1 L2L2 L3L3 x z 3 3 x z 4 4

Daphne Koller Robot Localization & Mapping Square Root SAM, F. Dellaert and M. Kaess, IJRR, 2006

Daphne Koller Eliminate Poses then Landmarks Square Root SAM, F. Dellaert and M. Kaess, IJRR, 2006

Daphne Koller Eliminate Landmarks then Poses Square Root SAM, F. Dellaert and M. Kaess, IJRR, 2006

Daphne Koller Min-Fill Elimination Square Root SAM, F. Dellaert and M. Kaess, IJRR, 2006

Daphne Koller Summary Variable elimination can be viewed in terms of transformations on undirected graph – Eliminating Z connects its current neighbors Sizes of cliques in resulting induced graph directly correspond to algorithm’s complexity Keeping induced graph simple provides useful heuristics for selecting elimination ordering

Daphne Koller END END END