Bayesian Networks Bucket Elimination Algorithm 主講人：虞台文 大同大學資工所 智慧型多媒體研究室

Content Basic Concept Belief Updating Most Probable Explanation (MPE) Maximum A Posteriori (MAP)

Bayesian Networks Bucket Elimination Algorithm Basic Concept 大同大學資工所 智慧型多媒體研究室

Satisfiability Given a statement of clauses (in disjunction normal form), the satisfiability problem is to determine whether there exists a truth assignment to make the statement true. Examples: A=True, B=True, C=False, D=False Satisfiable Satisfiable?

Resolution can be true if and only if can be true.   unsatisfiable

Direct Resolution Example: Given a set of clauses and an order d=ABCD Bucket A Bucket B Bucket C Bucket D Set initial buckets as follows:

Direct Resolution Bucket A Bucket B Bucket C Bucket D Because no empty clause (  ) is resulted, the statement is satisfiable. How to get a truth assignment?

Direct Resolution Bucket A Bucket B Bucket C Bucket D

Direct Resolution

Queries on Bayesian Networks Belief updating Finding the most probable explanation (mpe) – Given evidence, finding a maximum probability assignment to the rest of variables. Maximizing a posteriori hypothesis (map) – Given evidence, finding an assignment to a subset of hypothesis variables that maximize their probability. Maximizing the expected utility of the problem (meu) – Given evidence and utility function, finding a subset of decision variables that maximize the expected utility.

Bucket Elimination The algorithm will be used as a framework for various probabilistic inferences on Bayesian Networks.

Preliminary – Elimination Functions Given a function h defined over subset of variables S, where X  S, Eliminate parameter X from h Defined over U = S – {X}.

Preliminary – Elimination Functions Given a function h defined over subset of variables S, where X  S,

Preliminary – Elimination Functions Given function h 1,…, h n defined over subset of variables S 1,…, S n, respectively, Defined over

Preliminary – Elimination Functions Given function h 1,…, h n defined over subset of variables S 1,…, S n, respectively,

Bayesian Networks Bucket Elimination Algorithm Belief Updating 大同大學資工所 智慧型多媒體研究室

Goal Normalization Factor

Basic Concept of Variable Elimination Example: A B D C F G

Basic Concept of Variable Elimination Example:

Basic Concept of Variable Elimination G ( f ) D ( a, b ) F ( b, c ) B ( a, c ) C ( a )

Basic Concept of Variable Elimination Bucket G Bucket D Bucket F Bucket B Bucket C Bucket A

Basic Concept of Variable Elimination Bucket G Bucket D Bucket F Bucket B Bucket C Bucket A

Basic Concept of Variable Elimination f G (f ) +0.1  0.7

Basic Concept of Variable Elimination f G (f ) +0.1  0.7 ab D (a, b)

Basic Concept of Variable Elimination f G (f ) +0.1  0.7 ab D (a, b)  0.7  0.1  0.7  0.1  0.7  0.1  0.7  0.1 bc F (b, c)

Basic Concept of Variable Elimination f G (f ) +0.1  0.7 ab D (a, b) bc F (b, c) ac B (a, c)   0.400=   0.340=   0.400=   0.340=0.5020

Basic Concept of Variable Elimination f G (f ) +0.1  0.7 ab D (a, b) bc F (b, c) ac B (a, c) a C (a )   =   =

Basic Concept of Variable Elimination f G (f ) +0.1  0.7 ab D (a, b) bc F (b, c) ac B (a, c) a C (a ) aP(a, g=1)  =  = aP(a | g=1) / = / =

Bucket Elimination Algorithm

Complexity The BuckElim Algorithm can be applied to any ordering. The arity of the function recorded in a bucket – the numbers of variables appearing in the processed bucked, excluding the bucket’s variable. Time and Space complexity is exponentially grow with a function of arity r. The arity is dependent on the ordering. How many possible orderings for BN’s variables?

Determination of the Arity Bucket G Bucket B Bucket C Bucket D Bucket F Bucket A A B D C F G Consider the ordering AFDCBG. G B C D F A ,3,2,1

Determination of the Arity Given the ordering, e.g., AFDCBG G B C D F A G B C D F A A B D C F G Initial Graph Width of node Induced Graph Width of node d w(d): width of initial graph for ordering d. w*(d): width of induced graph for ordering d. w(d): width of initial graph for ordering d. w*(d): width of induced graph for ordering d. The width of a graph is the maximum width of its nodes. w(d) = 4w*(d) = 4

Definition of Tree-Width Goal: Finding an ordering with smallest induced width. NP -Hard Greedy heuristic and Approximation methods Are available.

Summary The complexity of BuckElim algorithm is dominated by the time and space needed to process a bucket. It is time and space is exponential in number of bucket variables. Induced width bounds the arity of bucket functions.

Exercises A B D C F G Use BuckElim to evaluate P(a|b=1) with the following two ordering: 1. d 1 =ACBFDG 2. d 2 =AFDCBG Give the details and make some conclusion. How to improve the algorithm?

Bayesian Networks Bucket Elimination Algorithm Most Probable Explanation (MPE) 大同大學資工所 智慧型多媒體研究室

MPE Goal: evidence

MPE Goal:

Notations xixi

MPE Let

MPE Some terms involve x n, some terms not. XnXn X n is conditioned by its parents. X n conditions its children.

MPE XnXn Not conditioned by x n Conditioned by x n Itself x n appears in these CPT’s

MPE Eliminate variable x n at Bucket n. Process the next bucket recursively.

Example A B D C F G

A B D C F G Bucket G Bucket D Bucket F Bucket B Bucket C Bucket A Consider ordering ACBFDG

Bucket Elimination Algorithm

Exercise Consider ordering ACBFDG

Bayesian Networks Bucket Elimination Algorithm Maximum A Posteriori (MAP) 大同大學資工所 智慧型多媒體研究室

MAP Given a belief network, a subset of hypothesized variables A=(A 1, …, A k ), and evidence E=e, the goal is to determine

Example A B D C F G Hypothesis (Decision) Variables g = 1

MAP Ordering Some of them may be observed

MAP

Bucket Elimination for belief updating Bucket Elimination for MPE

Bucket Elimination Algorithm

Example A B D C F G g = 1 Consider ordering CBAFDG Bucket G Bucket D Bucket F Bucket A Bucket B Bucket C

Exercise A B D C F G g = 1 Consider ordering CBAFDG Bucket G Bucket D Bucket F Bucket A Bucket B Bucket C Give the detail