Section 2 Appendix Tensor Notation for BP 1

In section 2, BP was introduced with a notation which defined messages and beliefs as functions. This Appendix includes an alternate (and very concise) notation for the Belief Propagation algorithm using tensors.

Tensor Notation Tensor multiplication: Tensor marginalization: 3

Tensor Notation 4 A real function with r keyword arguments Axis-labeled array with arbitrary indices Database with column headers A rank-r tensor is… == X XYvalue 1red12 2red20 1blue18 2blue30 Yvalu e red4 blue6 Tensor multiplication: (vector outer product) Y redblue X Y red4 blue6 Xvalu e 13 25

Tensor Notation 5 A real function with r keyword arguments Axis-labeled array with arbitrary indices Database with column headers A rank-r tensor is… == X a3 b5 Xvalu e a4 b6 Tensor multiplication: (vector pointwise product) Xvalu e a3 b5 X a4 b6 X a12 b30 Xvalu e a12 b30

Tensor Notation 6 A real function with r keyword arguments Axis-labeled array with arbitrary indices Database with column headers A rank-r tensor is… == Tensor multiplication: (matrix-vector product) Xvalu e X Y red blue X Y red blue X XY value 1red blu e XY value 1red21 2red40 1blu e 28 2blu e 48

Tensor Notation 7 A real function with r keyword arguments Axis-labeled array with arbitrary indices Database with column headers A rank-r tensor is… == Y redblue X Y redblue 810 XYvalue 1red blue4 2 6 Yvalu e red8 blue10 Tensor marginalization:

Input: a factor graph with no cycles Output: exact marginals for each variable and factor Algorithm: 1.Initialize the messages to the uniform distribution. 1.Choose a root node. 2.Send messages from the leaves to the root. Send messages from the root to the leaves. 1.Compute the beliefs (unnormalized marginals). 2.Normalize beliefs and return the exact marginals. Sum-Product Belief Propagation 8

9 Beliefs Messages VariablesFactors X2X2 ψ1ψ1 X1X1 X3X3 X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 X2X2 ψ1ψ1 X1X1 X3X3

Sum-Product Belief Propagation 10 Beliefs Messages VariablesFactors X2X2 ψ1ψ1 X1X1 X3X3 X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 X2X2 ψ1ψ1 X1X1 X3X3

X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 Sum-Product Belief Propagation 11 v.4 n 6 p 0 Variable Belief

X1X1 ψ2ψ2 ψ3ψ3 ψ1ψ1 Sum-Product Belief Propagation 12 Variable Message

Sum-Product Belief Propagation 13 Factor Belief ψ1ψ1 X1X1 X3X3 vn p d 180 n 00

Sum-Product Belief Propagation 14 Factor Message ψ1ψ1 X1X1 X3X3

Input: a factor graph with cycles Output: approximate marginals for each variable and factor Algorithm: 1.Initialize the messages to the uniform distribution. 1.Send messages until convergence. Normalize them when they grow too large. 1.Compute the beliefs (unnormalized marginals). 2.Normalize beliefs and return the approximate marginals. Loopy Belief Propagation 15