1. Introduction on Graphic Models
Jin Mao
Postdoc, School of Information, University of Arizona
Jan 20, 2015

2. Outline Probabilistic Graphical Models Bayesian Network Markov Network
Factor & Factor Graph

3. Probabilistic Graphical Models
Definition
Probabilistic graphical models use a graph-based representation as the basis for compactly encoding a complex distribution over a high-dimensional space.
Nodes: The Variables
Edges: Direct probabilistic interactions between the variables

4. Probabilistic Graphical Models
Features
Representationtransparent, in that a human expert can understand and evaluate its semantics and properties.
Inferenceposterior probability based on the structure.
Learningconstruct the model from observed data.

5. Probabilistic Graphical Models
Types
Bayesian Networks, directed
Markov Networks , indirected

6. Bayesian Networks
Definition
Bayesian networks (BNs) are graphical models for reasoning under uncertainty.
The nodes in a Bayesian network represent a set of random variables, X = X1,.. Xi,.. Xn, from the domain.

7. Bayesian Networks
Definition
Bayesian networks (BNs) are graphical models for reasoning under uncertainty.
The nodes in a Bayesian network represent a set of random variables, X = X1,.. Xi,.. Xn, from the domain.
Types of nodes:
Boolean: true/false
Ordered: low/medium/high
Integral: 1- 20 …

8. directed acyclic graphs, or simply dags.
Bayesian Networks
Definition
A set of directed arcs (or links) connects pairs of nodes, Xi Xj, representing the direct dependencies between variables.
Assuming discrete variables, the strength of the relationship between variables is quantified by conditional probability distributions associated with each node: P(Xj|Xi).
directed acyclic graphs, or simply dags.

9. Model by Knowledge Engineer
Bayesian Networks
Model by Knowledge Engineer
Identify variables of interests;
Define the states of variables:
both mutually exclusive and exhaustive.
To choose values that represent the domain efficiently, but with enough detail to perform the reasoning required;
Design the structure/topology.
Determine conditional probabilities.
Use it for prediction

10. Bayesian Networks
An example

11. Bayesian Networks
An example

12. Structure Terminology
Bayesian Networks
Structure Terminology
a parent of a child
an ancestor of a descendant
Root node, leaf node, intermediate node
Markov blanket
the node’s parents, its children, and its children’s parents.

13. Conditional Probabilities
Bayesian Networks
Conditional Probabilities
A conditional probability table (CPT), discrete variables.
First, look at all the possible combinations of values of those parent nodes. (instantiation)
For each distinct instantiation of parent node values, we need to specify the probability that the child will take each of its values by some means.
Pollution and Smoking and take the possible joint values < H;T >;< H;F >;< L;T >; < L;F >
< 0:05;0:02;0:03;0:001 >
Root nodes---prior probablities

14. Independence-maps (or, I-maps for short)
Bayesian Networks
The Markov property
there are no direct dependencies in the system being modeled which are not already explicitly shown via arcs.
Independence-maps (or, I-maps for short)
it is not generally required that the arcs in a BN
correspond to real dependencies in the system.
Redundant edges, minimal I-maps
Every arc happens to correspond to a real direct dependence---Dependence-Map
Both D-map and I-map, A perfect map.
no hidden “backdoor”

15. diagnostic reasoning:
Bayesian Networks
Types of reasoning
diagnostic reasoning:
reasoning from symptoms to cause, the opposite direction to the network arcs.

16. Predictive reasoning:
Bayesian Networks
Types of reasoning
Predictive reasoning:
reasoning from new information about causes to new beliefs about effects, following the directions of the network arcs.

17. intercausal reasoning: the mutual causes of a common effect
Bayesian Networks
Types of reasoning
intercausal reasoning:
the mutual causes of a common effect

18. Specific Evidence: observe a specific value
Bayesian Networks
Types of Evidence
Specific Evidence: observe a specific value
Negative Evidence: not a value
Virtual evidence, likelihood evidence, not so sure.
Suppose, for example, that the radiologist who has taken and analyzed the Xray in our cancer example is uncertain. He thinks that the X-ray looks positive, but is only 80% sure.

19. Bayesian Networks
Learning

20. AKA Markov random field (MRF)
Markov Networks
Definition
Undirected graphs can also be used to represent dependency relationships.
Useful in modeling domains where the interactions between the variables seem symmetrical and one cannot naturally ascribe a directionality to the interaction between variables
AKA Markov random field (MRF)

21. AKA Markov random field (MRF)
Markov Networks
Definition
Undirected graphs can also be used to represent dependency relationships.
Useful in modeling domains where the interactions between the variables seem symmetrical and one cannot naturally ascribe a directionality to the interaction between variables
AKA Markov random field (MRF)

22. Markov Networks Moralization: Moral graph of G
From BN to MN
Moralization:
Start with the DAG G, add an edge between every variable and each of its spouses, and finally drop the directionality of all edges in DAG G.
Moral graph of G
The moralization process ensures that every variable in DAG G will be connected to each variable in its Markov Blanket

23. Factor & Factor Graph A Factor(potential)
Factors
A Factor(potential)
A factor over a set of variables X is a function which maps each instantiation x of variables X to a non-negative real number, denoted.
The variables in a factor often
have dependency relationships.
The result of multiplying factors is
another factor.

24. Factor & Factor Graph
Factorization

25. Factor & Factor Graph
Factor Graph
A factor graph is an undirected graph containing two types of nodes: variable nodes and factor nodes.
The graph only contains edges between variable nodes and factor nodes.
Each factor node Vφ is associated with precisely one factor φ, whose scope is the set of neighbors of Vφ.

26. Factor & Factor Graph
Why Factor Graph
They are used extensively for breaking down a problem into pieces.
“We can simplify computations based on how variables are related to these factors. We’ll break up the joint distribution into a bunch of factors on a graph. ” (read:

27. Thank you!