Download presentation

Presentation is loading. Please wait.

Published byChrista Ogburn Modified over 2 years ago

1
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220: Reasoning and Decision under Uncertainty L09: Graphical Models for Decision Problems Nevin L. Zhang Room 3504, phone: 2358-7015, Email: lzhang@cs.ust.hk Home pagelzhang@cs.ust.hkHome page

2
CSIT 5220 L10: Graphical Models for Decision Problems l Introduction l Extending BN to Include a Single Decision l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 2

3
CSIT 5220 Probabilistic Reasoning and Decision l Method 1: Two-stage n In a BN, calculate posterior probabilities n Use the posteriors to make decisions l Method 2 n Combine the two stages n Extend BN to include decisions Better reveal structure of decision problem Compute optimal decisions directly from model l Reasoning: Jensen & Nielsen, Sections 9.1-9.4, 10.2, 11.1 Page 3

4
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include a Single Decision l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 4

5
CSIT 5220 Poker l From Lecture 04 Page 5 l Extend the model so that I can calculate the probability that my hand is better than the opponent’s hand l MH: My Hand l BH: Best Hand

6
CSIT 5220 Fold or Call Page 6

7
CSIT 5220 Fold or Call l Information that I have: FC, SC, MH Page 7

8
CSIT 5220 Modeling One Action l Start with a BN l Add the decision node and utility nodes n What information we have when making the decision n What chance and utility variables will the decision influence Page 8

9
CSIT 5220 Including More Decisions l Things become a bit more complicated. l Will see later. Page 9

10
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include Decisions l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 10

11
CSIT 5220 Decision Theory l Normative decision theory n How people should decide. (Rational agent) l Descriptive decision theory n How people actually decide. Page 11

12
CSIT 5220 Normative Decision Theory Page 12

13
CSIT 5220 Are you rational? l Lottery A: [$1mill] l Lottery B: 0.5[$2mill] + 0.5[$0mill] l Which one do you choose? l Most people would choose A U(1) > 0.5 U(2) + 0.5 U(0) l Most people are risk-averse, with concave utility function Page 13

14
CSIT 5220 Are your rational? l Suppose that you are $2mill in debt Page 14 l Lottery A: [$1mill] l Lottery B: 0.5[$2mill] + 0.5[$0mill] l Which one do you choose? l Probably B U(1) < 0.5 U(2) + 0.5 U(0) l You are being risk-seeking, with convex utility function

15
CSIT 5220 Utilities without Money Page 15

16
CSIT 5220 Utilities without Money Page 16

17
CSIT 5220 Marks as Utilities Page 17

18
CSIT 5220 Other Considerations l 2 is passing grade l If fail, can retake and hopefully get a better grade in transcript n In this case, 2 is the worst! Page 18

19
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include Decisions l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 19

20
CSIT 5220 Decision Trees l Classical way to represent decision problems with multiple decisions l Explicitly show all possible sequences of decisions and observations. l Example: Oil Wildcatter Page 20 A wildcatter is a person who drills wildcat wells, which are oil wells drilled in areas not known to be oil fields.drills wildcat wellsoil wellsoil fields Test on Seismic structure

21
CSIT 5220 Decision Tree for Oil Wildcatter Page 21

22
CSIT 5220 Decision Trees l Decision nodes: Rectangles l Chance nodes: ellipses l Utility values: at leaves, some times inside diamonds l To be read from root to leaves n Branches from a decision node: possible actions n Branches from a chance node: possible outcomes and probs n A decision node follows a chance node: The chance node is observed before the decision is made n No-forgetting Decision-maker remembers all the labels from root to a decision node l Game between decision maker and nature Page 22

23
CSIT 5220 Solution to a Decision Tree l Strategy: Which decision node to pick at each decision node Page 23

24
CSIT 5220 Solution to a Decision Tree l Optimal Strategy: The strategy with the highest expected utility Page 24

25
CSIT 5220 Solving Decision Trees Page 25

26
CSIT 5220 Example Page 26 77.59

27
CSIT 5220 Page 27

28
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include Decisions l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 28

29
CSIT 5220 Extending BN to Including one Decision Page 29 l Start with a BN l Add the decision node and utility nodes n What information we have when making the decision n What chance and utility variables will the decision influence l To include multiple decision nodes, n Need to consider the interactions among the decisions

30
CSIT 5220 Including Multiple Decisions l Two more decisions n MFC: my first change n MSC: my second change Page 30

31
CSIT 5220 Representing the Decision Sequence l First representation n All nodes observed before a decision are parents of that decision. n Information arcs. Page 31 l Assume that the decision maker doesn’t forget, then some links are redundant.

32
CSIT 5220 Representing the Decision Sequence l No-forgetting allows a more concise representation n Keep directed path going through all the decision node: Order of decision. n Arrows into a decision node only from those nodes observed immediately before that decision. n Implicit parents: parents of earlier decisions Page 32

33
CSIT 5220 Influence Diagram l A DAG with three types of nodes n Chance nodes, decision nodes, and utility nodes l There is a directed path containing all the decision nodes. l The utility nodes have no children. l Each chance node is associated with the conditional distribution given its parents. l Each utility node is associated with a utility function, a real-valued function of its parents. Page 33

34
CSIT 5220 Influence Diagram Page 34

35
CSIT 5220 l An influence diagram for the oil wildcatter problem n Decision: T: test = {y, n}; D: drill={y, n} n Utility: C: cost of test ; V: Benefit of drilling n Chance: O: Oil ={dry, wet, soaking} R: seismic structure {no-structure, open-structure, closed-structure, no-result} Influence Diagram Page 35

36
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include Decisions l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 36

37
CSIT 5220 Strategy (Policy) l A policy specifies what to do for each decision l It is a function of observed variables Page 37 l Different policies lead to different expected utility l Optimal policy: the Policy that yields the maximum expected utility. l How to find the optimal policy?

38
CSIT 5220 Finding Optimal Policy l First idea: n Convert to decision tree and solve it l How to convert influence diagram into decision tree 1. Draw tree nRoot: the thing that happens first nChildren of root: the thing that happens next n…n… 2. Figure out numerical information Page 38

39
CSIT 5220 l Order of events l Tree structure l Numerical info n Prob for branches from chance node n Utility for leaves

40
CSIT 5220 A Side Note l Two decision trees for Oil Wildcatter n First directly from problem specification. Asymmetric n Second from influence diagram Symmetric l Pro of ID: compact l Con of ID: cannot represent assymetry n Need to introduce artificial state R = no-result

41
CSIT 5220 Finding Optimal Policy l First idea: n Convert to decision tree and solve it l Exponential still! l Next: n Variable Elimination Algorithm for solving influence diagrams n Note n BN inference: All orderings give correct result, but might have different complexity n ID: Must use “strong elimination orderings”. Page 41

42
CSIT 5220 Temporal Order and Decisions andd Observations l Notations n Decision nodes have a temporal order: D 1, D 2, …, D n n T 0 : Set of chance nodes observed prior to any decision n T i : Set of chance nodes observed after D i is taken and before D i+1 is taken l Oil Wildcatter n D 1 = T; D 2 = D n T 0 = {}; T 1 = {R}; T 2 ={O} l Partial temporal order n T 0, D 1, T 1, D 2, T 2, …., D n, T n n Oil Wildcatter: T, R, D, O Page 42

43
CSIT 5220 Temporal Order l T 0 ={}, T 1 ={T}, T 2 ={A, B, C} l Partial temporal ordering n D 1, T, D 2. {A, B, C} n No ordering among A, B, C Page 43

44
CSIT 5220 Strong Elimination Ordering l Partial temporal order n T 0, D 1, T 1, D 2, T 2, …., D n, T n l Strong elimination orders n First eliminate variables in T n n Then eliminate D n n Then eliminate variables in T n-1 n Then eliminate D n-1 n ….. l Oil Wildcatter n Temporal order: T, R, D, O n Strong elimination ordering O, D, R, T Page 44

45
CSIT 5220 Strong Elimination Ordering l T 0 ={}, T 1 ={T}, T 2 ={A, B, C} l Partial temporal ordering n D 1, T, D 2. {A, B, C} n No ordering among A, B, C l Strong elimination orderings n A, B, C, D 2, T, D1 n B, C, A, D 2, T, D 1 n C, A, B, D 2, T, D 1 n …. Page 45

46
CSIT 5220 Variable Elimination l Two set of potentials (factors): l Eliminate decision and chance nodes one by one according to a strong elimination ordering. l When eliminate variable X Page 46

47
CSIT 5220 Variable Elimination on Oil Wildcatter l Strong Elimination Ordering: O, D, R, T Page 47

48
CSIT 5220 l Eliminate: O Page 48 Variable Elimination on Oil Wildcatter

49
CSIT 5220 Page 49

50
CSIT 5220 Page 50

51
CSIT 5220 Potentials after Eliminating O Page 51

52
CSIT 5220 Potentials after Eliminating O Page 52

53
CSIT 5220 Eliminating D l No probability potential involves D Page 53 l Optimal decision for D

54
CSIT 5220 Potentials after Eliminating D Page 54

55
CSIT 5220 Eliminating R Page 55

56
CSIT 5220 Potentials after Eliminating R Page 56

57
CSIT 5220 Eliminating T Page 57 l Optimal decision for T l Results same as those by decision tree

58
CSIT 5220 Solving Influence Diagram Using Netica

59
CSIT 5220 Solving Influence Diagram Using Netica l Netica cannot handle multiple utility l So, combine U and V

60
CSIT 5220 Solving Influence Diagram Using Netica

61
CSIT 5220 l Or, Get optimal action by trying each actions

62
CSIT 5220 L10: Graphical Models for Decision Problems l Extending BN to Include Decisions l Fundamentals of Rational Decision Making l Decision Trees l Influence Diagrams l Solving influence Diagrams l Value of information Page 62

63
CSIT 5220 Two types of Decisions l Action decisions n Result in significant state change of variables of interest n Example: D: Drill or not to drill l Test decisions n Look for more evidence n Example: T: Test of Seismic structure Page 63

64
CSIT 5220 Two types of Decisions l Typical scenario n Need to make one decision n Want to get more information before making the decision n Question Is it worthwhile to perform a particular test? Which test to choose if multiple tests are available? Page 64

65
CSIT 5220 Value of Information l What is the value of a test? n Create two influence diagrams n Solve both n Compare their values l Example: Oil wildcatter n Is it worthwhile to perform the seismic test? n ID1: without the test n ID2: with the test Page 65

66
CSIT 5220 Value of Information l Expected utility of ID2 n U(ID2) = 22.55 l What is the expected utility of ID1? Page 66

67
CSIT 5220 Expected Utility of ID1 l Temporal ordering: D, O l Elimination ordering: O, D l Eliminate O:

68
CSIT 5220 Expected Utility of ID1 l Potentials after eliminating O l Eliminate D l Expected utility of ID1 n U(ID1) = 20 Page 68

69
CSIT 5220 Value of Information l Difference in expected utility n U(ID2) – U(ID1) = 22.55 – 20 = 2.55 n The expected value of the seismic test is 2.55 n The test is worthwhile Page 69

70
CSIT 5220 Value of Information l If there are multiple tests n T1, T2, T3, … n Compute the value of each test, pick the best one n If the value of the best is positive, Pick the test among remain tests n Stop when value of the selected test is not positive Page 70

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google