1. Modeling Qualitative Preferences Using the CP-net Model

2. To make good decision, we must be able to assess and compare different alternatives. The ability to make decisions is a corner-stone of many AI applications: Decision-support expert systems Autonomous agents Configuration software …

3. Assessing Alternatives We compare alternatives based on their: Likelihood Desirability Our Focus: Assessing/specifying outcome desirability

4. Specifying Preferences Uncertainty Involved? Utility function desirable Enough to rank potential outcomes yesno Need to weigh the contributions of different outcome Need to recognize the best feasible alternative

5. Utility functions Qantify outcome desirability –Capture the difference in desirability between outcomes Necessary when: –Uncertainty is involved, or Drawbacks: –Complicated preference elicitation. –Generally hard optimization. Serious practical concern Serious comput. concern

6. When a utility function cannot be and/or need not be obtained, one should resort to other, more qualitative forms of preference representation.

7. What We Want from a Qualitative Preference Model Simple elicitation process based on intuitive and natural statements about preferences –No need for an expert decision analyst –Makes automatic online elicitation feasible As expressive as possible, subject to above Supports an efficient optimization process

8. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

9. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

10. Ceteris Paribus (cp) Statemenents Ceteris Paribus (Lat.) – all else being equal “ I prefer to have wine with my meal, all else being equal” That is: given two identical meals, one with wine and one without, I prefer the former.

11. Conditional CP Statements “ I prefer red wine to white wine with my meal, ceteris paribus, given that meat is served” That is: given two identical meals in which meat is served, I prefer red wine to white wine. Tells us nothing about two identical meals in which meat is NOT served.

12. Outcomes and Preferences 1.Domain variables. 2.Outcome space. 3.Preferences over the outcome space. Example on Product ( Computer System ) Configuration: Domain variables are the system’s properties: Processor Speed, Processor Manufacturer, Screen Size, etc. Dom( Screen Size ) = {15in, 17in, 19in, 21in} Possible preferential statements of a customer: I prefer 1000 MHz on 800 MHz I prefer 19in screen on 17in screen if video card is Sony’s

13. Preferential Independence If my preferences over the values of a variable v does not depends on the values of some other variables, then v is preferentially independent of all other variables. I prefer 1000 MHz to 800 MHz (all else being equal) If my preferences over the values of a variable v depends on the values of some other variables v 1, …,v k, then v is conditionally preferentially independent of all other variables V-{v 1, …,v k }, given an assignment on v 1, …,v k. I prefer 19in screen to 17in screen if video card is Sony’s (all else being equal)

14. Preferential Independence A subset of variables is preferentially independent of its complement if and only if, for all assignments holds Let be a partition of into three disjoint non-empty sets. and are conditionally preferentially independent given if and only if, for all holds

15. CP-nets (Boutilier, Brafman, Hoos, Poole, UAI ‘99) An intuitive, qualitative, graphical model of preferences, that captures statements of conditional preferential independence. DAG in which each node represents a domain variable. The immediate parents P(v) of a variable v in the network are those variables that affect user’s preference over the values of v. P(screen size) = { video card manuf. } P(operating system) = { processor speed, screen size }

16. A C DE F B Example of a CP-net

17. Any CP-net defines a partial order over the outcome space. A C B Consistency worst best

18. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

19. Preferential Optimization Finding the preferentially optimal outcome is straightforward! A C DE F B A C DE F B

20. Adding Constraints 1.Domain variables. 2.Outcome space. 3.Preferences over the outcome space. 4.Constraints on the domain variables. Constraints in Product ( Computer System ) Configuration: 17in screens are currently unavailable. 15in screens of NEC are incompatible with the Sony’s graphical card. Windows XP requires at least 800MHz processor speed.

21. Constraint-based Preferential Optimization (Boutilier et al. 97’) Branch & Bound algorithm for determining the set of feasible, preferentially non-dominated outcomes was suggested. Starting with an empty set of solutions, the algorithm continuously extends it by adding new non-dominated solutions. The current set of the solutions serves a lower bound for the forthcoming candidates.>   

22. Constraint-based Preferential Optimization EABF GC D H XYZ Initialize the set of local results R = . Choose any variable v with no parents in G. A Let v 1 >… > v k be the preference ordering of D(v) given the assignment on P(v) in K. K(x,y) : a 1 > … > a k

23. E aiai BF GC D H XYZ for i = 1 to k do v = v i K(x,y) : a 1 > …> a i > …> a k Strengthen the set of constraints C by v = v i to obtain C i if C i is inconsistent or exist j<i s.t. C j  C i continue with next iteration Reduce G to G’ by removing the variables assigned by K’. EBF GC D H Let G’ 1,…, G’ m be the strongly connected components of G’ for j= 1 to m do Search S j = Search ( G’ j, K  K’, C i ) else Let K’ be the partial assignment induced by v = v i and C i. aiai

24. for i = 1 to k do v = v i Strengthen the set of constraints C by v = v i to obtain C i if C i is inconsistent or exist j<i s.t. C j  C i continue with next iteration Reduce G to G’ by removing the variables assigned by K’. EBF GC D H Let G’ 1,…, G’ m be the strongly connected components of G’ for j= 1 to m do Search S j = Search ( G’ j, K  K’, C i ) else Let K’ be the partial assignment induced by v = v i and C i. E aiai BF GC D H XYZ aiai Add o to R return R if S j   for all j  m if for each o’  R holds K  o’  K  o foreach o  K’  S 1  …  S m do

25. Constraint-based Preferential Optimization

26. dominance testing So what is the price of comparison ( dominance testing ) between two outcomes?

27. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

28. Flipping Sequence - Example

29. Dominance Testing In (Boutilier et al.) dominance testing was treated as a search for a flipping sequence from the (purported) less preferred outcome to the (purported) more preferred outcome through a sequence of more preferred outcomes: Where, for, outcome differs from the outcome in the value of exactly one variable, given the values of in (and )

30. Dominance Testing for CP-nets with Binary Variables. Backtrack free algorithm in BBHP ‘99Remarks Linear? TreeComplexity CP-net graph

31. Complexity of Dominance Testing First result (Boutilier et al.): Dominance testing for binary, tree CP-nets is backtrack free.

32. Example: A C DE B

33. A C DE B

34. A C DE B

35. A C DE B

36. A C DE B

37. The algorithm for trees is not good for polytrees A C B

38. Dominance Testing for CP-nets with Binary Variables. Lower boundRemarks TreeComplexity CP-net graph

39. Dominance Testing for CP-nets with Binary Variables. Lower boundRemarks Tree PolytreeComplexity CP-net graph k - maximal indegree

40. Start of Analysis Denote by the maximal number of times that a variable may be required to flip its value on a irreducible flipping sequence from to. Lemma 1 Lemma 1: Given a dominance testing problem where is a singly connected, binary CP-net, for each variable we have thatolds:

41. Framework for Polytrees Using the upper bound established by Lemma 1, we provide a polynomial time procedure that determines the maximal number of feasible, possibly required value flips for a given variable. We provide a polynomial time algorithm that determines whether or not exist a flipping sequence from to. This algorithm is based on top-down execution of the previously defined procedure on the variables of the CP-net.

42. Dominance Testing for CP-nets with Binary Variables. k - maximal indegree Lower boundRemarks Tree Singly-connected DAG PolytreeComplexity CP-net graph NP-complete Reduction from 3SAT

43. Dominance Testing for CP-nets with Binary Variables. Minimal flipping sequences are polynomially bounded Reduction from 3SAT - maximal indegree Lower boundRemarks Tree NP-complete -connected DAG NP-completeSingly-connected DAG PolytreeComplexity CP-net graph

44. Dominance Testing for CP-nets with Binary Variables. EXPTIME or in NP? Minimal flipping sequences are polynomially bounded Reduction from 3SAT - maximal indegree Lower boundRemarks Tree ? DAG NP-complete -connected DAG NP-completeSingly-connected DAG PolytreeComplexity CP-net graph

45. Recall that... Starting with an empty set of solutions, the B&B algorithm continuously extends it by adding new non-dominated solutions. The current set of the solutions serves a lower bound for the forthcoming candidates. New candidate is compared to all solutions generated until now.    

46. The Big Picture Generally, COP is much harder than CSP. If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP. If some non-dominated solutions are required, and the CP-net is reasonably restricted, then the CP-net based COP is still not harder than the corresponding CSP. In general, if some (even just 2) non-dominated solutions are required, then the CP-net based COP may be much harder.

47. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

48. Any CP-net defines a partial order over the outcomes. A C B Recall that … worst best acyclic

49. Cyclic CP-nets are not necessarily consistent … Consistency of Cyclic CP-nets A B

50. Localizing the Analysis Strongly Connected Components Context variables of a SCC Contexts of a SCC = assignments to its context variables

51. Localizing the Analysis Theorem: A cyclic CP-net is consistent if and only if for every strongly connected component  and for every context  of , we have that  is consistent given . Conclusion: We can start by consistency analysis of only strongly connected CP-nets.

52. Basic SCCs – Simple Cycles, Binary Variables a cd b Simple cycle

53. Surprisingly … Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible! A B

54. Putting Things Together Theorem: A consistent specification of a binary, simple cycle CP-net over more than 2 variables is impossible! Surprising observation: There are many cyclic binary CP-nets with simple cycles SCCs that are consistent! Theorem: A cyclic binary CP-net is consistent if and only if for every its strongly connected component  and for every context  of , we have that  is consistent given . Disappointing?

55. Specific contexts break the cycles!

56. Context-dependent “acyclic” CP-nets

57. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. –Importance tradeoffs –Adding utilities Potential applications. Future research directions and open problems.

58. Adding Variable Importance into CP-networks

59. Relative Importance If it is more important to me that the value of X be high than the value of Y be high, then X is more important than Y. Processor type is more important to me than operating system ( all else being equal). Operating system is more important than processor type ( all else being equal), if the PC is used primarily for graphical applications. If, given z  Dom( Z ), it is more important to me that the value of X be high than the value of Y be high, then X is conditionally more important than Y.

60. A C B worst best

61. A B CD E nodes  variables cp-arcs (directed) i-arcs (directed) ci-arcs (undirected) cp-tables ci-tables

62. The same “forward sweep” algorithm works as is for TCP-nets! Outcome Optimization in TCP-nets 1.The relative importance relations do not play a role in this case. 2.The network is traversed according to a topological ordering induced by the CP-net part of the given TCP-net.

63. –Any acyclic CP-net induce a partial order over the variables, according to which the variables should be processed. –Any conditionally acyclic TCP-net induce a hierarchical set of partial orders over the variables. The central difference between CP-nets and TCP-nets with respect to constraint- based outcome optimization

64. Extension to conditionally acyclic TCP-nets If any non-dominated solution is enough, then the CP-net based COP is not harder than the corresponding CSP. “As is”, the Branch & Bound algorithm for CP-nets, executed on a conditionally acyclic TCP-net, loses its anytime behavior. We extended this algorithm to process TCP-net based constraint outcome optimization in an anytime fashion.

65. Introducing Utilities into CP-networks

66. Utility Functions Utility functions assign a real number to each outcome: u(o 1 )=0.4, u(o 2 )=4.3, … Actual u-values are significant: –U-values can be used to rank outcomes: u(o 1 ) < u(o 2 ) implies that o 1 is preferred to o 2 –But much more! we can assess any uncertain combination of outcomes: the utility of an action yielding with probability 0.4 and with probability 0.6 is 0.4*u(o 1 ) + 0.6*u(o 1 ) Dominance queries are easy Optimiziation may be difficult 

67. Generalized Additive Independ. The (possibly overlapping) sets X 1,...,X k of variables are generalized additive independent (GAI) if: –for any distributions P 1, P 2, with identical marginals over the X i, the expected utility w.r.t. U is the same for P 1 and P 2 X 1,...,X k are GAI iff U can be written as U(v) =  i f i (x i ) for some component utility factors f i

68. UCP-nets – CP nets + Utilities Utilities permit fine value tradeoffs and effective dominance tests CP-nets allows effective outcome optimization UCP-nets use the graphical structure of a CP-net, but quantify conditional utilities U(x) is sum of utility factors; U(abcd) = f A (a)+f B (b)+f C (abc)+f D (cd) = 5 + 4 +.2 +.9 = 10.1 Utility computation is linear AB C D a > a 5 2 b > b 4 3 a b.6.1 a b.2.8 a b.3.8 a b.9.3 c.9.8 c.2.3 c d

69. The CPI-Restriction So, far, directionality is not exploited –This is just a GAI decomposition with redundant factors (no need for factors f A or f B ; can be incorporated into f C ) We also require: DAG G is a valid CP-network for the preference order induced by U UCP-net is a specific form of GAI decomposition –Not all GAI-decompositions correspond to a UCP-net topology –Not all utility functions can be captured by UCP-nets: the CPI-conditions may not be satisfied

70. Testing CP-Ind. Condition CP-Ind conditions implies that not all quantifications lead to legitimate UCP-nets A local test exists to verify CPI conditions X dominates its children if for all x 1,x 2 s.t. f X (x 1,u)  f X (x 2,u),u, z, y : f X (x 1,u) - f X (x 2,u)   i f Yi (y i x 1 u i z i ) - f Yi (y i x 2 u i z i ) G is a UCP-net iff each variable dominates its children Domination is a local test Simpler sufficient conditions exist X Y1Y1 Y2Y2 UU2U2 U1U1 Z1Z1 Z2Z2

71. Tradeoff Weight Representation Let’s normalize utilities to the [0,1] range For each X with parents U and factor f X, –normalize each row of conditional “utility function” f X (—,u) to obtain local value function v u (X) –for each u  Dom(U), specify mulitplicative tradeoff weight  u and additive tradeoff weight  u –utility factor f X (x,u) =  u v u (x) +  u Every UCP-net has a normalized equivalent

72. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

73. Preference-based Configuration of Web Page Content

74. Our Goal Model for presenting the content of a web-page that: web-page that: Reflects the preferences of the web- page designer. Adjusts dynamically to the viewer’s current interests.

75. State of the Art – Categorization

76. Web page Article Component: Article Presentation alternatives: 1.Full content 2.Partial content 3.Heading only 4.Link only 5.Invisible 6.… Today: static, predefined appearance. Today: Any web page is a collection of components, each one with its static, predefined appearance.

77. Preference-based Configuration configuration spaceMulti-valued components of a web-page create a configuration space. Preference rankingPreference ranking is a total preorder over : means that configuration is subjectively equally or more preferred to.

78. The preference order represents the subjective preferences of the web page designer, not of its viewer. However, the actual choices of the viewer affect the choice of configuration.

79. Why qualitative decision making? Utility assessments unlikely to be intuitive in our setting. Ordering different options for each component is likely to be a relatively easy task.

80. Looking for a Model - Objectives Explicit ranking of all options – infeasible. Desiderata: –Intuitive. –Relatively efficient reasoning. –Capture conditional preference dependencies.

81. A B CD Each component of a web page can be represented by a variable in the CP-network.

82. Authoring Tool HTML DHTML Preference specification Web Browser 1. Interaction 3. Presentation 2. Preference-based reconfiguration of the web-page.

83. Plain HTML (DIV tags)

84. Parsed HTML

85. CP-network definition

86. Preference definition (1 - present, 0 - hide)

87. Authoring Tool HTML DHTML Preference specification Web Browser 1. Interaction 3. Presentation 2. Preference-based reconfiguration of the web-page.

89. Example Elections - Presenting is unconditionally preferred to hiding. Traffic Accident – Hiding is unconditionally preferred to presenting. New Airbag – Presenting is preferred only if Traffic Accident is presented, and Elections is hidden. NBA – Presenting is preferred only if Traffic Accident is not presented. NY Times comm. – Presenting is preferred only if both Elections and Traffic Accident are presented. Volvo comm. – Presenting is preferred only if either New Airbag or Traffic Accident is presented. Nike comm. – Presenting is preferred only if NBA is presented.

90. Traffic Accident Elections New Airbag NBA VolvoNike NY Times Corresponding CP-network

91. Traffic Accident Elections New Airbag NBA VolvoNike NY Times Initially optimal presentation

92. Traffic Accident Elections New Airbag NBA VolvoNike NY Times Viewer opened “Traffic Accident” Traffic Accident Elections New Airbag NBA VolvoNike NY Times

93. Viewer closed “Elections” Traffic Accident Elections New Airbag NBA VolvoNike NY Times Traffic Accident Elections New Airbag NBA VolvoNike NY Times

94. What if the event queue is bigger? Traffic Accident Elections New Airbag NBA VolvoNike NY Times Traffic Accident Elections New Airbag NBA VolvoNike NY Times

95. Integrating Global Constraints What if the constraints and the preferences are not defined over the same set of variables? No obvious way to utilize preferences for search space pruning … Example - User Interface Configuration –Preferences are defined over the content alternatives for the different UI components. –Geometric layout constraints.

96. Integrating Layout Constraints Layout constraints are represented as an integer linear program. Reconfiguration is adopted to look for a feasible, Pareto-optimal presentation. Generally, this task is computationally hard.

97. Highly Constrained Problems Reasonable approach: Forget about the CP-net, and generate all the solutions for the CSP. Using the CP-net, filter all the dominated solutions. Generally, filtering may be painful...

98. Given an outcome a, it is easy to determine the complete set of outcomes {a 1, …, a m } that are (preferentially) minimally worse than a. Preferentially optimal outcome Underconstrained Problems

99. Potential Applications Multi-agent meeting scheduler –CP-nets support efficient computation of Pareto- optimal solutions Filtering the results of search –Context can be use to disambiguate words, focus on the more important terms Scheduling problems –Various constraints are more important than others

100. Overview CP-net model for qualitative preferences. Preference Optimization. Complexity analysis of outcome comparisons. Consistency testing. Various Enhancements. Potential applications. Future research directions and open problems.

101. Possible connections with non-monotonic reasoning. Model acquisition from a restricted natural language text. –Possibility for an intuitive closed loop system … Analyzing domains in which qualitative modeling of preferences seem to be required. –Product configuration (!) –Computational Biology (?) Additional directions