1. Working with Preferences Ronen Brafman Computer Science Department Ben-Gurion University

2. Goal of our Work: Provide tools and build systems that can make, or can help users make, good choices. Such systems must understand the user’s preferences to be useful.

3. Applications of Interest optimal product/item selection optimal product/item selection Select a flight, a camera, a movie, … Select a flight, a camera, a movie, … optimal product configuration optimal product configuration Configure a PC, a vacation, a car, … Configure a PC, a vacation, a car, … personalization personalization Personalize content, interface, … Personalize content, interface, … guiding program choices guiding program choices Program will make choices based on a preference model provided by the designer Program will make choices based on a preference model provided by the designer We target lay users and application designers with no background in decision theory

4. Focus of this Talk : Provide an overview of an approach to preference handling based on the xCP-net models and algorithms, and discuss some applications

5. Constrained Optimization Process Manufacturer Constraints Preference specification Feedback  Spec Revision Optimal automobile Updated Choice

6. The Key Ideas Preference elicitation: Preference elicitation: Use natural language like statements – make it simple Use natural language like statements – make it simple Support heterogeneous types of statements Support heterogeneous types of statements Make scaling up to quantitative assessment possible Make scaling up to quantitative assessment possible Calculating optimal choices: Calculating optimal choices: Efficient methods for ordering sets Efficient methods for ordering sets Algorithms for constrained optimization Algorithms for constrained optimization Efficient data-driven incremental elicitation Efficient data-driven incremental elicitation

7. Elicitation and Modeling

8. Natural Preferences Statements Value preferences: Value preferences: Strict: I prefer an isle seat to a window seat Strict: I prefer an isle seat to a window seat Conditional: I prefer isle to window in economy class Conditional: I prefer isle to window in economy class Attribute importance: Attribute importance: Strict: Seat assignment is more important than airline Strict: Seat assignment is more important than airline Conditional: Seat assignment is less important than airline in domestic flights Conditional: Seat assignment is less important than airline in domestic flights

9. The Ceteris Paribus Semantics Ceteris Paribus = Latin: all else being equal Ceteris Paribus = Latin: all else being equal “I prefer an isle seat to a window seat”  “I prefer an isle seat to a window seat”  Given two flights that differ only in seat assignment, I prefer the one with an isle seat. Given two flights that differ only in seat assignment, I prefer the one with an isle seat. Otherwise, can’t infer a preference Otherwise, can’t infer a preference “Seat assignment is less important than airline in domestic flights”  “Seat assignment is less important than airline in domestic flights”  Given two domestic flights that differ in seat assignment and airline only, I prefer the one with a better airline Given two domestic flights that differ in seat assignment and airline only, I prefer the one with a better airline Says nothing about two international flights Says nothing about two international flights

10. CP-nets ( Boutilier, Brafman, Hoos, Poole 99, Boutilier, Brafman, Domshlak, Hoos, Poole 04) A qualitative, graphical model of preferences, that captures and organizes statements of conditional value preference. Each node represents a domain variable. Parents(v) are those variables that affect user’s preference over the values of v Parents(class) = {airline} Conditional preference table (CPT) associated with every node in the CP-net Provides an ordering over the values of the node for every possible parent context

11. A C DE F B Example of a CP-net

12. Any acyclic CP-net defines a (consistent) partial order over the outcome space. A C B Semantics and Consistency worst best

13. A C B worst best Importance Relations

14. Day of the flight Departure Time Stop-overs Class Airline More Complex Example

15. Day of the flight Departure Time Stop-over Class Airline

16. Day of the flight Departure Time Stop-oversClass Airline

17. A B CD E nodes  variables cp-arcs (directed) i-arcs (directed) ci-arcs (undirected) cp-tables ci-tables  

18. Role of Graphical Structure Users need not be aware of the underlying graphical structure Users need not be aware of the underlying graphical structure User employs statement templates or some other input interface User employs statement templates or some other input interface System constructs the network on the fly System constructs the network on the fly Graphical structure important for: Graphical structure important for: Analysis – query complexity related to structure Analysis – query complexity related to structure Algorithms – often use topological sort Algorithms – often use topological sort

19. Preferences  Plausibility CP = Conditional Plausibility (Jerome Lang) “p is more plausible than –p given q” (ceteris paribus): “p is more plausible than –p given q” (ceteris paribus): Given two worlds satisfying q that are identical except for the value of p, the one satisfying p is more plausible than the one satisfying –p. Given two worlds satisfying q that are identical except for the value of p, the one satisfying p is more plausible than the one satisfying –p. “p is more important, plausibility wise, than q” (ceteris paribus): “p is more important, plausibility wise, than q” (ceteris paribus): Worlds in which p has its more plausible value are more plausible than worlds in which q has its more plausible value Worlds in which p has its more plausible value are more plausible than worlds in which q has its more plausible value

20. UCP-Nets [Boutilier, Bacchus, & Brafman] Quantified CPT Quantified CPT Simplified semantics: sum of utility factors; U(abcd)= Simplified semantics: sum of utility factors; U(abcd)= f A (a)+f B (b)+f C (abc)+f D (cd) = f A (a)+f B (b)+f C (abc)+f D (cd) = 5 + 4 +.6 +.9 = 10.5 5 + 4 +.6 +.9 = 10.5 Linear time computation Linear time computation Linear time comparison Linear time comparison AB C D a 5 2 b 4 3 a b.6.1 a b.2.8 a b.3.8 a b.9.3 c.9.8 c.2.3 c d

21. Compiling Diverse Information into UCP-nets Diverse statement can be compiled into UCP-net: Diverse statement can be compiled into UCP-net: Independence information: maintained by graph Independence information: maintained by graph Value preferences and variables preferences Value preferences and variables preferences Two-way comparisons of complete outcomes Two-way comparisons of complete outcomes Quantitative information Quantitative information Simple compilation based on linear constraints Simple compilation based on linear constraints Good for incremental refinement: Good for incremental refinement: Start with qualitative model – no cold start! Start with qualitative model – no cold start! Refine with user feedback/additional observations Refine with user feedback/additional observations

22. Using (T)CP-nets Algorithms 1. Selecting the best element 2. Ordering and constrained optimization

23. 1. Ordering Elements Need: Order (the many) results of a query Need: Order (the many) results of a query Naïve solution: pairwise comparison -- o>o’ ? Naïve solution: pairwise comparison -- o>o’ ? Problem: NP-hard in general Problem: NP-hard in general Alternative solution: linearize the partial order Alternative solution: linearize the partial order Insight: some linearizations are easy to obtain Insight: some linearizations are easy to obtain Key: Key: 1. One of o e o’ or o’ e o can be decided quickly 2. Answers used to generate consistent ordering

24. 1. Preferential Optimization Finding the preferentially optimal outcome for an acyclic network is straightforward! A C DE F B A C DE F B

25. 2. Preference-Based Constrained Optimization (PCO) Given: Given: User preferences User preferences An implicitly specified set of feasible options An implicitly specified set of feasible options Find one/all/k optimal, feasible element(s) Find one/all/k optimal, feasible element(s) Applications: Applications: Product configuration (PC, vacation) Product configuration (PC, vacation) Optimal plan selection Optimal plan selection Content/display adaptation and personalization Content/display adaptation and personalization

26. Solving PCO: Ordered Generate & Test Generate outcomes in non-increasing order Generate outcomes in non-increasing order = linearize the partial order over all possible elements Test for feasibility Test for feasibility Check for optimality: Check for optimality: First feasible outcome is optimal! First feasible outcome is optimal! Need more? Need more? Maintain set of optimal solutions Maintain set of optimal solutions New solution is optimal if not dominated by any previously generated solution New solution is optimal if not dominated by any previously generated solution

27. Generating a Non-increasing Sequence of Outcomes Topologically sort the variables Topologically sort the variables Build an assignment (search) tree by instantiating variables according to this order Build an assignment (search) tree by instantiating variables according to this order Order variable values based on the CPT Order variable values based on the CPT Leaf nodes, ordered left to right (= depth-first traversal of the tree) correspond to a non- increasing sequence of outcomes Leaf nodes, ordered left to right (= depth-first traversal of the tree) correspond to a non- increasing sequence of outcomes

28. C B 1 2 3 A

29. Sorting? We just saw how we can order all outcomes We just saw how we can order all outcomes This method does not work for sorting a subset of all possible assignments This method does not work for sorting a subset of all possible assignments Fortunately, there is a method that can be used to sort n outcomes in time O(n log n) Fortunately, there is a method that can be used to sort n outcomes in time O(n log n)

30. Ordered Generate and Test  Efficient Constrained Optimization Tree search + favorite CSP pruning techniques Tree search + favorite CSP pruning techniques Equivalent to solving the CSP with meta-level constrains on variable and value ordering Equivalent to solving the CSP with meta-level constrains on variable and value ordering Branch and bound – eliminate sub-tree when: Branch and bound – eliminate sub-tree when: We assign a variable to a less preferred value We assign a variable to a less preferred value Current set of constraints as strong as for some previous value of this variable Current set of constraints as strong as for some previous value of this variable

32. Anytime Behavior First feasible solution is optimal! First feasible solution is optimal! No theoretical overhead beyond standard CSP solution No theoretical overhead beyond standard CSP solution No item withdrawn from set of current solutions No item withdrawn from set of current solutions To obtain more than one solution dominance testing required To obtain more than one solution dominance testing required Can lead to considerable computational overhead Can lead to considerable computational overhead

33. Applications

34. Flight Selection (Brafman, Domshlak, Kogan UAI’04) Instance of “selecting optimal element” problem Instance of “selecting optimal element” problem Approach: Approach: User supplies User supplies constraints (source, destination, etc.) constraints (source, destination, etc.) initial preferences initial preferences Preferences compiled into UCP-net Preferences compiled into UCP-net Top k results shown Top k results shown User provides feedback User provides feedback Which flight is most preferred among k best Which flight is most preferred among k best Any new preferences Any new preferences Top results revised accordingly Top results revised accordingly

35. Planning for Data Products (Golden et. al.) NASA collects much raw data about earth NASA collects much raw data about earth Requires extensive processing to be useful Requires extensive processing to be useful Each Earth scientists needs different processing Each Earth scientists needs different processing ImageBot generates data plans for such products ImageBot generates data plans for such products Data plans run scientific models, combine and transform data in order to achieve data goals Data plans run scientific models, combine and transform data in order to achieve data goals Many plans can generate one data product Many plans can generate one data product Each plan has different value Each plan has different value Using a simple preference language ImageBot planning algorithm is biased towards more preferred plans Using a simple preference language ImageBot planning algorithm is biased towards more preferred plans

36. Adaptive, Personalized Rich Media Presentations Presentations with diverse media elements requiring spatio-temporal synchronization Presentations with diverse media elements requiring spatio-temporal synchronization Target wide audiences Target wide audiences Audience members have different tastes, different network connections, and use different devices Audience members have different tastes, different network connections, and use different devices Goal: provide designers with tools for designing presentations that adapt to user and user context Goal: provide designers with tools for designing presentations that adapt to user and user context Demo Demo Demo

37. Example: ESPN Promo Presentation with five elements Presentation with five elements Video featuring upcoming broadcast Video featuring upcoming broadcast 2 image ads 2 image ads Video ad Video ad Running text with scores/news Running text with scores/news Each media element is a variable Each media element is a variable Additional variables denote user properties, device properties, bandwidth Additional variables denote user properties, device properties, bandwidth Variable values = different content and quality options Variable values = different content and quality options

38. How Does It Work? Variables denote different presentation elements (video, ads, running text) Variables denote different presentation elements (video, ads, running text) Additional variables denote context information Additional variables denote context information TCP-nets specify preference relation over presentations TCP-nets specify preference relation over presentations Author may specify additional constraints Author may specify additional constraints At download time combine: At download time combine: TCP-net TCP-net Information about user and user context Information about user and user context Resulting constrained optimization problem solved Resulting constrained optimization problem solved Output a SMIL presentation Output a SMIL presentation

39. GenderNationality CP-Net for ESPN Promo Ad1Sports VideoTextAd2  

40. Real-Time Content Selection for Command and Control Imagine a decision maker monitoring masses of data in a real-time command & control center for all rescue forces in Paris Imagine a decision maker monitoring masses of data in a real-time command & control center for all rescue forces in Paris Video streams Video streams Sensor data Sensor data Results of relevant queries Results of relevant queries Results of data analysis (e.g., simulations, risk assessment) Results of data analysis (e.g., simulations, risk assessment) Task: Which data to show at each point in time? Task: Which data to show at each point in time?

41. Possible Information Sources Cameras on fireman helmet Cameras on fireman helmet Fixed surveillance cameras Fixed surveillance cameras Heat sensors, smoke detectors, co 2 -levels Heat sensors, smoke detectors, co 2 -levels Area maps, building plans, driving distance, number of residents Area maps, building plans, driving distance, number of residents Simulation of structure strength, time to contain a fire as function of wind and other weather conditions, etc. Simulation of structure strength, time to contain a fire as function of wind and other weather conditions, etc. Demo Demo Demo

42. Our Proposal: Decision Theoretic Control Offline: build a preference model capturing the value of different choices Offline: build a preference model capturing the value of different choices Online: compute best choices Online: compute best choices In our case: which data to show In our case: which data to show In general: whatever choices the system must make In general: whatever choices the system must make Not new, but not too practical, so far Not new, but not too practical, so far Main obstacle: preference handling Main obstacle: preference handling

43. What Are the Challenges ESPN solution (using CP-nets) is static ESPN solution (using CP-nets) is static We know all the variables and their possible values at specification time We know all the variables and their possible values at specification time Our context is dynamic: Our context is dynamic: Different cities will have different relevant information streams Different cities will have different relevant information streams Same city will have different relevant information streams at each time Same city will have different relevant information streams at each time A single specification should handle LA, Paris, NY, etc. A single specification should handle LA, Paris, NY, etc. Our solution: relational preference rules Our solution: relational preference rules A relational model that generalizes of UCP-nets A relational model that generalizes of UCP-nets Assumes concepts/classes are fixed (fireman, fire, fire truck) Assumes concepts/classes are fixed (fireman, fire, fire truck) Does not assume anything about specific instances Does not assume anything about specific instances

44. Modeling a Fire Department Rules: Rules: Fireman(x) Λ fire(y) Λ x.location = y.location  x.camera.display [“on” 4, “off” 0] Fireman(x) Λ x.co 2 -level=high  x.camera.display [“on” 8, “off” 0]

45. Other Interesting Issues Set preferences – specifying and computing preferred subsets when items have synergy [AAAI’06,AAAI’07] Set preferences – specifying and computing preferred subsets when items have synergy [AAAI’06,AAAI’07] Browsing large heterogeneous databases Browsing large heterogeneous databases Joint work with Scott Klemer, Ron Yeh, and Yoav Shoham supported by NSF Joint work with Scott Klemer, Ron Yeh, and Yoav Shoham supported by NSF

46. Collaborators Carmel Domshlak – Technion Carmel Domshlak – Technion Craig Boutilier – University of Toronto Craig Boutilier – University of Toronto Holger Hoos, David Poole – UBC Holger Hoos, David Poole – UBC Doron Friedman – University College London Doron Friedman – University College London Solomon Shimony – Ben-Gurion University Solomon Shimony – Ben-Gurion University

47. Summary Much recent interest in work on preference representation, elicitation, and reasoning Much recent interest in work on preference representation, elicitation, and reasoning xCP-nets offer a simple preference language and convenient graphical and algorithmic tools xCP-nets offer a simple preference language and convenient graphical and algorithmic tools UCP-nets provide good target for knowledge compilation UCP-nets provide good target for knowledge compilation Many applications in e-commerce and user interfaces Many applications in e-commerce and user interfaces

48. Compare > If dominated …     Otherwise …