Preferences Toby Walsh NICTA and UNSW

Outline May 5,15:00-17:00  Introduction, soft constraints May 6, 10:00-12:00  CP nets May 7, 15:00-18:00  Strategic games, CP-nets, and soft constraints  Voting theory May 8, 15:00-18:00  Manipulation, preference elicitation May 9, 10:00-12:00  Matching problems, stable marriage

Motivation Preferences are everywhere!  Alice prefers not to meet on Monday morning  Bob prefers bourbon to whisky  Carol likes beach vacations more than activity holidays …

Major questions Representing preferences  Soft CSPs, CP nets, … Reasoning with preferences  What is the optimal outcome? Do I prefer A to B? How do we combine preferences from multiple agents? … Eliciting preferences  Users don’t want to answer lots of questions!  Are users going to be truthful when revealing their preferences?  …

Preference formalisms Psychological relevance  Can it express your preferences? Quantitative: I like wine twice as much as beer Qualitative: I prefer wine to beer Conditional: if we’re having meat, I prefer red wine to white …

Preference formalisms Expressive power  What types of ordering over outcomes can it represent? Total Partial Indifference Incomplete …

Preference formalisms Succinctness  How succinct is it compared to other formalisms? Can it (compactly) represent all that another formalism can? … Complexity  How difficult is it to reason with? What is the computationally complexity of ordering two choices? What is the computationally complexity of finding the most preferred choice? …

Utilities Map preferences onto a linear scale  Typically reals, naturals, … Issues  Cardinal or ordinal utility? Numbers meaningful or just ordering?  Different agents have different utility scales  Incomparability  Combinatorial domains First course x Main dish x Sweet x Wine x …

Ordering relation I prefer A to B (written A > B)  Transitive or not: if A > B and B > C then is A > C?  Total or partial: is every pair ordered?  Strict or not: A > B or A ≥ B  … Issues  Elicitation requires ranking O(m 2 ) pairs  Combinatorial domains  …

Case study: combinatorial auction Auctioneer  Puts up number of items for sale Agents  Submit bids for combinations of items Winner determination  Decide which bids to accept  Two agents cannot get the same item  Maximize revenue!

Case study: combinatorial auction Why are bids not additive?  Complements v(A & B) > v(A) + v(B) Left shoe of no value without right shoe  Substitutes v(A & B) < v(A) + v(B) As you can only drive one car at a time, a second Ferrari is not worth as much as the first  Auction mechanism that simply assigns items in turn may be sub-optimal How you value item depends on what you get later

Case study: combinatorial auction Winner determination problem  Deciding if there is a solution achieving a given revenue k (or more)  NP-complete in general Even if each agent submits jut a single bid And this bid has value 1

Case study: combinatorial auction Winner determination problem  Membership in NP Polynomial certificate Given allocation of goods, can compute revenue it generates

Case study: combinatorial auction Winner determination problem  NP-hard Reduction from set packing Given S, a collection of sets and a cardinality k, is there a subset of S of disjoint sets of size k? Items in sets are goods for auction One agent for each set in S, value 1 for goods in their set, 0 otherwise One other agent who bids 0 for all goods

Case study: combinatorial auction Winner determination problem  NP-hard One agent for each set in S, value 1 for goods in their set, 0 otherwise One special agent who bids 0 for all goods Allocation may not correspond to set packing Agents may be allocated goods with 0 value (ie outside their desired set) But can always move these goods over to special agent Revenue equal to cardinality of the subset of S

Case study: combinatorial auction Winner determination problem  Tractable cases Conflict graph: vertices = bids, edges = bids that cannot be accepted together If conflict graph is tree, then winner determination takes polynomial time Starting at leaves, accept bid if it is greater than best price achievable by best combination of its children

Case study: combinatorial auction Winner determination problem  Intractable cases Integer programming Heuristic search States = accepted bids Moves = accept/reject bid Initial state = no bids accepted Heuristics  Bid with high price & few goods  Bid that decomposes conflict graph

Case study: combinatorial auction Winner determination problem  Intractable cases Integer programming Heuristic search States = accepted bids Moves = accept/reject bid Initial state = no bids accepted Heuristics  Bid with high price & few goods  Bid that decomposes conflict graph

Case study: combinatorial auction Bidding languages  Used for agents to express their preferences over goods  If there are m goods, there are 2 m possible bids Many possibilities  Atomic bids  OR bids  XOR bids  OR* bids with dummy items  …

Case study: combinatorial auction Bidding languages: assumptions  Normalized v({})=0  Monotonic v(A) ≤ v(B) iff A  B Implies valuations are non-negative!

Case study: combinatorial auction Atomic bids  (B,p) “I want set of items B for price p” v(X) = p if X  B otherwise 0  Note this valuation is monotonic  Very limited range of preferences expressible as atomic bids  Cannot express even simple additive valuations

Case study: combinatorial auction OR bids  Disjunction of atomic bids (B1,p1) OR (B2,p2)  Value is max. sum of disjoint bundles v(X) = max { v 1 (X 1 ) + v 2 (X \ X 1 ) | X 1  X}  Not complete Can only express valuations without substitutes v(X u Y) ≥ v(X) + v(Y) Suppose you want just one item? v(S) = max{ v j | j  S }

Case study: combinatorial auction XOR bids  Disjunction of atomic bids but only one is wanted (B1,p1) XOR (B2,p2)  Value is max. of two possible valuations v(X) = max {v 1 (X), v 2 (X)}  Complete Can express any monotonic valuation Just list out all the differently valued sets of goods Hence XORs are more expressive than ORs

Case study: combinatorial auction XOR bids  Disjunction of atomic bids but only one is wanted (B1,p1) XOR (B2,p2)  Additive valuation requires O(2 k) XORs But only O(k) Ors  Thus, XORs are more expressive but less succinct than ORs

Case study: combinatorial auction OR/XOR bids  Arbitrary combinations of ORs and XORs  Bid := (B,p) | Bid OR Bid | Bid XOR Bid Recursively define semantics as before  B1 OR B2 v(X) = max { v 1 (X 1 ) + v 2 (X \ X 1 ) | X 1  X}  B1 XOR B2 v(X) = max { v 1 (X), v 2 (X) }

Case study: combinatorial auction Two special cases OR of XOR  Bid := XorBid | XorBid OR XorBid  XorBid := (B,p) | (B,p) XOR XorBid XOR of OR  Bid := OrBid | OrBid XOR OrBid  OrBid := (B,p) | (B,p) OR OrBid

Case study: combinatorial auction Downward sloping symmetric valuation  Items symmetric Only their number, k matters  Diminishing returns v(k)-v(k-1) ≥ v(k+1)-v(k) Using OR of XOR, such a valuation over n items is O(n 2 ) in size  Let p k = v(k)-v(k-1)  Then v(k) is ({x 1 },p 1 ) XOR.. XOR ({x n },p 1 ) OR ({x 1 },p 2 ) XOR.. XOR ({x n },p 2 ) OR.. OR ({x 1 },p n ) XOR.. XOR ({x n },p n )

Case study: combinatorial auction Downward sloping symmetric valuation  Items symmetric Only their number, k matters  Diminishing returns v(k)-v(k-1) ≥ v(k+1)-v(k) Using XOR of ORs (or OR) such a valuation is exponential in size  Need to represent all subsets of size k  OR of XORs is exponentially more succinct than XOR of ORs

Case study: combinatorial auction Monochromatic valuations  n/2 red and n/2 blue items  Want as many of one colour as possible v(X) = max {|X  Red|, |X  Blue|} With such a valuation  XOR of ORs is O(n) in size ({red 1,p}) OR.. OR ({red n/2 },p) XOR ({blue 1,p}) OR.. OR ({blue n/2,p})

Case study: combinatorial auction Monochromatic valuations  n/2 red and n/2 blue items  Want as many of one colour as possible v(X) = max {|X  Red|, |X  Blue|} With such a valuation  OR of XORs is O(2 n/2 ) in size Atomic bids in OR of XORs only need be monochromatic Removing non-monochromatic atomic bids will not change valuation of a monochromatic allocation Atomic bids need to have price equal to their cardinality Anything higher or lower will only value a monochromatic allocation incorrectly

Case study: combinatorial auction Monochromatic valuations  n/2 red and n/2 blue items  Want as many of one colour as possible v(X) = max {|X  Red|, |X  Blue|} With such a valuation  OR of XORs is O(2 n/2 ) in size There can be only a single XOR Suppose there are two (or more) XORs There are two cases:  One XOR is just blue, other is just red But then monochromatic valuation is not possible  One XOR is blue and red But then again monochromatic valuation is not possible

Case study: combinatorial auction Monochromatic valuations  n/2 red and n/2 blue items  Want as many of one colour as possible v(X) = max {|X  Red|, |X  Blue|} With such a valuation  OR of XORs is O(2 n/2 ) in size There can be only a single XOR This must contain all O(2 n/2 ) blue and O(2 n/2 ) red subsets XOR of ORs and OR of XORs are incomparable in succinctness

Case study: combinatorial auction OR* bids  Can modify OR bids so they can simulate XOR bids Recall that OR bids are not complete But XOR bids can be exponentially more succinct Get best of both worlds?  Introduce dummy items (which cannot be shared) to OR bids to make them simulate XOR (B u {dummy},p1) OR (C u {dummy},p2) is equivalent to (B,p1) XOR (C,p2)  Since XOR bids are complete, so are OR* bids

Case study: combinatorial auction OR* bids  Any OR/XOR bid of size O(s) can be represented as an OR* bid of size O(s) Homework exercise: prove this!  This bidding language still has limitations Majority valuation requires exponential sized OR* bid Any allocation of m/2 or more of the items has value 1 Any smaller allocation has value 0 No non-zero atomic bid in the OR* bid can have less than m/2 items Otherwise we could accept this set and violate majority valuation So we must have every n C n/2 possible subset of size n/2

Conclusions Wide variety of formalisms for representing preferences  Several dimensions along which to analyse them Completeness Succinctness Complexity of reasoning …