1. Expressiveness in mechanisms and its relation to efficiency: Our experience from $40 billion of combinatorial multi- attribute auctions, and recent theory Tuomas Sandholm Founder, Chairman, Chief Scientist CombineNet, Inc. Professor Computer Science Department Carnegie Mellon University

2. Outline Practical experiences with expressiveness Domain-independent measure of expressiveness – Theory on how it relates to efficiency Application of the theory to sponsored search Expressive ad (e.g., banner) auction that spans time

3. Sourcing before 2000 Structured, transparent Simultaneous negotiation with all suppliers Global competition Basic reverse auction Unstructured, nontransparent Sequential => difficult, suboptimal decisions 1-to-1 => lack of competition Expressive => win-win Implementable solution Manual negotiation ConsPros Bidding on predetermined lots is not expressive => ~ 0-sum game Lotting effort Small suppliers cant compete Unimplementable solution Bidding complexity & exposure

4. Expressive commerce Expressive bidding Expressive allocation evaluation

5. Expressive bidding Package bids of different forms Conditional discount offers of different forms (general trigger conditions, effects, combinations & sequencing) Discount schedules of different forms Side constraints, e.g. capacity constraints Multi-attribute bidding – alternates Detailed cost structures All of these are used in conjunction – Dont have to be used by all

6. Benefits of expressive bidding Pareto improvement in allocation 1.Finer-grained matching of supply and demand (e.g. less empty driving) 2.Exposure problems removed => better allocation & lower cost 3.Capacity constraints => suppliers can bid on everything 4.No need to pre-bundle => better bundling & less effort 5.Fosters creativity and innovation by suppliers 6.Collaborative bids => lower prices and better supplier relationships

7. Academic bidding languages unusable (in this application) OR [S. 99] XOR [S. 99] OR-of-XORs [S. 99] XOR-of-ORs [Nisan 00] OR* [Fujishima et al. 99, Nisan 00] Recursive logical bidding languages [Boutilier & Hoos 01] Fully expressive

8. Expressive allocation evaluation Side constraints – Counting constraints – Cost constraints – Unit constraints – Mixture constraints –…–… Expressions of how to evaluate bidder and bid attributes

9. Example of expressive allocation evaluation

10. Benefits of expressive allocation evaluation 1.Operational & legal constraints captured => implementable allocation 2.Can honor prior contractual obligations 3.Speed to contract: months weeks – $ savings begin to accrue earlier – Effort savings

11. Clearing (aka. winner determination) problem Allocate (& define) the business – so as to minimize cost (adjusted for buyers preferences) – subject to satisfying all constraints Even simple subclass NP-complete & inapproximable [S., Suri, Gilpin & Levine AAMAS-02] We solve problems ~100x bigger than competitors, on all dimensions: > 2,600,000 bids > 160,000 items (multiple units of each) > 300,000 side constraints > 1,000 suppliers Avg 20 sec, median 1 sec, some instances take days Speed & expressiveness: huge competitive advantage

12. CombineNet events so far > 500 procurement events – $2 million - $1.6 billion – The most expressive auctions ever conducted Total transaction volume > $40 billion Created 12.6% savings for customers – Constrained; Unconstrained was 15.4% Suppliers also benefited – Positive feedback (win-win, expression of efficiencies, differentiation, creativity) – Un-boycotting – They recommend use of CombineNet to other buyers

13. Applied to many areas Chemicals Aromatics Solvents Cylinder Gasses Colorants Packaging Cans & Ends Corrugated Boxes Corrugated Displays Flexible Film Folding Cartons Labels Plastic Caps/Closures Shrink/Stretch Film Ingredients/Raw Mat. Sugars/Sweeteners Meat/Protein Transportation Airfreight Ocean Freight Dray Truckload Less-than-truckload (LTL) Bulk Small Parcel Intermodal 3PLs Industrial Parts/Materials Bulk Electric Fasteners Filters Leased Equipment MRO Pipes/Valves/Fittings/Gauges Pumps Safety Supplies Steel Marketing Media buy Corrugate Displays Printed Materials Promotional Items Technology Security Cameras Computers Services Pre-press Temporary Labor Shuttling/Towing Warehousing Medical Pharmaceuticals Medical/surgical supplies Miscellaneous Office Supplies

14. Broader trend toward expressiveness

15. Amazon.c & New Egg o offer bundles of items (ca. 2000)

16. Facebook increases expressiveness of privacy control (2006) …we did a bad job of explaining what the new features were and an even worse job of giving you control of them…. This is the same reason we have built extensive privacy settings to give you even more control over who you share your information with.

17. CD+Tunes adds option for users to rent movies (2007)

18. Checked baggage Airlines charge extra for baggage, food & choice seats (2008)

19. Prediction/insurance markets becoming more expressive

20. Is more expressiveness always better? Not always for revenue! Expressive mechanism: v i ( )? v i ( )? v i ( )? An inexpressive mechanism: v i ( )?

21. Is more expressiveness always better for efficiency? And what is expressiveness, really? [Benisch, Sadeh & S. AAAI-08]

22. What makes a mechanism expressive? A straw man notion $5$2 Expression space 2 Item bid auction $5$2$6 Expression space 3 Combinatorial auction

23. What makes a mechanism expressive? Prop: Dimensionality of expression space does not suffice Expression space 1 ab Mapping Proof intuition [based on work of Georg Cantor, 1890] : Expression space 3 Work on informational complexity in mechanisms [Hurwicz, Mount, Reiter 1970s…] puts technical restrictions that preclude such mappings

24. Our notion: Expressive mechanisms allow agents more impact on outcome An agents impact is a measure of the outcomes it can choose between by altering only its own expression $X $Y $6 $4 A BD C $X$Y$6$4

25. Uncertainty introduces the need for greater impact $X$Y$6$4 $X $Y $4 $6 A,AA,CC,C C,D D,DB,D A,DA,B B,B

26. Uncertainty introduces the need for greater impact… $X$Y $3$7 $X $Y $3 $7 A,AA,CC,C C,D D,DB,D A,DA,B B,B

27. Uncertainty introduces the need for greater impact… $X$Y$6$4$3$7 $3 $4 $6 $7 $X $Y A,AA,AA,CA,CC,CC,C C,DC,D D,DD,DB,DB,D A,DA,DA,BA,B B,BB,B 10 outcome pairs but only 9 regions In this example the impact vector B,C cant be expressed

28. Expressive mechanisms $6$4$3$7 Our measure of expressiveness for one agent (semi-shattering): how many combinations of outcomes can he choose among Not just for combinatorial allocation problems because outcomes can be anything Captures multi-attribute considerations as well $X$Y$Z $X $Y D,DD,DB,DB,D A,BA,B B,BB,B B,CB,C A,AA,A Extra Region $X $Y A,AA,AA,CA,CC,CC,C C,DC,D D,DD,DB,DB,D A,DA,DA,BA,B B,BB,B Z=0 Some Z>0 $X $Y $Z In combinatorial auction all 10 pairs can be expressed $X $Y A,AA,AA,CA,CC,CC,C C,DC,D D,DD,DB,DB,D A,DA,DA,BA,B B,BB,B

29. An upper bound on a mechanisms best-case efficiency We study a mechanisms efficiency when agents cooperate It bounds the efficiency of any equilibrium It allows us to avoid computing equilibrium strategies It allows us to restrict our analysis to pure strategies only ??

30. Theorem: the upper bound on efficiency for an optimal mechanism increases strictly monotonically as more expressiveness (# of expressible impact vectors) is allowed (until full efficiency is reached) Proof intuition: induction on the number of expressible impact vectors; each time this is increased at least one more efficient outcome is allowed

31. Theorem: the upper bound on efficiency for an optimal mechanism can increase arbitrarily when any increase in expressiveness (# of expressible impact vectors) is allowed Proof intuition: construct preference distributions that ensure at least one type makes each combination of outcomes arbitrarily more efficient than any others

32. The bound can always be met Theorem: for any outcome function, there exists at least one payment function that yields a mechanism that achieves the bound's efficiency in Bayes-Nash equilibrium Proof intuition: if agents are charged their expected imposed externality (i.e., the inconvenience that they cause to other agents in the potentially inexpressive mechanism), then making expressions that maximize social welfare is an optimal strategy for each agent given that the others do so as well

33. Application to sponsored search [Benisch, Sadeh & S. Ad Auctions Workshop 2008]

34. Prototypical value advertiser Prototypical brand advertiser Bidder utility Rank percentile Heterogeneous bidder preferences

35. Mechanisms we compared Inexpressive mechanism $4 Rank 4 3 2 1 Premium mechanism $5 $4 4 3 2 1 Fully expressive mechanism $5 $4 $3 $2 4 3 2 1 Google, Yahoo!, Microsoft, … Our proposal Expressiveness

36. Inexpressive mechanism Premium mechanism Fully expressive mechanism

37. [Boutilier, Parkes, S. & Walsh AAAI-08] Expressive ad (e.g., banner) auctions that span time, and model-based online optimization for clearing

38. Prior expressiveness Typical expressiveness in existing ad auctions – Acceptable attributes – Per-unit bidding (per-impression/per-clickthrough (CT)) – Budgets – Single-period expressiveness (e.g., 1 day) Most prior research assumes this level of expressiveness

39. Campaign-level expressiveness Advertising campaigns express preferences over a sequence of allocations – Minimum targets: pay only if 100K impressions in a week – Tiered preferences: $0.20 per impression up to 30K, $0.50 per impression for more – Temporal sequencing: at least 20K impressions per day for 14 days – Substitution: either NYT ($0.90) or CNN ($0.50) but not both – Smoothness: impressions vary by no more than 20% daily – Long-term budget: spend no more that $250k in a month – Exclusivity Additional forms of expressiveness – Advertisers choice of impression/CT/conversion pricing (or combination) – Target audience (e.g., demographics) rather than indirectly via web site properties

40. Bidder 1: bids $1 on A, $0.50 on B, budget $50k Bidder 2: bids $0.50 on A, budget $20k Traditional first-price auction: $52.3k revenue Value of optimization under sequential expressiveness Supply of A Supply of B t0t0 t1t1 t2t2 50k0... 10k Bidder 2: 4.55kBidder 1: 45.45k Bidder 1: 9.09k

41. Bidder 1: bids $1 on A, $0.50 on B, budget $50k Bidder 2: bids $0.50 on A, budget $20k Optimal allocation: $70k revenue Supply of A Supply of B t0t0 t1t1 t2t2 50k0... 10k Bidder 1: 10k Bidder 1: 80k Bidder 2: 40k Value of optimization under sequential expressiveness

42. Stochastic optimization problem Advertising channels C Supply distribution of advertising channels P S Set of campaigns B Spot market demand distribution P D Time horizon T Can be modeled as Markov Decision Process (MDP) – But how do we make it scale?

43. Scalable optimization with real-time response Huge number of possible events => infeasible to compute full policy contingent on all future states Cannot reoptimize policy in real time at every event Optimize-and-dispatch architecture [Parkes and S., 2005] – Periodically compute policies with limited contingencies (e.g., stop dispatching when budget reached) – Dispatch in real time Policy form: x t i,j - fraction of channel i allocated to campaign j at time t Optimize over coarse time periods (e.g., minutes, hours) – Tradeoff between optimization speed and optimality – Finer-grained in near-term, coarse-grained in long-term

44. Channels A channel is an aggregation of properties (web pages or spots on them) Constructed automatically based on campaigns Lossless aggregation: two web pages are in the same channel if indistinguishable from the point of view of bids Example: – Bid 1: NY Times (NYT) – Bid 2: Medical article (Med) – Channels: (NYT Med), (NYT ¬Med), (¬ NYT Med) – Non-NYT pages grouped together, non-Med pages grouped together We can also perform lossy abstraction to avoid exponential blowup

45. Algorithms for stochastic problem Infeasible to solve the MDP – Huge state space – cross product of individual campaign states – High-dimensional continuous action space Our approaches: – Deterministic optimization – Online stochastic optimization

46. Deterministic optimization Replace uncertain channel supply with expectations Formulate the problem as a mixed-integer program (MIP) Solving a MIP is much faster than an MDP Our winner determination algorithms can solve very large problems [S. 2007] Solutions may be far from optimal if supply distributions have high variance – Does not adequately account for risk – Can be mitigated by periodic reoptimization

47. Sample-based online stochastic optimization [van Hentenryck & Bent 06] Compute only next action, rather than entire policy – Informed by what we might do in the future – Recompute at each time period Sample-based – Solve w.r.t. samples from distributions Extremely effective when good deterministic algorithms exist Requires that domain uncertainty is exogenous – Distribution of future events doesnt depend on decisions – Roughly true for advertising: allocation of ads should have little effect on supply of channel

48. REGRETS algorithm [Bent & van Hentenryck 04] Choose x t,i that maximizes f(x t,i ) Sample λ 1 Optimal solution...... Sample λ 2 Optimal solution Sample λ K Optimal solution λ 1 t+1 λ1tλ1t λ 1 t+2 λ1Tλ1T...λ 1 t+3 λ 1 t+4 λ 2 t+1 λ2tλ2t λ 2 t+2 λ2Tλ2T...λ 2 t+3 λ 2 t+4 λ K t+1 λKtλKt λ K t+2 λKTλKT...λ K t+3 λ K t+4 Time t ActionValue x t,1 f(x t,1 ) x t,2 f(x t,2 ) x t,n f(x t,n ) x t,3 f(x t,3 )............

49. Sample λ 1 Optimal solution...... Sample λ 2 Optimal solution Sample λ K Optimal solution λ 1 t+1 λ1tλ1t λ 1 t+2 λ1Tλ1T...λ 1 t+3 λ 1 t+4 λ 2 t+1 λ2tλ2t λ 2 t+2 λ2Tλ2T...λ 2 t+3 λ 2 t+4 λ K t+1 λKtλKt λ K t+2 λKTλKT...λ K t+3 λ K t+4 Lower bound on Q-values for action x t at time t...... REGRETS algorithm [Bent & van Hentenryck 04]

50. REGRETS doesnt apply to ad auctions Requires set of possible first-period decisions to be small Our dispatch policies are continuous Even a discretization of our continuous decision space would be huge: dimensionality = |C||B||Discretization|

51. Our extension of REGRETS to continuous action spaces Sample λ 1 Optimal solution from MIP:...... Sample λ 2 Optimal solution from MIP: Sample λ K Optimal solution from MIP: λ 1 t+1 λ1tλ1t λ 1 t+2 λ1Tλ1T...λ 1 t+3 λ 1 t+4 λ 2 t+1 λ2tλ2t λ 2 t+2 λ2Tλ2T...λ 2 t+3 λ 2 t+4 λ K t+1 λKtλKt λ K t+2 λKTλKT...λ K t+3 λ K t+4 Combining MIP......

52. Revenue: Flat bids MethodUnimodal supplyBimodal supply Bid-all25,687 ± 43614,004 ± 141 Myopic30,256 ± 43715,890 ± 175 Deterministic42,365 ± 58122,385 ± 227 Stochastic42,237 ± 58122,774 ± 238

53. Revenue: Bonus bids MethodUnimodal supplyBimodal supply Deterministic100,266 ± 3,55555,901 ± 1,887 Stochastic149,423 ± 3,20465,065 ± 2,356

54. Conclusions & future research Expressive mechanisms are practical & provide huge benefits Needed to develop natural concise expressiveness forms – Prior academic bidding languages not usable For efficiency, can add any expressiveness forms – Will help & can help an arbitrary amount; Bound can be met in BNE Uncertainty about others => need more expressiveness – Unlike in work solely on dominant-strategy mechanisms [Ronen 01], [Holzman et al. 04], [Blumrosen & Feldman 06] Sponsored search – GSP seems to run at a large inefficiency – Most of it fixable by our premium mechanism Expressive (banner) ad auctions that span time – Optimize-and-dispatch framework – Channel aggregation – Deterministic optimization with re-optimization – Online sample-based optimization – extended to continuous action space – Optimization provides significant benefits, even with no added expressiveness – Stochastic especially beneficial with non-linear preferences Most helpful expressiveness forms for other apps? – Agents preferences as input, our methodology can be used to evaluate different mechanisms – Psychological burden: expressing more vs. expressing strategically (e.g., chopsticks)