1. A tractable combinatorial market maker using constraint generation MIROSLAV DUDÍK, SEBASTIEN LAHAIE, DAVID M. PENNOCK Microsoft Research Thanks: David Rothschild, Dan Osherson, Arvid Wang, Jake Abernethy, Rafael Frongillo, Rob Schapire

2. A combinatorial question: How pivotal was Ohio? Day before the election: 83.1% chance that whoever wins Ohio will win the election If Obama wins Ohio, 93.9% chance he’ll win the election If Romney wins Ohio, 53.2% chance he’ll win the election

3. More fun election-eve estimates 22% chance Romney will win in Iowa but Obama will win the national election 75.7% chance the same party will win both Michigan and Ohio 48.3% chance Obama gets 300 or more Electoral College votes 12.3% chance Obama will win between 6 and 8 states that begin with the letter M

4. More fun election-eve estimates 22% chance Romney will win in Iowa but Obama will win the national election 75.7% chance the same party will win both Michigan and Ohio 48.3% chance Obama gets 300 or more Electoral College votes 12.3% chance Obama will win between 6 and 8 states that begin with the letter M

5. Where did you get these numbers? A: We crowdsourced them http://PredictWiseQ.com A fully working beta example of our technical paper in ACM EC’12

6. The wisdom of crowds

7. More: http://blog.oddhead.com/2007/01/04/the-wisdom-of-the-probabilitysports-crowd/ http://www.overcomingbias.com/2007/02/how_and_when_to.html Ignore crowd: if you’re in the 99.7th percentile

8. The wisdom of fools Create a predictor by averaging everyone who scored below zero – 62nd place out of 2231 ! – (the best “fool” finished in 934th place)

9. Can we do better? model it - baseline model it - baseline++ poll a crowd - mTurk pay a crowd - probSports contest pay a crowd - Vegas market pay a crowd - TradeSports market guess “Prediction market”

10. An Example Prediction A random variable, e.g. Will US go into recession in 2013? (Y/N)

11. An Example Prediction Market A random variable, e.g. Turned into a financial instrument payoff = realized value of variable $1 if$0 if I am entitled to: Will US go into recession in 2013? (Y/N) Recession in 2013 No Recession in 2013

12. 2012 November 28 5:49 a.m. ET

13. Between 17.3% and 20.7% chance

15. http://www.predictwise.com/maps/2012president 11-05-2012 10:09AM

16. Design for Prediction Auctions/FinancialPrediction Markets PrimaryGains from tradeInformation SecondaryInformationGains from trade

17. Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,...

18. Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,... Goals for prediction – Info aggregation – 1. Liquidity – 2. Expressiveness – Bounded budget – Indiv. rationality – Comp. complexity Equilibrium – Rational expectations Competes with: experts, scoring rules, opinion pools, ML/stats, polls, Delphi

19. Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,... Goals for prediction – Info aggregation – 1. Liquidity – 2. Expressiveness – Bounded budget – Indiv. rationality – Comp. complexity Equilibrium – Rational expectations Competes with: experts, scoring rules, opinion pools, ML/stats, polls, Delphi

20. Why Liquidity?

21. Low liquidity takes the prediction out of markets http://blog.oddhead.com/2010/07/08/why-automated-market-makers/ Between 0.2% and 99.8% chance

22. Why Expressiveness?

27. Call option and put options are redundant Range bets require four trades ( “ butterfly spread ” ) Bid to buy call option @strike 15 can ’ t match with ask to sell @strike 10 Can ’ t set own strike Bottom line: Lacks expressiveness

28. Why Expressiveness? Dem Pres, Dem Senate, Dem House Dem Pres, Dem Senate, GOP House Dem Pres, GOP Senate, Dem House Dem Pres, GOP Senate, GOP House... Dem Pres Dem House Dem wins >=270 electoral votes Dem wins >=280 electoral votes...

29. Industry Standard Ignore relationships: Treat them as independent markets Las Vegas sports betting Kentucky horseracing Wall Street stock options High Streetspread betting

30. NYSE 1926 http://online.wsj.com/article/SB10001424052748704858404576134372454343538.html

31. NYSE 1987 http://online.wsj.com/article/SB10001424052748704858404576134372454343538.html

32. NYSE 2006-2011 2011 Deutsche Börse AG 2007 Euronext 2006 Archipelago, ipo

33. NYSE 7pm Sep 10, 2012

34. New Markets – Same CDA

35. A Better Way (Or,... Bringing trading into digital age) Expressiveness – Linear programming – Bossaerts, Fine, Ledyard: Combined Value Trading Fortnow et al.: Betting Boolean Style – http://bit.ly/multipm Expressiveness + Liquidity – Automated market maker – Always quote a price on anything – Downside: requires subsidy/risk

36. Example: Liquidity and Expressiveness

37. Getting Greedy Design a market for information on exponentially many things “Combinatorial prediction market”

38. Combinatorial securities: More information, more fun Payoff is function of common variables, e.g. 50 states elect Dem or Rep

39. Combinatorial securities: More information, more fun Dem will win California

40. Combinatorial securities: More information, more fun Dem will lose FL but win election Dem will win >8 of 10 Northeastern states Same party will win OH & PA

41. Combinatorial securities: More information, more fun There will be a path of blue from Canada to Mexico

42. Some Counting 54 “states”: 48 + DC + Maine (2), Nebraska (3) 2 54 = 18 quadrillion possible outcomes 2 2 54  10 18008915383333485 distinct predictions More than a googol, less than a googolplex NOT independent

43. Overview: Complexity results PermutationsBooleanTaxonomy GeneralPairSubsetGeneral2-clauseRestrict Tourney GeneralTree Auction- eer NP-hard EC’07 NP-hard EC’07 Poly EC’07 NP-hard DSS’05 co-NP- complete DSS’05 ??? Market Maker (LMSR) #P-hard EC’08 #P-hard EC’08 #P-hard EC’08 #P-hard EC’08 Approx STOC’08 EC’12 #P-hard EC’08 Poly STOC’08 #P-hard AAMAS ‘09 Poly AAMAS ‘09

44. A research methodology DesignBuildAnalyze HSX NF TS WSEX FX PS

45. Examples Design Prediction markets – Dynamic parimutuel – Combinatorial bids – Combinatorial outcomes – Shared scoring rules – Linear programming backbone Ad auctions Spam incentives BuildAnalyze Computational complexity Does money matter? Equilibrium analysis Wisdom of crowds: Combining experts Practical lessons Predictalot Yoopick Y!/O Buzz Centmail Pictcha Yootles

46. http://PredictWiseQ.com

48. Automated Market Maker ExchangeMarket Maker IndependentTractable No risk No info propagation Industry standard Tractable Exponential loss bound No info propagation CombinatorialNP-hard No risk Full info propagation Major liquidity problem #P-hard Linear/Const loss bound Full info propagation Info propagation  Reward traders for information, not computational power

49. Automated Market Maker ExchangeMarket Maker IndependentTractable No risk No info propagation Industry standard Tractable Exponential loss bound No info propagation Our approachTractable Good loss bound Some info propagation CombinatorialNP-hard No risk Full info propagation Major liquidity problem #P-hard Linear/Const loss bound Full info propagation Info propagation  Reward traders for information, not computational power

50. Our Approach: Approx Combo Market Maker Independent market makers for securities and small groups Parallel constraint generation to find and remove arbitrage Embedded in a convex optimization framework Deterministic: Better user experience (Previous Predictalot: Monte Carlo)

51. Our Contributions Separates pricing (must be fast) and information propagation New method to derive loss bound Empirical evaluation on over 300 thousand complex predictions Building this this for real! WiseQ Game on PredictWiseQ.com for 2012 US Presidential election

52. Consistent pricing 0 1 01 A&B’&C Independent markets

53. Consistent pricing 0 1 01 A&B’&C Independent markets Prices p

54. Consistent pricing 0 1 01 A&B’&C Independent markets

55. Consistent pricing 0 1 01 A&B’&C B = 0.6 A = 0.8 C = 0.9 Independent markets

56. Consistent pricing 0 1 01 0.6B = 0.6 0.8A = 0.8 A&B’&C 0.9C = 0.9

57. Consistent pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

58. Consistent pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

59. Consistent pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9 A=B A=C A=B’

60. Consistent pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9 A&C = 0.5

61. Consistent pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

62. Approximate pricing 0 1 01 0.6B = 0.6 0.4 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

63. Approximate pricing 0 1 01 0.6B = 0.6 0.4 Prices p 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

64. Approximate pricing 0 1 01 0.5B = 0.5 0.5  Buy NotB Prices p 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

65. Approximate pricing 0 1 01 0.5B = 0.5 0.5 Prices p 0.8A = 0.8 0.8 A&B’&C 0.9C = 0.9 0.9

66. Approximate pricing 0 1 01 0.8 0.5 A = 0.8 B = 0.55 0.5 0.8 Prices p A&B’&C 0.9C = 0.9 0.9

67. For Election Create 50 states – initialize with prior Create all groups of 2 – init as indep For conjunctions of 3 or more, group with it opposite disjunction: A&B&C, A’|B’|C’ Each group is indep MM – fast In parallel: Generate, find, and fix constraints

68. Arbitrage and Constraints Possibility of risk-free profit: Execute trades: – Buy x shares of A – Buy x shares of B – Sell x shares of A  B Prob[A] + Prob[B] ≥ Prob[A  B] Price[A] + Price[B] − Price[A  B] ≤ 0 September 26, 2012Microsoft Research, New York City

69. Constraints Clique lower bound P(L1|...|Lm) ≥Σ C P(Li) – Σ C P(Li&Lj) Spanning tree upper bound P(L1|...|Lm) ≤ Σ P(Li) – Σ T P(Li&Lj) Threshold constraints TBA Choosing constraints is key! – Depends on bets (unlike Monte Carlo) – An art

70. Does it work? Tested on over 300K complex predictions from Princeton study Budget 10 States

71. Does it work? Tested on over 300K complex predictions from Princeton study Budget Log Score 50 States

72. Does it work? Tested on over 300K complex predictions from Princeton study Revenue

73. No really, does it work? http://PredictWiseQ.com

77. Predictalot Mar 16 2011 Over 4000 variations of this 3-team prediction were placed

78. Predictalot alpha

79. Further reading Blog post on PredictWiseQ http://blog.oddhead.com/2012/ 10/06/predictwiseq/ http://blog.oddhead.com/2012/ 10/06/predictwiseq/ Gory details: What is (and what good is) a combinatorial prediction market? http://bit.ly/combopm http://bit.ly/combopm Guest post on Freakonomics http://bit.ly/combopmfreak http://bit.ly/combopmfreak Our paper in ACM EC’12 http://research.microsoft.com/apps /pubs/default.aspx?id=167977 http://research.microsoft.com/apps /pubs/default.aspx?id=167977