A tractable combinatorial market maker using constraint generation MIROSLAV DUDÍK, SEBASTIEN LAHAIE, DAVID M. PENNOCK Microsoft Research Thanks: David Rothschild, Dan Osherson, Arvid Wang, Jake Abernethy, Rafael Frongillo, Rob Schapire

A combinatorial question: How pivotal was Ohio? Day before the election: 83.1% chance that whoever wins Ohio will win the election If Obama wins Ohio, 93.9% chance he’ll win the election If Romney wins Ohio, 53.2% chance he’ll win the election

More fun election-eve estimates 22% chance Romney will win in Iowa but Obama will win the national election 75.7% chance the same party will win both Michigan and Ohio 48.3% chance Obama gets 300 or more Electoral College votes 12.3% chance Obama will win between 6 and 8 states that begin with the letter M

More fun election-eve estimates 22% chance Romney will win in Iowa but Obama will win the national election 75.7% chance the same party will win both Michigan and Ohio 48.3% chance Obama gets 300 or more Electoral College votes 12.3% chance Obama will win between 6 and 8 states that begin with the letter M

Where did you get these numbers? A: We crowdsourced them A fully working beta example of our technical paper in ACM EC’12

The wisdom of crowds

More: Ignore crowd: if you’re in the 99.7th percentile

The wisdom of fools Create a predictor by averaging everyone who scored below zero – 62nd place out of 2231 ! – (the best “fool” finished in 934th place)

Can we do better? model it - baseline model it - baseline++ poll a crowd - mTurk pay a crowd - probSports contest pay a crowd - Vegas market pay a crowd - TradeSports market guess “Prediction market”

An Example Prediction A random variable, e.g. Will US go into recession in 2013? (Y/N)

An Example Prediction Market A random variable, e.g. Turned into a financial instrument payoff = realized value of variable $1 if$0 if I am entitled to: Will US go into recession in 2013? (Y/N) Recession in 2013 No Recession in 2013

2012 November 28 5:49 a.m. ET

Between 17.3% and 20.7% chance

:09AM

Design for Prediction Auctions/FinancialPrediction Markets PrimaryGains from tradeInformation SecondaryInformationGains from trade

Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,...

Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,... Goals for prediction – Info aggregation – 1. Liquidity – 2. Expressiveness – Bounded budget – Indiv. rationality – Comp. complexity Equilibrium – Rational expectations Competes with: experts, scoring rules, opinion pools, ML/stats, polls, Delphi

Design for Prediction Goals for trade – Efficiency (gains) – Inidiv. rationality – Budget balance – Revenue – Comp. complexity Equilibrium – General, Nash,... Goals for prediction – Info aggregation – 1. Liquidity – 2. Expressiveness – Bounded budget – Indiv. rationality – Comp. complexity Equilibrium – Rational expectations Competes with: experts, scoring rules, opinion pools, ML/stats, polls, Delphi

Why Liquidity?

Low liquidity takes the prediction out of markets Between 0.2% and 99.8% chance

Why Expressiveness?

Call option and put options are redundant Range bets require four trades ( “ butterfly spread ” ) Bid to buy call 15 can ’ t match with ask to 10 Can ’ t set own strike Bottom line: Lacks expressiveness

Why Expressiveness? Dem Pres, Dem Senate, Dem House Dem Pres, Dem Senate, GOP House Dem Pres, GOP Senate, Dem House Dem Pres, GOP Senate, GOP House... Dem Pres Dem House Dem wins >=270 electoral votes Dem wins >=280 electoral votes...

Industry Standard Ignore relationships: Treat them as independent markets Las Vegas sports betting Kentucky horseracing Wall Street stock options High Streetspread betting

NYSE

NYSE

NYSE Deutsche Börse AG 2007 Euronext 2006 Archipelago, ipo

NYSE 7pm Sep 10, 2012

New Markets – Same CDA

A Better Way (Or,... Bringing trading into digital age) Expressiveness – Linear programming – Bossaerts, Fine, Ledyard: Combined Value Trading Fortnow et al.: Betting Boolean Style – Expressiveness + Liquidity – Automated market maker – Always quote a price on anything – Downside: requires subsidy/risk

Example: Liquidity and Expressiveness

Getting Greedy Design a market for information on exponentially many things “Combinatorial prediction market”

Combinatorial securities: More information, more fun Payoff is function of common variables, e.g. 50 states elect Dem or Rep

Combinatorial securities: More information, more fun Dem will win California

Combinatorial securities: More information, more fun Dem will lose FL but win election Dem will win >8 of 10 Northeastern states Same party will win OH & PA

Combinatorial securities: More information, more fun There will be a path of blue from Canada to Mexico

Some Counting 54 “states”: 48 + DC + Maine (2), Nebraska (3) 2 54 = 18 quadrillion possible outcomes  distinct predictions More than a googol, less than a googolplex NOT independent

Overview: Complexity results PermutationsBooleanTaxonomy GeneralPairSubsetGeneral2-clauseRestrict Tourney GeneralTree Auction- eer NP-hard EC’07 NP-hard EC’07 Poly EC’07 NP-hard DSS’05 co-NP- complete DSS’05 ??? Market Maker (LMSR) #P-hard EC’08 #P-hard EC’08 #P-hard EC’08 #P-hard EC’08 Approx STOC’08 EC’12 #P-hard EC’08 Poly STOC’08 #P-hard AAMAS ‘09 Poly AAMAS ‘09

A research methodology DesignBuildAnalyze HSX NF TS WSEX FX PS

Examples Design Prediction markets – Dynamic parimutuel – Combinatorial bids – Combinatorial outcomes – Shared scoring rules – Linear programming backbone Ad auctions Spam incentives BuildAnalyze Computational complexity Does money matter? Equilibrium analysis Wisdom of crowds: Combining experts Practical lessons Predictalot Yoopick Y!/O Buzz Centmail Pictcha Yootles

Automated Market Maker ExchangeMarket Maker IndependentTractable No risk No info propagation Industry standard Tractable Exponential loss bound No info propagation CombinatorialNP-hard No risk Full info propagation Major liquidity problem #P-hard Linear/Const loss bound Full info propagation Info propagation  Reward traders for information, not computational power

Automated Market Maker ExchangeMarket Maker IndependentTractable No risk No info propagation Industry standard Tractable Exponential loss bound No info propagation Our approachTractable Good loss bound Some info propagation CombinatorialNP-hard No risk Full info propagation Major liquidity problem #P-hard Linear/Const loss bound Full info propagation Info propagation  Reward traders for information, not computational power

Our Approach: Approx Combo Market Maker Independent market makers for securities and small groups Parallel constraint generation to find and remove arbitrage Embedded in a convex optimization framework Deterministic: Better user experience (Previous Predictalot: Monte Carlo)

Our Contributions Separates pricing (must be fast) and information propagation New method to derive loss bound Empirical evaluation on over 300 thousand complex predictions Building this this for real! WiseQ Game on PredictWiseQ.com for 2012 US Presidential election

Consistent pricing A&B’&C Independent markets

Consistent pricing A&B’&C Independent markets Prices p

Consistent pricing A&B’&C Independent markets

Consistent pricing A&B’&C B = 0.6 A = 0.8 C = 0.9 Independent markets

Consistent pricing B = A = 0.8 A&B’&C 0.9C = 0.9

Consistent pricing B = A = A&B’&C 0.9C =

Consistent pricing B = A = A&B’&C 0.9C =

Consistent pricing B = A = A&B’&C 0.9C = A=B A=C A=B’

Consistent pricing B = A = A&B’&C 0.9C = A&C = 0.5

Consistent pricing B = A = A&B’&C 0.9C =

Approximate pricing B = A = A&B’&C 0.9C =

Approximate pricing B = Prices p 0.8A = A&B’&C 0.9C =

Approximate pricing B =  Buy NotB Prices p 0.8A = A&B’&C 0.9C =

Approximate pricing B = Prices p 0.8A = A&B’&C 0.9C =

Approximate pricing A = 0.8 B = Prices p A&B’&C 0.9C =

For Election Create 50 states – initialize with prior Create all groups of 2 – init as indep For conjunctions of 3 or more, group with it opposite disjunction: A&B&C, A’|B’|C’ Each group is indep MM – fast In parallel: Generate, find, and fix constraints

Arbitrage and Constraints Possibility of risk-free profit: Execute trades: – Buy x shares of A – Buy x shares of B – Sell x shares of A  B Prob[A] + Prob[B] ≥ Prob[A  B] Price[A] + Price[B] − Price[A  B] ≤ 0 September 26, 2012Microsoft Research, New York City

Constraints Clique lower bound P(L1|...|Lm) ≥Σ C P(Li) – Σ C P(Li&Lj) Spanning tree upper bound P(L1|...|Lm) ≤ Σ P(Li) – Σ T P(Li&Lj) Threshold constraints TBA Choosing constraints is key! – Depends on bets (unlike Monte Carlo) – An art

Does it work? Tested on over 300K complex predictions from Princeton study Budget 10 States

Does it work? Tested on over 300K complex predictions from Princeton study Budget Log Score 50 States

Does it work? Tested on over 300K complex predictions from Princeton study Revenue

No really, does it work?

Predictalot Mar Over 4000 variations of this 3-team prediction were placed

Predictalot alpha

Further reading Blog post on PredictWiseQ 10/06/predictwiseq/ 10/06/predictwiseq/ Gory details: What is (and what good is) a combinatorial prediction market? Guest post on Freakonomics Our paper in ACM EC’12 /pubs/default.aspx?id= /pubs/default.aspx?id=167977