An Experimental Test of Information & Decision Markets Robin Hanson, Takashi Ishikida and John Ledyard Caltech 2/4/ minute presentation!

DIMACS2 Information Markets Standard Information Markets seem to work. –Small but complete set of securities –Many informationally small and unbiased traders. Theory and evidence from experiments and applications are all positive. But all assume, require, use, ….. –Straight-forward behavior Price taking, honest revelation, etc. –Complete set of state dependent contracts –Common knowledge of all priors, … Even then we see “failures to fully aggregate information” –Incomplete Bayesian updating –Incompletely revealing Rational Expectations Equilibrium

DIMACS3 Decision vs Prediction A policy maker does not just want to know the probability that “terrorist attacks in the US will increase in 2005.” They want to know the probability that “terrorist attacks in the US will increase in 2005” if “US troops remain in Iraq for 2005.” With N events (attacks, troop size, …) and S outcomes for each (increase from 10-20%, decrease, …), a complete set of state dependent contracts requires N^S - 1 contracts. S = 2, N = 8 => 255 contracts

DIMACS4 Remember PAM? Goal: Collect accurate predictive information on political and economic stability in the middle east.

DIMACS5 Every nation*quarter: -Political stability -Military activity -Economic growth -US $ aid -US military activity & all combinations & ……… 8 nations, 5 indices, 4 quarters Remember PAM?

DIMACS6 Every nation*quarter: -Political stability -Military activity -Economic growth -US $ aid -US military activity & all combinations & ……… 8 nations, 5 indices, 4 quarters (N = 180) Even if we only use up-down questions, completeness requires 2^180 =1.5*(10^54) contracts. Remember PAM?

DIMACS7 Using Conditional Contracts The good news –There may more overall trading. Traders may know more about and be more willing to trade on the relatively more precise event {terrorism up | troops up} as opposed to the less precise {terrorism up}. The bad news –There may be thinner trading per security. Too many markets to pay attention to. –Thinner trading => bad price discovery and incomplete arbitrage => prices do not aggregate information.

DIMACS8 Decision Markets Markets for Decision Analysis will be thin. –Large and possibly incomplete set of securities –Few informationally large and biased traders Theory is unlikely to be a good predictor of behavior. Current applications and experiments may not be applicable to the thinner situation. How can we know what will actually work?

DIMACS9 Experimental Test Beds Create an environment that”captures” as much of the problem as possible (the econ wind tunnel) –Three traders, three events with 2 outcome each (8 states) –Common prior with asymmetric information 10 draws from one urn of 6 equally likely => (1,0,1), (1,1,0),….. Each trader sees only two entries of each draw: (1,0,x), (1,1,x),… Run different mechanisms and market designs Measure performance –How close are final prices to the fully informed posterior?

DIMACS10 Theory Benchmarks - 3 events posteriors priors uniform

DIMACS11 Individual Scoring Rule posteriors priors uniform

DIMACS12 Standard Markets posteriors priors uniform

DIMACS13 Design Matters Asking is not enough. “Let there be markets” is not enough. Conjecture: An IM will work better in thin situations, if we use (to “thicken” trading) –Conditional contracts and –a Combinatoric (package bid) Call Market Includes “no arbitrage” pricing but is intermittent Does not directly address “monopolistic agents”

DIMACS14 Combinatoric Call Market posteriors priors uniform

DIMACS15 Design Matters We are not yet at complete aggregation. Conjecture: An IM will work even better in thin situations, if we use (to “thicken” trading) –Conditional contracts and –A Combinatoric Sequentially Shared (Market) Scoring Rule Is continuous and directly addresses report manipulation But it involves a subsidy to traders.

DIMACS16 Shared Scoring Rule - w/CC posteriors priors uniform

DIMACS17 Tentative Conclusions Standard markets and surveys do not work will in thin situations. Using conditional contracts and assuming some self - selection, either combinatoric call markets or combinatoric sequentially shared scoring rules significantly improve performance over standard markets.

DIMACS18 Open Questions There are many others we did not test –Pari-mutuel mechanisms Economides, Lange, and Longitude (some combinatorics) Pennock - Dynamic Pari-mutel Market Plott - Auction then Pari-mutuel

DIMACS19 Open Questions There are many others we did not test –Pari-mutuel mechanisms Economides, Lange, and Longitude (some combinatorics) Pennock - Dynamic Pari-mutel Market Plott - Auction then Pari-mutuel –Others HP - ……….

DIMACS20 Some Open Questions There are many other mechanisms we did not test. –Pari-mutuel mechanisms Economides, Lange, and Longitude (some combinatorics) Pennock - Dynamic Pari-mutel Market Plott - Auction then Pari-mutuel –Others HP - ………. There are many other environments we did not test in. –Information monopolist –External incentives to manipulate internally –And for PAM -- do these results survive in an ultra-thin world?

DIMACS21 A Force 12 Storm Create an environment that really stress- tests the mechanisms –Six traders, 8 events w/ two outcomes each (256 states) –Common prior with asymmetric information 10 draws from one urn of 8! equally likely: – (1,0,1,0,1,1,0,0), (1,1,0,0,0,0,0,0),….. Each trader sees only 4 different entries: –(1,0,x,0,x,1,x,x), (1,0,x,0,x,1,x,x), …

DIMACS22 posteriors uniform priors

DIMACS23 Summary of Testing Thin: 3 traders, 3 events 7 independent prices from 3 people in 12 minutes Markets < Individual Scoring Rule < Call < SSSR SSSR ~ Call given that the group beats the prior –With selection, SSSR and Call Market do best. Ultra-Thin: 6 traders, 8 events 255 independent prices from 6 people in 12 min. Markets ~ Individual Scoring Rule ~ Call < SSSR –SSSR beats the priors at the top (60%) –Nothing else even beats the priors –SSSR is only one with any aggregation

DIMACS24 Final Thoughts Information Markets are possible and desirable. –Can improve our ability to identify and deal with uncertainty. Many policy applications will be in thin situations. Traditional market designs do not work in thin situations. –Information monopolies, adverse decisions, partial updating The SSSR (w/conditionals) definitely sharpens the signal/noise ratio in thin and ultra-thin markets over traditional markets. Can we do better? Undoubtedly.

DIMACS25