Game Theory and Empirics: What Can Go Wrong

Using Formal Theory to Identify and Characterize Biases James Fearon, “Signaling versus the Balance of Power and Interests” Kenneth Schultz, “Looking for Audience Costs” Alastair Smith, “To Intervene or Not to Intervene: A Biased Decision”

“Signaling versus the Balance of Power and Interests” Question: How do observable indicators of the balance of interests and capabilities influence the success of immediate extended deterrence? Intuition: The more the balance of interests and the balance of capabilities favor the defender, the more likely the defender’s deterrent threat will succeed. Findings: –Some indicators of defender interests (alliance, contiguity) predict immediate deterrence failure. –Past deterrent threat also predicts failure. –Inconsistent evidence on balance of capabilities.

The Theoretical Model

The Huth Deterrence Data Population: 58 cases of extended immediate deterrence –A challenger has made a threat against a protégé –A defender has made a deterrent threat to protect the protégé Data set records –Success or failure: Did challenger back down in the face of the defender’s threat? –Fight or not fight: If challenger did not back down, did defender make good on its threat?

Relationship of Data to Model Immediate deterrence

Possible Selection Effects The decision to challenge is conditioned on –the challenger’s type unobserved by analysts and the defender –the challenger’s prior beliefs about the defender’s type, esp. a D Observed, at best, with error by analysts common knowledge to participants –all other payoffs, including the stakes for the challenger, a C observed with error by analysts common knowledge to participants

Balance of Interests Hypothesis: Observable indicators of the defender’s interest in the protégé (which should increase a D ) will predict general deterrence success but immediate deterrence failure. Rationale: The challenger will only threaten a high-value protégé if it is highly motivated—i.e., a C is high. As a D increases, probability of challenge goes down, but probability that challenger will act given a threat and mobilization goes up.

A Note on Bias Is the estimated effect of alliance ties on deterrence outcomes biased? It is an accurate measure of the empirical association between these variables. It is biased estimate of the effect of alliance ties on the defender’s payoffs. –Table 6 seems to suggest that alliance ties decrease the credibility of the defender’s threat by decreasing a D or w D. –With the model, we see that, if alliance ties increase those parameters, the challenger’s self-selection on a C can reverse the sign in the observed relationship conditional on a challenge. –The bias arises because a C is an omitted variable that (conditional on a threat) is positively correlated with alliance ties but negatively correlated with deterrence success.

Past Deterrent Threats Hypothesis: A past deterrence crisis between the two states decreases the probability of immediate deterrence success. Rationale: If the defender made a previous deterrent threat or was previously a challenger, its type distribution going into the current crisis will have higher mean than if there was no prior crisis. As before, the more resolved the defender is initially expected to be, the more resolved the challenger must be in order to make a threat.

The Balance of Capabilities Hypothesis: The more the ex ante balance of capabilities favors the defender (increasing w D, decreasing w C ), the more likely it is that a deterrent threat will succeed. Rationale: Weak challengers only threaten strong defenders when they have a high value for the issue at stake. This should predict immediate deterrence failure. But: Weak challengers threaten on matters that they initially believe the defender does not care about. This means that a deterrent threat should lead to updating and immediate deterrence success.

Evidence Deterrence success more likely when –short-run and immediate balance of capabilities favors defender –major power defender vs. minor power challenger –defender possesses nuclear weapons (post 1945) Informal evidence that challenges in the face of large power asymmetries are “limited probes” on issues thought to be peripheral for stronger power. What we would like to see: Evidence that observable indicators of defender interests are negatively correlated with defender power advantage

Summary The fact that challengers are selecting into crises based on variables that we cannot directly observe (esp., a C, a D ) creates selection effects that complicate making inferences about –the effects of observable variables on game payoffs –whether our framework for understanding these kinds of interactions is correct This paper uses intuition from a formal model to think about the likely direction of the biases induced.

“Looking for Audience Costs” Questions: –Do state leaders incur domestic “audience costs” for backing down after making a threat? –Do democratic leaders incur higher costs than non- democratic leaders? Possible testing strategies –Indirect: Look for a correlation between variables thought to increase audience costs, e.g., democracy, and outcomes associated with higher audience costs, e.g., acquiescence by targets (Schultz 1999). –Direct: Look for whether leaders who backed down in a crisis face a higher risk of removal than those who did not. Can you make inferences about the distribution of audience costs (mean, covariates) by examining the sample of costs that are actually incurred?

The Theoretical Model C C T T (1,0) (0,1) (– a c,1) (1, – a t ) ( – w c, – w t ) Challenge Status Quo Concede Resist Back Down Back Down Stand Firm a c, a t known

Equilibrium Strategies wcwc wtwt kckc bcbc ktkt btbt CH/SF RS/SF War CH/BD RS/SF Challenger Backs Down SQ/BD RS/SF Status Quo CH/SF RS/BD Target Backs Down CH/BD RS/BD Challenger Backs Down SQ/BD RS/BD Status Quo CH/SF CD/BD Target Concedes CH/BD CD/BD Target Concedes SQ/BD CD/BD Status Quo Challenger’s audience costs observed Target’s audience costs observed

Strategic Censoring 1.b t is always decreasing in a c  probability of resistance decreases with the audience costs generated by the challenge 2.k c is nondecreasing in a c  probability that the challenger stands firm is nondecreasing in the audience costs it generated Therefore, increase in a c lowers the probability that a c is actually incurred.

wcwc wtwt kckc bcbc ktkt btbt Challenger’s audience costs observed Target’s audience costs observed Increasing the Challenger’s Costs

wcwc wtwt kckc bcbc ktkt btbt Challenger’s audience costs observed Target’s audience costs observed Increasing the Challenger’s Costs

Strategic Censoring 3.k c is nonincreasing in a t  probability that the challenger stands firm is nonincreasing in the audience costs generated by the target 4.k t is increasing a t  probability that the target stands firm is increasing in the audience costs it generated. Therefore, increase in a t lowers the probability that a t is actually incurred.

wcwc wtwt kckc bcbc ktkt btbt Challenger’s audience costs observed Target’s audience costs observed Increasing the Target’s Costs

wcwc wtwt kckc bcbc ktkt btbt Challenger’s audience costs observed Target’s audience costs observed Increasing the Target’s Costs

A Monte Carlo Demonstration Assume audience costs are generated as follows: Assume war costs are generated as follows:

A Monte Carlo Demonstration 1.Assume  c =  t = 0,  c =  t = 1,  = 1 audience costs same for targets and challengers ln a for democracies ~ N(1,1) ln a for non-democracies ~ N(0,1) 2.Generate 100 data sets of simulated data each set contains 50,000 dyad observations each state has 30 percent chance of being democratic for each observation, generate shocks and determine equilibrium outcome from the model 3.Compare summary statistics of actual and incurred audience costs (logged).

PopulationMeanStd. Deviation Full population All Democracies1.00 Nondemocracies Challengers that back down All Democracies Nondemocracies Targets that back down All Democracies Nondemocracies

Summary Efforts to find direct evidence of audience costs hampered by –partial observability: we can only measure them when they are incurred –strategic selection: the probability that they are incurred is a function of their value These problems tend to –lower the mean of observed audience costs –attenuate the effects of covariates on audience costs What is to be done?

“To Intervene or Not to Intervene: A Biased Decision” Questions: –Do states condition their behavior in crises on the expectation that allies will join in the event of war? –How reliable are alliance? Fact: Previous research finds that states come to aid of allies in war only about 25 percent of the time. If crisis behavior is conditioned on expectations of alliance reliability, then our estimates of reliability and its covariates can be biased.

The Theoretical Model A Attack Status Quo B C (an ally of B) Acquiesce Resist Honor Alliance Stay Out SQ ACQ INTBI-WAR

Modeling Reliability C’s decision to intervene: Can we estimate  c from observational data on whether C intervenes conditional on a war between A and B?

Sources of Selection Bias 1.B’s decision to resist is conditioned on C’s probability of honoring the alliance B is more likely to resist if C is more likely to intervene If B has information about C that we cannot control for, then this will bias upward our estimate of C’s reliability 2.A’s decision to attack is conditioned on C’s probability of honoring the alliance A is less likely to attack if C is more likely to intervene If A has information about C that we cannot control for, then this will bias downward our estimate of C’s reliability

How B’s Decision Introduces Bias B’s decision to resist: B’s decision to resist is conditioned on C’s reliability if Note: Does this make sense?

How B’s Decision Introduces Bias If B resists, If then Hence, in the sample of cases in which B has resisted, e c is not mean zero nor uncorrelated with X c.

A Test for Bias B does not condition on C if Smith shows that, if these conditions are met, then the distribution of X c is expected to be the same for all outcomes of the game. Hence, we can determine the distribution of reliability in the sample of cases that became wars, and test the null hypothesis that the distribution of reliability is the same in the sample of SQ and ACQ outcomes.

Data Observations on 1772 dyads with outcomes –SQ (no force used by either side): 975 –ACQ (force by one side only): 355 –War: 366 (Bigwar: 156) For each war, every ally of every participant in the war is an observation –Dependent variable: honor alliance or not –Independent variables: age of alliance, type of alliance, regime type, capabilities of C, expected utility indicators

Alliance reliability:

Summary Countries do condition behavior on expected reliability of allies. Average predicted probability of intervention at least.07 higher in cases in which B resists than in cases in which it does not. Evidence of bias, but what direction? What is to be done?