1. Advisor: Yeong-Sung Lin Presented by I-Ju Shih 2011/9/13 Modeling secrecy and deception in a multiple- period attacker–defender signaling game 1

2. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 2

3. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 3

4. Introduction 2011/9/13 Most applications of game theory to homeland-security resource allocation so far have involved only one-period games. Dresher (1961) was among the first researchers to apply game theory to military strategic interactions. However, he did not explicitly model deception and secrecy. Recent game-theoretic research has also indicated that publicizing defensive information instead of keeping it secret may help to deter attacks. 4

5. Introduction 2011/9/13 In practice, however, security-related information such as defensive resource allocations is often kept secret. There is a long tradition of deception in the military arena, as well as in business and capital ventures. Few of these studies have focused specifically on disclosure of resource allocations. Defenders might also have incentives to deceive by either overstating or understating their defenses, to deter or disinterest potential attackers, respectively. 5

6. Introduction 2011/9/13 Zhuang and Bier (2007) indicate that truthful disclosure should always be preferred to secrecy, which is not surprising, since their model is a game of complete information. Attacker uncertainty about defender private information can create opportunities for either defender secrecy or deception. Zhuang and Bier (2011) found that defender secrecy and/or deception could be strictly preferred in a one-period game in which the defender has private information (i.e., the attacker is uncertain about the defender type). 6

7. Introduction 2011/9/13 Secrecy has been sometimes modeled as simultaneous play in game theory, since in a simultaneous game, each player moves without knowing the moves chosen by the other players. Some researchers have modeled deception as sending noisy or imperfect signals to mislead one’s opponents. Hespanha et al. (2000) and Brown et al. (2005) defined deception in a zero-sum attacker-defender game as occurring when the defender discloses only a subset of the defenses, in an attempt to route attacks to heavily-defended locations. 7

8. Introduction 2011/9/13 By contrast, this paper defines deception as disclosing a signal (in the domain of the action space) that differs from the chosen (hidden) action. This paper applies game theory to model strategies of secrecy and deception in a multiple-period attacker-defender resource-allocation and signaling game with incomplete information. 8

9. Introduction 2011/9/13 Games are classified into two major classes: cooperative games and non-cooperative games. In traditional non-cooperative games it is assumed that 1. The players are rational. 2. There are no enforceable agreements between players. 3. The players know all the data of the game. However, real-game situations may involve other types of uncertainty. 9

10. Introduction 2011/9/13 In this paper they focus on the case where the defender does have private information, while the attacker does not. In this case, they allow two types of updates about the defender type – the attacker updates his knowledge about the defender type after observing the defender’s signals, and also after observing the result of a contest (if one occurs in any given time period). 10

11. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 11

12. Signaling game 2011/9/13 12 Games are classified into four major classes.

13. Signaling game 2011/9/13 13 A signaling game is a dynamic game of incomplete information involving two players, a Sender and a Receiver. It involves two players – one with private information, the other without – and two moves: first the informed player (Sender, she) makes a decision, she "sends a signal". then the uninformed player (Receiver, he) – having observed the informed player’s decision but not her private information – makes a decision, he "reacts to the signal".

14. Signaling game 2011/9/13 14 The timing of the game is as follows:  Nature selects a type t i for Sender from a set of feasible types T = {t 1,..., t I } according to a commonly-known probability distribution p(.), where p(t i ) > 0 (prior belief) for every i ∈ {1,...,I} and Σ p(t i ) = 1.  Sender observes t i and, on the basis of t i, chooses a message m j from a set of feasible messages M = {m 1,...,m J }.  Receiver observes m j and, on the basis of m j, selects an action a k from a set of feasible actions A ={a 1,...,a K }.  Payoffs are realised: if nature has drawn type t i, S has chosen message m j and R has selected action a k, then payoffs for S and R are u S (t i, m j, a k ) and u R (t i, m j, a k ).

15. Signaling game 2011/9/13 15

16. Signaling game 2011/9/13 16 Spence’s (1973) job market signalling model: Sender: a worker in search for a job. Receiver: a (potential) employer (or the market of prospective employers). Type: the worker’s productivity. Message: the worker’s education choice. Action: the wage paid to the worker.

17. Signaling game 2011/9/13 17 In a signaling game, there can be any or all of the following Perfect Bayesian Equilibrium (PBE): Pooling equilibrium: In a pooling PBE, both types of Sender choose the same message, so that they cannot be distinguished on the basis of their behavior. (pure strategy) Separating equilibrium: In a separating PBE, each Sender type chooses a different message, so that the message perfectly identifies the player type. (pure strategy) Semi-separating equilibrium: In a semi-separating PBE, one type of Sender plays a pure strategy while the other plays a mixed strategy. As a result, Receiver is able to imperfectly update his prior beliefs about Sender’s type. (mixed strategy)

18. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 18

19. Model formulation for repeated game 2011/9/13 19 This paper’s game has two players: an attacker (he, signal receiver, A); and a defender (she, signal sender, D). This paper’s model involves a N-period game with private defender information.

20. Model formulation for repeated game 2011/9/13 20 For simplicity, this paper considers only a two-type model; i.e., the defender type θ equals θ 1 with probability p 1 and θ 2 with probability 1-p 1. This paper assumes that p 1, the attacker’s prior probability at the beginning of the period 1, is common knowledge to both the attacker and the defender.

21. Model formulation for repeated game 2011/9/13 21 First, a defender of type θ chooses a strategy d t (θ) and a signal s t (θ) for θ = θ 1, θ 2. d t (θ) = 0 : The defender invests in short term expenses (such as police patrol) in period t. d t (θ) = 1 : The defender invests in capital defenses in period t. s t (θ) {0, 1, S} be the signal sent by a defender of type θ about its defensive choice.

22. Model formulation for repeated game 2011/9/13 22 The attacker observes the signal s t (θ), updates his belief from the prior p t to the posterior p' t, and chooses an attacker response a t (s t ). a t (s t ) = 0 is the decision to do nothing during period t. a t (s t ) = 1 represents the decision to launch an attack.

23. Model formulation for repeated game 2011/9/13 23 If both defender types send the same signal at equilibrium, s t (θ 1 ) = s t (θ 2 ), then p' t (posterior belief) = p t (prior belief). (Pooling equilibrium) If different defender types send different signals at equilibrium, s t (θ 1 ) ≠ s t (θ 2 ), then the attacker is able to recognize the defender type with certainty, in which case p' t = 1 with probability p t, and 0 with probability 1-p t. (Separating equilibrium)

24. Model formulation for repeated game 2011/9/13 24

25. Model formulation for repeated game 2011/9/13 25 This paper assumes for simplicity that the actual level of damage to the target is either 100% or zero. attacker defender Conditional probability that an attack would succeedattacker’s target valuations attack cost defender’s target valuationsConditional probability that an attack would succeed Defense cost

26. Model formulation for repeated game 2011/9/13 26 This contest success function is assumed to be of the form. where α > 1 is the effectiveness of defender short-term expenses relative to defender capital investment in security; ρ t-k is the fraction of defensive capital from period k that is still effective in period t. the effective defense short-term capital investment

27. Model formulation for repeated game 2011/9/13 27

28. Model formulation for repeated game 2011/9/13 28 Let β A and β D be the attacker and defender discount factors, respectively. (the current payoff, plus the discounted expected future equilibrium payoff) attacker defender

29. Model formulation for repeated game 2011/9/13 29 Definition 1. We call the collection {a*(s), d*(θ), s*(θ), p*, p'*} an equilibrium if the following four conditions are satisfied:

30. Model formulation for repeated game 2011/9/13 30 Definition 1. We call the collection {a*(s), d*(θ), s*(θ), p*, p'*} an equilibrium if the following four conditions are satisfied:

31. Model formulation for repeated game 2011/9/13 31 Definition 1. We call the collection {a*(s), d*(θ), s*(θ), p*, p'*} an equilibrium if the following four conditions are satisfied:

32. Model formulation for repeated game 2011/9/13 32 Definition 2. In an equilibrium {a*(s), d*(θ), s*(θ), p*, p‘*}, we say that in period t, a defender of type θ chooses: The cost of implementing truthful disclosure is lower than the costs of implementing secrecy and deception, respectively.

33. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 33

34. Attacker observes defensive investment from the previous period 2011/9/13 34 The model is under the assumption that the attacker can observe the previous period’s defensive choice, d t-1, at the beginning of period t. They still allow the defender’s private information to remain secret throughout the entire game, if not revealed by the defender’s choices. However, with this assumption, the defender cannot choose deception or secrecy at optimality for more than one time period.

35. Attacker observes defensive investment from the previous period 2011/9/13 35 For computational convenience, they assume that capital can be carried over only to the immediate next period. (ρ k = 0 for k ≥ 2, and ρ 1 = ρ)

36. Attacker observes defensive investment from the previous period 2011/9/13 36

37. Attacker observes defensive investment from the previous period 2011/9/13 37 Case A (p t = 0 or p t =1): In this case, at the beginning of period t, the attacker already knows whether the defender is of type θ = θ 2 or θ = θ 1.

38. Attacker observes defensive investment from the previous period 2011/9/13 38 For all 48 cases, we calculate e t using Eq. (11), and let p' t (posterior belief) = p t+1 (prior belief) = p t (prior belief). The attacker and defender total expected payoffs are calculated as the sum of the current payoff plus the discounted future equilibrium payoff:

39. Attacker observes defensive investment from the previous period 2011/9/13 39 Case B (0 < p t < 1): In this case, at the beginning of period t, the attacker is uncertain about the defender type, and we have a three player, 8*6*6 game. For all 288 cases, we calculate e t (θ) using Eq. (11), and then determine p' t stochastically as a function of s t (θ), s t (θ 2 ), and p t, using condition 3 of Definition 1.

40. Attacker observes defensive investment from the previous period 2011/9/13 40 the attacker payoff is given by: the payoff to a defender of type h is given by:

41. Attacker observes defensive investment from the previous period 2011/9/13 41 In the examples in the following sections, we use the following baseline parameter values: N = 2; p 1 = 0.9; β A = 0.9; β D (θ 1 ) = β D (θ 2 ) = 0.9; ρ(θ 1 ) = ρ(θ 2 ) = 0.5; α(θ 1 ) = α(θ 2 ) = 2; v A (θ 1 ) = v A (θ 2 ) = 20; v D (θ 1 ) = v D (θ 2 ) = 20. Moreover, we use the following baseline costs:

42. Attacker observes defensive investment from the previous period 2011/9/13 42 1. Effectiveness of expenses as defender private information Here, we let α(θ 1 ) = 2 and α(θ 2 ) = 4 be the defender private information. Defender’s strategyDefender’s signal

43. Attacker observes defensive investment from the previous period 2011/9/13 43 1. Effectiveness of expenses as defender private information Here, we let α(θ 1 ) = 2 and α(θ 2 ) = 4 be the defender private information. Defender’s strategyDefender’s signal θ1θ1 θ2θ2

44. Attacker observes defensive investment from the previous period 2011/9/13 44 1. Effectiveness of expenses as defender private information Here, we let α(θ 1 ) = 2 and α(θ 2 ) = 4 be the defender private information. Defender’s strategyDefender’s signal

45. Attacker observes defensive investment from the previous period 2011/9/13 45 1. Effectiveness of expenses as defender private information Here, we let α(θ 1 ) = 2 and α(θ 2 ) = 4 be the defender private information. Defender’s strategyDefender’s signal

46. Attacker observes defensive investment from the previous period 2011/9/13 46 2. Target valuation as private information We consider α(θ 1 ) = α(θ 2 ) = 1.5; v A (θ 1 ) = v D (θ 1 ) = 10 and v A (θ 1 ) = v D (θ 2 ) = 20. Defender’s strategyDefender’s signal θ1θ1 θ2θ2

47. Attacker observes defensive investment from the previous period 2011/9/13 47 3. Defender costs as private information We consider α(θ 1 ) = α(θ 2 ) = 2 and the defender of type θ 2 has higher costs for all signals than the defender of type θ 1 when the defenses are given by d = 0.

48. Attacker observes defensive investment from the previous period 2011/9/13 48 3. Defender costs as private information We consider α(θ 1 ) = α(θ 2 ) = 2 and the defender of type θ 2 has higher costs for all signals than the defender of type θ 1 when the defenses are given by d = 0. Defender’s strategyDefender’s signal

49. Attacker observes defensive investment from the previous period 2011/9/13 49 4. Other parameters as defender private information In cases where the defender’s private information is associated only with future payoffs (such as the carry-over coefficients ρ k and the discount rate β D ), they have not found deception or secrecy in their numerical model, despite an extensive computer search.

50. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 50

51. Attacker does not observe defensive investment 2011/9/13 51 For simplicity, this paper also assumes that the attacker does not observe the result of the contest from the previous period. Therefore, we need to solve a three-player 8 N *6 N *6 N game, where N is the number of periods. We let the cost be the defender’s private information.

52. Attacker does not observe defensive investment 2011/9/13 52

53. Agenda 2011/9/13 Introduction Signaling game Model formulation for repeated game Attacker observes defensive investment from the previous period Attacker does not observe defensive investment Conclusions and future research 53

54. Conclusions and future research 2011/9/13 54 This work uses game theory and dynamic programming to model a multiple-period, attacker– defender, resource-allocation and signaling game with incomplete information. This paper’s numerical examples show that defenders can sometimes achieve more cost-effective security through secrecy and deception in a multiple-period game. One limitation to this paper is that their algorithm does not automatically identify mixed strategies.

55. Conclusions and future research 2011/9/13 55 Although they found secrecy and deception as equilibrium strategies, which is somewhat unusual in the literature, such equilibria were relatively rare and difficult to obtain in our model, compared to the frequency with which secrecy and deception are observed in practice. They suspect that this may be at least in part because of some of the more unrealistic assumptions of game theory (e.g., common knowledge, full rationality).

56. Thanks for your listening. 2011/9/13 56