1. How Shall We Play a Game? A Game-theoretical Model for Cyber-warfare Games
Tiffany Bao∗, Yan Shoshitaishvili†, Fish Wang†
Christopher Kruegel†, Giovanni Vigna†, David Brumley∗
∗Carnegie Mellon University, †UC Santa Barbara

2. Cyber Grand Challenge (CGC)
First Place: $2,000,000
Second Place: $1,000,000
Third Place: $750,000
One year ago, DARPA launched the Cyber Grand Challenge, which is a hacking competition for security systems to automatically discover vulnerabilities, patch vulnerabilities and attack each other.
In this competition, the first place won 2 million dollars, the second place won 1m dollars and the third place won 750 thousand dollars.

3. Strategy Matters First Place: $2,000,000 Second Place: $1,000,000
Third Place: $750,000
… if you choose to do nothing.
After the competition, people reviewed the game, and they found that in order to get the third place, all you need to do is do nothing. This observation is quite interesting. It shows to us how important to make the right decision. Even though you don’t have good technical skills, as long as you have a good strategy, you could still get a pretty good position.

4. Real World
National Security Agency discloses 91% of the zero-day vulnerabilities (that it discovers in software made and/or used in the U.S. to developers).
Admiral Michael Rogers, Director of the NSA
Looking beyond, the decision making for vulnerabilities is happening not only in hacking competitions, but also in reality, where the players are individuals, parties and countries.
For example, NSA makes strategic decision for undisclosed software vulnerability, aka zero-day vulnerabilitlies. ‘Admiral MG, the director of the NSA, stated that
There are many factors we need to consider for making the decision. In my talk, I will focus on three elements as follows.

5. 1. Action Sequence + For a zero-day vulnerability Withhold and Attack
Disclose and Patch +
First, we need to consider the actions over time.
For a zero-day vulnerability, we need to decide whether or. However, this is not a binary choices between a and p. We could combine these two together and have a strategy such as attack-then-disclose.
when to xx does make the outcome of the game different.

6. 1. Action Sequence Player 1 attacks Player 2
We could even have a strategy such as patch-then-attack. This is due to the fact that patching costs time for players, and the opponents could attack the players while patching is incomplete.

7. 2. Uncertainty of the Other Players
Has another player discovered the vulnerability yet? How likely will another player discover the vulnerability in the future?
The difference between knowing and not knowing about the other players is similar to the difference between chess and poker games. bishop poker games, such as Texas holdem, you don’t know what card they have, so you have to think about the odd that they have better cards.

8. 3. Ricochet & Patch-based Exploit Generation (PEG)
The Ricochet attack: to generate an exploit based on a receiving exploit [1]. The Patch-based Exploit Generation (PEG): to generate an exploit based on a receiving patch.
collateral damage
side effect
especially with the 1 factor. patching is not the end of the game. attack is not the end of the game. The side effect of the previous action might change the final outcome of the game.
[1] T. Bao, Y. Shoshitaishvili, R. Wang and D. Brumley. Your Exploit is Mine: Automatic Shellcode Transplant for Remote Exploits, Proceedings of the 38th IEEE Symposium on Security and Privacy, 2017.

9. Uncertainty of the other players
Previous Work
Cyber-hawk[2]
Schramm et al.[3]
Our Work
Action Sequence No
Yes
Uncertainty of the other players
Ricochet + PEG
[2] T. Moore, A. Friedman, and A. D. Procaccia. Would a ‘cyber warrior’ protect us? Exploring trade-offs between attack and defense of information systems. In Proceedings of the Workshop on New Security Paradigms, pages 85–94, 2010
[3] H. C. Schramm, D. L. Alderson, W. M. Carlyle, and N. B. Dimitrov. A game theoretic model of strategic conflict in cyberspace. Military Operations Research, 19(1):5–17, 2014.

10. Our Work: the Cyber-warfare Model
Scope
One vulnerability
Independent and rational players
Outline
One player: the player model
Multiple players: the game model
Nash equilibrium

11. Knowing a Zero-day Vulnerability
Player Model
Player
Knowing a Zero-day Vulnerability
Action
We do not consider secret patching
to attack, the player must generate the exploit for attack.
Player’s Machines

12. Player Model Player Action Player’s Machines Discover by self
Observe disclosure from the others
Action
Detect exploits from the others
We do not consider secret patching
Player’s Machines

13. Patch-based Exploit Generation
Player Model
Player
Exploit Generation
Discover by self
Patch-based Exploit Generation
Observe disclosure from the others
Action
Detect exploits from the others
The Ricochet Attack
We do not consider secret patching
Player’s Machines

14. Patch-based Exploit Generation
Player Parameters
Player
Exploit Generation
Discover by self
Attack
Patch-based Exploit Generation
Observe disclosure from the others
Patch
Detect exploits from the others
The Ricochet Attack
Parameters represent the capability of the technical components
Player’s Machines

15. Player State and Player Action
Player States
Not Discovered a zero-day vulnerability
Discovered a zero-day vulnerability
Player Actions
: Nop
: Attack, Patch, Stockpile

16. Player State and Player Action
Discovered
Not discovered
Collect Information
Make a Decision
Player state and action in one round
Attack
Stockpile
Patch
Nop
End

17. Multiple Players Not discovered Discovered Player 1 Player 2 Player 2
Nop
Attack
Stockpile
Patch
Nop
Player 2
Attack
Stockpile
Patch
Multiple players in each round
Nop
Attack
Stockpile
Patch
Player 1
Discovered
Not discovered

18. Rounded Game: Game Tree
Player 1
Player 2
Player 2
A, N
S, N
P, N
Nop
Attack
Stockpile
Patch
meaning that players in the game has incomplete information, they know their own state, but they may not certain about the other player’s state.
In each round
Player 1
Discovered
Not discovered

19. Stochastic Game
N, N
S, N
P, N
A, N

20. Incomplete Information
Player 1
Player 2
Nop
Attack
Stockpile
Patch
Nop
Player 2
Attack
Stockpile
Patch
Multiple players in each round
Nop
Attack
Stockpile
Patch
Player 1
Discovered
Not discovered

21. Player 1’s Perspective Not discovered Discovered Attack Stockpile
Patch
Multiple players in each round
Nop
Attack
Stockpile
Patch
Discovered
Not discovered

22. Player 2’s Perspective Not discovered Discovered Attack Stockpile
Patch
Nop
Attack
Stockpile
Patch
Multiple players in each round
Discovered
Not discovered

23. Ricochet + PEG Player Player Player 1 Player 2 Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
Player
Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
We do not consider secret patching
Player 1
Player 2

24. Ricochet Player Player Player 1 Player 2 Attack Attack
Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
Player
Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
Attack
We do not consider secret patching
Attack
Player 1
Player 2

25. Patch-based Exploit Generation
Player
Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
Player
Exploit Generation
Automatic Patch-based Exploit Generation
The Ricochet Attack
Attack
We do not consider secret patching
Patch
Player 1
Player 2

26. Game Model Therefore, we model the game as: a stochastic game, and
an incomplete information game.
Partial-observation Stochastic Game (POSG).

27. Computing Nash Equilibrium
Nash equilibrium: the strategy profile where all players play their optimal strategy.
Computing the Nash equilibrium for POSG is known to be intractable[4].
no analytical results
[4] L. MacDermed, C. L. Isbell, and L. Weiss. Markov games of incomplete information for multi-agent reinforcement learning. In Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence, pages 43–51, 2011.

28. Computing Nash Equilibrium
For the Cyber-warfare game, we observe:
Players infer the the other player’s state by player’s parameters.
Assuming the parameters are accessible, thus the inference is also public.
Convert from POSG to Stochastic Game (SG)
Compute the Nash equilibrium for SG using the Shapley Method (dynamic programming).
we assume that parameters are public. if not, players can estimate the parameters -> robustness

29. Evaluation 1: Review Previous Conclusions
Cyber-hawk[2]
Schramm et al.[3]
Our Work
Action Sequence No
Yes
Uncertainty of the other players
Ricochet+PEG
Conclusion
The attacking player(s) should attack right away.
It is possible that neither player wants to attack.
At least one player wants to attack.
Introduce by column
Each player decides a single action, either to attack or to disclose.
Each player do not know whether the other player has learned the same vulnerability.
[2] T. Moore, A. Friedman, and A. D. Procaccia. Would a ‘cyber warrior’ protect us? Exploring trade-offs between attack and defense of information systems. In Proceedings of the Workshop on New Security Paradigms, pages 85–94, 2010
[3] H. C. Schramm, D. L. Alderson, W. M. Carlyle, and N. B. Dimitrov. A game theoretic model of strategic conflict in cyberspace. Military Operations Research, 19(1):5–17, 2014.

30. Neither Player Attacks
Player 1 discovers the vulnerability
Player 2 generates the exploit
player 1 should never choose to ATTACK because he will suffer a greater loss if player 2 launches ricochet attacks. Player 1 should also never choose to STOCKPILE, because player 2 may re-discover the vulnerability and then ATTACK. Therefore, player 1’s best strategy is to PAT C H once he discovers the vulnerability. After player 1 discloses a vulnerability, player 2 receives the patch and generates exploits based on the patch, which costs him δ2 rounds. Within the rounds, player 1 would have completely patched his own machines, which makes any future attack from player 2 valueless.

31. Evaluation 2: Cyber Grand Challenge
Strategic-Shellphish: Shellphish + strategy based on the Cyber-warfare model.
Consider all the teams as one player.
Strategic-Shellphish
268543
Shellphish
254452
Downloads/cfe-submission

32. Conclusion
Cyber-warfare game, which addresses the limitations of previous work regarding:
Actions over time
Ricochet and Patch-based exploit generation
Uncertainty of the other player
We find a method to compute the Nash Equilibrium of the Cyber-warfare game.
Applications:
We observe that Ricochet may lead to neither players attack.
We could help teams such as Shellphish with more scores.
We proposed

33. Questions?

34. END

35. Multiple Players’ Actions over Time
T0. A vulnerability is introduced.
T1. Player 1 realizes the vulnerability.
T2. Player 1 launches an attack.
T3. Player 1 starts to patch and Player 2 realizes the vulnerability.