1. Ho-Chul Cho, Kyung-Joong Kim, Sung-Bae Cho
Replay-based Strategy Prediction and Build Order Adaptation for StarCraft AI Bots
Ho-Chul Cho, Kyung-Joong Kim, Sung-Bae Cho
Dept. of Computer Engineering, Sejong University, South Korea
Dept. of Computer Science, Yonsei University, South Korea

2. Outline Motivation Proposed Methods Experimental Results
Replay Analysis with Fog-of-War
Feature Expansion
Build Order Change
Experimental Results
Conclusions and Future Works

3. Motivation

4. StarCraft (a Popular RTS Game)
Units
Resource
Buildings

5. StarCraft AI Bot StarCraft AI Competitions BWAPI State of the Art
AIIDE (2010~)
CIG (2011~)
SSCAI (2012~)
BWAPI
State of the Art
Scouting : Early Stage of Development
Strategy Adaptation : Weak Points
Strategy Prediction : Low Performance
Combat Skills : Partially Successful
Mass Production : Good Points

6. Xelnaga (Developed by Authors)
Focus of this research

7. Fast construction of Air unit
Build Order as a Strategy
Units
Resource
Factory
Factory + Factory
Buildings
Fast Attack
Starport
Factory + Starport
Fast construction of Air unit
Command Center
Factory + Command Center
Gathering more resource

8. Relationship among Build Orders
Resource
We need to predict enemy’s build order and adapt build order of AI bot
Player
Opponent
Fast DT
Obs
Reaver
Drop
Expand
Unknown
0.50
0.59
0.67
0.64
0.00
0.41
0.52
0.49
0.29
0.33
0.48
0.71
0.36
0.51
0.31
1.00
0.69
Winning ratio of each strategy (0~1)

9. Scouting with Fog-of-war
Hostile units/buildings
Resource
Units
Buildings
Invisible area
Visible area
Scouting enemy’s
base by AI bots
Fog-of-War

10. Related Works Replay Mining
B. Weber, and M. Mateas, “A data mining approach to strategy prediction”
in IEEE Symposium on Computational Intelligence and Games

11. Overview of the Proposed Methods
Replay Mining without
Fog-of-War
Replay Mining with Fog-of-War
Low Prediction Accuracy
Improving Accuracy with Feature Expansion
No Build Order Adaptation
Build Order Adaptation

12. Proposed Methods
8~9분정도 나올듯 동영상까지.

13. Proposed Methods

14. Collecting Replays and Extraction
Collecting replays from StarCraft community sites
The “labeling” of strategy on the replays can be done by human experts
Game event logs are extracted from replays using BWAPI (with Fog of War) or Lord Martin Replay Browser
Game Event Logs (Raw data)
122 Protoss_Probe 138
123 Protoss_Probe 138
123 Protoss_Nexus 139
124 Protoss_Probe 138
124 Protoss_Probe 144
124 Protoss_Gateway 147
124 Protoss_Probe 143
124 Protoss_Probe 141
125 Protoss_Gateway 147
...

15. Realistic Game Log Extraction

16. Feature Vector Generator
A subset of an example feature vector
Attribute
Game Time
Pylon
1:20
Gateway
2:05
Gas
2:40
Expansion
11:00
Second Expansion
15:11
Third Expansion
18:45
Fourth Expansion
0:00
Second Gas

17. Feature Expansion Feature Expansion Replay1 112 170 220 Fast Legs… N
Strategy
Replay1
112
170
220
Fast Legs
Replay2
110
222
Fast DT
Replay3
120
190
Fast Expand
ReplayM
100
165
230
Unknown
Feature Expansion
X1 < X2
X1 < X3 …
XN-1>XN
Strategy
Replay1
True
False
Fast Legs
Replay2
Fast DT
Replay3
Fast Expand
ReplayM
Unknown

18. Feature Expansion Making new feature with time order of game events
The number of features is 0.5 * N * (N-1)
The new feature has one of “true” or “false” value

19. Feature Expanded Decision Tree
An example of standard and feature-expanded decision tree
(a) Standard decision tree
(b) Feature-Expanded decision tree

20. Strategy Prediction
Using machine learning algorithms (Nnge, KNN, J48, Bagging , Random Committee, Random Forest, and Rotation Forest)
Using FEDT(Feature Expanded Decision Tree)

21. Adaptation to Opponent’s Strategy
Resource 1
Prediction
(Machine learning)
Enemy
AI bot
Hidden Strategy
Strategy 1
Strategy 2 …
Strategy N 2
Which build order
is strong to opponent’s strategy?
(Statistic Analysis)
Player
Opponent
Fast DT
Obs
Reaver
Drop
Expand
Unknown
0.50
0.59
0.67
0.64
0.00
0.41
0.52
0.49
0.29
0.33
0.48
0.71
0.36
0.51
0.31
1.00
0.69 3
Changing AI Bot Strategy
(Can we generate build order from the models?)
(Feature Expanded Decision Tree)
Counter Strategy

22. Experimental Results

23. dd
Experimental Setup
The number of samples (FOW = Fog-of-War) (P = PROTOSS, T = TERRAN, Z = ZERG)
The number of strategies (class label) for each race is seven.
(ex. Protoss : Fast Dark Templar, Fast Observer, Fast Expansion, Fast Legs, Reaver Drop, Carrier, and Unknown)
Types
FOW
Raw Replays
# Samples
YGOSU.com
P vs. P O
1140 -
Weber et al.
542
P vs. T
1139
P vs. Z
1024
T vs. T
628
T vs. Z
1150
Z vs. Z
1010

24. Strategy Prediction by FEDT
Comparison of standard DT and the feature-expanded DT in terms of accuracy and the size of model (the number of leaves and the size of the tree) (W =Weber dataset, Y=YGOSU.com)
Race (Source)
Standard DT
Feature-Expanded DT
Accuracy (%)
Size
P (Y)
89.49
(157, 313)
99.73
(15, 29)
P (W)
89.68
(125, 249)
99.77
(14, 27)
T (W)
91.05
(122, 243)
99.96
(11, 21)
Z (W)
95.76
(72 , 143)
100.0
(10, 19)
Average
91.50
(119, 237)
99.87
(13, 24)
IF (FirstExpansion <= Stargate){
IF( RoboBay <= FirstExpansion){
IF(Citadel <= RoboBay){
IF(Legs <= Archives){
IF( FourthExpansion <= Legs)
“Unknown”
ELSE “Fast Legs” …
FEDT can make interpretable trees for human
FEDT can be easily converted into
programming codes
FEDT can make rules to label the replays
Conversion of feature-Expansion decision tree into a programming code

25. Standard Decision Tree

26. Feature Expanded Decision Tree

27. Strategy Prediction during the Game
P vs. P (YGOSU.com)
(without fog-of-war)

28. Strategy Prediction during the Game
P vs. P (YGOSU.com)
(with fog-of-war)

29. Build Order Adaptation
Successful Prediction
Unsuccessful Prediction
α is the maximum winning ratio if the player changes the current build order into new one (from the statistics) β is 0.5
Player
Opponent
Fast DT
Obs
Reaver
Drop
Expand
Unknown
0.50
0.59
0.67
0.64
0.00
0.41
0.52
0.49
0.29
0.33
0.48
0.71
0.36
0.51
0.31
1.00
0.69
Winning ratio of each strategy (0~1)
Expected win (0~1) from the prediction accuracy (Rotation Forest + FEDT) and the winning ratio of each strategy (P vs. P, YGOSU.com data)

30. Conclusions and Future Works

31. Conclusions and Future Works
The new framework for build order prediction and adaptation
Strategy prediction with realistic data and introduction of feature expansion.
Strategy adaptation by combining the statistical winning ratio and prediction accuracy
We need to improve accuracy of prediction with fog-of-war and apply this research to AI bots.

32. Thank You