1. CS510 AI and Games Final Report on Dec. 09 2008 Juncao Li

2. juncao@cs.pdx.eduComputer Science, Portland State University2 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

3. juncao@cs.pdx.eduComputer Science, Portland State University3 Advanced Protection (AP) Author: Soren Johnson Turn-based strategy game –Player makes strategy each turn –AI plays against player’s strategy Static AI V.S. adaptive AI –Player learns the strategy of static AIs –Adaptive AI adapts based on users performance

4. juncao@cs.pdx.eduComputer Science, Portland State University4 AP (Cont.)

5. juncao@cs.pdx.eduComputer Science, Portland State University5 AP (Cont.)

6. juncao@cs.pdx.eduComputer Science, Portland State University6 AP (Cont.)

7. juncao@cs.pdx.eduComputer Science, Portland State University7 AP (Cont.) Features –Each minion has a brain (automaton) encoded by a 128-bit string –Four behaviors depending on the input: Move forward, turn right, turn left and do actions –250 brains for a minion to choose 20 hardcoded, 230 from genetic algorithm During the play, brains are rated based on their performance against the player –Dynamically choose best-fit brains Each player has their own easy and hard minion brains Players does not loose or win too much

8. juncao@cs.pdx.eduComputer Science, Portland State University8 Minion Inputs

9. juncao@cs.pdx.eduComputer Science, Portland State University9 Why Brain Matters

10. juncao@cs.pdx.eduComputer Science, Portland State University10 Why Brain Matters (Cont.)

11. juncao@cs.pdx.eduComputer Science, Portland State University11 Why Brain Matters (Cont.)

12. juncao@cs.pdx.eduComputer Science, Portland State University12 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

13. juncao@cs.pdx.eduComputer Science, Portland State University13 My Goal & Why Design AIs that play as players –Try to learn the player’s strategy –Try to perform best on all 250 brains Because Chaos selects brains based on player’s strategy Win (defined later) brains as much as it could Design benchmark for the AIs on each side –How well each AI performs Learning about players helps game designs –Player will not like a game that they can never win! –Player will not like a game that they can easily win!

14. juncao@cs.pdx.eduComputer Science, Portland State University14 Finite State Machine (FSM) Code the game strategy in a FSM Inputs or what matters –Treasury: Human and Chaos –Terrain and current human nodes on the map Output –Where to place the units –What units to place –Based on current state Allocate more farmers for earning money Allocate more infantry for attacking Chaos

15. juncao@cs.pdx.eduComputer Science, Portland State University15 FSM (Cont.)

16. juncao@cs.pdx.eduComputer Science, Portland State University16 FSM (Cont.)

17. juncao@cs.pdx.eduComputer Science, Portland State University17 FSM (Cont.)

18. juncao@cs.pdx.eduComputer Science, Portland State University18 FSM (Cont.) My FSM is static Hard to make it dynamic and adaptive –Too many possibilities (24*24 size map plus human ……) Works well as a benchmark –How well the Chaos AI performs Train Neural Network (NN) –Hopefully, teach the NN to recognize the map

19. juncao@cs.pdx.eduComputer Science, Portland State University19 Use Neural Network (NN) Input nodes: MAP_SIZE+2 –Map size: 24*24 = 576 –Human+Chaos treasury: 1+1 Output nodes: MAP_SIZE –Human units placement One hidden layer: MAP_SIZE+2 Normalize the inputs/outputs range: [0.0, 1.0] Use other’s NN code: –AI Game Engine: well developed, but not efficient –Tim Jones book: simple and efficient NN

20. juncao@cs.pdx.eduComputer Science, Portland State University20 Train the NN Firstly, use the static FSM to train the NN –Random generated maps –Random initial Human/Chaos treasury –FSM generates outputs as the training case Secondly, improve the NN –Randomly generate strategy –If it performs better than current NN –Train NN with the strategy

21. juncao@cs.pdx.eduComputer Science, Portland State University21 Understand the NN Outputs NN doesn’t always tell the exact answer So we have to “guess” Human_T MatchHuman(float type){ float threashold = 0.2; type = type*HUMAN_TYPES; if(type HUMAN_TYPES+threashold) return NO_HUMAN; if(fabs(type-float(DRONE))<threashold) return DRONE; else if(fabs(type-float(MINE))<threashold) return MINE; else if(fabs(type-float(FARMER))<threashold) return FARMER; else if(fabs(type-float(INFANTRY))<threashold) return INFANTRY; else if(fabs(type-float(SETTLER))<threashold) return SETTLER; else if(fabs(type-float(ARMOR))<threashold) return ARMOR; else if(fabs(type-float(JAMMER))<threashold) return JAMMER; else if(fabs(type-float(ARTILLERY))<threashold) return ARTILLERY; return NO_HUMAN; } CNNPlayer::DoTurns(){ …… //normalize the output float low = actual[0]; float high = actual[0]; float avg = 0; for(i=0; i<OUTPUT_NEURONS; i++){ if(low>actual[i]) low=actual[i]; if(high<actual[i]) high=actual[i]; avg+=actual[i]/OUTPUT_NEURONS; } for(i=0; i<OUTPUT_NEURONS; i++){ if(actual[i]<=0) actual[i] = 0; else{ actual[i] = (actual[i]-low)/(high-low); //actual[i] = actual[i]/avg; } …… }

22. juncao@cs.pdx.eduComputer Science, Portland State University22 NN Outputs (4 hour & 1M inter.)

23. juncao@cs.pdx.eduComputer Science, Portland State University23 Recall the FSM

24. juncao@cs.pdx.eduComputer Science, Portland State University24 A Random NN Give You This!

25. juncao@cs.pdx.eduComputer Science, Portland State University25 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

26. juncao@cs.pdx.eduComputer Science, Portland State University26 Evaluations Test based on each turn –Only for each turn, the Chaos’ brain is certain –My AI performs well (win) if and only if: Given random initial treasury and map It has advantage against most Chaos’ brains (money earned) Good performance on each turn leads to final win

27. juncao@cs.pdx.eduComputer Science, Portland State University 27 Statistics of My FSM Initial Treasury We want this! Not well dealing with money between this range

28. juncao@cs.pdx.eduComputer Science, Portland State University 28 Statistics of My NN AI Initial Treasury We want this!

29. juncao@cs.pdx.eduComputer Science, Portland State University29 Agenda About Advanced Protection My Design and Work Results and Evaluations Conclusion and Discussion

30. juncao@cs.pdx.eduComputer Science, Portland State University30 Conclusion Studied AP –Adaptive turn-based strategy game Static FSM –Benchmark –Train NN NN –Developed a demo –Trained by FSM –Need more time to train it

31. juncao@cs.pdx.eduComputer Science, Portland State University31 Discussion Static FSM –Efficient to code –Pros: do exactly what you want –Cons: do exactly what you want NN –Like dealing with a child Patient Be careful what you feed to it Try to understand a child’s language More trained, usually better performance

32. juncao@cs.pdx.eduComputer Science, Portland State University32 Discussion (Cont.) Coding & debugging issues –Error in coding NN is not easy to be detected Wrong training data Inconsistent between training inputs and using inputs –Be careful about important parts Training data Training process –Set breakpoint to check –Code review –Train it by the same example, see if it adapts as expected

33. juncao@cs.pdx.eduComputer Science, Portland State University33 Questions ?