Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology 1 Evolving Reactive NPCs for the Real-Time Simulation Game Advisor ： Dr. Hsu Reporter ： Wen-Hsiang Hu Author ： JinHyuk Hong and Sung-Bae Cho IEEE Symposium on Computational Intelligence and Games

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 2 Outline Motivation Objective Introduction The game: Build & Build Basic behavior model Co-evolutionary behavior generation Experiment and Results Discussion Conclusion Personal Opinion

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 3 Motivation  AI in computer games has been highlighted in recent, but manual works for designing the AI cost a great deal.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 4 Objective  Designing NPCs’ behaviors without relying on human expertise.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 5 Basic behavior model  Two different grid scales are used for the input of the neural network such as 5×5 and 11×11. five neural networks are used to decide whether the associating action executes or not. The game: Build & Build random action probability: 0.2

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 6 Co-evolutionary behavior generation  We use the genetic algorithm to generate behavior systems that are accommodated to several environments.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 7 Experiment and Results  5×5 obtains lower winning averages for complex environment, while it performs better when the environment is rather simple.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 8 Introduction  It is challengeable for many researchers to apply AI to control characters. (AI produce more complex and realistic games.)  Finite state machines and rule-based systems are the most popular techniques in designing the movement of characters.  While neural networks, Bayesian network, and artificial life are recently adopted for flexible behaviors.  Evolution generates useful strategies automatically.  This paper proposes a reactive behavior system composed of neural networks is presented, and the system is optimized by co-evolution.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 9 Rule based approach  AI of many computer games is designed with rules based techniques such as finite state machines (FSMs) or fuzzy logic.  FSMs have a weak point of its stiffness; however, the movement of a character is apt to be unrealistic. ─ there is a trend towards fuzzy state machine (FuSM).

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 10 Adaptation and learning: NNs, EAs, and Artificial life  The adaptation and learning in games will be one of the most major issues making games more interesting and realistic.  Neural network, and evolutionary algorithms (e.g. genetic algorithm) are promising artificial intelligence techniques for learning in computer games. ─ NN - is badly trained ─ GE - required too many computations and were too slow to produce useful results.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 11 Co-evolution  By simultaneously evolving two or more species with coupled fitness.  Superior strategies for an environment have been discovered by co-evolutionary approaches.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 12 Reactive behavior  Reactive model performs effectively since it considers the current situation only.  Neural networks and behavior-based approaches are recently used for the reactive behavior of NPCs keeping the reality of behaviors.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 13 The game: Build & Build  ‘Build & Build’ developed in this research is a real- time strategic simulation game, in which two nations expand their own territory.  Each nation has soldiers who individually build towns and fight against the enemies, while a town continually produces soldiers for a given period.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 14 The game: Build & Build

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 15 Designing the game environment  The game starts two competitive units in a restricted land with an initial fund.  The units are able to take some actions at the normal land but not at the rock land.  A unit can build a town when the nation has enough money, while towns produce units using some money.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 16 Designing the game environment (cont.)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 17 Designing NPCs  NPC can move by 4 directions as well as build towns, attack units or towns, and merge with other NPCs.  The attack actions are automatically executed when an opponent locates beside the NPC.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 18 Designing NPCs (cont.)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 19 Designing NPCs (cont.)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 20 Basic behavior model (cont.)  Two different grid scales are used for the input of the neural network such as 5×5 and 11×11.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 21 Basic behavior model (cont.)  In order to actively seek a dynamic situation, the model selects a random action with a probability (in this paper, a = 0.2) in advance. five neural networks are used to decide whether the associating action executes or not.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 22 Co-evolutionary behavior generation  We use the genetic algorithm to generate behavior systems that are accommodated to several environments.  Two pair-wise competition patterns are adopted to effectively calculate the fitness of an individual.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 23 Co-evolutionary behavior generation (cont.)  The fitness of an individual is measured by the scores against randomly selected M opponents.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 24 Experiment and Results  Four different battle maps => demonstrate the proposed method in generating strategies adaptive to each environment.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 25 Experiment and Results (cont.)  The case with 11×11 shows more diverse behaviors than that with 5×5, since it observes information on a more large area.  5×5 obtains lower winning averages for complex environment, while it performs better when the environment is rather simple.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 26 Experiment and Results (cont.) Fig. 8. Winning rate between 5×5 behavior and 11×11 behavior at each generation on map type 3. The 11×11 shows the better performance than the 5×5, since it considers more various input conditions so as to generate diverse actions.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 27 Experiment and Results (cont.)  For the plain map, 5×5 behavior system shows a simple strategy that tries to build a town as much as possible. Building a town leads to generate many NPCs so as to slowly encroach on the battle map as showns in Fig. 9.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 28 Discussion  The reactive system shows good performance on simple environments like the plain map, but it does not work well for complex environments.  Also, the amount of input information is important for the reactive system when the environment is not simple.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 29 Conclusion  A reactive behavior system was presented for the flexible and reactive behavior of the NPC.  Co-evolutionary approaches have shown the potentialities of the automatic generation of excellent strategies corresponding to a specific environment.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 30 Personal Opinion  Strength ─ Designing NPCs’ behaviors without relying on human expertise.  Weakness ─ the limitation of direction  Application ─ real-time strategic simulation game