A review of A Panorama of Artificial and Computational Intelligence in Games G. N. Yannakakis & J. Togelius October 2014 Elizabeth Camilleri

Overview I. Introduction II. 3 Panoramic views of Game AI Research 10 main Game AI reserach areas II. 3 Panoramic views of Game AI Research The method (computer) perspective The end user (human) perspective The player-game interaction perspective III. Interconnections between the 10 areas IV. Summary & Conclusions

I. Introduction (1) Artificial Intelligence (AI) – methods based on logic eg. Planning & reasoning Computational Intelligence (CI) – nature-inspired methods eg. Evolutionary computation & ANNs No official agreement on meanings “Game AI” + “AI in games” used interchangeably = both areas of AI + CI i.e. the field covering everything that requires some form of intelligence in games. Although there is no official agreement as to what Artificial Intelligence (AI) and Computational Intelligence (CI) really mean, it has been generally understood over the years that AI refers to methods based on logic such as planning and reasoning while CI refers to more nature-inspired methods such as evolutionary computation and neural networks. While their definitions are somewhat debatable, these research area names as well as the terms “game AI” and “AI in games” will be assumed in this paper as referring to the same field, i.e. the field covering everything that requires some form of intelligence in games.

I. Introduction (2) Dagstul Seminar on Artificial and Computational Intelligence in Games Several papers about (10) specific AI areas In contrast, this paper aims to: Connect the areas together Study interconnections actual potential Propose a taxonomy for a common understanding & vocabulary in Game AI/CI. Following the Dagstuhl Seminar on Artificial and Computational Intelligence in Games, several papers were written about the different research areas in AI. Each of these papers covered the current state-of-the-art of one research area in detail. As opposed to this approach, this paper was written in order to examine existing relationships and interconnections between the different research areas and to propose new potential interconnections with the aim of trying to connect the different areas together as much as possible and to help researchers understand how their research area relates to others as well as to give current and prospective researchers new research ideas. “We aim to facilitate and foster synergies across active research areas through placing all key research efforts into a taxonomy with the hope of developing a common understanding and vocabulary within the field of AI/CI in games.”

10 main Game AI research areas Non-player character (NPC) behaviour learning [NPC] Search and planning [S&P] Player modeling [PM] Games as AI benchmarks [AI Bench.] Procedural content generation [PCG] Computational narrative [CN] Believable agents [BA] AI-assisted game design [AI-Ass.] General game AI [GGAI] AI in commercial games [Com. AI] ((EXPLAIN EACH AREA IN SOME DETAIL)) The Dagstuhl Seminar highlighted ten main research areas in AI/CI which this paper addresses one by one and also collectively. The ten areas are: Obviously, as research in the area of AI/CI increases, new ideas and questions will surface and new methods will be conceived while older questions and methods may slowly fade out of consideration. In other words, the research areas will change and therefore the proposed taxonomy should in no way be considered as fixed. Even more so considering that the partitioning of the AI/CI area into ten independent yet interconnecting sections is essentially arbitrary should not be taken too concretely.

[NPC] – using reinforcement learning techniques to learn NPC behaviours that play games well. [S&P] – search is fundamental to computer science with many core algos being search algos (eg. Dijkstra). Planning is an application of search in state space – planning algos search for the shortest path from one state (start) to another (end). [PM] – computational models creating for detecting how the player perceives and reacts to gameplay through physiological measurements or questionnaires.

4) [AI-Bench. ] - games or parts of games (eg 4) [AI-Bench.] - games or parts of games (eg. levels or tasks) that offer a way to evaluate the performance of external AI systems on the task(s) associated with the benchmark. 5) [PCG] – the automatic creation of game content (eg. levels, maps, items, quests & textures). 6) [CN] – focuses on the representational & generational aspects of stories that can be told via a game. 7) [BA] – the study of mechanims for the construction of agent architectures that appear to have believable or human-like characteristics.

8) [AI-Ass.] - development of AI-powered tools that support the game design and development process – can assist in the creation of game content varying from levels & maps to game mechanics & narratives. 9) [GGAI] – the study of having game agents to competently play a large variety of games and not just one in particular. 10) [Com. AI] – AI that does not necessarily provide general solutions to deep problems (as in Academic Research but works well enough and looks good to the player. Eg. acceptable to give AI players extra information, teleport characers or invent units out of nothing.

II. 3 panoramic views of Game AI Research Authors first view Game AI from 3 different perspectives each with a different focus: The method (computer) The end user (human) The player-game interaction The paper views each of the different Game AI areas as a group of subareas with interconnections and interdependencies between them and other Game AI areas. The authors use three different perspectives to do so: one which focuses on the computer (i.e. the methods used), one which focuses on the human (i.e. the end users) and another which focuses on both (i.e. the human-computer interaction).

1. The methods (computer) perspective 6 main AI methods Evolutionary computation Reinforcement learning Supervised learning Unsupervised learning Planning Tree search Which methods are dominant or secondary in each of the 10 Game AI areas?

Some observations: Games as AI Benchmarks omitted as all methods are applicable. PM, BA, AI-Ass. = top 3 areas with most varied methods. PCG = area with least methods. Empty cells = potential new intersections between AI areas and methods In Table I, we may see which methods are dominant (i.e. are most commonly used in the available literature) and which methods are secondary (i.e. have been considered in a significant number of studies but are still not dominant) for each of the ten research areas previously defined. From the table we can notice some interesting relationships from the point of view of the methodologies. (See paper for more detailed analysis of relationships). The empty cells in Table 1 present several relationships between methodologies/techniques and Game AI areas which have not been explored before, or at least not in much depth. As we can see from the table, the space for new possibilities is quite large.

2. The end user (human) perspective 3 core dimensions involved in AI: The process that AI follows The context under which algorithms operate The end user type that benefits from this outcome Serves as framework to classify the 10 Game AI areas. Each area follows a process under a context for a particular end user type. The authors identify three core dimensions involved in AI: the process that the AI follows, the game context under which algorithms operate and the end user type that benefits most from the resulting outcome. In this perspective, the authors analyse the relationships between these three dimensions, particularly focusing on the end users. These relationships form the basis of the proposed taxonomy. This taxonomy serves as a framework for classifying the ten Game AI areas identified in the above sections. Each Game AI area follows a process under a context for a particular end user.

What can AI methods model, generate & evaluate? What can AI do within games? For whom? The authors identify modeling, generation and evaluation as the main forms of AI processing (i.e. what can AI do within games?) and content and behaviour as the main contexts of such processing (i.e. what can AI methods model, generate and evaluate?). Finally, they divide end users into four main types; the designer, the player, the AI researcher and the producer/publisher and identify which types of end users (for whom?) are affected by which combinations of process and context as we can see in Fig.1 below.

3. The player-game interaction perspective Focus on interaction between player (human) and game (computer) Use 6 Game AI areas that affect the Player end user type (see prev. Slide) Present the relationships involved in a player- game interaction framework In this perspective, the authors focus on the interaction between the player (human) and the game (computer). Using the results from Section II-B, they analyse how each of the six Game AI areas that affect the Player end user type and present a player-game interaction framework to illustrate the relationships involved as seen in Fig.2 below. The remaining four Game AI areas were not included in the framework since they have little to no effect on player-game interaction.

III. Interconnections between the 10 areas Analysis of how the areas influence each other Only direct influences (102 – 10 = 90 impractical) Existing and strong Outgoing: black area reached by arrow Incoming: thick solid red line around area Existing and weak Outgoing: dark grey area reached by arrow Incoming: solid red line around area Potentially strong Outgoing: light grey area reached by arrow Incoming: dotted red line around area In this section, the authors give an insight to each of the ten Game AI research areas and analyse how they inform/influence each other. They only look into direct influences as listing all possible influences would be impractical since 102 – 10 = 90. A direct influence can be existing and strong (●), existing and weak (○) or potentially strong (?). Influences that do not fall under these categories were not considered. Additionally, a figure is provided for every section (game AI area) which describes both its outgoing and incoming influences. Outgoing influences can be existing and strong (black area reached by arrow), existing and weak (dark grey area reached by arrow) or potentially strong (light grey area reached by arrow). Similarly, incoming influences can be existing and strong (thick solid red line around area), existing and weak (solid red line around area) and potentially strong (dotted red line around area).

1. NPC behaviour learning

2. Search and planning

3. Player modeling

4. Games as AI benchmarks

5. Procedural Content Generation

6. Computational Narrative

7. Believable Agents

8. AI-assisted game design

9. General Game AI

10. AI in commerial games

IV. Summary & Conclusions Identified 10 most active Game AI areas Results: Dominant algorithms + potential new methods in each area Different impact of each area on different end user types Influence of different areas on the player, the game and their interaction Placed on 3 holistic frameworks: AI method mapping End user taxonomy Player-game interaction loop

Most influential areas: Summary (contd.): Detailed analysis of the 10 key Game AI areas and their interconnections Most influential areas: Games as AI benchmarks NPC behaviour learning PCG General Game AI Most influenced areas: Commercial Game AI

Influences show much room for further exploration in research: Conclusions: Influences show much room for further exploration in research: Existing and strong (6) Existing and weak (33) Potentially strong (13) Areas currently very active: NPC behaviour learning + S&P +GGAI PM + PCG

Unexploited/underexploited or potentially strong connections: Conclusions (contd.): Currently strong areas (via clustering trending topics in recent conferences): PM PCG Narritive generation Unexploited/underexploited or potentially strong connections: PM -> BA BA -> PCG & AI-Ass. game design GGAI -> AI-Bench. PM, AI-Ass. game design, CN -> Com. AI

Apparent shift in the use of Game AI Conclusions (contd.): Apparent shift in the use of Game AI from NPC control and playing board games well (game agents) (more than 75% of conference papers links in 2005) to more non-traditional applications (52% links in 2011 excluding NPC & game agents) for the development of better games.

Thank you.