1. 10.3 Understanding Pattern Recognition Methods Chris Kramer

2. Pattern Recognition is... Abstracting relevant information from game world Constructing concepts or models and deducing patterns for higher-level reasoning and decision-making systems. necessary especially when game world is not deterministic built-in randomness player actions.

3. Approaches to Understanding Functional Derived from the roles Pattern Recognition has in games. Methodological Applying computer science concepts to PR in games

4. Functional Approach Think about role of PR in an actual game Sports game: “read” the match RTS: identify threats; strategize 1 on 1: react to opponent's favored moves 3 Aspects determine role of a PR system Level of decision making Stance toward player Use of modeled knowledge

5. Level of Decision Making 3 classic military levels: Strategic, Tactical, Operational Strategic: Lots of data and time; Many speculative decisions; cost of single poor decision is high Tactical: Mediator between Strategic and Operational; Accomplish strategic plan; Coordinate groups of entities; More real- time Operational: Very concrete; Many short-term, reactive decisions; individual unit actions

6. Stance toward player Enemy: provide a challenge; be purposeful, if not intelligent Ally: assist player; communicate in an accessible format; be consistent Neutral: fairness; especially in case of observer – AI which governs camera movement in sports game

7. Use of Modeled Knowledge Generators and Symbols – game world seen as a machine generating a series of different states or actions Symbols are the fundamental reasoning unit resulting from PR Sequence of symbols used in 2 ways: Prediction and Production Prediction: model player as generator and predict next symbol Production: AI models self as generator and determines best symbol to execute

8. Methodological Approach Apply general computer science innovations to pattern recognition in games 3 aspects: Optimization, Adaptation, Uncertainty Optimization: mathematical maximization of some objective function Adaptation: create model based on known results of previous models Uncertainty: applying methods which account for uncertainty

9. Optimization 3 mathematical elements: objective function, variables, constraints Techniques are iterative, time-consuming and usually offline Problem: like any math function, there are local optimums in addition to the global optimum. Many approaches to find global optimum – most focus on multiple traces of search space Genetic algorithms – best when variables are independent Swarm algorithms – iterators “fly” in the search space

10. Optimization Issues Usually multi-dimensional (many variables) – impossible to visually represent Computational difficulty can be eased with heuristic weakening of optimality criteria Real-time usage must be linear heuristic search with few variables.

11. Optimization Example: Age of Empires AoE: RTS game with many varied units and teams Genetic algorithm used to balance units Set up mock battles, use results of battles to guide selection Objective function: want both sides to have equal numbers of wins and losses

12. Adaptation Recognizing changing circumstances Best for indeterminate or unknown factors Methods include: Reinforcement Learning, Influence Maps, Neural Networks

13. Uncertainty Soft Computing refers to methods that account for uncertainty: probabalistic reasoning, fuzzy logic Fuzzy Sets: items have partial or probabalistic membership in a set. Fuzziness can be incorporated with other methods to better deal with uncertainty: Fuzzy genetic algorithm, Fuzzy Neural Nets

14. 10.5 Getting Around the Limits of Machine Learning Method of Analysis: 4 Questions Analysis of Machine Learning in 3 actual games Specific Limits to learning

15. Analysis Method 4 Questions: 1. Cheap to recognize what to learn from? 2. Cheap to store the knowledge? 3. Cheap to use the knowledge? 4. Does game benefit from learning? Major divisions of the problem: implicit vs. explicit online vs. offline

16. Black & White Player dictates good or bad creature behavior Explicit Online learning Answer to all 4 Questions: YES! #2 might have been problem in past Testing started early and was integral to project Innate programmed behaviors constrained learning

17. Command and Conquer: Renegade Unimplimented feature: if player goes between areas by an unkown path, AI will learn that path Online Implicit Learning 4 Questions: yes Path is already processed, Storage of new pathnodes is negligible

18. Re-Volt Genetic algorithm used during development to tune parameters of car AIs Offline Implicit Learning Lap Times provided simple criteria for selection

19. Limits to Learning Context of first 3 questions Recognizing something to learn from More realism in games results in problems similar to machine learning in the real world Noisy Inputs: Lots of data, Little useful data Time Dependencies Some algorithms to costly for online learning Use of knowledge can be expensive also

20. Good Knowledge Representation Helps Everywhere Preprocessing input signals and boiling them down to easier-to-use units High level of representation is easier to work with Beware loss of information MIDI vs. Raw audio

21. Seeing More Clearly Learning from the present: modern games are good at this Learning from past: Causal Chains Recognizing key precursors to learning event Keep detailed history to learn from Never been implemented

22. Seeing the Wrong Thing in the Right Examples Instance-based learning models often learn the wrong things Military System: detect if a tank is present in an aerial photograph Performed well on training and test sets But very poorly on real data All training photos of tanks were on sunny days, and photos of nothing were on cloudy days Must have “difficult” data

23. Storing New Knowledge Find ways to incorporate new knowledge online Complex algorithms – recalculation unfeasible Cannot lose performance against original data set Over-fitting Existing algorithm for neural nets: Temporal Differences

24. Conclusion Question #4 is the most important!