Presentation is loading. Please wait.

Presentation is loading. Please wait.

Planning and Learning in Games Michael van Lent Institute for Creative Technologies University of Southern California.

Similar presentations


Presentation on theme: "Planning and Learning in Games Michael van Lent Institute for Creative Technologies University of Southern California."— Presentation transcript:

1 Planning and Learning in Games Michael van Lent Institute for Creative Technologies University of Southern California

2 Business of Games 60% of Americans play video games $25 Billion dollar industry worldwide (2004) $11 Billion dollars in the US (2004) $6.1 billion in 1999, $5.5 billion in 1998, $4.4 billion in 1997. One day sales records Halo 2: $125 million in a single day Harry Potter (Half-blood Prince): $140 million single day Consoles dominate the industry 90% of sales (Microsoft, Sony, Nintendo) Average age of game players is 29 Average age of game buyers is 36 59% of game players are men

3 Game AI: A little context History of game AI in 5 bullet points Lots of work on path planning Hand-coded AI Finite state machines Scripted AI Embed hints in the environment Things are starting to change Game environments are getting more complex Players are getting more sophisticated Development costs are sky rocketing Incremental improvements are required to get a publisher Game developers are adopting new techniques Game AI is becoming more procedural and more adaptive

4 Scripted AI: Example 1 Age of Kings Microsoft ; The AI will attack once at 1100 seconds and then again ; every 1400 sec, provided it has enough defense soldiers. (defrule (game-time > 1100) =>(attack-now) (enable-timer 7 1100)) (defrule (timer-triggered 7) (defend-soldier-count >= 12) =>(attack-now) (disable-timer 7) (enable-timer 7 1400))

5 Scripted AI: Example 2 Age of Kings Microsoft (defrule(true)=> (enable-timer 4 3600) (disable-self)) (defrule (timer-triggered 4) => (cc-add-resource food 700) (cc-add-resource wood 700) (cc-add-resource gold 700) (disable-timer 4) (enable-timer 4 2700) )

6 The SIMS Maxis Procedural AI: The Sims

7

8

9

10 Two Adaptive AI Technologies Criteria First-hand experience Support procedural and adaptive AI Early stages of adoption by commercial developers

11 Two Adaptive AI Technologies Criteria Deliberative Planning F.E.A.R. (Monolith/Vivendi Universal for PC) Condemned (Monolith/Sega for Xbox 2)

12 Two Adaptive AI Technologies Criteria Deliberative Planning Machine Learning Long considered scary voodoo Decision tree induction & neural nets in Black & White Drivatar in Forza Motorsport

13 Why Planning and Learning? Improving current games More variable & replayable More immersive & engaging More customized experience More robust More challenging Improved profits More sales Marketing Cheaper development New elements of game play and whole new genres Necessary as games advance

14 Why not Planning and Learning? Costlier development Is the expense worth the result? Greater processor/memory load AI typically gets 10-20% of the CPU That time comes in frequent small slices Harder to control the players experience Harder to do quality assurance Double the cost of testing Adds technical risk Programmers need to spin up on new technologies Designers need to understand whats possible Designers create the AI; Programmers implement it Marketing backlash Once game is stable its too late to add a major feature

15 Why Planning and Learning? Improving current games More variable & replayable More immersive & engaging More customized experience More robust More challenging Improved profits More sales Marketing Cheaper development New elements of game play and whole new genres Necessary as games advance

16 Blah Blah blah Blah? Blah blah blah Blah blah & blah Blah blah blah Blah blah Improved profitsImproved profits Blah blah Blah Blah blah Blah blah blah blah blah blah blah blah blah Blah blah blah blah

17 Deliberative Planning What is deliberative planning? If you know the current state of the world and the goal state(s) of the world and the actions available When each can be done How each changes the world then search for a sequence of actions that changes the current state into a goal state. Deliberative planning is just a search problem When to plan? Off-line: Before/after each game session Real-time: During the game session During development: Not part of shipped product

18 Deliberative Planning Domain independent planning engine Abstract problem description Goal world state (Mission objective) secure(building1) clear(building1) & clear(building2) & clear(building3) captured(OpforLeader) or killed(OpforLeader)

19 Deliberative Planning Domain independent planning engine Abstract problem description Goal world state (Mission objective) Operators Team-Move (opfor,L?) Checkpoint (u1) Checkpoint (u2) Checkpoint (u3) (opfor at L?)(mobile opfor) (mobile u1) (mobile u2) (mobile u3) (u2 at L?) (u3 at L?) (u1 at L?)

20 Deliberative Planning Domain independent planning engine Abstract problem description Goal world state (Mission objective) Operators (base-secure) Secure-Base-Against-SW-Attack (at-base u?,u?,u?) Defend-Building (u?, b14) Secure-Perimeter-Against-SW-Attack (opfor) (u? at b14) Ambush (u?, sw-region) Patrol (u?, s-path) (u? at s-path) (u? at sw-region) (at-base u?,u?) (perimeter-secure)

21 Deliberative Planning Domain independent planning engine Abstract problem description Goal world state Operators Initial world state Deliberative Planning: Find a sequence of operators that change the initial world state into a goal world state.

22 Strategic Planning Example GoalInit (base-secure) Secure-Base-Against-SW-Attack Team-Move (opfor) (opfor at base) Checkpoint (u1) Defend-Building (u1, b14) Secure-Perimeter-Against-SW-Attack (opfor) Ambush (u3, sw-region) Patrol (u2, s-path) Checkpoint (u2) Checkpoint (u3) (u1 at b14) (u2 at s-path) (u3 at sw-region) (mobile opfor)

23 Plan Execution Execute atomic actions from plan Move from abstract planning world to real world Real-time interaction with environment 10+ sense/think/act cycles per second Ambush (u3, sw-region) Select-ambush-locMove-to-ambush-locWait-to-ambushAmbush-attackReport-success Report-failure Abandon-ambush Defend

24 Machine Learning: Behavior Capture Also called: Behavioral Cloning Learning by Observation Learning by Imitation A form of Knowledge Capture Learn by watching an expert Experts are good at performing the task Experts arent always good at teaching/explaining the task Learn believable, human-like behavior Mimic the styles of different players When to learn? During development Off-line

25 Drivatar Check out the revolutionary A.I. Drivatar technology: Train your own A.I. "Drivatars" to use the same racing techniques you do, so they can race for you in competitions or train new drivers on your team. Drivatar technology is the foundation of the human- like A.I. in Forza Motosport. Collaboration between Microsoft Games and Microsoft Research

26 Learning to Fly Learn a flight sim autopilot from observing human pilots 30 observations each from 3 experts 20 features (elevation, airspeed, twist, fuel, thrust…) 4 controls (elevators, rollers, thrust, flaps) Take off, level out, fly towards a mountain, return and land Key idea: Experts react to the same situation in different ways depending on their current goals Divide a flight sim task into 7 phases Learn four decision trees for each stage (one per control) Second key idea: Dont combine data from multiple experts Sammut, C. Hurst, S., Kedzier, D., and Michie, D. Learning to fly. In Proceedings of the Ninth International Conference on Machine Learning, pgs. 385-393, 1992.

27 KnoMic (Knowledge Mimic) Learn air combat in a flight sim and a deathmatch bot in Quake II Dynamic behavior against opponents Cant divide the task into fixed phases Key idea: Experts dynamically select which operator theyre working on based on opponent and environment Also learn when to select operators (pre-conditions) and what those operators do (effects) Second key idea: Experts annotation observations with their operator selections van Lent, M. & Laird, J. E., Learning Procedural Knowledge by Observation. Proceedings of the First International Conference on Knowledge Capture (K- CAP 2001), October 21-23, 2001, Victoria, BC, Canada, ACM, pp 179-186.

28 The Future

29 Where to learn more AI and Interactive Digital Entertainment Conference Marina del Rey, June 2006 Journal of Game Development Charles River Media Game Developer Magazine August special issue on AI Game Developers Conference AI Game Programming Wisdom book series Historical: 2005 IJCAI workshop on Reasoning, Representation and Learning in Computer Games AAAI Spring Symposiums 1999 – 2003 2004 AAAI Workshop

30 Interesting observations A few of my own: The most challenging opponent isnt the most fun. Never stupid is better than sometimes brilliant. Never underestimate the players ability to see intelligence where there is none. Game companies arent a source of research funds A few of Will Wrights: Maximize the ratio of internal complexity to perceived intelligence. The player will build an internal model of your system. If you dont help them build it, theyll probably build the wrong one. The flow of information about a system has a huge impact on the players perception of its intelligence. From the players point of view there is a fine line between complex behavior and random behavior.


Download ppt "Planning and Learning in Games Michael van Lent Institute for Creative Technologies University of Southern California."

Similar presentations


Ads by Google