Artificial Intelligence in Game Design Probabilistic Finite State Machines and Markov State Machines

Randomness Inside State Randomness in actions taken by NPC Randomness inside update method Can depend on current state Confident Angry Frightened Attack Left 40% 60% 30% Attack Right 35% 20% Defend 5% 50%

Randomness Inside State Randomness in Initial Setup Randomness in enter method Example: choice of weapon in Fight state 60% 35% 5%

Randomness in Transitions Same current state + same stimuli = one of several possible next states Possibly including current state Performing different tasks at random Guard Door and Shout for Help Player visible 60% Patrol in front of Door 40% Player visible Chase Player

Random Behavior Timeouts Continue strong emotional behavior for random number of steps Predator seen Predator seen Wander Flee Predator not seen 10% Predator not seen 90%

Unpredictability of World Small chance of “unexpected” occurrence Adds “newness” to game even after multiple plays Adds to “realism” of world Target in sights 98% Reload Aim Fire Normal case Finished reloading Target in sights 2% Gun Jam Gun cleared Unexpected case

Randomness in Emotional States Emotional transitions less predictable Effect of “delayed reaction” Small hit by player 75% Player HP < 10 40% Player HP < 10 60% Confident Angry Small hit by player 25% Heavy hit by player 70% My HP < 10 50% Heavy hit by player 30% My HP < 10 50% Heavy hit by me 30% Frightened Heavy hit by me 70%

Probabilities and Personality NPCs with probabilities can give illusion of personalities Differences must be large enough for player to notice in behavior Small hit by player 10% Player HP < 10 80% Player HP < 10 20% Confident Angry Small hit by player 90% Heavy hit by player 70% My HP < 10 80% Heavy hit by player 30% Orc with anger management issues Heavy hit by me 30% My HP < 10 20% Frightened Heavy hit by me 70%

Dynamic Probabilities Likelihood of transition depends on something else More realistic (but not completely predictable) Can give player clues about state of NPC % of bullets left Firing Reload Player not firing 1- % of bullets left Guard Door and Shout for Help 1- Energy % Player visible Patrol in front of Door Energy % Chase Player

Emergent Group Behavior Each NPC in group can choose random behavior Can appear to “cooperate” Half of group fires immediately giving “cover” to rest If player shoots firing players, rest will have time to reach cover Fire 50 % Player visible Patrol Cover reached 50 % Take Cover

Emergent Group Behavior Potential problem: Possibility all in group can choose same action All either shoot or take cover No longer looks intelligent Can base probabilities on actions others take Fire 1 - % of other players firing Player visible Patrol Cover reached Take Cover % of other players firing

“Markov” State Machines Tool for decision making about states Give states a “measure” describing how good state is Move to state with best measure Key: Measure changes as result of events Possibly returns to original values if no events occur Based (sort of) on Markov probabilistic process (but not really probabilities)

“Markov” State Machines Example: Guard choosing cover Different cover has different “safety” measures Firing from cover makes it less safe (player will start shooting at that cover) Represent safety as vector of values 1.0 trees 1.5 wall 0.5 brush

“Markov” State Machines Assign transition “matrix” to each action Defines how each state affected by action Multiplier < 1 = worse Multiplier > 1 = better Example: fire from trees Trees less safe Other positions marginally safer (player not concentrating on them) 0.1 1.2 1.2

“Markov” State Machines “Multiply” current vector by matrix to get new values Note: real matrix multiplication requires 2D transition matrix 1.0 0.1 0.1 = 1.5 1.2 1.8 0.5 1.2 0.6 0.1 0 0 0 1.2 0 0 0 1.2

“Markov” State Machines Further events modify values Example: Now fire from behind wall 0.1 1.2 0.12 = 1.8 0.2 0.9 0.6 1.2 0.72

“Markov” State Machines Note “total safety” (as sum of values) decreasing 3  2.5  1.74… May be plausible (all cover becoming less safe) Can normalize if necessary Can gradually increase values over time Usually result of time/turns without event Example: player leaves area 0.12 1.1 0.132 = 0.9 1.1 0.99 0.72 1.1 0.792