1. Designing AI (character behaviour)
Game Design
Vishnu Kotrajaras, PhD

2. Challenge the players Shooting: overwhelm by opponents
AI can be easy, the numbers will provide the challenge anyway
First-person-shooter:
player can run out of ammo and can’t see in the dark (enemies do not have this disadvantage)
Also, there are lots of enemies
Therefore, each of them does not have to be smart.
Player will still be challenged at this difficulty

4. Challenge the players(2)
RTS: AI has to do everything the player does, and has to seem smart
We can help it a bit by
AI can see the entire map
AI has a larger number of starting unit
AI has a bigger pool, or easier way to get resources

5. Challenge the players(3)
Turn-based strategy game:
Most difficult to make AI
Chess!
Mostly, the AI cheated a bit….

7. Do not do stupid things
Do not just fall off a cliff…err..that’s Lemmings
If the enemy is supposed to be human, make them clever
If you can’t make them clever, just make them look like robots!
Different acceptable levels for different games
Metal Gear Solid, enemies are clever but their eyesight are extremely short
But if they have the same viewing range as you… the game will be too difficult

9. Be unpredictable
Players want AI to try to defeat them in ways players never think of
Of course, games can surprise players with story, bosses, next level.
Ai adds replayability
Uses random number and probability (weight) for choosing possible actions
This is fuzzy decision making
Some choice of actions may seem stupid, but real human does this things when panicked
The player will be the one that thinks the AI is clever
Unpredictability must not break other AI goals

10. Assist Storytelling
For example, NPC in a particular town runs away from strangers
This makes the player wants to investigate
AI of team mates helps telling the story of each character
Action, speech
Dynamic storytelling
Players can influence the NPC’s mood
Action somewhere else in the world may affect his attitude towards the player

11. Create a living world
Add life that do not directly interact with players
Ambient life
Birds
Cats
Dogs

12. Should the AI not have any advantage?
AI may not be challenging anymore, because naturally, man is more clever
Programmers will spend too much time on AI
Remember, the goal is not the use of AI techniques, but the challenge the player can get
Players love to beat tons of enemies, it feels great. Therefore few clever enemies may not be good enough

13. Too real? AI that runs away when wounded
If the AI is faster, or navigate better, the player will never outrun it
Now it is a chase game, forever chase
AI that always cover weak points when players shoot
No way to win!

14. AI and levels Must be developed together
AI that works on every level is hard
Problem is, you may have 2 teams, one working on AI and the other on levels
Must work together
Create simple AI that works in some situation
Then level designer create a level based on this simple AI
Keep looking at each other’s work and communicate.
Both must adjust their work
Levels may have to sacrifice some coolness in order for AI to work well
AI may not be able to swim, so the level must not have river
Must prevent NPC getting stuck at corner

15. Good enough? Teammates AI Don’t fail the player
Especially, don’t fail when the player need it the most

16. scripting Use dynamic AI, together with pre-made script Half-life
Use scripted path (rule-based behaviour)
The AI do things depending on situation (reacting)
Tester is the one who work out what the AI should do

19. Dynamic scripting
Maintain several rule bases, one for each opponent type.
Each time a new opponent is encountered, rules are extracted from the rule base of that creature type.
The probability that one rule is selected depends on a weight value that is associated with each rule.
Weights of all rules are updated after a completion of an encounter.
the order which the rules are placed depends on application.

20. Advantage of dynamic scripting
Fast
Extraction of prepared rules from rule base.
Update weight only once per encounter.
Still intelligent
Every rule is from the prepared rule bases.
Robust
Rules are not removed immediately when punished.

21. Example of dynamic scripting
Opposing party encounter (AI and player)
Each party has 2 fighters and 2 wizards of the same level.
Weapons are static.
Can select 2 types of potion.
Wizards know 7 spells.

22. The dynamic scripting language
(Optional conditional statement) and an action.
Conditional statement
One or more conditions.
Combined with logical AND or OR.
Condition examples
Distance separating characters.
Character’s health.
Spell effects currently on characters.
Actions (5 actions in this experiment)
Attack an enemy.
Drink potion.
Casting spell.
Moving.
Passing the turn.

23. The dynamic scripting language(2)
Things can be specific
Cast spell “magic missile” at closest enemy wizard.
Or general
Cast any offensive spell at random enemy.
Or somewhere in between
Cats the strongest damaging spell available at the weakest enemy.

24. Rules are executed in sequential order (condition of that rule must also satisfy.).
If at the end of all chosen rules, no action is selected, a pass turn is chosen as a default action.
Chosen rules are selected according to their corresponding weight.
Chosen rules are ordered according to their priority. If they have equal priority, then they are ordered by their weight.

25. Total rules in Baldur’s gate experiment
Fighter
20 rules in total.
The program choose 5 rules to form a script.
Wizards
50 rules in total.
10 are chosen to form our sequence rule script.

26. The weight update function
Party fitness
Range 0..1
0 if party has lost the fight.
0.5 + half the average remaining health of all party members.

27. Health of “n” at the end
Health of “n” at the start of the fight

28. Character fitness Range 0..1 Based on
Avg. remaining health of the party.
Avg. damage done to the opposing party.
Remaining health of the character (if character dies, we use the time of death instead).
Party fitness.

29. Time of death
Large reward to party victory.
Smaller reward to the individual’s survival.
Even smaller reward for friend’s survival and the damage they did to the enemy.

30. The actual weight update function
Translated from character fitness.
Only the rules executed were rewarded or penalized.
Max. penalty
Break even point
Max. reward
Max. weight value

31. In the trials
MP=30
MR=100
MW =2000
b=0
All weights are originally 100

32. Static scripts to fight against
Must get changed (to reflect player’s change in strategy).
Offensive
Fighter attacks nearest enemy.
Wizards use most powerful spell to attack weakest enemy.
Disabling
Fighter first drink potion that prevent status effects, then attack the nearest enemy.
Wizards use all spells that can disable enemies for a few rounds.
Cursing
Wizards try all spells that harm enemy.
Defensive
Fighter drinks potion that reduce damage first, then attacks nearest enemy.
Wizards use all defensive spells.

33. Composite tactics Random party tactics Random character tactics
Select 1 from 4 tactics at each encounter.
Random character tactics
Consecutive party tactics
Continue using the same party strategy if still win.
Most resemble human tactic.

34. Test run Basic tactics 21 tests, each test we run until we find
Average turning point (the first time that dynamic scripting win for 10 consecutive encounters).
Absolute turning point (first encounter which, following this encounter, consecutive loses of dynamic script is never more than consecutive win).

35. Avg is much higher than the median due to rare occurrences of high turning points (early divergence makes unlearn hard).
Easy to defeat
Hardest to defeat, but still learn to defeat it.

38. Current NPC Do things only in the area that a player is active.
Static agent
wait for player to interact with it.
Or play through a scripted sequence.
Proactive Persistent agents (proposed in the paper)
Have needs, beliefs, desires of their own.
Can initiate action on their own.
They are modeled regardless of a player’s location in the game.

39. PPA requirements Believable Configurable Scalable Able to switch mode
Only 1 or 2 is enough. The rest can be old static.
Able to switch mode

40. Current agent architectures
Reactive (like if-then-else), mostly implemented by FSM.
advantages
Very fast.
Simple to program and understand.
Completely under control.
Require little support from infrastructure.
Disadvantages
Must try to code every possible situation that it will encounter.
Stupid.

41. Deliberative Build model of he world. Form plans to achieve goals.
Prolog-like
But
Maybe very slow to decide things.
Require constant maintenance of a knowledge base.
Cannot be used in fast changing game.

42. Hybrid agent
Use reactive for time-critical behavior, such as avoiding collision.
Long term planning can use deliberative system.

43. PPA requirement in detail
Autonomy
Social ability (human-like)
Reactivity
Pro-activeness
Persistence
Scalability
Extendability
Believability
configuability

44. Proposed architecture
System composed of
Behaviour system
Social system
Goal based planner
Schedule

45. What agent experiences in the world
Ordinary behaviour
Human-like communication & relationship between characters
A task for an agent at particular time

46. Sweetser’s PhD. Thesis
Intelligent AI – adaptable to game environment in a strategy game.
Each cell on the map has low level properties (Physics)
Heat capacity, ignition point and burning rate.
Wetness and fluid flow.
Pressure flow.
Explosion on a map will now cause natural fire. River realistically blocks out the fire.

47. Object has
Low level properties just like a cell. (Object is treated as a cell, but has only 1 adjacent cell, that is, the cell it is in.)
High level properties
Structures- is_open, has_volume
So an object can contain water, pressure, etc. realistically.
Agent
Has low and high level properties like an object, but can move.
Calculate comfort value of its current cell and its surrounding cell.
<0.1 -Comfortable
Slowly move to a more comfortable cell.
–distress and run from the cell.
>0.6 –panic and run

48. Neighborhood size =1
Neighborhood size =2

49. Agent experiments Exp1 neighborhood size
1: can avoid immediate danger, but get stuck in a larger hazard. Or it runs into another danger.
3: picking good destination. More agents move to the same destination.
But run through hazards.

50. Exp2 optimising agent navigation
First select a goal, using neighborhood size =3.
Then evaluate cells in its immediate neighborhood, using comfort value and distance from the goal.
Comfort 50%, distance 50%- appear intelligent
Comfort 25%, distance 75%- still run through hazard
Comfort 75%, distance 25%- appear stupid, moving randomly

51. Exp3 combing comfort and desire
Propagate the desire out from the goal position, with a factor of 0.7
4.9 7 7 7
4.9 7 10 7
4.9 7 7 7
4.9

52. Next, an agent calculate comfort values for neighborhood size =3.
Agent select the best cell to move to, based on desirability and comfort.
50% desire, 50% comfort
75% desire, 25% comfort
25% desire, 75% comfort
Agent then chooses a cell to move to in its immediate neighborhood (comfort 50%, closeness 50%)
Result
best, but only about half the agents found the goal, as they opted for comfort over goal.
Appeared less intelligent, moving randomly. Only about half the agents found the goals, why????
More cycles to converge. No significant difference between the number of agents that found the goal, compared to previous results.
Increasing propagation constant to 0.8 helps improve agents.
Agents can wait for hazards to pass before moving further.
But the agents were still not successfully able to find the goal.
Why????????

53. Answer by Sweetser
It's not that the agents don't "survive", but rather that they stop in a cell and do not reach the goal.
This means that they cannot see a better cell near them to move towards and thus stay in their current location.
The reason why more agents don't reach the goal is due to the fact that they are considering both the dangers of the environment and trying to reach the goal.
There are also lots of dangers for the agents to face on the map.
So more agents would reach the goal with a higher desire weighting or if the map were less dangerous.
I think it would be interesting to include some real-time adaptation for the agents and certainly basing their decision on their attributes would make sense.

54. Last exp: multiple goals
Agents were much better in finding goals.
But maybe because there are more goals within the same physical area.
Or the desirability of many goals accumulate, allowing influence to propagate further.
But did not appear as intelligent or realistic.
What has not been done from this thesis.
Agent can run away, but it does not use environment any more than that, such as burning forests or using water to put out fire. (There are no moving objects in Sweetser’s thesis.)
Maybe adding objects that can be moved between cells (by agents) is another step.
Agent sees only cell, not objects in the cell.
It will be better if we can have agents neutralizing hazards.