1. Managing Complexity in the Halo2 AI Damián Isla Bungie Studios

2. Managing Scalability in the Halo2 AI Damián Isla Bungie Studios

3. Scalability?
No! Scalability!

4. Why Scalable AI? More is BETTER
The brute force approach to common sense: the more unique situations you can recognize and react to, the more common-sensical you are.
(e.g. Cyc, OpenMind projects)

5. 3 Dimensions of Scalability
heretic grunt
elite major
flood
swarms
hunters
hell-jumpers
brute captain
Miranda
sniper marine
DRAMA!
pacing
story
challenge
Variety
Variation
drive the warthog
Shoot the needler
melee attack
fight
perch
hide
boarding
Strafe target
Volume

6. The Ultimate Goal Designer I want X to do Y whenever Z! Engineer
Okay. (To himself) Hmm … and I know exactly where to put that …
I’m happy now. (Trundles back to design-bubble)
Two points:
A great architecture has one PERFECT place for each new feature
X,Y,Z – as in almost all of the designer’s requests – describe something observable by the player. It is not smart, it is in fact a predictable rule that will always happen. The only question is whether it will play nice with the other 500 XYZs the designer has asked for.

7. AI Design Requirements
Transparency
Facilitate the ongoing narrative in the player’s head
Coherence
Focused action, right priorities
Directability
For the designer
Workability
For the engineer
See [Butcher & Griesemer]
GDC 2002
Transparency – often about clear animation, clear lines of dialogue, etc.
NEXT SLIDE: TITLE SCALABLE DECISION MAKING

8. Managing Scalable Decision-Making in the Halo2 AI Damián Isla Bungie Studios

11. Behaviors
Behavior: a program that takes temporary control of the actor in order to get something done
Behaviors are action over time:
Move to a point
Orient
Shoot
Crouch
Play dialogue
Trigger “actions”
Behaviors have notion of relevancy and duration
e.g.,
Fight
Vehicle entry
Throw grenade
Find cover
Cover friend
Check body
Wake up a sleeping grunt
Etc.
Halo2: ~130 in total

12. The Combat Cycle
Massive amounts of complexity hiding in here!

13. Decision-Making Behavior Tree (or DAG) Good Bad Intuitive Modular
Scalable
Bad
Never really that simple
Debugging is hard
Memory is hard

14. Decision routines
Parent custom
Child-competitive

15. Given A, B or C, always choose D (all of the above)
Decision routines
Child-competitive strategies
Analog Relevancy
Binary Relevancy
Prioritized-list (root, engage)
Sequential
Sequential-looping (search)
Randomized
One-off randomized (postcombat)
DP #1: customizability
Priorities list is most common
COHERENCE IS ABOUT PRIORITIES
Design Principle #1 :
Given A, B or C, always choose D (all of the above)

16. Unless the player is in vehicle, in which case...
Impulses
Problem: What happens (with a prioritized list) when the priority is not constant?
Unless the player is in vehicle, in which case...

17. Design Principle #2 : Formalize complexity (don’t hide it)
Impulses
Solution: Separate alternative trigger conditions out into separate impulse
Two execution options
In-place
Redirect
Design Principle #2 : Formalize complexity (don’t hide it)

18. Design Principle #3: Embrace the hackery
Impulses
The other purpose of impulses:
HACKS
Or, “localized code with ad-hoc
Functionality”
e.g.
crouch_on_danger impulse
dive impulse
Design Principle #3: Embrace the hackery
Two things demanded of the architecture:
Allow us to do it.
Help us contain/catalog/keep track of it

19. Behavior Metadata
Problem: In determining relevancy, we check the same conditions over and over and over and over
Actor’s vehicle status (infantry, driver, passenger)
Actor’s alert status (idle, in combat, visible target)
Solution: Use metadata to describe execution conditions
Force all behaviors to declare their execution conditions
Halo2: “conditions” bitvector compared to “actual state” bitvector
Acts as a behavior/impulse mask (modifying the basic structure of the tree itself)

20. Design Principle #4 : Variation from a stable base
Custom Behaviors
Problem: character-type specific behaviors
Don’t want all characters evaluating behaviors that only one type can do
E.g. grunt’s “retreat from scary enemy” impulse
Solution: attach custom behaviors to tree
ALTERNATIVE: New TREE

21. Stimulus Behaviors
Problem: rare event-driven behaviors tested for every tick
e.g., Grunts flee when their leader dies
Solution: Stimulus Behaviors
Behaviors or impulses dynamically and asynchronously placed into tree at specified location
“Actor Died”
1 sec.
Why not just force the actor to start running the behavior???
Maybe a higher-priority XYZ is running
The tree ensures that the priorities of the designers are respected.
TREE PLACEMENT as much a part of the decision as the RELEVANCY function

22. Joint Behaviors Remember Design Principle #1! (always choose D,
all of the above)

23. Behavior Summary Behavior DAG Binary decision routines Impulses
Metadata
Custom Behaviors
Stimulus Behaviors
Joint behaviors
The behavior tree is fundamentally a DYNAMIC structure!

24. Memory is a problem fundamental to the behavior tree structure.
No system is complete with an explicit memory solution.

25. Memory and Memory
Problem: Persistent behavior state is impossible (too much data to keep around!)
Solution: Only keep around state for behaviors that are running
Problem: What about state we need to keep around regardless of whether we’re running the behavior or not? (e.g. last grenade-throw time)
Part of maintaining coherence is remembering what I’ve been doing!!!

26. The Anatomy of Memory …memory is a complicated thing. Store memory as…
Behavior state (short-term)
Persistent per-behavior
Persistent per-target
Props are also
Perception caches
Our knowledge model
“Props”

27. (formalize complexity,
The Anatomy of Memory
e.g. Search
Aquire target 1
Lose sight of target 1
Search target 1
Last known location inspected
New search point selected
Aquire target 2
Search deactivated, short-term state discarded
Target 2 killed
Switch to target 1
Where do we search?
Existing search point used
EXPERIENCE is VERY coherent
No tree architecture is complete without a memory solution
Remember
Design
Principle #2
(formalize complexity,
don’t hide it)

29. Designer Direction Variety Variation Character definition
Model, animation graph
Behavior parameters
Static control
Level-scripting
Dynamic Control
Variation

30. Static Control: Variety
Character types
Elites
Grunts
Marines
Variants
Gold elite
Red elite
Honor guard elite
Jetpack elite
Each variant gets a .character definition file
A LOT of parameters to author and maintain!
64(variants) x
80 (behaviors/variant) x
3 (parameters/behavior) =
15,360

31. Static Control: Parameters
Character files divided into parameter blocks
Vitality properties
Perception properties
Search properties
Weapon usage
Etc.
Blocks can be instantiated or not
Hierarchy: children characters instantiate only significant variations from parent
Root of tree is the “generic” character, which provides “reasonable” values for all parameters
Remember Design
Principle #4!
(Variation from a
stable base)

32. Static Control: Styles
Styles are cool:
Behavior mask
Bias control parameters
Superposable
(Another reason why Impulses are great)
VARIETY
Another reason why separating different trigger conditions into different impulses is a great idea: expose those triggers to the designers
Back to vehicle entry behavior

33. Dynamic Control: Organization
Halo1:
“Encounters”
Groups of actors
Groups of areas
Problem: moving actors from one encounter to the next was “architecturally discouraged”
Halo2:
“Squads”
Groups of actors
“Zones”
Groups of areas
“Orders”
A mapping between the two

34. Dynamic Control: Orders
The Metaphor:
“Take that hill!”
“Hole up in that bunker!” Or
“Occupy this space and behave this way!”
Position direction
Behavior direction

35. Blah blah blah Actor is a member of a squad Squad gets an order
Actors get free range inside of region

36. Orders and Behavior Behavior direction
Vehicle behaviors enabled/disabled
Active camouflage enabled/disabled
Follow player enabled/diabled
Rules of engagement
Style
Behavior mask
Bias control parameter
A way that orders can directly constrain the decision space of the actors
Tricky to author … but the nice thing is that you only need a few of them, like (assaulting, bunkering, harassing…)
Only about 5 or 6 used in Halo2
NO SCRIPTING
“Alright, men, engage active camouflage and move forward cautiously. And don’t open fire until the Master Chief does!”

37. Design Principle #5 : Work with your representations
In summary
Design Principle #5 : Work with your representations

38. Recap Design Principles: Always choose D (all of the above)
Formalize complexity (don’t hide it)
Embrace the hackery
Variation from stable base
Work with your representations

39. Technical Takeaways Dynamic behavior tree Binary relevancy
Behavior masking
Principled approach to memory
Take smarts out of the behavior and put it in the knowledge model

40. Conclusions
All systems need to “play ball” with the core decision mechanism
Not looking for “smart” architecture, looking for expressive architecture

41. The Ultimate Goal
Designer
I want X to do Y whenever Z!

42. Questions?
Damián Isla