4 Why Scalable AI? More is BETTER The brute force approach to common sense: the more unique situations you can recognize and react to, the more common-sensical you are.(e.g. Cyc, OpenMind projects)
5 3 Dimensions of Scalability heretic gruntelite majorfloodswarmshuntershell-jumpersbrute captainMirandasniper marineDRAMA!pacingstorychallengeVarietyVariationdrive the warthogShoot the needlermelee attackfightperchhideboardingStrafe targetVolume
6 The Ultimate Goal Designer I want X to do Y whenever Z! Engineer Okay. (To himself) Hmm … and I know exactly where to put that …I’m happy now. (Trundles back to design-bubble)Two points:A great architecture has one PERFECT place for each new featureX,Y,Z – as in almost all of the designer’s requests – describe something observable by the player. It is not smart, it is in fact a predictable rule that will always happen. The only question is whether it will play nice with the other 500 XYZs the designer has asked for.
7 AI Design Requirements TransparencyFacilitate the ongoing narrative in the player’s headCoherenceFocused action, right prioritiesDirectabilityFor the designerWorkabilityFor the engineerSee [Butcher & Griesemer]GDC 2002Transparency – often about clear animation, clear lines of dialogue, etc.NEXT SLIDE: TITLE SCALABLE DECISION MAKING
8 Managing Scalable Decision-Making in the Halo2 AI Damián Isla Bungie Studios
11 BehaviorsBehavior: a program that takes temporary control of the actor in order to get something doneBehaviors are action over time:Move to a pointOrientShootCrouchPlay dialogueTrigger “actions”Behaviors have notion of relevancy and duratione.g.,FightVehicle entryThrow grenadeFind coverCover friendCheck bodyWake up a sleeping gruntEtc.Halo2: ~130 in total
12 The Combat CycleMassive amounts of complexity hiding in here!
13 Decision-Making Behavior Tree (or DAG) Good Bad Intuitive Modular ScalableBadNever really that simpleDebugging is hardMemory is hard
15 Given A, B or C, always choose D (all of the above) Decision routinesChild-competitive strategiesAnalog RelevancyBinary RelevancyPrioritized-list (root, engage)SequentialSequential-looping (search)RandomizedOne-off randomized (postcombat)DP #1: customizabilityPriorities list is most commonCOHERENCE IS ABOUT PRIORITIESDesign Principle #1 :Given A, B or C, always choose D (all of the above)
16 Unless the player is in vehicle, in which case... ImpulsesProblem: What happens (with a prioritized list) when the priority is not constant?Unless the player is in vehicle, in which case...
17 Design Principle #2 : Formalize complexity (don’t hide it) ImpulsesSolution: Separate alternative trigger conditions out into separate impulseTwo execution optionsIn-placeRedirectDesign Principle #2 : Formalize complexity (don’t hide it)
18 Design Principle #3: Embrace the hackery ImpulsesThe other purpose of impulses:HACKSOr, “localized code with ad-hocFunctionality”e.g.crouch_on_danger impulsedive impulseDesign Principle #3: Embrace the hackeryTwo things demanded of the architecture:Allow us to do it.Help us contain/catalog/keep track of it
19 Behavior MetadataProblem: In determining relevancy, we check the same conditions over and over and over and overActor’s vehicle status (infantry, driver, passenger)Actor’s alert status (idle, in combat, visible target)Solution: Use metadata to describe execution conditionsForce all behaviors to declare their execution conditionsHalo2: “conditions” bitvector compared to “actual state” bitvectorActs as a behavior/impulse mask (modifying the basic structure of the tree itself)
20 Design Principle #4 : Variation from a stable base Custom BehaviorsProblem: character-type specific behaviorsDon’t want all characters evaluating behaviors that only one type can doE.g. grunt’s “retreat from scary enemy” impulseSolution: attach custom behaviors to treeALTERNATIVE: New TREE
21 Stimulus BehaviorsProblem: rare event-driven behaviors tested for every ticke.g., Grunts flee when their leader diesSolution: Stimulus BehaviorsBehaviors or impulses dynamically and asynchronously placed into tree at specified location“Actor Died”1 sec.Why not just force the actor to start running the behavior???Maybe a higher-priority XYZ is runningThe tree ensures that the priorities of the designers are respected.TREE PLACEMENT as much a part of the decision as the RELEVANCY function
22 Joint Behaviors Remember Design Principle #1! (always choose D, all of the above)
23 Behavior Summary Behavior DAG Binary decision routines Impulses MetadataCustom BehaviorsStimulus BehaviorsJoint behaviorsThe behavior tree is fundamentally a DYNAMIC structure!
24 Memory is a problem fundamental to the behavior tree structure. No system is complete with an explicit memory solution.
25 Memory and MemoryProblem: Persistent behavior state is impossible (too much data to keep around!)Solution: Only keep around state for behaviors that are runningProblem: What about state we need to keep around regardless of whether we’re running the behavior or not? (e.g. last grenade-throw time)Part of maintaining coherence is remembering what I’ve been doing!!!
26 The Anatomy of Memory …memory is a complicated thing. Store memory as… Behavior state (short-term)Persistent per-behaviorPersistent per-targetProps are alsoPerception cachesOur knowledge model“Props”
27 (formalize complexity, The Anatomy of Memorye.g. SearchAquire target 1Lose sight of target 1Search target 1Last known location inspectedNew search point selectedAquire target 2Search deactivated, short-term state discardedTarget 2 killedSwitch to target 1Where do we search?Existing search point usedEXPERIENCE is VERY coherentNo tree architecture is complete without a memory solutionRememberDesignPrinciple #2(formalize complexity,don’t hide it)
29 Designer Direction Variety Variation Character definition Model, animation graphBehavior parametersStatic controlLevel-scriptingDynamic ControlVariation
30 Static Control: Variety Character typesElitesGruntsMarinesVariantsGold eliteRed eliteHonor guard eliteJetpack eliteEach variant gets a .character definition fileA LOT of parameters to author and maintain!64(variants) x80 (behaviors/variant) x3 (parameters/behavior) =15,360
31 Static Control: Parameters Character files divided into parameter blocksVitality propertiesPerception propertiesSearch propertiesWeapon usageEtc.Blocks can be instantiated or notHierarchy: children characters instantiate only significant variations from parentRoot of tree is the “generic” character, which provides “reasonable” values for all parametersRemember DesignPrinciple #4!(Variation from astable base)
32 Static Control: Styles Styles are cool:Behavior maskBias control parametersSuperposable(Another reason why Impulses are great)VARIETYAnother reason why separating different trigger conditions into different impulses is a great idea: expose those triggers to the designersBack to vehicle entry behavior
33 Dynamic Control: Organization Halo1:“Encounters”Groups of actorsGroups of areasProblem: moving actors from one encounter to the next was “architecturally discouraged”Halo2:“Squads”Groups of actors“Zones”Groups of areas“Orders”A mapping between the two
34 Dynamic Control: Orders The Metaphor:“Take that hill!”“Hole up in that bunker!”Or“Occupy this space and behave this way!”Position directionBehavior direction
35 Blah blah blah Actor is a member of a squad Squad gets an order Actors get free range inside of region
36 Orders and Behavior Behavior direction Vehicle behaviors enabled/disabledActive camouflage enabled/disabledFollow player enabled/diabledRules of engagementStyleBehavior maskBias control parameterA way that orders can directly constrain the decision space of the actorsTricky to author … but the nice thing is that you only need a few of them, like (assaulting, bunkering, harassing…)Only about 5 or 6 used in Halo2NO SCRIPTING“Alright, men, engage active camouflage and move forward cautiously. And don’t open fire until the Master Chief does!”
37 Design Principle #5 : Work with your representations In summaryDesign Principle #5 : Work with your representations
38 Recap Design Principles: Always choose D (all of the above) Formalize complexity (don’t hide it)Embrace the hackeryVariation from stable baseWork with your representations
39 Technical Takeaways Dynamic behavior tree Binary relevancy Behavior maskingPrincipled approach to memoryTake smarts out of the behavior and put it in the knowledge model
40 ConclusionsAll systems need to “play ball” with the core decision mechanismNot looking for “smart” architecture, looking for expressive architecture
41 The Ultimate GoalDesignerI want X to do Y whenever Z!