“Low Level” Intelligence for “Low Level” Character Animation Damián Isla Bungie Studios Microsoft Corp. Bruce Blumberg Synthetic Characters MIT Media Lab

“Low level” Animation …? Animation not having to do with gross body movement or “behavior” Animation not having to do with gross body movement or “behavior” –Eye gaze –Facial expression –Ambient / idling animation –Animation style –Speech? Interesting because an “internal life” is implied Interesting because an “internal life” is implied

Cognitive Modeling CM: Giving characters an internal life Too much autonomy? ProsUnpredictabilityResponsiveness Leverage animation ConsUnpredictabilityReproducibilityControllability

“Low level” Cognition …? A class of abilities that are relevant to, but independent of, high-level action Perception Perception Knowledge modeling Knowledge modeling Attention Attention Memory Memory Emotional reaction Emotional reaction Motion quality Motion quality …

Example 1: AlphaWolf Emotional memories: player has total control, but wolves react to instructions based on past experience Emotional memories: player has total control, but wolves react to instructions based on past experience B. Tomlinson, “Synthetic Social Relationships for Computational Entities”, PhD. Thesis, MIT Media Lab 2002 B. Tomlinson, “Synthetic Social Relationships for Computational Entities”, PhD. Thesis, MIT Media Lab 2002 Wolves maintain their own cognition, memory and emotion models Wolves maintain their own cognition, memory and emotion models

Example 2: Object Persistence Piaget: The persistence of a mental image after the sensory stimulus has been removed Piaget: The persistence of a mental image after the sensory stimulus has been removed Object Persistence = location expectation formation Object Persistence = location expectation formation Focus on search tasks (where do I expect the sheep to be?) Focus on search tasks (where do I expect the sheep to be?)

Spatial Expectations Probabilistic Occupancy Map –Discrete spatial probability distribution –Uncertainty through discrete diffusion

POM Algorithm If target observed:Find closest node n* Otherwise:Divide map nodes into visible (V) and nonvisible (N) sets Either way:Diffuse Probability

Emergent Look-Around Also: Emergent Search Also: Emergent Search Simple rule: always direct gaze towards most likely location of the target Simple rule: always direct gaze towards most likely location of the target

Expectations and Emotions Observations can have emotional impact –Wanted to see something but didn’t  confusion –Saw something where you didn’t expect it to be  surprise –Having trouble finding the target  frustration … plus variations –Target desired + confusion  disappointment –Target feared + surprise  panic –Target desired + surprise  delight Emotions may –Focus attention (salience) –Bias behavioral choices / Affect decision-making parameters –Affect animation (facial and parameterized) –Act as a debugging channel!

Expectations and Emotions Emotional Autonomic variable Emotional Autonomic variable Surprise (unexpected observation ) Surprise (unexpected observation ) Confusion (negated expectation) Confusion (negated expectation) –Proportional to amount of culled probability Frustration (consistently negated expectations) Frustration (consistently negated expectations)

Results: Duncan the Highland Terrier Duncan: Virtual sheep-herding Virtual sheep-herding Layered behavior system Layered behavior system Synthetic vision Synthetic visionResults: Emergent look-around Emergent look-around Emergent search Emergent search Salient Moving objects Salient Moving objects Distribution-based object-mapping Distribution-based object-mapping Emotional reactions Emotional reactions –Surprise –Confusion –Frustration Video

Conclusions “Low Level” Conclusions “Low Level” Conclusions –A model of Object Persistence –Simple mechanism, complex results  Simple implementation  Intuitive “High Level” Conclusion “High Level” Conclusion –Intelligence >> Action-selection  You control the wolves, but what they feel matters  You control Duncan, but what he knows matters

Questions? Damián Isla Bruce Blumberg Synthetic Characters