1. Rose / Metoyer 09/08/06Oregon State University Evolving Character Controllers for Collision Preparation Robert Rose 09/08/06

2. Rose / Metoyer 09/08/06Oregon State University Agenda Motivation Overview Background Character Model Evolving Character Controllers for Collision Preparation Results Conclusion

3. Rose / Metoyer 09/08/06Oregon State University Motivation Video Games –Games have many collisions with human characters –Recent focus has been on collision response: Ragdoll (pure physics) Hybrid Response (Zordan05, Mandel04) –Little attention has been given to collision preparation –A complete preparation and response system would increase a game’s realism dramatically DOA4 - TecmoNCAA 2003 - EA ABC Television

4. Rose / Metoyer 09/08/06Oregon State University Motivation Exploration of Evolutionary Robotics –Developing controllers for robots and physically simulated characters by hand is a difficult problem –Evolving Virtual Creatures (Sims94) –Evolved human locomotion (Smith98, Wolff03) –A strong sense… Holy grail: complete dynamics-based control of human characters Evolutionary techniques must be the key! Sims94 Smith98Wolff03

5. Rose / Metoyer 09/08/06Oregon State University Motivation Video

6. Rose / Metoyer 09/08/06Oregon State University Overview Physically simulate character model Control system –Desired joint angles Test environment –Throw a projectile at the character Evaluation system –Measure “pain” perceived by character Optimization system –Genetic algorithms

7. Rose / Metoyer 09/08/06Oregon State University Background Motion Capture vs. Physical Simulation Physically Simulated Humans Controlling Physically Simulated Humans –Special-purpose controllers Bruderlin89, Raibert91, Laszlo96, Takashima90, Hodgins95 –Combining controllers de Garis90, Faloutsos01 –Developing controllers by hand is not optimal Time consuming Re-targeting difficult (or not possible)

8. Rose / Metoyer 09/08/06Oregon State University Background Evolution of Human Character Controllers –Genetic Algorithms Roberts03, Wyeth03 –Genetic Algorithms + Neural Networks de Garis90, Smith98, Nolfi00 –Genetic Programming Gritz95/97, Wolff03 Gritz95Roberts03

9. Rose / Metoyer 09/08/06Oregon State University Background Collision Response –Motion tracking - Zordan02 Applies torques to character to track motion capture data Weaken control after impact, then gradually strengthen –Fall control - Mandel04 Transitions to fall controller after impact or tripping Motion capture resumes after character is in “rest” state

10. Rose / Metoyer 09/08/06Oregon State University Background Collision Response –Hybrid Control Dynamic Response for Motion Capture Data, Zordan05 Use control system to bring character to nearest motion data Blend out of control, into motion capture after a short period of time

11. Rose / Metoyer 09/08/06Oregon State University Background Collision Response –Endorphin - Natural Motion Offline solution; “multi-pass simulation technique” for building “uncapturable” motion capture sequences Contains 40-50 preprogrammed controllers that can be applied to your character. Ex: Stagger backwards and fall over All controllers end in a rest state (ragdoll)

12. Rose / Metoyer 09/08/06Oregon State University Character Model Physically Simulated Character Model –9 segments, 8 joints, 18 DOF –Uniform density segments –Simulated in Novodex

13. Rose / Metoyer 09/08/06Oregon State University Character Model 1 DOF Joint Control –PD-servos - “spring” controller Ball and Socket Joint Control –Our solution is an extension of PD-servos to 3 dimensions Controller Stability –Precision loss is our worst enemy Causes high-frequency oscillations - smoothing outputs helps –Overpowered on the twist axis Damping the twist axis torque helps

14. Rose / Metoyer 09/08/06Oregon State University Character Model Pain Mesh –Color codes areas of the character model according to “pain regions”

15. Rose / Metoyer 09/08/06Oregon State University Evolving Character Controllers Testing Controllers –Evolutionary process generates a lot of controllers –Fitness measures performance of controller during a single test –Multiple tests are necessary to ascertain “total fitness” Evaluating Controllers –Fitness metrics: pain, energy, distance Generating Controllers –Chromosome format –Selection process –Mutation and crossover

16. Rose / Metoyer 09/08/06Oregon State University Testing Controllers 36 tests per controller –9 tests according to the threat grid –x 4 tests for each mode of joint failure Threat grid –We desire “general” solutions rather than solutions that train for blocking a specific case –9 tests with “jitter”

17. Rose / Metoyer 09/08/06Oregon State University Testing Controllers Joint Failure –Humans typically block with both hands and duck the head –We desire solutions that use all faculties of the character model –Joint failure forces the character to defend itself using: Both arms Only the right arm, then only the left arm No arms at all

18. Rose / Metoyer 09/08/06Oregon State University Evaluating Controllers Evaluation of a test run requires a fitness function –Fitness = pain + energy + distance Pain Metric –How effectively did the controller minimize the amount of pain perceived by the character? Energy Metric –How much energy did the controller use to defend the character? Distance Metric –How far away did the controller block the projectile from the character’s head?

19. Rose / Metoyer 09/08/06Oregon State University Evaluating Controllers Pain Metric –Goal: Find controller that cause the least amount of harm to come to the character –Premise: Getting hit in some parts of your body hurts more than others –Method: Take the point of impact p i and use it to look up the “pain region” the character was hit in -- each region has a pain multiplier Apply the pain multiplier to the force of the impact Sum for every point of impact

20. Rose / Metoyer 09/08/06Oregon State University Evaluating Controllers Pain Metric –Pain multipliers developed ad hoc

21. Rose / Metoyer 09/08/06Oregon State University Evaluating Controllers Energy Metric –Goal: We want to find “realistic” solutions –Premise: Humans don’t “over-compensate” movement –Method: Penalize controller based on energy consumption Picking a large value for c t prevents solutions that over-compensate But, picking too large value has undesirable consequences… –Energy metric overtakes the pain metric –Converges on solutions that only minimize energy c t = 0.001

22. Rose / Metoyer 09/08/06Oregon State University Evaluating Controllers Distance Metric –Goal: We want to find different styles of poses –Premise: Humans sometimes block far from their head –Method: Penalize controllers that block close to the head Take the distance from the first point of impact (p 0 ) to the head (h) Round this to the nearest integer Apply distance multiplier c d c d = 0.015

23. Rose / Metoyer 09/08/06Oregon State University Generating Controllers To generate controllers, we use Genetic Algorithms –Encode desired joint angles as a chromosome –Tournament selection process –Perform mutation and crossover to generate new chromosomes –Slope of solutions over a window determines end

24. Rose / Metoyer 09/08/06Oregon State University Results

25. Rose / Metoyer 09/08/06Oregon State University Genetic Algorithm Parameters Before we could begin testing our pain measurement theories we needed to stabilize the parameters to the GA Tested four mutation schemes (below) Tested two tournament sizes (4, 8) Tested eight population sizes (100-800 increments of 100)

26. Rose / Metoyer 09/08/06Oregon State University Genetic Algorithm Parameters Mutation Schemes A-D Fitness vs. Time

27. Rose / Metoyer 09/08/06Oregon State University Genetic Algorithm Parameters Tournament Sizes 4,8 - Population Sizes 100-800 Fitness vs. Time

28. Rose / Metoyer 09/08/06Oregon State University Genetic Algorithm Parameters Conclusion: –Mutation Scheme A Clear winner, others didn’t converge as reliably –Tournament size didn’t seem to matter Others have observed the same behavior (Gritz97) 4 was chosen arbitrarily –Population size not so clear Population sizes of 400 and over always converged on a solution Population sizes under 400 converged, but more sporadically Ultimately, convergence time was the decider: 400 seemed optimal Footnote: –Early in this work I used culling as a selection process… –Simulations took overnight to converge on a solution!

29. Rose / Metoyer 09/08/06Oregon State University Results Evolution in progress

30. Rose / Metoyer 09/08/06Oregon State University Results Results at various angles

31. Rose / Metoyer 09/08/06Oregon State University Results Hand emphasis: hands-only collision

32. Rose / Metoyer 09/08/06Oregon State University Results Video

33. Rose / Metoyer 09/08/06Oregon State University Conclusion Collision Preparatory Poses –We successfully generated preparatory poses automatically –Tweaking the fitness function and collision space gave us different styles of results –Our system occasionally gave us some odd solutions Blocking with the back of the hands - Ow! The mysterious “inverted elbow” pose - I wish I could do that! Opportunities –Energy metric enhancements Consider the direction torque is being applied in - no back of hands –Joint limit penalties Penalize the system for placing the character in positions that would fracture a bone on impact

34. Rose / Metoyer 09/08/06Oregon State University Conclusion Evolving Character Controllers –Automated development of human character controllers is a hard problem! Developing realistic controllers requires a large search space Evaluation of human movement requires codifying what is “human” Our problem domain works well because of its simplicity –Ultimately, it’s about what the artist wants Artist codification of a problem, combined with pain measurement could be a powerful tool