1. Retargetting Motion to New Characters Michael Gleicher

2. Homework Keep working on next homework  Due on Tuesday  Build a mocap data previewer mocap.cs.cmu.edu (see Tools and Resources)  Load data file  Create favorite data structures  Use DOFs to animate a simple character

3. Homework Grading  Demo your IK solution Sunday or Monday  Room 002a this time!  Sign up sheet up front

4. Readings Interactive Control of Avatars Animted with Human Motion Data (Lee et al.) Motion Graphs (Kovar et al.)

5. Spacetime Constraints Homeworks look good  Let’s discuss for a moment Sequential Quadratic Programming aka Newton’s method and Newton-Lagrange Wilson (1963) and Beale’s SOLVER (1967)  A member of the descent family of constrained optimization techniques  Makes critical assumption that approximating constraints with linear functions is meaningful  Cost function must be quadratic

6. Spacetime Constraints This is just a search method  Penalty Methods permit constraints to be violated when searching for optimality Non-linear Least Squares used by Gleicher  Conjugate gradient  Simulated annealing loses benefit of derivatives  What about GA?

7. Spacetime Constraints The building blocks define SQP The implementation of SQP can vary  Matrix inversion  Special routines for handling sparse or positive definite matrices

8. Motivation: Why Do We Care? Have one animated motion, want another For characters with identical structure, but different proportions Manual tweaking is slow and difficult Want to preserve high-level properties that are hard to define

9. Simple Try 1 Just reuse parameters of original motion in new motion with different character hmm … something’s not quite right …

10. Simple Try 2 Need to meet constraints, for instance foot touches floor No problem, just use inverse kinematics to establish constraints each frame … better, but not very smooth …

11. What’s the problem? No temoporal notion of constraints Solving locally for each frame generates unwanted artifacts, can’t plan ahead Problem: introduces high frequency motions

12. Try again with Filtering Apply a low-pass filter to remove unwanted high frequencies But, may violate constraints Want to preserve high frequencies in original motion, without introducing new ones OriginalSmoothed IKIKRetargetted

13. Enough Failed Attempts Need a way to solve the constrained optimization problem that takes the whole motion into account Where have we seen this before?  Spacetime constraints  Through-the-lens camera control  Neuroanimator  Virutal Creatures

14. Spacetime Constraints Mathematically encode all constraints and objectives Not so easy to capture desirable aspects of original motion mathematically More constraints and more complicated objective functions take longer to solve  Don’t use physics constraints on every step

15. Tools for Traveling through Space and Time Constraints  Identify features that must be present

16. Constraints q t i = parameters of motion (joint angles and root position) at time t i Define constraint as: f(q t i ) = c  c is a constant Constraint examples  Parameter’s value is in a certain range (joint limits)  Point on character is in specific location or within a range  Two points are a certain distance apart  Vector between points has a certain orientation

17. Tools for Traveling through Space and Time Objective Functions  Stay close to the original (especially high frequency motion)  Use constraints to keep objective function from being too simple minimize g(x) subject to f(x) = c

18. Objective Functions Goal: Minimize noticeable change  Hard to define, choose something simple instead Minimize differences between new motion and original motions Define functions of motion parameters  Original motion:  New (retargeted) Motion:  Difference between the two: Objective Function:

19. Tools for Traveling through Space and Time Initial Solution for Solver  Some basic scaling of original data to get a good guess

20. Starting Point Good initial motion can speed numerical solver Start just by scaling motion to match scaled character If needed, translate motion to get as close as possible to satisfying constraints

21. Representation Want to minimize introduction of high frequencies Choose representation of to achieve this Cubic B-splines  Control point spacing determines frequency limit  Try having control points every 2, 4 or 8 frames (uniform spacing)

22. Retargeting Procedure 1. Take original motion and identify constraints 2. Scale and translate to find initial estimate for solution, m 1 (t) 3. Choose representation for d(t) – what is the control point spacing 4. Solve non-linear constraint problem: find d(t) that will satisfy constraints when added to motion estimate from step 2 (use spacetime) 5. If constraints are not well satisfied, solve again using m 1 (t) + d(t) as initial motion in step 2 and choose a denser spacing in step 3

23. Solving the Constrained Optimization Problem Want to solve for control point values (call the vector of these x) Define constraints and objective functions in terms of these control points  minimize g(x) subject to f(x) = c  g(x) = ½ xMx (a weighted sum of squares, diagonal matrix M gives weights) In addition to SQP could use a non-linear least squares solver to minimize constraint residuals (distance from meeting constraints)

24. The examples 120 Hz mocap data downsampled to 30 Hz (one example from text was rotoscoped) Maker positions converted to articulated character parameters Euler angles for joints (3 DOF)  Except elbow, knee, ankles Frequently, data for hand and feet is unavailable

25. Getting 60% shorter Gray character results in foot skate  Left shows strides getting longer (black)  Right shows adjusting foot plants (black)

26. 2-D Walking 82 frames / 15 fps 14 DOFs (2 pos + 12 ang) 146 constraints on heel/toe 328 inequality constraints to keep feet above ground 1968 joint limit constraints

27. 3-D Walking 34 DOF, 112 Frames 4193 scalar constraints (~354 active at any one time… computation in seconds

28. Ladder Their custom SQP, LMULT permits some constraints to be violated by ½ in. Least squares optimization results in errors less than ¼ in.

29. Dancing Woman might lift off ground Male permitted to move upper body 1200 constraints 14 seconds to solve w/ 1/8 in. errors

30. Extensions Characters with different structure  First use standard retargeting to a character whose dimensions roughly match those of the new character  Find correspondences (manually) between features of original and new character, make these constraints  Use retargeting with spacetime constraints again, to find a new motion that satisfies these constraints

31. Conclusions Basic retargeting works for characters with same structure, different limb lengths Control point spacing is important for preserving desired frequencies  Enforcing uniformity across time is a limitation Enforcing certain features of original motion may not produce natural retargeted motion

32. Morphing simulated characters Hodgins and Pollard  Adapting Simulated Behaviors For New Characters  Siggraph 97

33. Morphing simulated characters We know how character has changed in size and mass We have no control of joint positions explicitly We adapt character through control algorithms  Timing parameters  Muscle stiffnesses and damping

34. Morphing simulated characters

35. We know a lot about scaling Animation of Dynamic Legged Locomotion – SIGGRAPH 91, Raibert and Hodgins  Scale kangaroo by L It will have to jump L times as high Because gravity is unchanged  more time will pass before landing (for L >1) Have cadence 1/sqrt(L) of original But because limbs are longer (L>1)  travel sqrt (L) as fast

36. We know a lot about scaling

37. Motion Graphs Introduction to the paper by Kovar, Gleicher, and Pighin Motion can be “sampled” just as video clips were in Video Textures

38. Sampling mocap Transitioning from frame i to frame j White are good transitions Green are local min

39. One change vis. Video Textures Multiple timesteps are grouped into a clip The clips are connected to one another in a graph Transitioning from one clip to another is not as easy as cross-fade or morph from image domain

40. General outline Build motion graph Remove dead ends (find strongly connected components) Search for paths that produce desired motion (branch and bound search) Blend clips to form a smooth motion