Turning to the Masters: Motion Capturing Cartoons Christopher Bregler, Lorie Loeb, Erika Chuang, Hrishi Deshpande SIGGRAPH ’02

Motivation Animation has two dimensions: Visual Style Motion Style Style of Drawing or Model Rendering etc Motion Style How Characters Move Amount of Exaggeration Use of Cartoon Physics

Examples

Motivation Motion Capture restricted to a very small region (Green) Try to isolate the motion style of cartoon animation and apply it to different visual styles

Challenges: Capture Start with a 2-dimensional animated video Cartoon characters have no markers Low Frame Rate makes tracking difficult Identifying limb locations in cartoon characters is difficult Cartoon Objects undergo large degrees of non-rigid deformation throughout the sequence

Challenges: Retargeting Most retargeting techniques based on skeletal models Can capture only 2D information. Retargeting domain may be 3D models

Main Idea: Parameterize cartoon motion with a combination of affine-transformations and key-weight vectors. Claim: Describe a wide variety of motion and non-rigid shape deformation.

Affine Deformations

Affine Deformations Important part of cartoon motion comes from Velocity of entire body Stretch and Squash in different directions S is a 3xN shape matrix encoding N points in homogenous form, si=[xi,yi,1]T The ball shape V(t) at time-frame t is defined with the equation on left

Key-Shape Deformations For more complicated motion use a set of characteristic key-shapes Si . The model then extends to

Extended Linear Warping Space In-Between shapes produce undesirable visual artifacts Can be avoided by using a large number of intermediate shapes Can use non-linear interpolating functions but then inverse calculation is non-trivial

Extended Linear Warping Space.. Solution: For every combination of hand-picked key-shapes generate M in-between key-shapes. For K shapes: (K-1)*(K-2)*M shapes now Use PCA to give: Mean shape M Eigen Vectors E1…EL. Using Sl=M+El and Sl+1=M gives the extended linear space

Contour Capture Given sequence of cartoon contours: V(1)…V(t) and labeled key-shapes S1….Sk need to find affine matrix and the weights

Contour Capture Done by a two-step process. Estimate the affine parameters. Estimate the key-weights Iterate. Some constraints include all key-shapes adding up to 1. Reason: Good at interpolation and bad at extrapolation

Video Capture Input: Sequence of images instead of contours Extend the model to directly model image pixel variations.

Video Capture: Affine case S2xN=[s1,s2…sN] :contains (x,y) coordinates of all pixels in cartoon image region I(si): intensity at pixel si Io: Image Template (Base Key-Shape)

Approximation

Video Capture: Affine and Key-Shape Io: Was the image template (Base Key-Shape) in affine case Replace it with linear combination of key-shapes. The error function thus becomes:

Results from Video-Capture

Results (contd..)

Retargeting Motion Key-Shape based Correspondence also specified between control points etc:.

Results Video

Pros and Cons Advantages Disadvantages Attempts the problem of cartoon capture Motion Style that is expressive is taken into consideration Adopts simple models and hence can be fast Disadvantages Adopts very simple models. Input Key-Shapes and Correspondences need to be provided. Adopts interpolation Retargeting to 3D not handled completely Most Importantly: No temporal constraint used. Hence jitter would be a major problem.