2014 Animation Programming for Music Video Games Jessica Scott Harmonix Music Systems, Inc. October 10, 2014 #GHC

About me −Lead engineer at Rock Band 2 & 3 The Beatles: Rock Band Dance Central 1, 2 & Spotlight −B.S. Computer Science & Studio Art, Williams College −M.S. Visualization, Texas A&M University

2014 Why animation for games? −Why use character animation? Help people identify with characters Brings life to games and game worlds Can be used to show user what to do (Kinect games) The art department says so

2014 3D animation: the basics Artists create or process motion capture in 3D programs (Maya, 3D Studio Max, Motion Builder) Animation is exported into a format that a game engine can play back This is usually custom per game engine

2014 Why animation programming? −Why does it require programmers? Getting data out of animation programs isn’t magic Games have special requirements for memory/performance/response to user input

2014 Computer Representation −Skinning/bones Don’t want to represent the position of every mesh vertex at every point in time Standard practice: make a “skeleton”, store positions & rotations of “bones” at certain times Attach vertices to bones with offsets/weights: this is called skinning −Keyframes Way of storing animation poses at points in time Animation programs will interpolate between them

2014 Animation Playback −Set a series of in-game bones to the positions/rotations that were exported −The code to determine when to play what animation is another story…

2014 Memory & Performance  Animation often takes up a large percentage of a game’s memory & CPU budgets  Animation storage −Store rotation, position, scale for each bone  File size/memory usage reduction techniques −Compression −Blending

2014 Game Animation Special Needs  Why game animation is more complicated than movie animation −Choice: player control of characters −Real-time playback −Dynamic elements of game worlds (or, in Harmonix’s case, the music) determine what animations to play

2014 Blending  Basic technique needed for dynamic choice of animations −Combining multiple animations either within one frame or over time −We need it for natural-looking yet automatic transitions between different animations −Allows us to reuse shorter, smaller pieces of animation

2014 Blending  At one point in time, for one bone We do a lot of interpolating between rotations Rotations should be stored in local space instead of world

2014 Memory & Performance  Rotation storage: Quaternions −Better than rotation matrices Four floating point numbers  Interpolation of quaternions is fast, so blending is cheaper

2014 Blending  Time-based −Once you know how to blend within a single frame, the next step is to cross-fade between animations over time.

2014 Inverse Kinematics  Modifying a character’s joint positions and rotations based on physics constraints  Mathematical calculations are applied after regular skeleton posing  Don’t forget to modify the whole joint chain, or else...

2014 Game Examples  Dance Central: foot slide during blending Consider two dance moves that you want to blend together nicely. If a foot is on the ground and has weight on it at the end of one move, you don’t want to simply blend its position to the position in the new move, because it will look unrealistic and weird as it slides across the floor. Instead, you would either want the foot to remain in the same spot without moving, or to lift and then move to the new position- and if the character’s weight is on the foot, it shouldn’t lift.

2014 …

Game Examples  The Beatles: Rock Band: microphone lean-in We wanted to have the Beatles lean together into the microphone when they were singing harmony Added an IK constraint that pulled a character’s head to be six inches away from the microphone, then adjusted the neck and spine accordingly…

2014 Game Examples  And quickly realized: only turn it on if the character’s very close to the microphone.

2014 In conclusion…  Animation programmers have the best bugs

2014 Thank you for coming!  Meanwhile… Harmonix is hiring engineers. If you know anyone who might like to work on animation, gameplay, or network code with us, please have them me:

2014 Got Feedback? Rate and Review the session using the GHC Mobile App To download visit