1. Helper Joints: Advanced Deformations on RunTime Characters
Jason Parks
Character Technical Director
Sony Computer Entertainment America

2. Contacts, Reference, Credits
Webpage (Helper Joints-GDC 2005):
PAN Arts AVIs by Warwick Mellow
Human Model by Sven Jenson via Miller/Thuriot’s MasterClass 2003 (Character Creation Toolkit)
BodyBlend render by Zach Gray
Wrinkle R&D by Tyler Crook (SCEA)

3. Intended Audience
“Advanced – Requires experience and familiarity with subject.”
Maya Technical Artists who ‘Bind Characters’ for RunTime or Offline usage
Paint weights (Artisan and Component Editor)
Apply deformers
Use Set Driven Keys
Write Expressions
Use Utility Nodes

4. Topics What are Helper Joints? Why use Helper Joints?
Definition
Why use Helper Joints?
New uses for Extra Joints
Smooth binding sucks & Where to use Helper Joints
How to use Helper Joints?
Our Goals, Disclaimer, Fusiforms
Research: The Basic Building Blocks
RunTime Rigs
SetDrivenKeys
Expressions
Constraints
Non-RunTime Rigs
Baking Out
Spline IK
Jiggle Deformer
Automating Production
Scripting
Workflow
Muscle Systems
Procedurally generated Helper Joints

5. Stop Talking, No more text, show some Moving Pictures!

6. Helper Joints Defined
“Helper Joints” are secondary joints offset from the standard hierarchy which will translate, rotate, or scale to achieve a “hand sculpted” affect on an area of vertices.
They can be within the primary hierarchy (parented, grouped, rigged), or outside of the primary hierarchy (point or parentConstrained)
They can be controlled by SetDrivenKeys, Expressions, Simple Rigs (constraint system), or Super Advanced Rigs (really any simulation you can think of) that are baked-out.

7. Why use any Extra Joints?
Unlike characters rigged, bound, and rendered in Maya or any other high-end 3D package, your characters at run-time need to follow a very specific rule set depending on your Game Engine. Because all run-time engines already support joints, it is a tool we can exploit and push to its limits.
Traditional usage of Extra Joints to animate Props and Accessories
Extra “Helper Joints” to fix Smooth binding
Extra “Helper Joints” to create the next level in realism

8. Traditional Extra Joints
Traditional usage of Extra Joints for Props and Accessories:
Props, Rope
Hair, Ponytails
Capes, Necklaces, Clothing
Fat

9. New Ideas for Extra Joints
Extra “Helper Joints” to create the next level in realism:
Skin sliding
Wrinkles
Fat/Muscle/Flesh Jiggle
Muscle bulging

10. Fix Smooth binding
Extra “Helper Joints” to make skin transform properly:
Volume maintenance
Crease fixing
Complex Areas
Multiple joints affecting a single vert

11. Smooth Binding Limited
Vertex deforms by a weighted combination of Linear Translations around a joint rotating

12. Smooth Binding Shoulder

13. Smooth Binding Problem Areas
Elbow and Knee:
Interpenetration on the inside of the crease
Loss of volume on the outside of the crease (i.e. – the elbow tip or knee cap do not become ‘pointy’ when the elbow or knee bends)
Wrist/Forearm:
Loss of volume when the forearm/wrist is twisted too much down the length of the arm. (i.e. - ‘Candy-wrapper’ affect)
Upper Thigh/Gluteus:
Interpenetration at the bottom of the butt cheek when leg goes too far back
Shoulders:
Interpenetration in armpit
Interpenetration at the neck/shoulder junction area (trapezius)
Lack of representation of scapula movements

14. Our Goals – The “Ideal” What are we really trying to get?
Reference with Motion
Books
“Anatomy of Movement” by Blandine C. Germain
Internet
PAN Arts by Warwick Mellow
Software
Absolute Character Tools for 3DSMAX

15. Our Goals
PAN Research
Pectorals
Shoulders

16. Our Goals
PAN Research
Clavicle
Forearm

17. Our Goals
Muscles – Independent Research

18. Our Goals
Muscles - Absolute Character Tools

19. Disclaimer: RunTime Rig Support
Individual Developer code will dictate what will be evaluated in Real-Time and what type of relation you can specify between your primary skeleton’s animation and your Helper Joints’ transformation.
Hopefully you can get 1 or more of these RunTime Rigging methods supported:
SetDrivenKeys
Expressions
Constraints

20. Disclaimer: RunTime Rig Support
NO:
Source Code provided for engine side.
YES:
Techniques for art-side production of rigs (in Maya)
Applicable to all games on all consoles

21. How To Get There
Muscles are the drivers in reality so lets create Helper Joints that mimic the way muscles work.
Simple FusiForm muscles are easy to mimic in Maya so lets create those and bind them to our skin

22. Research – FusiForm Muscle
“FusiForm” muscle: Quick Definition
‘Tapering at each end; spindle-shaped.’
Explain basic fusiForm muscle

23. Research – The Building Blocks
Let’s pick the Biceps as our test case to create a muscle and mimic its action with a Helper Joint being driven by SetDrivenKeys:
Rotating
Scaling
Translating
The research explores the underlying mechanisms of the rigs
The ‘failed’ attempts still illustrate how rigs can be put together
Explain basic fusiForm muscle

24. Research – The Building Blocks
UNDERSTANDING HOW THE RIGS WORK
IS MORE VALUABLE
THEN MY SUGGESTED SOLUTIONS
Explain basic fusiForm muscle

25. Research – Biceps (NURBs)
Muscle as Influence Object - NURBs
Explain basic fusiForm muscle

26. Research – Biceps (Polys)
Muscle as Influence Object - Polygons

27. Research – Biceps (Short)
SetDrivenKey w/ Single Driver Axis
Rotating
Short
Notes test

28. Research – Biceps (Long)
SetDrivenKey w/ Single Driver Axis
Rotating
Long

29. Research – Biceps (Scale)
SetDrivenKey w/ Single Driver Axis
Scaling

30. Research – Biceps (Translate)
SetDrivenKey w/ Single Driver Axis
Translate

31. Research – Upper Arm (Translate)
Single Driver Axis
Translate
Works pretty good for Single driving axis!
Warning: Not anatomically correct!

32. Research – Biceps (RollBone)
SetDrivenKey w/ Single Driver Axis
Translate
Add Roll
BREAKS!

33. Research – Biceps (RollBone)
SetDrivenKey w/ Two Driver Axis
Translate
Add Roll
w/ Extra Keys compensating for roll
Only fixes in when arm is straight out
When arm is bent it still BREAKS!

34. Research – Biceps (RollBone)
Add parent joint w/ another set of driven keys for roll
Rotation of Helper’s parent joint is percentage of primary hierarchies rotation

35. Research – Shoulder (Multiple Drivers)
SetDrivenKey w/ Single Driver Axis from Two bones
Translate
Combination is difficult

36. Research – Shoulders (Multiple Drivers)
SetDrivenKey w/ Two Driver Axis from Two bones
Translate
2 Driving bones = BREAKS!(kinda)
+ 2 Driving axis = BREAKS!BAD!

37. RunTime Rigs – SetDrivenKeys
Summation
SetDrivenKeys are fast, easy to use and work great for “simple” Helper Joints
1 driveR joint and 1 driveR axis per driveN Helper Joint
Nest hierarchies of driveN Helper Joints if you have more than 1 driveR joint or axis

38. RunTime Rigs – SDKs for Volume Preservation
Forearm Helper Joint w/ SDK’s for Scale and rotation

39. RunTime Rigs – SDKs for Wrinkles
Scaling of Helper Joints (R&D by Tyler SCEA)

40. RunTime Rigs – SDKs for Wrinkles
Joints are scaling on 2-axis (bulging), no scaling on length-wise axis
No translation or rotation

41. RunTime Rigs – SDKs for Wrinkles
Weighting is very tedious

42. RunTime Rigs – SDKs for Wrinkles
For shirt it takes many more joints because of 3 D.O.F. and larger surface.
6 joints to do it right

43. RunTime Rigs – SDKs for Wrinkles
Different joints scale for each direction in each D.O.F.
Weighting is time consuming

44. RunTime Rigs – SDKs for Wrinkles
Incorporation:
Could theoretically add a scaling joint as a child of the helper joints
Then just weight individual vertices to these scaling joints to cause ‘wrinkling’
Or just make separate scaling wrinkle joints wherever needed

45. RunTime Rigs
Done with SDKs
Start Expressions

46. RunTime Rigs – Expressions (Simple:Elbow)
Simple Helper Joints which only need a Translation Expression
Based on 1 axis from driving bone
dj_helper.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))

47. RunTime Rigs - Expressions (Complex:Biceps)
Complex Helper Joints which need Translation & Rotation Expression
Translation: Use another expression to translate dj_bicep joint (parented to Bicep_Helper) for bulge based on elbow rotation.
Expression 2 – Describes bulge from rotation of elbow’s Z-axis
dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))
base.ty = 5, maxOffset = 1.8, maxAngle = 110
Rotation: Joint needs to be parented to both shoulder and bicep-roll joints. Use Expression to take percentage.
Expression 1 – Describes rotation of parent joint around bicep-roll joint. (60% from shoulder, 40% from bicep-roll)
Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))

48. RunTime Rigs - Expressions (Complex:Biceps)
Rotation: Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))
Translation: dj_bicep.ty = (base.ty + (maxOffset * (elbow.rz /maxAngle)))
base.ty = 5, maxOffset = 1.8, maxAngle = 110

49. RunTime Rigs – Expressions & Constraints (Complex:Biceps)
Complex Helper Joints which need Scale & Rotation Expressions but looking at Distance
Requires 6 extra parent helpers (can be joints or ‘nulls’)
Component 1 – Scale (Bulge)
Handled by expression based on distance of ‘shock-absorber’
bicepHelper.sz = (scaleFactor*(1/(boneLength))) : boneLength = sqrt ((foreArmOffset.tx – bicepOffset.tx)² + (foreArmOffset.ty – bicepOffset.ty)² + (foreArmOffset.tz – bicepOffset.tz)²)
Component 2 – Rotate
Handled by expression base on % of angle of bicep_roll
Bicep_Helper.rx = ((.4 * bicep.rx) + (.6 * shoulder.rx))
Requires 2 pointConstraints

50. RunTime Rigs – Expressions & Constraints (Complex:Biceps)
Scale Expression based on distance

51. RunTime Rigs - Expressions
Summation
Very difficult to describe a motion of a Complex Helper Joint based on 2 or more axis from driving joint. Even MORE difficult when there are multiple bones acting as drivers.

52. RunTime Rigs
Done with Expressions
Start with Constraints

53. RunTime Rigs - Constraints
pointConstraint
Good for position only
orientConstraint
Good for orientation only
parentConstraint
Good for both position and orientation
aimConstraint
Robust and elegant
vertexConstraint
Allows the use of muscles

54. RunTime Rigs - Constraints
AIMCONSTRAINT IS
DA BOMB!

55. RunTime Rigs – Constraints (Complex:Biceps)
aimConstraint pointing at forearm
Arc problem
Handles rotation of bicep roll on extension but not flexion

56. RunTime Rigs – Constraints (Complex:Biceps)
aimConstraint flexion problem
Handles rotation of bicep roll on extension but not flexion

57. RunTime Rigs – Constraints (Complex:Biceps)
aimConstraint flexion problem handled on XZ plane w/ expression for Bulge
Bicep_Helper’sParent.scale = (baseScale + (maxOffset * (elbow.rz /maxAngle)))
i.e. - baseScale = 1, maxOffset = .3, maxAngle = 110

58. RunTime Rigs – Constraints (Complex:Biceps)
aimConstraint still broken on Y plane during flexion

59. RunTime Rigs – Constraints (Complex:Pectoralis)
Requires 2 extra parent helpers (can be joints or ‘nulls’)
Requires 1 aimConstraint

60. RunTime Rigs – Constraints (Complex:ArmPit)
Requires 2 extra parent helpers (can be joints or ‘nulls’)
Requires 1 aimConstraint

61. RunTime Rigs – Constraints (Complex:Pectoralis & Armpit)
Requires 3 extra parent helpers (can be joints or ‘nulls’)
Requires 2 aimConstraints

62. RunTime Rigs – Constraints (Simple:Neck)
Look at normal smooth bind

63. RunTime Rigs – Constraints (Simple:Neck)
NURBs muscle as Influence Object

64. RunTime Rigs – Constraints (Simple:Neck)
Requires 1 helper joint and 1 target null for aiming
Requires 1 aimConstraint

65. RunTime Rigs – Constraints (Simple:Neck)
Comparison

66. RunTime Rigs – Constraints (Simple:Neck - Back)
Requires 1 helper joint and 1 target null for aiming
Requires 1 aimConstraint

67. RunTime Rigs – Constraints (Simple:Neck - All)
Requires 4 helper joints w/ 4 extra target nulls
Requires 4 aimConstraints

68. RunTime Rigs – Constraints (SuperComplex – Scapula)
Look at normal smooth bind

69. RunTime Rigs – Constraints (SuperComplex – Scapula)
What is scapula really doing?

70. RunTime Rigs – Constraints (SuperComplex – Scapula)
Comparison
With Rig
Normal Binding

71. RunTime Rigs – Constraints (SuperComplex – Scapula)
Shrugging

72. RunTime Rigs – Constraints & (Exp./SDK) (SuperComplex – Scapula)
Requires 2 helper joints and 2 target nulls for aiming
Requires 2 aimConstraints & 1 pointConstraint
Requires expression w/ “if” statement:
If (lShoulder.rotateY >0) {
l_scapulaConst.rotateX = ( $scaleFactor* (lShoulder.rotateY/$maxAngle)); }
Else
l_scapulaConst.rotateX = 0;
Ex: $scaleFactor = 30 degrees and $maxAngle =90 degrees.
OR SetDrivenKey

73. RunTime Rigs – Constraints & (Exp./SDK) (SuperComplex – Scapula)
Explanation of (SDK or “If” statement in expression)

74. RunTime Rigs – Constraints & (Exp./SDK) (SuperComplex – Scapula)
Pros: Full Range Looks Great
Cons: Complex and Weighting is Hard!

75. RunTime Rigs – Expression/SDK (Complex – ArmPitBack)
Normal armpit crease

76. RunTime Rigs – Expression/SDK (Complex – ArmPitBack)
New Helper Joint added

77. RunTime Rigs – Expression/SDK (Complex – ArmPitBack)
Translation of Helper Joint tip driven by expression or SetDrivenKey from shoulder rotation
Requires 1 extra parent Helper Joint w/ percentage rotation of shoulder driven by Rotate expression or SetDrivenKey

78. RunTime Rigs – Expression/SDK (Complex – ArmPitBack)
armPitParent Joint rotating 60% of shoulder joint

79. RunTime Rigs – Exp./SDK (Complex – ArmPitBack)
armPitTip joint translating down its local X & Y as a function of shoulder rotation

80. RunTime Rigs – Exp./SDK (Complex – ArmPitBack)
Requires 1 simple expression for parent Joint’s rotation:
HelperParent.rotateY = (offset angle + ((% arm rotate) * lShoulder.rotateY)))
Ex - dj_armPitBackParent.rotateY = (-35 * (.6 * lShoulder.rotateY));
Requires 1 expression for translation of tip w/ “if” statement:
If (lShoulder.rotateY <0){
HelperTip.translateX = (base.tX + ($scaleFactorX*(lShoulder.rY/$maxAngle)));
HelperTip.translateY = (base.tY + ($scaleFactorY*(lShoulder.rY/$maxAngle)));
}Else{
HelperTip.translateX = baseX;
HelperTip.translateY = baseY; }
Ex: $scaleFactorX = 2 units, $scaleFactorY = -4 units and $maxAngle =65 degrees.

81. RunTime Rigs – Exp./SDK (Complex – ArmPitBack)
Weighting is powerful and tricky. Some vertices now have 5 different weights.

82. RunTime Rigs – Scaling Entire Rig
What if character needs to be scaled? (i.e.- player defined proportions)
A simple ‘Group’ node above the rig mechanism in Maya’s hierarchy will scale easily
The Trick is figuring out how much to scale. This is dependent on what ‘space’ the helper joint deforms, and how that space has been scaled.

83. RunTime Rigs
Done with Constraints
Start with Muscles

84. RunTime Rigs - Muscles
Muscles at RunTime allows you to constrain Helper Joints to the muscle surface and get the effect of the muscle
Requires some sort of geometryConstraint

85. RunTime Rigs – Constraints & Bind & Blendshape (Muscles:Biceps)
Deformer Object is low-count poly muscle rigid (or smooth) bound to two joints to handle twisting of bicep
Requires a blendShape on deformer object to handle bulge.
*Note: blendShape is not required on main skinning surface.
Complex Helper Joint is ‘VertexConstrained’ to vertex on Deformer Object

86. RunTime Rigs – Constraints & Bind & Blendshape (Muscles:Biceps)
VertexConstrained

87. RunTime Rigs – Constraints & Bind & SDK (Muscles:Biceps)
Poly Muscle (Simplest Form)
1 triangle Rigid (or Smooth) Bound
Handles rotation of bicep
SDK (or simple expression [1 driving bone/axis, 1 driven translating axis]) driving single vertex for bulge
Complex Helper joint ‘VertexConstrained’ to single vert
Finally skinning layer

88. RunTime Rigs – Constraints & Bind & SDK (Muscles:Biceps)
1 Poly (Simplest Form)
VertexConstraint

89. RunTime Rigs – Constraints & Bind (Muscles:Pectoralis)
Deformer Object is low-count poly muscle rigid (or smooth) bound to three joints to handle complex Pectorals motion
Requires MuscleDriver parent joints to be offset from main skeletal joints
MuscleDriver child joints are ‘pointConstrained’ to parent joints to avoid rotation of parent joints
Complex Helper Joint is ‘VertexConstrained’ to vert on Deformer Object

90. RunTime Rigs – Constraints & Bind (Muscles:Pectoralis)
PointConstraints & VertexConstraints

91. RunTime Rigs to Non-RunTime

92. Non-RunTime Rigs Helper Joints Definition:
They can be controlled by SetDrivenKeys, Expressions, Simple Rigs (constraint system), or Super Advanced Rigs (really any simulation you can think of) that are baked-out.

93. Non-RunTime Rigs - Baked Out
BakeOut examples
Bicep
Complex Helper Joints constrained to nurbsMuscle influence object, baked out, 1 key per frame, scene cleaned.

94. Non-RunTime Rigs - Baked Out
BakeOut examples
Pectorals
Complex Helper Joints constrained to nurbsMuscle influence object, baked out, 1 key per frame, scene cleaned.

95. Non-RunTime Rigs - Baked Out
BakeOut examples
Pectorals
Lots!

96. Non-RunTime Rigs - SplineIK
Great Skin-sliding effect

97. Non-RunTime Rigs - Jiggle
Basic Jiggle is fairly easy to create in Maya, either with a:
Simple Expression
frameCache node
Could be turned into RunTime if your code supported it

98. Non-RunTime Rigs - Jiggle
Create a 1 (or more) frame lag in global space for a joint
BaseJoint = ‘Goal’
Blue joint parented to spine
EndJoint = ‘Weighted’
Yellow joint lagging behind Blue joint by 1 frame

99. Non-RunTime Rigs - Jiggle
Expression:
int $frame = frame; // get the current frame
float $tx_0 = BellyJiggleBase.translateX;
float $tx_1;
// reset the attributes on start frame
if ($frame == 1){ // start frame
$tx_1 = $tx_0;
BellyJiggleEnd.translateX = $tx_0; }
else{
BellyJiggleEnd.translateX = $tx_1;
Repeat for Y & Z axis

100. Non-RunTime Rigs - Jiggle
frameCache Node

101. Non-RunTime Rigs - Jiggle
frameCache Expression

102. Non-RunTime Rigs - Jiggle
Incorporation:
Could theoretically add a jiggle joint as a child of the helper joints
Then just weight individual vertices to these jiggle joints to cause ‘jiggling’
Or just make separate jiggle joints and their ‘base’ joint wherever needed
Where are they needed?

103. Non-RunTime Rigs to Automation

104. Automating Production
Scripts
Workflow
Offline Muscle Systems
Procedurally generated Helper Joints

105. Automating Production
Scripts: Tools you’ll need
Script to auto-build entire Helper Joint System on your character
Selectable body parts
Scripts to automate editing of Rigs, SetDrivenKeys, & Expressions for the artists
Scripts to do lots of mirroring:
Helper Joint Rigs
Weighting
Set Driven Keys
Export and Import Set Driven Keys
Hint: look at the animCurves and add custom attrs for connections to them upon export
Demo

106. Automating Production
Workflow
Duplicate your primary skeleton and skin
Apply your muscle system to this duplicate character
Constrain duplicate skeleton to primary skeleton
Hide duplicated skeleton, muscles, & rigs. Template duplicated skin.

107. Automating Production
Workflow

108. The Ultimate Reference
Muscle Systems

109. The Ultimate Reference
Muscle Systems

110. The Ultimate Reference
Muscle Systems
Auto imports muscle system
Scaling to your skeleton
AutoFits muscles to duplicated skin
AutoSkins muscles to duped skin

111. The Ultimate Reference
Once you have the “Ultimate Reference”, you now have the data to let a programmer put you out of a job. All (s)he needs to figure out is:
How many joints?
Where to place them?
How do they need to transform?
What does their weighting need to be?

112. Mohr/Gleicher Method
Siggraph 2003 : “Building Efficient, Accurate Character Skins from Example”
White Paper on auto-computing placement, animation, and weighting of Helper Joints.

113. Conclusion
We’ve covered just a few examples of how to use SetDrivenKeys, Constraints, and Expressions.
My solutions are just first shot attempts. With refinement, you can come up with much better solutions.
With these simple tools you can use them in many combinations to achieve nearly any effect you need.
Of course everything is RunTime code dependent. As Character T.D.s we need to fight for:
SetDrivenKeys
Constraints
Point, orient, !!AIM!!, parent, geometry
Expressions
Spline IK
frameCache (jiggle)
Helper Joints are just a temporary solution until we can get more advanced deformers at RunTime to simulate Muscles
Per-vertexDeformer (influence objects)
WrapDeformer (lattices)
fusiForm deformer
. . . Talk more about this next year?

114. Thanks
… for your time.
Please fill out surveys so I can make a better one next year.
Peace out

115. Contacts, Reference, Credits
Webpage (Helper Joints-GDC 2005):