2. Handrix: Animating the Human Hand George ElKoura Karan Singh University of Toronto

4. Motivation How do we get from a piece of music: To finger motions that play the music:

5. Overview System Architecture OverviewSystem Architecture Overview Guitar AlgorithmGuitar Algorithm Anatomy of the HandAnatomy of the Hand Hand ModelHand Model ResultsResults

6. System Architecture

11. Related work AnatomyAnatomy –Häger-Ross and Schieber, 2000 VisionVision –Lee and Kunii, 1995 GraphicsGraphics –Rijpkema and Girard, 1991 MusicMusic –Quine, 1990

12. A Bit of Anatomy Why are fingers interdependent?Why are fingers interdependent? –“One to many” muscle insertion sites –Close proximity of tendons –Neurological constraints A clear anatomical understanding is still being developedA clear anatomical understanding is still being developed

13. Hand Model Goal:Goal: –Map partially specified hand positions to realistic hand postures Approach:Approach: –Motion Capture lots of hand positions –Nearest Neighbors, RBFs to approximate hand-pose space

14. Interdependence Assumptions Assumptions to reduce number of parameters:Assumptions to reduce number of parameters: 1.Independence of the Thumb 2.Invariance of Hand Dominance 3.Independence of Adduction/Abduction 4.Irrelevance of Frequency 5.Locality: Posture of wrist and arm do not affect digit independence These assumptions reduce the problem to 12 DOFs.These assumptions reduce the problem to 12 DOFs. Reference: Häger-Ross and Schieber, Journal of Neuroscience 20(22), 2000 © Babette Adrian

15. Interdependence Reconstruction Allow the animator/controller to partially specify a hand posture (  0,  1, …  n )Allow the animator/controller to partially specify a hand posture (  0,  1, …  n )

16. Interdependence Reconstruction Allow the animator/controller to supply weights for each joint indicating the importance of the supplied joint angle (  0,  1, …,  n )Allow the animator/controller to supply weights for each joint indicating the importance of the supplied joint angle (  0,  1, …,  n ) 1  0 = 1 0  n = 0 …

17. Interdependence Reconstruction Correct the user-supplied hand posture using the given weights and the database of physically possible posturesCorrect the user-supplied hand posture using the given weights and the database of physically possible postures

18. Reconstruction Method The database is a collection of joint angles represented by 12D vectors:  IThe database is a collection of joint angles represented by 12D vectors:  I Compute the weighted distance by:Compute the weighted distance by: The k-Nearest Neighbors interpolation weights are computed by:The k-Nearest Neighbors interpolation weights are computed by:

19. Interdependence Results (No Reconstruction)

20. Interdependence Results (With Reconstruction)

21. Interdependence Results (Comparison)

22. The Guitar 6 strings numbered from 1 to 6 (heavy to light) Frets delimit strings lengths producing audibly different notes Frets are numbered in increasing order from the nut (fret 0)

23. Tablature Common guitar music notation Common guitar music notation Tells musician what frets to hold Tells musician what frets to hold Does not tell musician what fingers to use Does not tell musician what fingers to use Simple ASCII, readily available (OLGA) Simple ASCII, readily available (OLGA) Adaptable to specifying multi reaching goals over time Adaptable to specifying multi reaching goals over time

24. Fretting Design Objectives 1. Economy of effort 2. Read-ahead control 3. Maintain natural motion of fingers 4. Capture differences in finger dexterity and strengths 5. Fingertips touch guitar at the correct fret Reference: Hector Quine, Guitar Technique, Oxford University Press, 1990

25. Guitar Space

26. Cost Functions Assign a different cost function to each finger, wristAssign a different cost function to each finger, wrist Each cost function can be a sum of specific cost functions to separately handle dexterity, strength and so forthEach cost function can be a sum of specific cost functions to separately handle dexterity, strength and so forth Overall cost is computed, and cheapest fingering winsOverall cost is computed, and cheapest fingering wins

27. Finger Resolution Algorithm

28. Finger Collisions Finger collisions are handled in a simple way When a collision is detected, the colliding finger is moved backwards on the fretWhen a collision is detected, the colliding finger is moved backwards on the fret

29. Results – C-Major Scale

30. Results – Chromatic Scale (fails)

31. Results – Chromatic Scale (passes)

32. Results – Chord Transitions

33. Results – More Chords

34. Results – Bar Chords

35. Results – C-Major Scale (3 Fingers)

36. Results – Stairway to Heaven

37. Future Work Apply to other areas:

38. Future Work Slides, bends, hammersSlides, bends, hammers Frequency of motionFrequency of motion Motion Dynamics (strumming hand)Motion Dynamics (strumming hand) Anatomical Hand ModelAnatomical Hand Model Extension to other areas (e.g. locomotion)Extension to other areas (e.g. locomotion)

39. Summary System Architecture Overview Guitar Algorithm Anatomy of the Hand Hand Model Results For More Information Please Visit: http://www.dgp.toronto.edu/~gelkoura/handrix/index.html

40. Acknowledgements DGP Lab, University of Toronto Chris Landreth Dave Baas Side Effects Software, Inc. Alias Systems NSERC For more information please visit: http://www.dgp.toronto.edu/~gelkoura/handrix/index.html