Presentation on theme: "Choreographer- mathematician collaboration developing machine segmentation techniques for motion capture analysis of dance."— Presentation transcript:
Choreographer- mathematician collaboration developing machine segmentation techniques for motion capture analysis of dance
Associate Professor Kim Vincs Dr Wai Kuan Yip Dr Vicky Mak Ms Kim Barbour
Motion capture & dance analysis Motion capture offers precision and true 3D tracking of dance movement Recent development over the last 10 years Relatively few motion capture labs with dance focus A new gold standard for quantitative dance analysis?
Challenges in motion capture dance analysis Analysis needs to be culturally and artistically relevant (deLahunta) Pattern recognition in dance crosses multiple frames of reference both between and within genres (?ref) Approach to capture protocol, marker set design and feature extraction need to be appropriate to the needs of end-users, i.e. dance artists (Norman)
‘Capturing Dance’ project at Deakin Motion.Lab Aims to isolate features of motion capture data that are artistically useful to choreographers Features may be different for different dance genres and different dance artists Will work with artists across 3 dance genres; contemporary dance, ballet and Australian indigenous dance over 3 years
Project team Associate Professor Kim Vincs Dr Vick Mak Associate Professor Richard Smith Dr Wai Kuan Yip Ms Kim Barbour Mr Daniel Skovli Mr Peter Driver Ms Lisa Bolte Ms Carlee Mellow Ms Phoebe Robinson Ms Mee Young Yuk
This paper Analysis of a single, complex contemporary dance phrase 20 repeats of the phrase by a single dancer 64 marker-set adapted to capture spine-arm-foot relationships important to dance Approaches to segmentation into dance-meaningful chunks for analysis Necessity of choreographer-mathematician collaboration in developing appropriate analysis techniques
Challenges in segmenting dance Dance phrases tend to overlap Different body segments need different levels of smoothing Both large and small movements may be artistically significant Different combinations of body segments may initiate new movements Dance artists themselves don’t agree on how phrases are segmented (deLahunta 2005)
Some previous approaches to segmenting dance movement Inter-limb correlation (Nakata 2007) Laban Movement Analysis, (Bouchard & Badler 2007) Minimum velocity (Hachimura & Nakamura 2001) Triple minima; force, kinetic energy & momentum (Kahol, Tripathi & Panchanathan 2004)
Preliminary data – manual segmentation of a ‘tendu’ Expert segmentation problematic as depends on weighting conflicting factors Surprising variability within samples (5 samples, elite ballet dancer) Highlights difference between dancers’ conceptual map of the movement and the detailed ‘motion capture view’
Our approach Primary question is what segmentation is artistically meaningful? We used a collaborative, practice-based approach to develop segmentation ‘schema’ for the phrase Choreographer, 2 dancers and 2 mathematicians developed definition of segments based on kinematics Aimed to create an automated system based on the artistic schema
Problems identified Dancers used multiple conceptual frameworks, e.g. velocities, heights, correlations of body parts Dancers’ framework is ‘procedural’ – segment definitions only make sense in relation to the preceding movement Segmentation is phrase-specific and cannot be generalized
Segmentation steps Conversion from positional information to hierarchical (parent-child) translation and rotation format to make analysis scale & position invariant Smoothing to remove noise – Butterworth 6Hz, moving average 20-50 window, Gaussian smoothing
Segmentation steps Polygonal approximation to estimate gradients of consecutive points to find local minima 3 possible scenarios; Negative gradient positive gradient Zero-gradient positive gradient Positive gradient zero gradient
Examples of polygonal approximation Important to select the right threshold to estimate local minima Above: Maximum tolerable threshold 5*SD (just nice) Above Right: 15*SD (too loose) Right: 30*SD (way too loose)
Local minimum segmentation Root location Right elbow location Absolute rotational velocity Local minimum segmentation
Example of compliant sample (#11) Example of non-compliant sample (#2)
Local minimum segmentation Example of compliant sample (#17) Example of non-compliant sample (#12)
Summary Method works very well for movements that rely on body-shape change, but breaks down for large loco-motor and turning movements Large variation in ‘dancer compliance’ Variation or ‘dancer compliance’ does not correlate with dancer/choreographer ratings – variation in inherent in the style
Conclusions Dancer-mathematician collaboration useful for identifying recognition problems that need to be solved Dance recognition process needs to be able to deal with a high level of intra-performer variability Different types of movement may require different approaches to segmentation Further techniques for optimizing machine segmentation (eliminating false segmentation points) needed to supplement dance-generated phrase descriptions
References Bouchard, D & Badler, N 2007, 'Semantic Segmentation of Motion Capture Using Laban Movement Analysis', in C. Pelachaud et al. (ed.), Intelligent Virtual Agents, Springer-Verlag, Berlin Heidelberg, vol. 4722, pp. 37-44. DeLahunta, S & Barnard, P 2005, 'What's in a Phrase?', in J Birringer & J Fenger (eds), Tanz im Kopf: Jarbuch 15 der Gesellschaft für Tanzforschung, LIT Verlag, Münster. Hachimura, K & Nakamura, M 2001, 'Method of generating coded description of human body motion from motion-captured data', Robot and Human Interactive Communication, 2001. Proceedings. 10th IEEE International Workshop on, pp. 122-7. Kahol, K, Tripathi, P & Panchanathan, S 2004, 'Automated gesture segmentation from dance sequences', Proceedings, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 17-19 May pp. 883-8. Nakata 2007, 'Temporal Segmentation and Recognition of Body Motion Data Based on Inter-Limb Correlation Analysis', Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), Oct. 29 2007-Nov. 2 2007 pp. 1383 - 8 Norman, SJ 2006, 'Generic Versus Idiosycratic Expression in Live Performance Using Digital Tools', Performance Research, vol. 11, no. 4, pp. 23-9.
Your consent to our cookies if you continue to use this website.