1. Rowing Motion Capture System Simon Fothergill Ph.D. student, Digital Technology Group, Computer Laboratory Jesus College graduate conference May 2009

2. Overview The Bigger Picture Previous work Problem Process Data Capture System Results Future work

3. The Bigger Picture Sentient Computing! Computer Vision Pattern Recognition & Machine learning A long way to go!

4. The Bigger Picture – Watching Humans Physical Performances Heath care What are they doing? How well are they doing it? How should be improved? How should they be told?

5. Previous Work - Activity / Gesture recognition Motion capture methods have included: Blob tracking Point trajectories Recognition techniques have included: Single frame Multiple frame Parametric

6. Learn the quality of a performance from body part trajectories Minimise markers using redundancy Complex trajectories, continuous score Flexible rubrics require learning Different types of expert labelling: Explanations Non-specific / specific Different granularities of quality Which sections of the trajectory are how relevant? One section of a can depend on many aspects

7. Process Learning Judging Performance Capture motion Expert coach labels with their judgement Trajectories Inference model Learn Video Performance Capture motion TrajectoriesInference model Features Extract and select features Features Extract and select features Judgement Evaluate Capture video

8. Data Capture System Erg Power Control Motion sensitive LED markers ECS Wii controllers

9. Data Capture System - Architecture Nintendo Wii controller Bluetooth IR 1024x768 camera (100Hz) Nintendo Wii controller IR 1024x768 camera (100Hz) PC Wii library Bluetooth library C server Bluetooth PC Java / C client Video camera (30Hz) Fire wire TCP/ IP C server Buffer Wii controller

10. Data Capture System – Calibration and operation Server Triangulation Stereo calibration Client 4 x 2D coordinates 4 x 3D coordinates Erg calibration Label markers Transform to ECS Update ECS if necessary ECS Detect strokes Log data Log files Save picture Encodes video Calibrate labeller Calibrate WMCS StorageBatch Display on GUI Calculate stats Control camera Calibration Live operation

11. Data Capture System Example video

12. Preliminary Results Preliminary results have been obtained using a dataset of 6 rowers and the complete trajectory of the erg handle only. Binary classification over stroke quality was done using tempo-spatial features of the trajectory and a neural network. Two training methods were compared. Classification accuracy across given number of performers, for quality of individual aspects of technique.

13. Summary and Further Work Data capture system and how it fits into the bigger picture More information is available on the feature extraction & selection and inference algorithms. A larger data set would allow conclusive results to be obtained Feature extraction and selection methods that address using the relevant segments of the relevant trajectories More sophisticated modelling based on particle filters Supports multiple body parts and labelling methods Uses a distribution of motion vectors to probabilistically track the “quality so far” as the stroke evolves.

14. In Conclusion Advertisement! Acknowledgements Professor Andy Hopper, Dr Sean Holden, Dr Robert Harle Members of the DTG and Rainbow groups, Computer Laboratory Jesus College, JCBC and the Graduate society References Optical tracking using commodity hardware, Hay, S.; Newman, J.; Harle, R.; ISMAR 2008. Page(s):159 - 160 Thank you! Questions? Please come down to the boathouse and use the data capture system!