1. Centre for Vision, Speech & Signal Processing University of Surrey Adrian Hilton, John Collomosse Peng Huang, Dan Casas, Ed Brookes, Chris Budd, Marco Volino(BBC icase)

2. 4D performance modelling & animation Performance Capture 4D Model Interactive Animation

3. Vision, Speech & Signal Processing CVSSP Director: Professor Josef Kittler, FREng ‣ IET Faraday Medal 2008 ‣ IAPR KS Fu Award 2006 ‣ Distinguished Professor 2004 Research Focus: Multidimensional signal processing, interpretation and understanding Signals: audio, speech, images, video, 3D video, 3D medical images, 3D volumetric sequences ‣ Founded in 1986 ‣ 110 people (15 academics, 30 post-docs, 65 PhDs) ‣ Top ranking in RAE 2008 assessment (Electronic Eng.)

4. Vision, Speech & Signal Processing CVSSP Structure A-lab: Machine Audition C-Lab: Cognitive Vision B-Lab: Biometrics I-Lab: Multimedia Communications 34 M-Lab: Medical Imaging Communicatio n V-Lab: Visual Media ImagiCommuni cation Core Science Signal Processing Image & Video Processing Pattern Recognition Computer Vision Machine Learning

5. Vision, Speech & Signal Processing V-Lab: Visual Media Prof. Adrian Hilton, Dr. John Collomosse, Dr. Krystian Mikolajczyk Enabling technologies for next generation communication & entertainment ‣ Visual content production ‣ Audio-visual search and retrieval ‣ Visual interaction & communication ‣ 3D and free-viewpoint video ‣ Visual content analysis ‣ Video-based animation production ‣ Human shape & motion analysis Film, broadcast and interactive entertainment, fashion, biomechanics Visual Media Platform Grant (EPSRC) BBC Audio-Visual Partnership - audio-visual 3D media production Body shape recognition for online fashion (EPSRC) RE@CT - Production and delivery of interactive 3D content (EU) SCENE - Novel scene representation for 3D media (EU) SyMMM - Onset metadata in movie production (TSB) VBAP - Video-based animation production (EPSRC) Collaborators: BBC, Sony, Framestore, DNeg, The Foundry, Filmlight, Vicon, Bodymetrics 3D video Free-viewpoint video video stylisation visual searchobject tracking

6. Vision, Speech & Signal Processing Visual Content Production

7. Performance Capture Multiple view video 3D video [Starck IEEE CGA’07]

8. SurfCap 3D Video Database [Starck et al. CGA’07] I: Studio Capture

9. II: Global Non-rigid Alignment 3D video sequences Shape similarity tree4D model [Huang CVPR’11, Budd 3DIMPVT’11]

11. Interactive Animation Interactive control of character animation: editing motion high-level parameterisation of motion transitions between motions 4D model Interactive Animation

12. Skeletal Character Animation Motion Editing Motion Parameterisation Motion Graphs Heck & Gleicher’07 Kovar&Gleicher’02 Witkin’95 Brundelin’95 Rose’98 Arikan&Forsyth’02

13. 3D video concatenation [Huang et al. CVPR’09] Text key startkey end surface motion graph representation key-frame animation [Starck SCA’05,CGA’07]

14. 3D video concatenation [Huang et al. CVPR’09]

15. 4D motion editing Goal: Interactive editing of 4D models Space-time key-frame editing Laplacian deformation framework learnt 4D motion space [Tejera CVMP’11]

16. III: 4D motion editing [Tejera et al. CVMP’11]

17. III: 4D motion parameterisation High-level real-time motion control parameters: walk speed/direction, jump height etc combine multiple skeletal sequences [Rose’98] solution: mesh sequence blending ie walk/run walk run

18. 4D motion parameterisation Mesh sequence blending (1) temporal alignment: dynamic time warp (II) blend corresponding frames non-linear blending (Laplacian): ~100ms/frame linear blending: <8ms/frame but unrealistic solution: hybrid non-linear blending 10ms/frame [Casas MIG‘11]

19. source target [Casas et al. MIG’2011]

20. [Casas MIG’11,I3D’12] 4D motion parameterisation

21. III: Interactive Animation interactive motion transitions skeletal motion graph [Gleicher’02,Arikan’02] [Casas MIG’11] solution: 4D parametric motion graph real-time transitions using shape similarity

23. Summary Performance capture 3D video capture indoor/outdoor joint segmentation & reconstruction Structured representation Global non-rigid alignment shape similarity tree 4D models Interactive Animation 4D motion parametrisation 4D parametric motion graphs space-time editing

24. RE@CT: Surrey Pipeline for production of interactive 4D models integration of skeletal control with surface motion graphs (WP3) parametric control for general motion (WP4) interactive control of motion constraints (foot slide, hand-contact etc.) (WP4) interactive motion stylisation (WP4)

25. WP3T5 Surface motion graph extraction & development automatic graph construction similarity measures for dynamic shape & appearance efficient representation D3.1 (month 12): Initial actor model skeleton based motion graph

26. WP4: Actor Animation Algorithms Interactive animation API using surface motion graphs T4.1 Specification (all) T4.2 Parametric surface motion graph (Surrey,BBC) T4.3 Integration of face and body (HHI,BBC,INRIA,ART) T4.4 Artistic editing and control (Surrey) T4.5 Adaptation & manipulation of appearance (HHI) D4.1 (month 12): Initial RE@CT animation engine based on skeletal surface motion graph WP3

27. WP4T1: Specification (All) Data formats for parameterised motion graph Interface for face & body animation Interface for authoring content Representation for parameterised 4D models Interfaces and representation for appearance 1. input from all partners on current/proposed rep. 2. draft data/API specification (month 3, Surrey) 3. working data/API specification (month 6, Surrey)

28. Vision, Speech & Signal Processing 3D Sports Production

29. Vision, Speech & Signal Processing 3D Sports Production

30. Vision, Speech & Signal Processing Outdoor 3D Production