1. Multimedia System and Networking Lab @ UTD Slide- 1 University of Texas at Dallas B. Prabhakaran Computer Animation Main Text: –"Computer Animation: Algorithms & Techniques“, Rick Parent, Morgan Kaufman publishers. –"3D Computer Graphics: A Mathematical Introduction with OpenGL", Samuel R. Buss, Cambridge University Press

2. Multimedia System and Networking Lab @ UTD Slide- 2 University of Texas at Dallas B. Prabhakaran Course Outline 1.Skeletons 2.Quaternions 3.Skinning 4.Facial Animation 5.Advanced Skinning 6.Channels & Keyframes 7.Animation Blending 8.Inverse Kinematics 9.Locomotion 10.Particle Systems 11.Cloth Simulation 12.Collision Detection 13.Rigid Body Physics

3. Multimedia System and Networking Lab @ UTD Slide- 3 University of Texas at Dallas B. Prabhakaran Contact Information B. Prabhakaran Department of Computer Science University of Texas at Dallas Mail Station EC 31, PO Box 830688 Richardson, TX 75083 Email: bprabhakaran@utdallas.edu Fax: 972 883 2349 URL: http://www.utdallas.edu/~praba Phone: 972 883 4680 Office: ECSS 3.706 Office Hours: Tuesdays/Thursdays 10.30-11.15am Other times by appointments through email Announcements: Made in class and on course web page. TA: TBA.

4. Multimedia System and Networking Lab @ UTD Slide- 4 University of Texas at Dallas B. Prabhakaran Prerequisites CS 2236 (CS2), CS/SE 3345 (Data structs & Alg. analysis), Math 2418 (Linear Algebra). Familiarity with –Vectors (dot products, cross products…) –Matrices (4x4 homogeneous transformations) –C++ or Java –Object oriented programming –Basic physics

5. Multimedia System and Networking Lab @ UTD Slide- 5 University of Texas at Dallas B. Prabhakaran Evaluation 1 or 2 Homeworks 1 Final Exam: 75 minutes or 2 hours (depending on class room availability). Mix of MCQs and Short Questions. Programming Projects

6. Multimedia System and Networking Lab @ UTD Slide- 6 University of Texas at Dallas B. Prabhakaran Grading 70% Projects 5% Homeworks 25% Final

7. Multimedia System and Networking Lab @ UTD Slide- 7 University of Texas at Dallas B. Prabhakaran Schedule Final Exam: Last week of class OR As per UTD schedule Projects and homework(s) schedules will be announced in class and course web page, giving sufficient time for submission.

8. Multimedia System and Networking Lab @ UTD Slide- 8 University of Texas at Dallas B. Prabhakaran Programming Projects No copying/sharing of code/results will be tolerated. Any instance of cheating in projects/homeworks/exams will be reported to the University. No copying code from the Internet. 2 individual students copying code from Internet independently: still considered copying in the project !! Individual projects. Deadlines will be strictly followed for projects and homeworks submissions. Projects submissions through eLearning. Demo may be needed

9. Multimedia System and Networking Lab @ UTD Slide- 9 University of Texas at Dallas B. Prabhakaran Cheating Academic dishonesty will be taken seriously. Cheating students will be handed over to Head/Dean for further action. Remember: home works/projects (exams too !) are to be done individually. Any kind of cheating in home works/ projects/ exams will be dealt with as per UTD guidelines. Cheating in any stage of projects will result in 0 for the entire set of projects.

10. Multimedia System and Networking Lab @ UTD Slide- 10 University of Texas at Dallas B. Prabhakaran Proposed Projects 3 projects Encourage you to come up with your own project proposal too Announcements will be made soon Use OpenGL Possible use of Autodesk 3D Max Or other public domain software C/C++ mostly

11. Multimedia System and Networking Lab @ UTD Slide- 11 University of Texas at Dallas B. Prabhakaran Applications Special Effects (Movies, TV) Video Games Virtual Reality Simulation, Training, Military Medical Robotics, Animatronics Visualization Communication

12. Multimedia System and Networking Lab @ UTD Slide- 12 University of Texas at Dallas B. Prabhakaran Computer Animation Kinematics Physics (a.k.a. dynamics, simulation, mechanics) Character animation Artificial intelligence Motion capture / data driven animation

13. Multimedia System and Networking Lab @ UTD Slide- 13 University of Texas at Dallas B. Prabhakaran Animation Process while (not finished) { DrawEverything(); MoveEverything(); } Interactive vs. Non-Interactive Real Time vs. Non-Real Time

14. Multimedia System and Networking Lab @ UTD Slide- 14 University of Texas at Dallas B. Prabhakaran An Example

15. Multimedia System and Networking Lab @ UTD Slide- 15 University of Texas at Dallas B. Prabhakaran The process involves… Motion Capture (Data Acquisition)

16. Multimedia System and Networking Lab @ UTD Slide- 16 University of Texas at Dallas B. Prabhakaran Human Motion Capture

17. Multimedia System and Networking Lab @ UTD Slide- 17 University of Texas at Dallas B. Prabhakaran UTD’s Motion Capture Facility…

18. Multimedia System and Networking Lab @ UTD Slide- 18 University of Texas at Dallas B. Prabhakaran Captured 3D Motion: E.g., 1

19. Multimedia System and Networking Lab @ UTD Slide- 19 University of Texas at Dallas B. Prabhakaran Captured 3D Motion: E.g., 2

20. Multimedia System and Networking Lab @ UTD Slide- 20 University of Texas at Dallas B. Prabhakaran Motion Capture Matrix Frame pelvis 1-4.62953-36.2313176.458590.269166.422797.569 2-4.65407-36.2417176.453590.039166.612797.706 3▪▪▪▪▪▪ Pelvis Joint Data:

21. Multimedia System and Networking Lab @ UTD Slide- 21 University of Texas at Dallas B. Prabhakaran Applying Motion Data to 3d Model

22. Multimedia System and Networking Lab @ UTD Slide- 22 University of Texas at Dallas B. Prabhakaran Animated 3D Model

23. Multimedia System and Networking Lab @ UTD Slide- 23 University of Texas at Dallas B. Prabhakaran Animation: Applications & Possibilities Using an Expert to Train Animation Toolkit –Content Based Retrieval of 3D Models & Motions Networked 3D Games –Streaming 3D Models and Motions Copyright / Content Protection Collision Detection

24. Multimedia System and Networking Lab @ UTD Slide- 24 University of Texas at Dallas B. Prabhakaran Application: Improve Your Game !

25. Multimedia System and Networking Lab @ UTD Slide- 25 University of Texas at Dallas B. Prabhakaran 3D novice motion & 2D expert motion We can get novice pitching data using motion capture system There are bunch of videos include expert’s pitching motion 2D video data has expert’s stylistic actions 3D motion data compensate for the incompleteness of 2D data

26. Multimedia System and Networking Lab @ UTD Slide- 26 University of Texas at Dallas B. Prabhakaran 2D Motion Analysis position time Motion analysis by tracking the object (e.g. right hands)

27. Multimedia System and Networking Lab @ UTD Slide- 27 University of Texas at Dallas B. Prabhakaran Constraint of 2D motion Analysis 3D motion capture data 2D video motion analysis data We need to compare the dissimilarity between 2D & 3D data However, 2D data from a single camera doesn’t have enough information for comparing with 3D motion capture data

28. Multimedia System and Networking Lab @ UTD Slide- 28 University of Texas at Dallas B. Prabhakaran Reconstruction 3D from 2D using HMM Calculate most probable style-path given 2D observations Argmax P(Q|O1O2…OT) Red: 3D novice motion data Blue: reconstructed 3D motion data

29. Multimedia System and Networking Lab @ UTD Slide- 29 University of Texas at Dallas B. Prabhakaran Resynthesis Following to the reconstructed 3D expert style. Red: 3D novice motion data Blue: reconstructed 3D motion data

30. Multimedia System and Networking Lab @ UTD Slide- 30 University of Texas at Dallas B. Prabhakaran Another Fun: 3D Tennis Game Realistic Tennis game – Topspin (EA Sports)

31. Multimedia System and Networking Lab @ UTD Slide- 31 University of Texas at Dallas B. Prabhakaran Application of learning expert style motion to 3D Tennis Games Tennis novice can learn by comparing style with realistic professional player Motion capture system can capture novice’s naïve actions (serve, swing, volley..) We can build realistic professional expert’s actions by motion resynthesis (3D motion reconstruction from 2D video data)

32. Multimedia System and Networking Lab @ UTD Slide- 32 University of Texas at Dallas B. Prabhakaran Parallel FSM (Finite State Machine) Motion capture data is not high-level semantic data (sequences, not segmented data) To identify “high-level action”, we prepare neural network and Parallel FSM Parallel FSM is needed since human actions happened not in a separate way

33. Multimedia System and Networking Lab @ UTD Slide- 33 University of Texas at Dallas B. Prabhakaran Behavior Modeling: Neural Network & Parallel FSM Sensor layer : two input nodes which notice the object’s movement & boolean value of range respectively. Control layer : works as a hidden layer Stand, Straight and Grab nodes (output nodes) also initial states of each FSMs.

34. Multimedia System and Networking Lab @ UTD Slide- 34 University of Texas at Dallas B. Prabhakaran High-level behavior recognition using Motor-graph To interpret low-level actions to high-level behaviors Motor-graph is designed with states of FSMs Nodes : each state, edges: state transitions (c) subsumed by (b) sub-graph

35. Multimedia System and Networking Lab @ UTD Slide- 35 University of Texas at Dallas B. Prabhakaran Translate into “serve” action by Parallel FSM & Motor Graph Locomotion FSM Head FSM Arm Hands FSM A0 H0 L0A1 A2 L1L2 A4 A5 H1 H2 H3 Neural Network sensors the participant’s action and hand it to FSMs Each FSM recognize the state-transition and draw it to motor-graph This action motor-graph is subsumed by “serve” minimum motor graph  translate this action as “serve” !!

36. Multimedia System and Networking Lab @ UTD Slide- 36 University of Texas at Dallas B. Prabhakaran System Architecture: Analysis novice’s style & feedback expert-like action Behavior Translation Neural Network & FSM Motor Graph Hands style Head styleLocomotion style Style Analysis Matched expert’s serve Showing a developed serve with user’s style Novice’s behavior

37. Multimedia System and Networking Lab @ UTD Slide- 37 University of Texas at Dallas B. Prabhakaran Animation Toolkit Animation Authoring Through Reuse: –Motion mapping –Inverse kinematics Example: –GET walking FROM Andy –GET waving FROM Andy –JOIN Andy.walking WITH Andy.waving

38. Multimedia System and Networking Lab @ UTD Slide- 38 University of Texas at Dallas B. Prabhakaran Animation Authoring Toolkit

39. Multimedia System and Networking Lab @ UTD Slide- 39 University of Texas at Dallas B. Prabhakaran Animation Query Handling Partial Fuzzy Query Resolution: –Primary attribute centric query resolution insert an animation sequence where Mickey Mouse is walking slowly in a park with a fountain or a statue in the background –Heuristics for retrieving top k objects Maximal grade based approach Maximal attributes based approach Threshold algorithm –Decent precision and recall shown. “Partial Fuzzy Query Resolution for Animation Authoring” (Phanivas Kotharu, MS Thesis, UTD).

40. Multimedia System and Networking Lab @ UTD Slide- 40 University of Texas at Dallas B. Prabhakaran Network CaptureIndexing ……….. Index Tree Metadata based Query Query by Example Compression Query / Data Processor Deliverable Data Animation Toolkit

41. Multimedia System and Networking Lab @ UTD Slide- 41 University of Texas at Dallas B. Prabhakaran Shape Analysis of 3d models Applications -Categorization of shapes -Indexing techniques of 3d models -Querying techniques for 3d model database Ultimate goal: Content based 3d model search

42. Multimedia System and Networking Lab @ UTD Slide- 42 University of Texas at Dallas B. Prabhakaran Streaming 3D Games Over the Internet 3D Streaming Server Rendering Client Network

43. Multimedia System and Networking Lab @ UTD Slide- 43 University of Texas at Dallas B. Prabhakaran 3D Model Streaming Advantages: –1 Base Mesh + M Refinements = Original Mesh –Bandwidth Friendly Drawbacks: –Intolerant to Transmission errors –Not friendly for Real Time 3D Streaming Base Mesh Faces: 4281 Vertices: 2249 Size:131KB Mesh after 5 Batches Faces: 23675 Vertices: 11946 Size: 748KB Original Mesh Faces: 69451 Vertices: 35947 Size: 3MB 1. 2. 3.

44. Multimedia System and Networking Lab @ UTD Slide- 44 University of Texas at Dallas B. Prabhakaran Content Protection of 3D models and MoCap Data 3D models and MoCap Data - Commercial value (“money”) - Requires lot of human effort Tampering and piracy of data: –loss of information, with ultimate loss of time and money. –Faulty training & education How do we do content protection to avoid piracy and tampering ?

45. Multimedia System and Networking Lab @ UTD Slide- 45 University of Texas at Dallas B. Prabhakaran Tamper Proofing Game Data Secure data used for driving the game (different from outcome data) Tamper proofing –Detect (and possibly correct) attacks on data Water marking (more to do with copyrighting) Focus both on 3D models, motion, apart from other data

46. Multimedia System and Networking Lab @ UTD Slide- 46 University of Texas at Dallas B. Prabhakaran Collision Detection  Authoring operations may lead to unintentional collisions.  Collision detection: alert authors on possible collision detection and suggest possibilities for avoiding them.  Previously used approaches for Collision Detection can be classified into 3 categories.  Cell Based  Tree Based  Bounding Object Based

47. Multimedia System and Networking Lab @ UTD Slide- 47 University of Texas at Dallas B. Prabhakaran Cell-based Approach  D ivide the entire search space into a number of cells and a collision possibility is triggered if two objects come in one cell.  Disadvantages: High memory usage. Tough to correctly determine the size of each cell. Too small a cell: objects occupying many cells and hence more collision tests. Too big a cell: unnecessary tests being carried out.

48. Multimedia System and Networking Lab @ UTD Slide- 48 University of Texas at Dallas B. Prabhakaran The motion of Object A causes a rippling effect on Object C after colliding with Object B. The possibility of Collision of Object C can be detected when the bounding sphere of Object A encompasses C. This helps in Early detection of the Collision.

49. Multimedia System and Networking Lab @ UTD Slide- 49 University of Texas at Dallas B. Prabhakaran Course Outline 1.Skeletons 2.Quaternions 3.Skinning 4.Facial Animation 5.Advanced Skinning 6.Channels & Keyframes 7.Animation Blending 8.Inverse Kinematics 9.Locomotion 10.Particle Systems 11.Cloth Simulation 12.Collision Detection 13.Rigid Body Physics