1. Lecture 15-16 Pose Estimation – Gaussian Process Tae-Kyun Kim 1 EE4-62 MLCV

2. MS KINECT http://www.youtube.com/watch?v=p2qlHoxPioM http://www.youtube.com/watch?v=p2qlHoxPioM 2 The KINECT body pose estimation is achieved by randomised regression forest techniques. EE4-62 MLCV

3. We learn pose estimation as a regression problem, and Gaussian process as a cutting edge regression method. 3 EE4-62 MLCV We see it through the case study (slide credits to) : Semi-supervised Multi-valued Regression, Navaratnam, Fitzgibbon, Cipolla, ICCV07, where practical challenges addressed are 1)multi-valued regression and 2)sparsity of data.

4. 4 EE4-62 MLCV A mapping function is learnt from the input image I to the pose vector, which is taken as a continuous variable.

5. 5 EE4-62 MLCV Given an image (top left), e.g. a silhouette (top right) is obtained by background subtraction techniques (http://en.wikipedia.org/wiki/Background_subtraction).http://en.wikipedia.org/wiki/Background_subtraction The estimated 3d pose is shown at two camera angels (bottom left and right).

6. 6 EE4-62 MLCV We attempt to map 2D image space to 3D pose space. There is inherent ambiguity in pose estimation (as an example in the above).

7. 7 EE4-62 MLCV Eye tracking can be tackled as a regression problem, where the input is an image I and the output is a eye location.

8. 8 EE4-62 MLCV Typical image processing steps: Given an image, a silhouette is segmented. A shape descriptor is applied to the silhouette to yield a finite dimensional vector. (Belongie and Malik, Matching with Shape Contexts, 2000)

9. 9 EE4-62 MLCV The output is a vector of m joint angles.

10. 10 EE4-62 MLCV

11. 11 EE4-62 MLCV

12. 12 EE4-62 MLCV

13. 13 EE4-62 MLCV

14. 14 EE4-62 MLCV

15. Gaussian Process 15 EE4-62 MLCV

16. 16 EE462 MLCV i.i.d. (independent identical distributed) Review of Gaussian density estimation (lecture 1,2) We want to find the Gaussian parameters from the given data. The problem is to find the parameters by maximising the posterior probability

17. 17 EE462 MLCV Review of Gaussian density estimation (lecture 1,2) We do not have priors on the parameters and data, thus we maximise the (log) likelihood function instead. Maximum Likelihood (ML) vs Maximum A Posterior (MAP) solutions: P(X|Y) P(X|Y)P(Y) (e.g. Gaussian Process)

18. EE462 MLCV Review of polynomial curve fitting (lecture 1,2) We want to fit a polynomial curve to given data pairs (x,t). where The objective ftn to minimise is

19. Gaussian Processes 19 EE4-62 MLCV

20. 20 EE4-62 MLCV

21. 21 EE4-62 MLCV

22. 22 EE4-62 MLCV

23. Gaussian Processes for Regression 23

24. 24 EE4-62 MLCV

25. 25

26. EE4-62 MLCV 26

27. 27 EE4-62 MLCV

28. 28 EE4-62 MLCV

29. 29 EE4-62 MLCV

30. 30

31. 31 EE4-62 MLCV

32. Gaussian Process Matlab Toolbox http://www.lce.hut.fi/research/mm/gpstuff/i nstall.shtml (try demo_regression_robust.m, demo_regression1.m) http://www.lce.hut.fi/research/mm/gpstuff/i nstall.shtml 32

33. Learning Hyperparameters 33 EE4-62 MLCV

34. 34

35. Automatic Relevance Determination 35 EE4-62 MLCV

36. 36

37. 37

38. Back to the pose estimation problem 38

39. 39 EE4-62 MLCV

40. It’ll never work… – is multivalued – and live in high dimensions 40 EE4-62 MLCV

41. 41 EE4-62 MLCV

42. 42 EE4-62 MLCV

43. 43 EE4-62 MLCV

44. 44 EE4-62 MLCV

45. 45 EE4-62 MLCV

46. 46 EE4-62 MLCV

47. 47 EE4-62 MLCV GMM : Gaussian Mixture Model GPLVM : Gaussian Process Latent Variable Model

48. 48 EE4-62 MLCV

49. 49 EE4-62 MLCV

50. 50 EE4-62 MLCV

51. 51 EE4-62 MLCV

52. 52 EE4-62 MLCV

53. 53 EE4-62 MLCV

54. 54 EE4-62 MLCV

55. 55 EE4-62 MLCV

56. 56 EE4-62 MLCV

57. 57 EE4-62 MLCV

58. 58 EE4-62 MLCV

59. 59 EE4-62 MLCV

60. 60 EE4-62 MLCV

61. 61 EE4-62 MLCV t: a continuous latent variable

62. 62 EE4-62 MLCV

63. 63 EE4-62 MLCV

64. 64 EE4-62 MLCV

65. 65 EE4-62 MLCV

66. 66 EE4-62 MLCV

67. 67 EE4-62 MLCV

68. 68 EE4-62 MLCV

69. 69 EE4-62 MLCV

70. 70 EE4-62 MLCV

71. Real-Time Human Pose Recognition in Parts from Single Depth Images (J. Shotton et al, 2011) Key features – Depth image as input – Real-time by Random Forest, and Part-based 71 EE4-62 MLCV

72. Progressive Search Space Reduction for Human Pose Estimation (Ferrari et al, 2008) 72 EE4-62 MLCV