1. HCI/ComS 575X: Computational Perception
Instructor: Alexander Stoytchev

2. Brief OpenCV Tutorial

3. Motion Energy & Motion History
HCI/ComS 575X: Computational Perception
Iowa State University
Copyright © Alexander Stoytchev

4. F. Bobick and J.W. Davis ``An appearance-based representation of action''. In Proceedings of IEEE International Conference on Pattern Recognition 1996, August 1996, pp

5. Davis, J. and A. Bobick ``The Representation and Recognition of Action Using Temporal Templates'', In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 1997, pp

6. Readings for Next Time (Face Recognition: Eigenfaces)
M. Turk and A. Pentland (1991). ``Eigenfaces for recognition''. Journal of Cognitive Neuroscience, 3(1). (Class Web)
Dana H. Ballard (1999). ``An Introduction to Natural Computation (Complex Adaptive Systems)'', Chapter 4: Data, pp 70-94, MIT Press. (WebCT)

7. The Authors
Aaron Bobick
James Davis

8. What is this?

9. What action is being performed?

10. Motion Energy Image (MEI)[

11. Average MEI for various viewing angles

12. Motion History Image (MHI)[

13. Definitions Image Sequence Binary Images indicating regions of motion
Binary Motion Energy Image

14. Motion Energy

15. Motion History
The result: more recently moving pixels appear brighter

16. [

17. MHI pyramid [

18. Motion templates for finishing LEFT-ARM-RAISE and FAN-UP-ARMS.[

20. Aerobics Dataset

21. Results for 30o off center camera location

22. OpenCV Demo
If you want to try this at home just compile and run the motempl.c file in the ../samples/c directory

23. Applications

24. Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y
Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, and A. Wilson (1999) ``The Kidsroom: A Perceptually-Based Interactive and Immersive Story Environment" Presence: Teleoperators and Virtual Environments, Vol. 8, No. 4, 1999, pp

25. The Kid’s Room
[Bobick et al. 1996]

26. The Kid’s Room [

27. The Kid’s Room [

28. The Kid’s Room [

29. Monsters [

30. Scavenger Hunt [

31. Scavenger Hunt [

32. The Blue Monster [

33. Spin in place [

34. 3 Kids Spin on a Rug [

35. The River Vorls [

36. The Technology [

37. Motion History Templates
Making a ‘Y’
Flapping
Spinning [

38. Detecting the Bed [

39. Man Overboard Detector[

40. Movies The quality of the movies is not very good
You can download them from:
demos/kidsroom/kidsroom.html

41. J. Davis and A. Bobick ``Virtual PAT: A Virtual Personal Aerobics Trainer'', Workshop on Perceptual User Interfaces, November 1998, pp

42. Interactive Virtual Aerobics Trainer[

43. Interactive Virtual Aerobics Trainer[

44. Interactive Virtual Aerobics Trainer[

45. Movies

46. The Personal Pet Project
Pepe
Joint work with
Rawesak Tanawongsuwan

47. Tanawongsuwan, R. , Stoytchev, A. , and Essa, I
Tanawongsuwan, R., Stoytchev, A., and Essa, I., "Robust Tracking of People by a Mobile Robotic Agent", Technical Report GIT-GVU

48. Project Goals build an intelligent, adaptive, user-friendly agent
build an agent that has a personality
make the interaction between the user and the agent as natural as possible

49. Face and Hands Segmentation
Gesture Recognition
Pointing gesture
Motion Profile Calculation
Face and Hands Segmentation
Neural Network
Hidden Markov Models
Gesture Recognition

50. Motion History/Energy
Perform a frame-by-frame subtraction (image differencing) and accumulate the results over the history window.
Motion energy is a binary version of motion history
Technique from Bobick and Davis.

51. Gesture recognition

52. Gesture Recognition

53. Gesture recognition

54. Movie

55. THE END