1. PortableVision-based HCI A Hand Mouse System on Portable Devices 連矩鋒 (Burt C.F. Lien) Computer Science and Information Engineering Department National Taiwan University

2. Problems A Portable Vision-based HCI –Hand mouse operating on a projected interface –Real-time detection of user hand motion from a user PDA/SmartPhone’s video camera (target platform) A part of “mTeller” project –Natural storytelling support system

3. Why important Vision-based HCI is a more instinct way to manipulate data Mobility –Steerable interface everywhere

4. Related Works I A Portable System for Anywhere Interactions –Sukaviriya et al., IBM Research Real-time hand tracking using a set of cooperative classifiers based on Haar-like features –Barczak1 et al., Institute of Information & Mathematical Sciences Massey University

5. Everywhere Display (IBM) Figure 1: Interactive store application

6. Related Works II Real-Time Hand-Arm Motion Analysis using a single Video Camera – Hienz et al. MMX-Accelerated Real-Time Hand Tracking System –Liu et al. (2001)

7. Related Works III Rapid Object Detection Using a Boosted Cascade of Simple Features. –Viola, P., & Jones, M. (2001). Robust real-time object detection. –Viola, P., & Jones, M. Robust real-time face detection –P. Viola and M. Jones. Adaboost-based real-time pedestrian detection –P. Viola, M. Jones, and D. Snow.

8. Target Devices

9. System Configuration Hand motion capture and interpretation Wireless projector data transmission Interactive Interface

10. What is new A portable vision-based HCI on handheld device Design for general data manipulation

11. System Implementation Platform (prototype) –“Laptop” + “Low Cost Camera (USB)” Software tools –“MS VC++” + “Intel OpenCV library”

12. Technical Challenges and approaches Real-time hand motion tracking under an intense lighting condition (projector light) –Adaboost Hand position vs. Corresponding programs on host –Comparison between PDA’s screen and the projected screen(??) An efficient algorithm to run detection system on handheld devices (limited computation) –Lower framing rate (2fps?) + fewer classifier Noise filter –A sliding window or a simple filter to filter misbehaving hand motion

13. Expect Result A generic vision-based HCI software program running on a laptop (1st phase) A user can manipulate the laptop directly on the projected image –Hand mouse (one-click function only)

14. The End