1. 2009 International Symposium on Ubiquitous Computing System at Beijing, China Hiroaki Kimura, Sota Matsuzawa, and Tatsuo Nakajima Department of Computer Science Waseda University, Japan

2. UCS 2009 at Beijing, China 08/24/2009Outline 1. Introduction 2. Approach 3. Experiment 4. Discussion 5. Conclusion

3. UCS 2009 at Beijing, China 08/24/2009  Interaction for Ubiquitous Computing  Support different kinds of interaction techniques.  Not a pair of mouse and keyboard.  Provide high-quality user experiences.  Ideal interaction  Intuitive  High-learnability  Can be used by everyoneIntroduction The interaction using Gesture Recognition is one of the effectual choice.

4. UCS 2009 at Beijing, China 08/24/2009Introduction Nintendo Wii Toshiba Qosmio G50

5. UCS 2009 at Beijing, China 08/24/2009 Introduction - Issue  An issue of designing gestural interface Task ： play video Shake left to right ？ Draw a triangle ？ Clap hands ？ … What kind of gesture is the most suited? For example... No guideline, UI toolkit, reference point

6. UCS 2009 at Beijing, China 08/24/2009  Offer information for designers and developers. 1. Kind of task can be operated with gesture? 2. Type of gesture appropriated for a particular task?  To achieve our goal  Observe subjects dealing with experimental applications.  Extract some valuable elements. Introduction - Our goal

7. UCS 2009 at Beijing, China 08/24/2009Approach  We developed an experimental application.  Using a 3-axis accelerometer  Defining gestures by example  Setting corresponding gesture to each task

8. UCS 2009 at Beijing, China 08/24/2009Approach Circle...? Triangle!!! Hmm... DefineUsere-Define Subjects can re-define gestures as many times as needed.

9. UCS 2009 at Beijing, China 08/24/2009Approach  Transition of subjects’ behavior Before (at the beginning) After (at the end)

10. UCS 2009 at Beijing, China 08/24/2009Approach Intuitive High- learnability  It must have some elements which subjects can imagine the gesture from corresponding task. The common defined gesture is suited for the task can be used for one of the primitive gestures.

11. UCS 2009 at Beijing, China 08/24/2009Experiment  Target Applications  Questionnaire results ⇒ Movie Player / Picture Viewer  Analyzed these applications tasks.  Movie Player ： play, stop, pause, fast forward, rewind, volume up, volume down, full screen, download  Picture Viewer ： zoom in, zoom out, rotate, send (mail)  general tasks ： apply (OK), up, down, left, right 19 Tasks

12. UCS 2009 at Beijing, China 08/24/2009Experiment  Implementation  Use 3-axis accelerometer for recognition.  Assume on mobile phone.  We adopted Sun SPOT (Sun Microsystems, Inc.) as sensor device.

13. UCS 2009 at Beijing, China 08/24/2009Experiment  Sensor ： Sun SPOT  Data Manager ： Java  Presentation ： Adobe AIR

14. UCS 2009 at Beijing, China 08/24/2009Experiment  Screenshots

15. UCS 2009 at Beijing, China 08/24/2009  Subjects and procedure  8 university students, 4 days  Each subject has to... 1. use the applications at least 15 minutes per day. 2. redefine hard to use gesture. 3. keep a log of shapes of redefined gestures.Experiment

16. UCS 2009 at Beijing, China 08/24/2009Experiment  Questionnaire to each subject on the web. (scale of 1-7)  Q1: Could you remember gestures which you defined corresponding to each task?  Q2: Do you think gestural interface is an intuitive way for using these applications?  Q3: Is gesture re-definition by example system necessary?  Q4: How comfortable are the applications?  Q5: Please tell us possible situation using with gestures

17. UCS 2009 at Beijing, China 08/24/2009Discussion  Found common gestures in...  play, fast forward, rewind, volume up, volume down  zoom in, zoom out, rotate  ok, up, down, left, right  Not found common gestures in...  stop, pause, full screen, download  print, send mail

18. UCS 2009 at Beijing, China 08/24/2009Discussion  Fast Forward (movie player) Shake right twice ：５ Imitation of icons used in audio visual equipments.

19. UCS 2009 at Beijing, China 08/24/2009Discussion  PLAY (movie player) Circle ：４ Shake right ：３ Simple and short gesture for frequently- used tasks

20. UCS 2009 at Beijing, China 08/24/2009Discussion  DOWNLOAD (movie player) No commonality Can not feel intuitive to control this task with gesture Need to improve design

21. UCS 2009 at Beijing, China 08/24/2009Discussion  Recognizing gestures  ( 1 ) simple, ( 2 ) short, ( 3 ) uncompetitive  Show the recognition rate to users.  Learning cost  Users can remember their defined gestures easily.  However difficult to remember infrequently tasks.  Possible object/area using with gestures  Apply to 3D user interfaces  As temporal interaction in public space  User identification e.g.

22. UCS 2009 at Beijing, China 08/24/2009Conclusion  Observe how performed two casual applications using with gestures.  Questionnaire to subjects for acquiring qualitative data.  Common gestures corresponding each tasks, and extracted some valuable points for gestural interface.

23. UCS 2009 at Beijing, China 08/24/2009 Thank you very much. Distributed and Ubiquitous Computing Lab. http://www.dcl.info.waseda.ac.jp