1. © 2001-2002 Sony Computer Science Laboratories, Inc. Jun Rekimoto Interaction Laboratory SONY Computer Science Laboratories, Inc. SmartSkin ： An Infrastructure for Freehand Manipulation on Interactive Surfaces

2. © 2001-2002 Sony Computer Science Laboratories, Inc. Background Various types of “realworld” surfaces become interactive media –Ex: DigitalDesk [Wellner] –Support collaborative work –Fusion of virtual & real Camera-based input technologies –Sensitive to lighting conditions, occlusion –Camera cannot be integrated in the surface Possibility of other principle ？

3. © 2001-2002 Sony Computer Science Laboratories, Inc.

4. Lev Theremin 1896-1993

5. © 2001-2002 Sony Computer Science Laboratories, Inc. Lev Theremin 1896-1993

6. © 2001-2002 Sony Computer Science Laboratories, Inc. SmartSkin: sensing principle transmitter receiver transmitter receiver 2πfCV

7. © 2001-2002 Sony Computer Science Laboratories, Inc. SmartSkin: sensing principle transmitter receiver Conductive & Grounded Object (Human Body) transmitterreceiver

8. © 2001-2002 Sony Computer Science Laboratories, Inc. wave signal To host PC low-pass filter Analog-Digital converter receiver reference signal amp reference signalanalog switch signal from electrode to ADC

9. © 2001-2002 Sony Computer Science Laboratories, Inc. Prototype I: Interactive Table An wooden table covered by 8×9 electrodes. 30fps

10. © 2001-2002 Sony Computer Science Laboratories, Inc. Prototype I: Interactive Table An wooden table covered by 8×9 electrodes. 30fps

11. © 2001-2002 Sony Computer Science Laboratories, Inc.

13. Multiuser, Multihand interactions

14. © 2001-2002 Sony Computer Science Laboratories, Inc. Distance measurement

15. © 2001-2002 Sony Computer Science Laboratories, Inc. Input by hand(arm) shape

16. © 2001-2002 Sony Computer Science Laboratories, Inc. Peak position estimation a potential field created by bicubic interpolation peak positionsensor values

17. © 2001-2002 Sony Computer Science Laboratories, Inc. Potential Driven Movement

18. © 2001-2002 Sony Computer Science Laboratories, Inc. PrototypeII: GesturePad 24 (transmitters) × 32 (receivers) (1cm pitch)

19. © 2001-2002 Sony Computer Science Laboratories, Inc.

21. Multiple finger detection Various image processing algorithm can be applied. Connected region labeling Pattern matching Object tracking Etc.

22. © 2001-2002 Sony Computer Science Laboratories, Inc.

23. Interactions by Multi-fingers

24. © 2001-2002 Sony Computer Science Laboratories, Inc. Interaction by physical objects electrodes As a platform of tangible interfaces.

25. © 2001-2002 Sony Computer Science Laboratories, Inc. SmartSkin Capacitive sensing (c.f., Thermin) Mesh-shaped electrodes Multiple object tracking 2.5D (2D position, distance) ＋ shape Fingertip detection by interpolating Can be embedded in various objects Low-cost, scalable

26. © 2001-2002 Sony Computer Science Laboratories, Inc. Discussions ： analogy between biological “Skin” and artificial surfaces form Sensors on the surface Input （ outside → skin ） –Touch, pressure, temperature Output （ skin → outside ） –image (e.g., Chameleon) –Tactile communication Artificial surface -> Skin

27. © 2001-2002 Sony Computer Science Laboratories, Inc. Future Directions Interaction techniques featuring multi hands/fingers, shapes Non planar surface Integration with a flat-panel display Data communication –Ex: table becomes a communication medium Special Thanks to … –Takaaki Ishizawa 、 Asako Toda （ Keio University ） –Kentaro Fukuchi (Tokyo Institute of Technology)

28. © 2001-2002 Sony Computer Science Laboratories, Inc. Smart Keyboard Two hand input → Text Input Single Hand → Command

29. © 2001-2002 Sony Computer Science Laboratories, Inc. Interactive Clothing transmitter receiver body shield layer fabric

30. © 2001-2002 Sony Computer Science Laboratories, Inc.

32. DiamondTouch [MERL2001] Identifies individual users Requires external electrodes (chairs)

33. © 2001-2002 Sony Computer Science Laboratories, Inc. Near-Field data transmission

34. © 2001-2002 Sony Computer Science Laboratories, Inc. Hand tracking without camera

35. © 2001-2002 Sony Computer Science Laboratories, Inc. 動作原理（ 2 ） Transmitter (AVR) I-V amp A/D Converter LPF 100-500KHz (5Vp-p) 1-10μA analog switch

36. © 2001-2002 Sony Computer Science Laboratories, Inc. Wireless.11, BT etc. ID

37. © 2001-2002 Sony Computer Science Laboratories, Inc. 物体を介した操作 電極

38. © 2001-2002 Sony Computer Science Laboratories, Inc. Application Domains Where to install? –Furniture, Floor, Wall –Wearables -- Clothing, Bag, Shoes –Tablet, Keyboard, Mouse, Gamepad –Mobile Devices –Robot How to use –as a primary input device –to augment existing interaction

39. © 2001-2002 Sony Computer Science Laboratories, Inc. Multi-Hand/Finger Input

40. © 2001-2002 Sony Computer Science Laboratories, Inc. 複数の指による入力

41. © 2001-2002 Sony Computer Science Laboratories, Inc. 距離の判定

42. © 2001-2002 Sony Computer Science Laboratories, Inc.

43. The Concept Beyond The GUI dogma (Mouse+Display) –Inspired by analogies between biological skin & artificial surfaces As An Input Device –Touch, Proximity, Digital Data (Electrostatic/magnetic coupling), energy As An Output Device –Pixel Spatial Pattern, Pixel Temporal Pattern (beacon), Digital Data, energy, tactile, sound? As A Communication Device –Information, Motion, Gesture, Emotion, Intimacy

44. © 2001-2002 Sony Computer Science Laboratories, Inc. なぜ Skin? 生物の「皮膚」の機能 – 保護、形態保持 – 入力（触覚） – 出力（表示）