1. Tangible User Interfaces (TUI’s)

2. What are Tangible User Interfaces? Physical WorldDigital world TUI’s 2

3. GUI vs TUI (Ishii 2008) 3

4. Precursors E.g. Marble answering machine, Durrell Bishop, Royal College of Art, Interaction Design, 1992. http://vimeo.com/19930744http://vimeo.com/19930744 4

5. Early Work Fitzmaurice et al. 1995 – Graspable user interfaces http://www.youtube.com/watch?v=V-TGEe- Imro http://www.youtube.com/watch?v=V-TGEe- Imro Ishii et al. 1997 - Tangible bits 5

6. Recent work E.g. Lumino, Baudish et al, 2010 http://www.youtube.co m/watch?v=tyBbLqViX7g http://www.youtube.co m/watch?v=tyBbLqViX7g E.g. Portico, Avrahami et al, 2011 http://vimeo.com/29359 319 http://vimeo.com/29359 319 6

7. Benefits of TUI’s (Ishii 2008) 1.Double interaction loop - immediate tactile feedback 2.Persistency of tangibles 3.Coupled input/output space 4.Special vs generic purpose 5.Space-multiplexed vs time- multiplexed input Also fun + engaging! 7

8. Sense position & orientation on touch technology Add intelligent drawing support How can they be best combined with multi-touch surfaces for enjoyable and productive interaction? Can we connect these physical drawing tools to the digital space? 8

9. Our Approach Design – Tangible hardware Implementation – Recognizer – Drawing application Usability evaluation Capacitive Touch (CapTUI)Infrared Touch (TanGeo) – Ryan Tan & Bryan Chen – Rachel Blagojevic – Jacky Zhen 9

10. CapTUI Technology – Capacitive Small touch screens e.g. iPad, smart phones etc… Touch detection via electrical pulse from fingers/conductive material 10

11. Design: Tangible Hardware 11

12. Final Design 12

13. Implementation: Tangible Recognition Tangible ID – 3 point (min) unique patterns Valid patternsInvalid patterns 13

14. Implementation: Tangible Recognition Learning phase Recognition phase – Touch point detection – Match point distances to saved tangible ID’s – No way of knowing which part of the touch point is in contact (+/- error) 14

15. Implementation: Drawing Application Beautification Ink-to-edge snapping Corner snapping & Length visualization 15

16. Implementation: Drawing Application Visual drawing guides Tangible outline Angle visualization 16

17. First iteration: Video Second iteration: Demo 17

18. Evaluation First iteration: usability – Simple drawing tasks Second iteration: comparative study – Recognizable vs non recognizable drawing tools on screen 18

19. Usability Evaluation Can users construct simple drawings using the tangibles? Is the system usable? 10 participants 5 simple drawing tasks 19

20. Usability Evaluation: Results First exploration – Technology works – Is usable for simple drawings Tangible detection problems – Stability – Consistent circuit – Finger to tangible contact – Friction with screen – Comfortable drawing Tangible outline helpful – recognition indicator Drawing guides needed 20

21. Comparative Study Does CapTUI assist users to easily draw precise geometric drawings Recognizable vs non recognizable drawing tools on screen 12 Participants 21

22. Comparative Study: Results CapTUI rated significantly higher than Paint overall Visual guides helpful for precise drawing – significantly lower average angle error. Participants enjoyed using CapTUI significantly more Participants believed that CapTUI produces significantly more tidy drawings than Paint. Making fine grained movements with the tangibles difficult Tangible design still needs work – consistent detection – accurate positioning 22

23. Tangeo Technology – Infrared Table tops e.g. Microsoft Surface 2.0 PixelSense - Touch detection via infrared reflection for each pixel Image processing on detected pixels Identifies finger/blobs/tags 23

24. Design: Tangible Hardware 24

25. Final Design 25

26. Implementation: Tangible Recognition Learning phase Recognition phase – Detection via custom tags – Use tag location to get tangible outline – Use thresholds for blob sizes 26

27. Implementation: Drawing Application 27

28. Implementation: Drawing Application Visual Guides – Tangible outline – Angle visualisation – Length visualisation – Ink beautification corner snapping ink-to-edge snapping 28

29. Usability Evaluation Can users construct simple geometric drawings using Tangeo? Is the system usable? 2 phase cycle 8 participants 4 drawing tasks 29

30. Usability Evaluation: Results Enjoyable / easy to use the tangibles Good recognition Visual guides helpful and easy to understand Drawing accuracy – less positive perception Add stylus for drawing 30

31. References Marble answering machine - Crampton Smith, G. The Hand That Rocks the Cradle. I.D., May/June 1995, pp. 60-65. Fitzmaurice G. W., H. Ishii, and W. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), ACM Press/Addison- Wesley Publishing Co., New York, NY, USA, 442-449. Ullmer B. and H. Ishii. 1997. The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software and technology (UIST '97). ACM, New York, NY, USA, 223-232. Ishii H., B. Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States Ishii H., 2008. Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI '08). ACM, New York, NY, USA, xv-xxv. Baudisch P., T. Becker, and F. Rudeck. 2010. Lumino: tangible building blocks based on glass fiber bundles. In ACM SIGGRAPH 2010 Emerging Technologies (SIGGRAPH '10). ACM, New York, NY, USA, Article 16, 1 pages. Avrahami D., J. Wobbrock, and S. Izadi. 2011. Portico: tangible interaction on and around a tablet. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, 347-356. Blagojevic R., X. Chen, R. Tan, R. Sheehan, and B. Plimmer. 2012. Using tangible drawing tools on a capacitive multi-touch display. In Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (BCS-HCI '12). British Computer Society, Swinton, UK, UK, 315-320. Zhen, J. S., R. Blagojevic and B. Plimmer (2013). Tangeo: Geometric Drawing with Tangibles on an Interactive Table- Top. CHI 2013. Paris France, ACM. WIP: in press. Shaer O. and E. Hornecker (2010) "Tangible User Interfaces: Past, Present and Future Directions", Foundations and Trends® in Human-Computer Interaction: Vol. 3: No 1-2, pp 1-137. 31