1. L1 Sketch Tools Advanced HCI Beryl Plimmer

2. Agenda What’s the difference between a keyboard and a pencil?

3. While creating a visual design the design talks back to you – called ‘backtalk’ Our short term memory is too limited for us to hold complex design in our heads We tend to iterate around the design and design ideas Sometimes starting again This is a very rapid process!

4. The physical interaction Direct interaction Visual feedback at input position Kinaesthetic Proprioception

5. Cognitive process Computer tool –Choose type of control –Tap, drag –Align –Name –Borders –… Pen –Draw Imagine you are creating a user interface design. You need to add another set of controls

6. The visualization produced Messy makes us think more –Usually this is a good thing! –But not acceptable as finished visualization

7. What is the problem with computer design tools? Computer based tools are too constraining –Attention should be on the big picture –But it gets diverted to the detail Designers (nearly) always user a pen and paper/whiteboard first. This is independent of the discipline!

8. So why use computers at all? Editing Archiving Sharing Translation between tools and formats ‘save time’

9. Proof We know this for sure for designing things like User interfaces We don’t know if it is true for more abstract models such as uml models What about planning an –Essay –Algorithm What do you do?

10. Why is this so? The psychologist don’t really know. There seems to be a different brain process going on when we are using a pen There are things about the interaction feedback loop I also wondered whether the ‘natives’ - i.e. your generation would overcome this –It seems not

11. Experiment Comparing formal and informal presentations These two designs are the same (bad) solutions to a problem –Sketch 8.6 changes –VB Form6.5 changes

12. So what happens if you partly tidy? Louise Yeung, 2007, ‘Exploring Beautification and the effects of designs’ level of formality on design performance during the early stages of the design process, MSc Thesis, UoA

13. What happened? Number of changesEnjoyment

14. But Computers are much better for –Editing –Archiving –Versioning –Remote collaboration And Interpreted sketches can be –Animated/Executed –Beautified –Translated

15. Diagramming Tools Support early design –Paper like Quick No decisions No rules InkKit

16. InkKit Why a toolkit? There are quite a number of sketch tools around –including Freeform There is a lot of commonality Progress at exploring the more interesting issues is slow because of the effort required to get to first base. Cross-domain studies better if the variables are more controlled

17. Framework Sketch spaces –Generic (paper like) –Flexible Recognition –Smart –Trainable –Overridable Extensible –For domain specifics

18. Framework - Sketch pages One big page or lots of little ones? Usual computer editing (copy, paste, undo redo)! Flexible arrangement Connections between parts of pages and pages Denim Freeform

19. Implementation - environment Tablet PC –Nice hardware interface (though rather small) –OS support for inking and character recognition C#

20. Implementation – User Interface PagesPortfolio

21. Implementation – recognition Divider Tablet OS’s Recognition Engine Basic Shape Recognition Component Recognition Component examples Domain specific Domain independent Components Joiner Strokes

22. Implementation – Extensibility Interpreter ~ 200 – 300 lines –Basic info about diagram (has connectors) – defines all the components – names and containment (parent/child) –Domain specific rules – like fill list box Examples Translator ~ 300+ lines –Takes recognized sketch –Generates output data Other –execution

23. Does it work?

24. Organization Chart Contents Person "boss" is a superior of "Lackey" is a superior of "Minion" has no superiors Person "Lackey" has no inferiors is an inferior of "boss" Person "Minion" is a superior of "Worm" is a superior of "Molecule" is an inferior of "boss" Person "Worm" has no inferiors is an inferior of "Minion" Person "Molecule" has no inferiors is an inferior of "Minion" Person "Nobody" has no inferiors has no superiors

25. Student add-ins

26. Lines of Code DomainInterpreterToOutput Venn Diagram102Powerpoint293 UML Class Diagram 264Java830 UML Activity Diagram281Visio Hierarchical Visual Modeling 275HTML255 Music459Lilypond510

27. Multi-Domain http://www.cs.auckland.ac.nz/research/hci/digital_ink/sketch_tools/inkkit.shtml

28. SketchNode / SketchSet Very simple diagrams –Therefore few recognition errors Focus on user experience –Editing support –Keeping ‘hand drawn’

29. Intelligent editing support 29

30. Maintain a formal visualization SketchNode Ideas: Helen Purchase and Beryl Plimmer Programming: Hong Yul Yang Studies: various students

31. Euler Diagrams

32. Other sketch tools

33. What next? Recognition engines Annotation Tools