2. Interaction Models I Marti Hearst (UCB SIMS) SIMS 213, UI Design & Development March 9, 1999

3. Last Time: Heuristic Evaluation l A “discount” usability testing method l UI experts inspect an interface –prototype initially –later, the full system l Check the design against a list of design guidelines or heuristics l Present a list of problems to the UI designers and developers, ranked by severity l See homework assignment (Due Mar 18)

4. Outline l Windowing systems l Events overview l Event dispatching and handling l For more information, see Olsen 98, Developing User Interfaces

5. Windowing Systems l Provide infrastructure to support common UI-related services l Provide functionality for –Input/Output device handling –Which window gets which input/output l Window manager »create and organize windows »implement interaction in those windows

6. Adapted from slide by James Landay Windows l Top level windows known as root windows –provide an abstraction to separate applications –windowing system arbitrates resources among these l Each root window belongs to an app. –all descendant windows belong to same application –not used by OLE (ActiveX)

7. Adapted from slide by James Landay Networked Windowing Systems l XWindows designed to allow applications to run on remote machines l Uses client-server model –(X reverses standand usage of client/server terminology) X Server std system software Client app software Network User

8. Adapted from slide by James Landay XWindows l Note backwards terminology –User is on “server” not “client” l X Server –interprets X commands and can send events –determines which window receives events and forwards over network to proper client l X Client –software interface to X (Xlib) –assembles the output from Xlib routines into packets for transmission to server

9. Adapted from slide by James Landay Events l An event is something “interesting” that happens in the system –Mouse button goes down –Item is being dragged –Keyboard button was pressed

10. Adapted from slide by James Landay Window Events l User interacts with input device –action translated into software events –must be distributed to the appropriate window l Events contain information on –type –mouse position or character key –the window the event is directed to

11. Adapted from slide by James Landay Input Events l Mouse button events –depress/release mouse key –modifier (shift keys, etc.) –double click (X doesn’t have this  fakes it) l Mouse movement events –implement painting with mouse –mouse drag l Mouse enter and exit events –e.g. if you entered / exited a button region

12. Adapted from slide by James Landay Button Events Button mouse enter mouse exit (But using mouse move events within a button is unnecessary, due to the semantics of buttons.)

13. Adapted from slide by James Landay Events (cont.) l Keyboard events –translate raw “scan codes” into ASCII

14. Adapted from slide by James Landay Events (cont.) l Windowing events on window –creation / destruction –opening / closing –iconifying / deiconifying –selection / deselection –resize –redraw » (window manager tells the application to redraw within a certain region; the application does the actual redrawing)

15. Adapted from slide by James Landay Widgets that can register specific kinds of events close box title bar folder scroll bar size control

16. Adapted from slide by James Landay Major Issues l How to decompose the UI into interactive objects (widgets)? l How to distribute inputs to the interactive objects? l Contrast –Sequential (standard) program flow –Event-driven program flow

17. Sequential Programs l Program takes control, prompts for input l Examples include –command-line prompts (DOS, UNIX) –LISP interpreter l The user waits on the program –Program tells user it’s ready for more input –User enters more input

18. Adapted from slide by James Landay Sequential Programs (cont.) l The user waits on the program –1. Program tells user it’s ready for more input –2. User enters more input –3. Program responds and goes back to 1. l But how to model the many actions a user can take? –For example, a word processor? –Need to do editing, inserting, etc.

19. Adapted from slide by James Landay Event-Driven Programming l Instead of the user waiting on program, have the program wait on the user l All communication from user to computer is done via “events” l An event is something “interesting” that happens in the system –Mouse button goes down –Item is being dragged –Keyboard button was pressed

20. Adapted from slide by James Landay Interactor Tree l Decompose interactive objects into a tree –based on screen geometry of objects –use nested rectangles l Used for dispatching events –Events are dispatched (sent) based on code associated with the widget –The code responds appropriately to the event l Variety of methods for dispatching events –Return to this later

21. Adapted from slide by James Landay Interactor Tree Display Screen “F:\cs160\Public” window Inner Window title bar horizontal scroll bar contents area “CDJukebox” folder “Home Ent…” folder … size control … “Web Newspaper” window …

22. Adapted from slide by James Landay Interactor Tree Display Screen Outer Win [white] 7 8 9 4 5 6 0 + - 1 2 3 = 93.54 ENT ?????

23. Adapted from slide by James Landay Interactor Tree Display Screen Outer Win [white] Result Win [tan] Result String Inner Win [green] Keypad [grey] - button + button 0 button = button 7 8 9 4 5 6 0 + - 1 2 3 = 93.54 ENT

24. Adapted from slide by James Landay Main Event Loop l Main event loop Initialization While (not time to quit) { Get next event E Dispatch event E } l The meat of the program is in the code that handles the “dispatch”

25. Adapted from slide by James Landay Event Queues l Input events are placed in a queue –Ensures events are processed in order l Main event loop removes them from the queue (get-next-event) & dispatches for processing Mouse move (22, 33) Mouse move (40, 30) Mouse down left (45, 34) Mouse up left (46, 35)

26. Adapted from slide by James Landay Event Queues (cont.) l Can use ignore unwanted events –e.g., ignore mouse moves in a forms-based program »just get enter/exit events –however, do want to track mouse moves in a drawing program

27. Adapted from slide by James Landay Event Dispatch Dispatch (event E) { switch (E.window) {... case FIVE-KEY: if (E.type == left-down){ cur = 5 + 10*cur; display (cur); last-op = NUMBER; }... 7 8 9 4 5 6 0 + - 1 2 3 = 0 ENT Hit the ‘5’ key

28. Adapted from slide by James Landay Event Dispatch Dispatch (event E) { switch (E.window) {... case TWO-KEY: if (E.type == left-down) { cur = 2 + 10*cur; display (cur); last-op = NUMBER; }... 7 8 9 4 5 6 0 + - 1 2 3 = 5 ENT Hit the ‘2’ key

29. Adapted from slide by James Landay Event Dispatch Dispatch (event E) { switch (E.window) {... case ENTER-KEY: if (E.type == left-down){ push (cur); cur = 0; last-op = COM; }... 7 8 9 4 5 6 0 + - 1 2 3 = 52 ENT Hit the ‘enter’ key

30. Adapted from slide by James Landay Event Dispatch Dispatch (event E) { switch (E.window) {... case SIX-KEY: if (E.type == left-down) { cur = 6 + 10*cur; display (cur); last-op = NUMBER; }... 7 8 9 4 5 6 0 + - 1 2 3 = 0 ENT Hit the ‘6’ key

31. Adapted from slide by James Landay Event Dispatch Dispatch (event E) { switch (E.window) {... case PLUS-WIN: if (E.type == left-down){ if (last-op == NUMBER) push (cur); result = pop() + pop(); push (result); display (result); cur = 0; last-op = COM; } 7 8 9 4 5 6 0 + - 1 2 3 = 6 ENT Hit the ‘+’ key

32. Adapted from slide by James Landay Dispatching Events l If user scrolls the text, the software must: –direct the mouse events to the scroll bar –update the scroll bar display during the drag –notify the text editing window it needs to scroll itself so that the text appears to have moved

33. Adapted from slide by James Landay Dispatching Events (cont.) l Algorithm selects the bottom-most, front- most region in the interactor tree l Called bottom-first event dispatching –scroll bar or contents over outerwin (bottom- most) –scroll bar over contents (front-most) –Advantage: »each window need only consider its own events »simple –disadvantage: »difficult to impose a high level of control »inflexible

34. Adapted from slide by James Landay Dispatching Events (cont.) l Top-down event dispatching –events passed to top-most, front-most window –it dispatches to one or more of its children

35. Adapted from slide by James Landay Event Focus l Where should keyboard events go? –mouse-based »attach mouse position to all key events and dispatch events in the same way as mouse events –click-to-type »send all keyboard events to last window where mouse-down occurred

36. Adapted from slide by James Landay Event Focus l Mouse focus –scrollbar example: retain the focus on the scrollbar widget until the mouse button is released –compensates for difficulty of keeping the mouse within the narrow scrollbar

37. Adapted from slide by James Landay Event Handling l Event tables (in the early days…) –indexed by event types (integer from 0 - 255) –hold pointers to functions that handle each event –one table per / window –lots of things to maintain when attached to a widget that you want to make reusable l Callbacks –each kind of widget defines a set of named callbacks which will be run when the widget receives an appropriate event

38. Adapted from slide by James Landay Callback Example l How do we notify text window to scroll when the scroll bar is moved? –create a vertical scroll bar widget –write a callback procedure which has code to notify text windows of their new position –register the callback as being the program to invoke when the scroll bar is moved –register the text window as the data for the callback »so the system knows which window to scroll

39. Summary l Windowing systems –software to manage and arrange windows »keeps track of current focus –software to support handling of user-created events »complex GUIS are built up of hierarchically nested windows l Events –associated with various types of input devices and actions –are handled one by one from a queue