1. INTERACTION MODELS Lecture 3

2. Agenda CLG – designers’model of UI Users’models Hierarchical Models Linguistic Models Physical Models Laws in HCI (pointing, choosing, learning…)

3. CLG Represents the designers’ conceptual model in interface implementation The main purpose –to allow the designer to build a conceptual model that is acceptable for users needs

4. CLG 3 components: Conceptual – abstract concepts in system organization Semantic – concepts representations in the system Task – task domain Communication – the command language Syntactic – command argument structure Interaction – dialogue structure Physical – the devices with which the user interacts Spatial layout – the arrangement of input-output devices Devices

5. CLG Each level must be seen as a top-down design sequence After creation of a conceptual model, a command language is constructed in order to implement the conceptual model, then a display layout is built to illustrate the command language 3 points of view: Linguistic – structural aspects of UI Psychological – the knowledge the user has about the system (mental model) Design – contribute to design specifications understanding

6. CLG HCI Conceptual Component Semantic LevelTask Level Communication Component Syntactic LevelInteraction Level Physical Component Physical LayoutDevices 05/06/2016 Proiectarea interfeţelor om-calculator

7. CLG Sequence of four levels, each a refinement of the preceding level Each level is a complete system description Mappings go between levels Linguistic model of interfaces 05/06/2016 Proiectarea interfeţelor om-calculator

8. CLG Task level Designer: analyze users’ needs in terms of tasks to be accomplished Produces task description: decomposition of user’s task domain into subtasks User view: how can I use the system to do X (e.g., edit a letter/ delete )?

9. CLG - example NEW_MAIL=(A TASK (* check for new UNREAD_MESSAGES, if any, read them) (DO (SEQ: (CHECK_FOR_NEW_MAIL) (READ_NEW_MAIL) ) UNREAD_MESSAGE = (AN ENTITY NAME = “ Unread_message” (*comments…) ) MESSAGE = (AN ENTITY REPRESENTS (AN UNREAD_MESSAGE) NAME= “message” TYPE= (ONE OF: OLD NEW) (*comments) )

10. CLG Semantic level Designer: Objects and operations needed by user to accomplish task described in task level Produces semantic description: conceptual model that defines functionality of system and maps between task and syntactic levels User view: how do I make the system do X (e.g., delete a sentence)?

11. CLG example SEM_M2 = (A SEMANTIC METHOD FOR CHECK_FOR_NEW_MAIL DO (SEQ: (START PINE_PROGRAM) (SHOW INBOX) (LOOK IN INBOX FOR (A MESSAGE TYPE= NEW)) )

12. CLG Syntactic level Designer: embeds conceptual model of system into a language structure so that the users can communicate with system. Designs dialogues and information displays. Produce syntactic description: maps between semantic and interaction levels User view: how do I mark the sentence? What is the delete command?

13. CLG example SYN_M2 = ( A SYNTACTIC METHOD FOR CHECK_FOR_NEW_MAIL DO (ENTER PINE/INBOX) )

14. CLG Interaction level Designer: describes display actions taken by the system and dialogue conventions. Defines exact input and output sequences for particular input and output devices. Produces interaction specification: maps to syntactic level User view: Which keys (or mouse buttons, etc...) should I push to mark the sentence and then delete it?

15. CLG Example IA_M2 = (AN INTERACTION METHOD FOR CHECK_FOR_NEW_MAIL DO(KEY:”PINE-I” RETURN) )

16. CLG User: Must express the task in computer’s language; physical actions Perceives physical signals from the computer and transforms them in symboic structures with signifiance Translates the systems answers according the delegated task

17. CLG Task level: Goal – to analyze users’ needs and to structure domain tasks such that computer can participate to task accomplishment Describes task structure delegated to the system The task must be “translated”” in operations that can be perfomed by the computer Semantic level: Describes the set of objects, attributes, and operations used in the communications between the user and the computer in order to accomplish a task; on the system side: data structures and procedures On the user side: conceptual entities and operations on them

18. CLG Syntactic level: Describes the conceptual entities and operations that can be used in a given context or system state and the way these can be accessed (commands and sintactical rules) Interaction level: Described the way command names and objects manipulated in the physical actions and interaction structure (includes typing rules and manipulation convention rules and system responses)

19. CLG how a CLG representation might be analyzed to predict or evaluate aspects of the system’s usability, such as performance times, memory load and learning? it carries many of the drawbacks of a strictly top-down design process and leaves little room for design iterations. CLG cannot be used to describe the relation between the tasks and the information on the screen provides a valid framework to model (competent) users’s knowledge – at least from the point of view of the system designer

20. Users Model in Design Users representations in interaction with an interface; create models of aspects related to understanding, knowledge, intentions and the processes needed in interaction with a system. Hierarchical models – task and goals hierarchies: GOMS, CCT Linguistic models – user-system grammar: BNF, TAG Physical models – human motor abilities:KLM Developed by psychologists + analystsi/ designers in order to understand users

21. Models classification Competence models – anticipates users legal behavioral sequences, without considering if they can be really performed (what is expected from users). Performance models - describe required behavioral sequences, required users knowledge and the way the knowledge is used in order to perform tasks

22. HIERARCHICAL MODELS

23. GOMS – Goals, Operators, Methods, Selection rules (Card, Moran, Newell) The GOMS Model is a human information processing model that predicts what skilled users will do in seemingly unpredictable situations. Most used and influential task analysis method (task based dialog description technique) Describes the steps in a dialog needed to accomplish a task using a specific interface Goal – users performance estimation based on system description, before being developed 05/06/2016 Proiectarea interfeţelor om-calculator

24. GOMS Goals – something that the person wants to accomplish. Can be high level (e.g. WRITE-PAPER) to low level (e.g. DELETE CHARACTER) Operators – basic perceptual, cognitive, or motor actions used to accomplish goals, or actions that the software allows user to make (e.g. PRESS-ENTER-KEY or CLICK-MOUSE) Methods – procedures (sequences) of subgoals and operators that can accomplish a goal Selection rules – personal rules users follow in deciding what method to use in a circumstance 05/06/2016 Proiectarea interfeţelor om-calculator

25. Operators 3 types: External – perception and motor actions Mental Predefined : RECALL, RETAIN, FORGET, RETRIEVE, DECIDE Defined by analysts: FIND_MENU_ITEM « Cut » Primitives – derived from hardware (ex: mouse) Used as a prediction base for users performence (each operator has a predefined execution duration) 05/06/2016 Proiectarea interfeţelor om-calculator

26. GOMS Analysis Model of mental processes where the user accomplishes goals by solving subgoals in divide et impera manner GOMS analysis consists in a high level goal which is decomposed in a set of unit tasks that can be further decomposed in base operators 05/06/2016 Proiectarea interfeţelor om-calculator

27. GOMS Analysis Example- Text selection Selection rule set for goal: select text If text-is word, then Accomplish goal:select word. If text-is arbitrary, then Accomplish goal:select arbitrary text. Return with goal accomplished. Method for goal: select word Step 1. Locate middle of word. (M) Step 2. Move cursor to middle of word. (P) Step 3. Verify that the correct word is located. (M) Step 4. Double click mouse button. (BB) Step 5. Verify that the correct text is selected. (M) Step 6. Return with goal accomplished. 05/06/2016 Proiectarea interfeţelor om-calculator

28. GOMS Analysis Example (cont) Method for goal: select arbitrary text Step 1. Locate beginning of text. (M) Step 2. Move cursor to beginning of text. (P) Step 3. Verify that the correct beginning is located. (M) Step 4. Press mouse button down. (B) Step 5. Locate end of text. (M) Step 6. Move cursor to end of text. (P) Step 7. Verify that correct text is marked. (M) Step 8. Release mouse button. (B) Step 9. Verify that the correct text is selected. (M) Step 10. Return with goal accomplished. 05/06/2016 Proiectarea interfeţelor om-calculator

29. GOMS Analysis Tha analysis of goal structure provides performance information: The goals stack depth provides information about the working memory load 90% correct predictions of the commands executed by users Execution time estimated with an error of 33% 05/06/2016 Proiectarea interfeţelor om-calculator

30. GOMS lacks Analysis granularity level Needs expert behavior, error is not considered 05/06/2016 Proiectarea interfeţelor om-calculator The method is good to describe the way EXPERTS perform a routine task

31. GOMS Limitations Task in question must be usefully analyzed in terms of the procedural (how to do it) knowledge. Represents only skilled behavior. Not useful for ill-defined problem solving, exploration, etc. Cognitive walkthrough is useful for exploratory behavior by novices. Need to start with a list of top-level tasks or user goals. List must be provided outside of GOMS.

32. CCT – Cognitive Complexity Theory implementation of GOMS models 2 parallel descriptions: Users goals - similar to goals hierarchies in GOMS, but using production rules like If condition then action, where: Condition – statement about users working memory content ; if it is true, action executes Action – consists of one or more elementary actions that might be changes in working memory or external actions (key pressing) Software system (device) – state-transition diagrams

33. CCT example – Delete a character (PRDELC1 IF (AND (TEST-GOAL delete character) (NOT (TEST-GOAL move cursor to %LINE %COL)) (NOT (TEST-CURSOR %LINE %COL))) THEN ( (ADD-GOAL move cursor to %LINE %COL)) ) (PRDELC2 - IF (AND (TEST-­GOAL delete character) (TEST­-CURSOR %LINE %COL)) THEN ( (DO­KEYSTROKE DELETE) (WAIT) (DELETE­GOAL delete character) (UNBIND %LINE %COL)) )

34. State-transitions Diagram state transition User action System action 1: display document 2: update cursor position 3: store the position of the mistyped character 4: show correct document (without mistyped character)

35. CCT Predictions about the execution time of a given task By analyzing what the user should do in order to operate on a software system, comparison between system is possible Production rules based representation allows the analysis of task-equipment mapping, showing the difference between user tasks hierarchy (how to do it) and transitions between system’s states (how it works)

36. LINGUISTIC MODELS

37. Linguistic Models: BNF, TAG Understanding the user's behavior and cognitive difficulty based on analysis of language between user and system. Similar in emphasis to dialogue models BNF - Backus-Naur Form TAG – Task-Action Grammar HCI – a communication process using a language Goal : understanding human behaviour and interface cognitive complexity analysis

38. BNF A BNF specification is a set of derivation rules, written as ::= where is a nonterminal consists of sequences of symbols and/or sequences separated by the vertical bar, '|’ indicating a choice, or the plus, ’+’, indicating a sequence, the whole being a possible substitution for the symbol on the left. Symbols that never appear on a left side are terminals.

39. BNF Very common notation from computer science A purely syntactic view of the dialogue Terminals lowest level of user behavior e.g. CLICK-MOUSE, MOVE-MOUSE Nonterminals ordering of terminals higher level of abstraction e.g. select-menu, position-mouse

40. BNF example – drawing a line draw_line::= select_line+choose_points+last_point select_line::=position_mouse+CLICK_MOUSE choose_points::=choose_one|choose_one+choose_points choose_one::=position_mouse+CLICK_MOUSE last_point::=position_mouse+DOUBLE_CLICK_MOUSE position_mouse::=empty|MOVE_MOUSE+position_mouse Interface complexity estimated by : Number of rules Number of « + » and « | » operators Number og basic actions needed to complete a task represents only users actions, not users perception on system answer

41. BNF Complications same syntax for different semantics no reflection of user's perception minimal consistency checking

42. BNF Example US postal address ::= ::= [ ] | ::= | ".“ ::= [ ] ::= ","

43. TAG – Task-Action Grammar BNF extension + parameterized production rules (focuses on the consistency in language stucture and embodies users knowledge about the world (ex : « up » is the opposite of « down ») Ex: cp (copy file), mv (move file) – Unix commands BNF: copy::= ‘cp’ + filename + filename | ‘cp’ + filenames + directory move::= ‘mv’ + filename + filename | ‘mv’ + filenames + directory

44. TAG Example TAG: file_op[Op] := command[Op]+ filename+filename | command[Op]+filenames+directory command[Op=copy]:=‘cp’ command[Op=move]:=‘mv’ Knowledge about the world – moving an object using command line interaction movement[Direction]:=command[Direction] + distance +RETURN command[Direction=forward]:=‘FORWARD’ command[Direction=backward]:=‘BACKWARD’ command[Direction=left]:=‘LEFT’ command[Direction=right]:=‘RIGHT’ Use of “known-item” – information that the user knows before using the system and does nor have to learn it movement[Direction]:=command[Direction] + distance +RETURN command[Direction]]:=known-item[Type=word, Direction]

45. TAG Example *command[Direction=forward]:=‘FORWARD’ *command[Direction=backward]:=‘BACKWARD’ *command[Direction=left]:=‘LEFT’ *command[Direction=right]:=‘RIGHT’ Rules denoted by “*” may be generated from the second rule using knowledge about the world and that are not considered when computing the complexity of the interface

46. Grammar pros and cons Pros: Aspects can be formally written down Verification of completeness and correctness Cons: Doesn’t scale well Difficult to apply for GUIs – usually describe sequence of actions Transition diagrams Menu-tree structures Statecharts Graphical specifications

47. PHYSICAL MODELS

48. KLM (Keystroke Level Model) Related to GOMS Refers to unit tasks (simple command sequences : find/replace, change font) with execution time smaller than 20 seconds supposes that complex tasks are decomposed in subtasks before being executed Tasks are executed in two steps : Task acquisition- when the users build the tasks mental model Task execution – using system functionalities

49. KLM Refers only to the execution part of task perfomance six execution phase operators Physical motor : K - keystroking P - pointing (use Fitt’s Law) H - homing D - drawing Mental M - mental preparation System R - response (may be omitted if the user doesn’t have to wait for response - copy) times are empirically determined. T execute = TK + TP + TH + TD + TM + TR

50. KLM example GOAL: ICONISE-WINDOW [select GOAL: USE-CLOSE-METHOD. MOVE-MOUSE-TO- FILE-MENU. PULL-DOWN-FILE-MENU. CLICK-OVER-CLOSE-OPTION GOAL: USE-CTRL-W-METHOD PRESS-CONTROL-W-KEY] For a particular user: Rule 1: Select USE-MENU-METHOD unless another rule applies Rule 2: If the application is GAME, select CTRL-W-METHOD compare alternatives: USE-CTRL-W-METHOD vs. USE-CLOSE-METHOD assume hand starts on mouse USE-CLOSE-METHOD P[to menu] 1.1 B[LEFT down]0.1 M 1.35 P[to option]1.1 B[LEFT up]0.1 Total 3.75 s USE-CTRL-W-METHOD H[to kbd] 0.40 M 1.35 K[ctrlW key]0.28 Total 2.03 s

51. KLM Example – delete a character 1.Move hand to mouse H[mouse] 2.Move mouse cursor before the mistyped character PB[LEFT] 3. Move hand to keyboard H[keyboard] 4. Delete character MK[DELETE] 5. Type correct character K[char] 6. Go back where you have been before H[mouse]MPB[LEFT] T execution =T K + T B + T P + T H + T D + T M + T R = = 2t K + 2t B + t P + 3t H + 0+ 2t M + 0

52. KLM – Operators execution times CodeOperationTime K Key press and release (keyboard) Best Typist (135 wpm)0.08 seconds Good Typist (90 wpm)0.12 seconds Poor Typist (40 wpm)0.28 seconds Average Skilled Typist (55 wpm) 0.20 seconds Average Non-secretary Typist (40 wpm) 0.28 seconds Typing Random Letters0.50 seconds Typing Complex Codes0.75 seconds Worst Typist (unfamiliar with keyboard) 1.20 seconds PPoint the mouse to an object on screen1.10 seconds BButton press or release (mouse)0.10 seconds HHand from keyboard to mouse or vice versa0.40 seconds MMental preparation1.20 seconds T(n)Type string of charactersn × K seconds W(t)User waiting for the system to respond D(n0, l0) Draw n0 straight line segment with mouse of total length l0 centimeters

53. KLM A mouse click would be written "BB" (button press, button release) AA sequence of 3 keyboard key presses is "AAA" or "T(3)" ("T" stands for "type"). A common operation that involves pointing and clicking something on the screen would be written "MPBB" (think, point, press, release). KLM is not the only technique for evaluating interfaces, but it can be used to compare the speed of two different interfaces designed to accomplish the same task.

54. KLM The most difficult to predict is M – the user know what he/she has to do M - pauses in the stream of actions (about 1 second) and that are associated with routine acts - remembering a filename or finding something on the screen (.6 - 1.35 sec; use 1.2 sec)) M represent routine thinking, not complex, lengthy, problem-solving, racking the brain, or creative meditations KLM - recommended to compare systems, even if they do not exist yet, or to compare alternatives from the same system Predictions are very accurate (20% error)

55. Case study Describe and compare using KLM the task of creating a new text file in Windows Explorer/Total Commander.

56. Fitts’s Law proposed by Paul Fitts in 1954 predicts that the time required to rapidly move to a target area is a function of the distance to the target and the size of the target. used to model the act of pointing, either by physically touching an object with a hand or finger, or virtually, by pointing to an object on a computer monitor using a pointing device.

57. Fitt’s Law T is the average time taken to complete the movement. a represents the start/stop time of the device (intercept) and b stands for the inherent speed of the device (slope). D is the distance from the starting point to the center of the target. (Traditionally, researchers have used the symbol A for this, to mean the amplitude of the movement.) W is the width of the target measured along the axis of motion. W can also be thought of as the allowed error tolerance in the final position, since the final point of the motion must fall within ± W ⁄ 2 of the target's center.

58. Fitts’s Law The first HCI application of Fitts' law was by Card, English, and Burr (1978), who used the index of performance (IP), defined as 1 ⁄ b, to compare performance of different input devices, with the mouse coming out on top

59. Fitts’s Law applies under a variety of conditions, with many different limbs (hands, feet, head-mounted sights, eye gaze), manipulanda (input devices), physical environments (including underwater), and user populations (young, old, special educational needs, and drugged participants). the constants a, b, IP have different values under each of these conditions. in GUIs, Fitts' law has been applied to tasks where the user must position a mouse cursor over an on-screen target, such as a button or other widget.

60. Fitts’s Law Fitts' law models both point-and-click and drag-and- drop actions. Dragging has a lower IP associated with it, because the increased muscle tension makes pointing more difficult. Fitts' law has also been shown to model target-directed hand and head motions in a virtual environment

61. Fitts’s Law Consequences on Design Buttons and other GUI controls should be a reasonable size; it is relatively difficult to click on small ones. Edges and corners of the computer monitor (e.g., the location of the Start button, Taskbar and a maximized window's Close button in Microsoft Windows; or the menus and Dock of Mac OS X) are particularly easy to acquire with a mouse, touchpad or trackball. Because the pointer remains at the screen edge regardless of how much further the mouse is moved, they can be considered as having infinite width. This doesn't apply to touchscreens, though.

62. Fitts’s Law Consequences on Design top-of-screen menus (e.g., Mac OS) are sometimes easier to acquire than top-of-window menus (e.g., Windows OS). Pop-up menus can usually be opened faster than pull- down menus, since the user avoids travel: the pop-up appears at the current cursor position. Pie menu items typically are selected faster and have a lower error rate than linear menu items, for two reasons: because pie menu items are all the same, small distance from the center of the menu; and because their wedge- shaped target areas (which usually extend to the edge of the screen) are very large.

63. Fitt’s Law Demo http://ei.cs.vt.edu/~cs5724/g1/tap.html

64. Meyer’s Law Meyer’s Law is a refinement of Fitts’ Law for predicting the time it takes for rapid aimed movements, such as hitting a button on the screen by moving a mouse to it. (A and B are constants which may vary with the input device.) Meyer’s Law can be used to make predictions of how much time it will take for a user to accomplish a task involving selection of targets on the screen (such as icons, menus, or hypertext links).

65. Meyer’s Law T = time to move to a target D = distance to target W = width of target A ~ -13 msec B ~ 108 msec

66. Hick’s Law

67. Hick’s Law According to Card, Moran, and Newell, the +1 is "because there is uncertainty about whether to respond or not, as well as about which response to make." The law can be generalized in the case of choices with unequal probabilities p i of occurring, to where H is the information-theoretic entropy of the decision, defined as T = k*H

68. Hick’s Law Consequences on Design It is better to have large menus, than deep menus, because the search is made from a global perspective

69. The Power Law of Practice Alternative execution time estimation method considering the expertise level of the user T n = T 1 * n -a, a ~ 0.4, where: T n = execution time after n trials T 1 = execution time on first trial n = executions number Task execution time exponentially decreases with the number of trials

70. Project Recommandation Use CogTool to compare different design alternatives.CogTool 05/06/2016 Proiectarea interfeţelor om-calculator