1. Interaction

2. Contents Understanding & Conceptualizing Interaction
Interaction models
Interaction styles
Interaction paradigms

3. Understanding & Conceptualizing Interaction
It is tempting to design an interface by going directly to the nuts & bolts
Whether to use touch screen
GUI details
Whether speech should be used
While these decisions are necessary, they should be done after understanding the problem space

4. Understanding the Problem Space
Before designing a solution, you need to understand the problem
What is the problem with the current interface?
What do the users find difficult?
What is the functionality which the users want?
Do all users have the same requirements?
Are all users willing to put time into understanding the problem?

5. Example: Understanding Browser Deficiencies
A major web browser has been losing market share to competitors
The development team meets to discuss the problem
After asking users, it is found that many have never used bookmarks
They zero in on the bookmarks as a potential problem area

6. Example: Understanding Browser Deficiencies
There are various ideas to explain the problem
Organizing bookmarks requires dragging URLs to folders which is tedious and error-prone
Another member agrees with this assessment based on feedback
A software engineer, who scrolls through history rather than using bookmarks, assumes others do the same. He recommends that the history facility be removed due to it being overly complex and unused.

7. Example: Understanding Browser Deficiencies
The result of this discussion is:
They have various theories as to what the problem is
They still do not understand the real problem
They must go back, ask more users for feedback, and think about the problem for a longer time
You cannot create a solution until you understand the problem !

8. Conceptualizing the Design Space
“A conceptual model is a high-level description of how a system is organized and operates” Johnson & Henderson
The conceptual model is how the user will see and understand the system

9. Conceptualizing the Design Space
“The most important thing to design is the user’s conceptual model. Everything else should be subordinated to making the model clear, obvious, and substantial. This is almost exactly the opposite of how most software is designed” Liddle

10. The Conceptual Model Outlines what people can do with a product
Presents the concepts necessary to interact with the product
The metaphors and analogies used to convey an understanding of the product to the user

11. The Conceptual Model The model
Gives the user an understanding of how to use the product
Forms a shared vision between all of the designers
Should be designed after the problem space is understood but before the rest of the solution is designed

12. Contents Understanding & Conceptualizing Interaction
Interaction models
Interaction styles
Interaction paradigms

13. Interaction Models
Interaction models can help us understand the interaction between the user and the system
Terminology
Domain
Important concepts about the problem space
In graphic design these would be shapes, drawing tools and a drawing surface

14. Interaction Models Tasks Goals Intention Task analysis
Operations to manipulate the concepts in the domain
Goals
The desired result of a task
Intention
A specific action designed to achieve a goal
Task analysis
The identification of the problem space in terms of domain, tasks, goals and intentions

15. Interaction Models Core language Task language
A language used to describe computational aspects of the state of the system
Task language
Describes psychological aspects of the domain relevant to the state of the user

16. The Execution-evaluation Cycle
This is also called Norman’s model of interaction
It is probably the most influential model
It is broken into two major phases
Execution
Evaluation
These are further broken into a total of 7 stages

17. The Execution-evaluation Cycle
Establishing the goal
Forming the intention
Specifying the action sequence
Executing the action
Perceiving the system state
Interpreting the system state
Evaluating the system state with respect to the goals and intentions

18. The Execution-evaluation Cycle
Each of these stages is performed by the user
The user determines a goal, figures out how to achieve it, performs the action, and then looks at the system to see if it worked as expected
Norman uses this model to explain why users have problems with interfaces

19. The Gulf of Execution
Remember that the system operations are defined in terms of the core language and the user describes actions in terms of the task language
Since these languages are not the same, it can result in misunderstandings
This means that there is a gulf between the users actions to achieve a goal and the systems interpretation of those actions

20. The Gulf of Evaluation
The gulf of evaluation is the distance between the actual state of the system and the state expected by the user
The more difficulty the user has interpreting the system state, the less effective the interaction
Norman’s model provides a simple framework for the understanding of interaction

21. The Interaction Framework
Norman’s model is for understanding the interaction
It does nothing to consider the system’s side of the communication
This is addressed by an extension to Norman’s model by Abowd and Beale
This extension is the interaction framework

22. The Interaction Framework
This breaks the system into 4 major components
The system
The user
The input
The output
It uses the core language and the task language as well as languages for both input and output

23. The Interaction Framework
There are 4 steps in the interaction cycle
The user formulates a goal and a task to achieve it. This is expressed in the input language.
Input languages is translated into core language
System transforms itself into a new state
The state of the system is then rendered in the output language and sent to the user O
output S U
core
task I
input

24. The Interaction Framework
There are 4 translations involved in the interaction
The users task must be articulated in the input language
The input language is then translated into stimuli for the system
Once the system state is updated, evaluation begins with the presentation of results in the output language
Finally, observation translates the output language into understanding by the user O
presentation
observation
output S U
core
task I
articulation
performance
input

25. Articulation Poor articulation
A user wants to turn on lights at one end of a room
The user is confronted with a bank of unlabeled switches
The user has no idea of which switch controls the lights at the end of the room
The problem is that the user cannot articulate the goal (turn on lights at one end) in the language provided (unlabeled switches)

26. Articulation Good articulation Virtual reality systems use data gloves
This allows every motion of the user’s hand to be translated directly into operations to be performed by the system
This is an example of a direct manipulation interface
It provides an excellent translation from the task language to the core language

27. Output Consider writing a paper with a word processor
You must be able to see the effects of your actions
Often, you can see only the immediate paragraph without the surrounding text where there might be changes as well
This means that all of the state changes cannot be be displayed in the output language

28. Using the Interaction Framework
The interaction framework can be used to judge the usability of the interface
By concentrating on translations, it allows us to determine if concepts are being communicated correctly
This might be used as the basis for choosing one tool over another

29. Contents Understanding & Conceptualizing Interaction
Interaction models
Interaction styles
Interaction paradigms

30. Interaction Styles
The choice of interaction style has a profound impact on the nature of the interaction
We will look at
Command line interface
Menus
Natural language
Question/answer dialog
Forms
WIMP
Point and click
Three-dimensional interfaces

31. Command Line Interfaces
The earliest interfaces were command line
Despite the availability of more modern interfaces, they continue to be used
Command line interfaces
Provide abbreviations and function keys to speed use

32. Command Line Interfaces
Provide direct access to commands without having to navigate menus
Allow options to be used to modify command behaviour
Are very powerful
Have a steep learning curve before they can be used effectively
This can be lessened by using meaningful command names

33. Menus These display the available choices
The user can select one by mouse, numeric entry, or arrow keys
Larger menus are organized hierarchically so not all of the menu is displayed at once
This makes navigation more difficult since the user cannot see where they want to go

34. Menus
The advantage of menus is that they lessen the load on the user’s memory since commands no longer have to be remembered
This still means that the commands must be meaningful and easy to understand

35. Natural Language
On the surface it is attractive to allow instructions in a natural language like English
Problems are soon encountered due to the ambiguity of natural language
The boy hit the dog with the stick
Did the boy use a stick to hit a dog or did the boy hit a dog carrying a stick?

36. Natural Language
There is also ambiguity in the meaning of individual words as well as the sentences
Although research continues into natural language understanding, it remains problematic for the near term

37. Question/answer Dialogs
The user is asked a series of questions and provided with a series of answers from which to choose
These are easy for novices to use
They restrict queries to those supported by the system
General purpose query languages provide more flexibility but are more difficult to learn and use

38. Forms
This is used primarily in data entry and occasionally in data retrieval
The user is presented with a form that looks like a familiar paper form
Advanced forms allow the user to leave fields blank and to fill in the fields in different orders
A spreadsheet is a special type of form in which the results of changes can be seen immediately

39. WIMP Interface
This is the familiar windowing system consisting of Windows, Icons, Menus and Pointers
This has become a standard interface supported by many computer systems including
Microsoft Windows
X Windows
Apple Macintosh

40. WIMP Interface Windows Icons
Areas of the screen where text and or graphics can be rendered
Icons
Small pictures representing concepts or actions
Used for buttons, labels, and representing windows in collapsed form

41. WIMP Interface Pointers
Pointing devices such as mice, touchpads, trackballs, tablets, etc.
These are essential to WIMP interfaces
The pointer is represented on screen by a cursor which can take on a variety of shapes
Often the shape of the cursor is used to indicate state information about the system

42. WIMP Interface
Menus
These allow the user to issue commands by selecting from menus
There are pull-down menus from a menu-bar as well as pop-up menus
They all reduce the load on the user’s memory
There are also menus which are circular as well as ones which pick up gestures by the user with the pointing device

43. WIMP Interface Buttons
Buttons are small windows which can be clicked to perform an action
Buttons can be labeled with text, graphics or both
Buttons can change appearance to indicate their state
Radio and checkboxes are buttons with a slightly different behaviour and appearance

44. WIMP Interface Toolbars
This is an area of the screen where buttons and icons are collected
It acts similarly to a menu by allowing the user to select from many commands
It often makes selections more obvious by showing them all at once
Often, the user can customize their display by showing a selection of multiple toolbars which are available

45. WIMP Interface Palettes
A palette is a collection of controls which can alter the state of the system
They usually select values for various options
They provide immediate feedback on the state of these options
Graphics systems often use palettes to se4lect the drawing color, line thickness, etc.

46. WIMP Interface Dialog boxes
A dialog box is a window which conducts an interaction with the user to
Provide information
Notify the user of an error
Solicit input from the user
Often, these dialogs are modal so that the user must interact with them before continuing
System modal locks the entire windowing system and are usually a bad idea as the user cannot interact with other programs

47. Contents Understanding & Conceptualizing Interaction
Interaction models
Interaction styles
Interaction paradigms

48. Paradigms for Interaction
This section will look at the key technologies which have contributed to interactivity
This will be partially historical in nature so that you can see how it evolved over time

49. Time Sharing
Initially, computers were batch systems used by a single user
In the 1960s advances in computing power made time sharing possible
This allowed a single computer to be used by several individuals
They were able to interact with it via teletypewriters
Many people at the time were excited by the possibilities

50. Video Display Units
The teletypewriter gave way to the video display unit
Ivan Sutherland invented Sketchpad, a program which allowed you to draw on the screen and have it stored in memory
This formed the basis for all modern windowing systems and established VDUs and the main display technology

51. Programming Toolkits
Douglas Engelbart had the idea that the computer should be used to augment human capabilities, not just for data processing
He created systems which were forerunners of modern word processors
He advocated the creation of software toolkits which could be used to easily build ever more complex software

52. Personal Computing The 1970s saw computing power offered to the masses
The LOGO system was designed to allow children to draw simple graphics using a turtle which draw with its tail
Alan Kay at Xerox Parc had a vision of the Dynabook, a handheld personal computer which is more advanced than what we have today
The tablet PC is the closest thing to a Dynabook

53. Windowing Systems
The success of personal computing with the introduction of the PC created a need for richer interaction
Kay developed the WIMP interface at Xerox which produced the Xerox Star in 1981
One of the programmers from the project was hired by Apple…and the Mac was born
Soon WIMP came to predominate

54. The Metaphor
Metaphors are a good way to introduce users to new technology
They allow people to understand something new based on something they already understand
LOGO used the metaphor of a turtle dragging its tail on the ground
Many systems are based on the metaphor of the office desktop

55. The Metaphor
Although the metaphor is useful to get users used to a new system it can be harmful if carried too far
the word processor looks and seems to act just like a typewriter
Few people know that they should format paragraphs rather than putting an extra space between them
Few people realize that space is a character, since it is just empty space on a typewriter

56. The Metaphor
The most extreme example of the metaphor is the virtual reality system
This places the user in the metaphor, creating an alternate world
These interpret the users normal actions as instructions to manipulate virtual objects in the virtual world

57. Direct Manipulation
Direct manipulation interfaces are characterized by rapid feedback on every action performed by the user
This makes it appear that the user is manipulating an object in real time
This is a very satisfying experience that mirrors what happens in the real world

58. WYSIWYG What You See Is What You Get
These are interfaces that show you what the end result will look like in real time
They are ubiquitous in word processing
The trouble is that a lot of processing power is spent on it and sometimes it is less convenient to include diagrams this way than to use other techniques

59. Language vs. Action
While direct manipulation is good for many tasks, it can be less efficient when applied to repetitive tasks
Languages are better for this sort of task
The user can describe the action to be performed and the system will interpret and perform the action as many times as desired
This is why command languages are still used for certain types of tasks

60. Hypertext Vannevar Bush proposed the idea of interlinked text in 1945
Since then, it has been used for
Linking of text in help systems
Linking of documents in the Web
Among the problems of hypertext is the tendency for users to get lost and not be able to navigate back to a specific point or to forget their location

61. Multi-modality
Normally humans communicate with computers using two modes
Typing, a haptic sense
Reading, a visual sense
Multi-modality uses the other senses including haptic and auditory
This can allow more information to be conveyed at once

62. Computer Supported Cooperative Work
The development of the network led to the desire for people to work on shared documents remotely
This created
Chat systems
Shared whiteboards
Systems for the delivery of lectures remotely
This area is also called groupware

63. The World Wide Web
The introduction of the web in 1989 is the most important change in computing
It moved the internet out of the research lab and made it available to all
It formed a union of computing and communications
It made everyone aware of computers, not just the few who used them on a daily basis

64. Agent-based Interfaces
An agent is something which performs some action on your behalf
An filter is an agent which identifies and flags junk
Help agents watch the user’s actions and make suggestions or provide help
Some agents search the internet to find specific items
Others can be dispatched to sites on the net to perform actions on your behalf

65. Ubiquitous Computing
Ubiquitous or pervasive computing moves it out of beige boxes so that computing permeates our environment
We have refrigerators which can surf the net for menus
We have appliances which can be programmed from a PC anywhere on the internet
We are developing RFID tags where items can report their location to a central computer

66. Sensor-based Interaction
We are used to entering data into a computer
What if the computer had sensors which gathered information from the environment?
This happens with lights which detect someone entering a room
Computers can have sensors which let them find out about the real world and respond appropriately

67. Context-aware Interaction
Once the computer knows about the state of the environment, it can make decisions as to what is appropriate for the environment
This can use artificial intelligence to take actions which are appropriate to what the user is currently doing