ICS 463, Intro to Human Computer Interaction Design: 9 “Theory”. Input and Output Dan Suthers

Input Devices Discrete Entry Devices Continuous Entry (Pointing) Devices Natural Language Input

Discrete Entry Devices Keyboards: Hardware –Membrane Spill resistant No built in feedback –Electromechanical “Click” feedback Sensitive to spills etc.

Discrete Entry Devices Keyboards: Layout –QUERTY Designed to slow you down! –Dvorak Designed for efficient and less stressful typing Available since 1930s yet not used due to cost of retraining!!!

Dvorak

Discrete Entry Devices Keyboards continued –Chord Multiple simultaneous keystrokes make a letter Can type with one hand Mouse buttons Selection or indicating position in combination with mouse pointing Switches Useful for disabled

Continuous Entry (Pointing) Devices These range from indirect cursor control to direct manipulation of display surface 2D: position  motion –Cursor keys Activation maps to motion Best for horizontal and vertical movements –Joysticks Position maps to direction and speed Best for controlling direction and speed, e.g., games!

Continuous Entry (Pointing) Devices 2D: position  indirect position –Mouse Best for moving objects and indicating position Need adequate 2D surface Mechanical and optical variants –Trackpads, trackballs Best for indicating position. Little space needed. –Graphics Tablets Like writing with pen on paper, but less natural due to indirect display of results

Continuous Entry (Pointing) Devices 2D: position  direct position –Mimeo, Smartboard Like whiteboard but writing is recorded More natural than graphics tablets because writing is displayed on same surface Can be used as output device with projection –Touch Screens Direct manipulation, easy to learn, durable Lack of precision (if using finger) Large targets and untrained users in public Palm Pilot: the exception? (uses stylus)

Continuous Entry (Pointing) Devices 2D: position  direct position cont. –Stylus Screens Palm Pilot etc. More precise than touch screens May require training (handwriting recognition) –Eye Tracking Based on reflection of light Requires stable head Involuntary eye movements may be a problem Potentially useful for disabled

Continuous Entry (Pointing) Devices Three dimensional –3D Trackers Magnetic, ultrasound or radio Track in 3D relative to fixed reference point Head Trackers used for disabled –Flexion Sensors (Dataglove) Track finger motions –Computer Vision Sophisticated algorithms track location of objects in space

Natual Language Input Handwriting Recognition –Natural handwriting (Newton) Requires training of the computer –Stylized writing (Palm Pilot) Requires training of the user Less error prone (sort of)

Natual Language Input Speech Recognition –Isolated Word versus Continuous –Speaker Dependent versus Independent –Advantages Easier to train users Can do other things at the same time Accessible to disabled –Disadvantages Error prone Vulnerable to interference from noise Natural language is not precise

Input Device Design Issues Match the device to Physiological and psychological characteristics of users Tasks to be performed Intended work environment

Matching Devices to Tasks Identify the task requirement –What kind of motion and manipulation is needed? –What kind of feedback is needed? Identify the expressiveness of the devices –How can one manipulate the device? How natural is the mapping? –Does the manipulation of the device correspond well to the manipulation of the artifacts? –Does the feedback match the user’s mental model?

Output Devices Visual –Data Visualizations –Visual Interaction –Dynamic Visualization and Animation Auditory Speech Tactile

Visual Output Most common Large literatures on perceptual issues, e.g., dark characters on light background optimize contrast and minimize reflections Uses –Data visualization –Visual feedback of interaction –Dynamic visualization of models or data –Interactive animations

Data Visualization Large literature on visual display of data Parallel versus serial detection: –Parallel: color, value, angle, sloope, length, texture –Serial: shape, area, curvature, connection, containment No one representation is optimal for all tasks; e.g. Lohse’s example of line graph, bar graph, and table (retrieving values, finding maximums, comparison of trends) Match characteristics of data to those of representation (e.g., continuous versus categorical): refer back to Ch. 4 (table 4.2)

Visual Interaction Continuous feedback on processes and state maintains user’s mental model and feeling of control –Progress through a process –Prompting for input –Confirming input –Indicating errors –Location in the artifact –Location of next action

Dynamic Visualization & Animation Data visualization: external data is displayed Model-based visualization: computer model is displayed Animation: (inter)active visualization of process Advantages over noncomputational media: –Apply algorithm to new data easily –Interactive –Flexible

Design recommendations Use paper-based prototypes Consider a wide range of tasks and test with a large range of data Carry out tests on many real users

Auditory Uses of sound are expanding –Alerting to errors –Drawing attention in complex environment or when eyes may be away from screen –Monitoring background events –Subtle audio enhancements make widget manipulations more “physical” –Data sonification: provide audio coding; rely on audio pattern recognition to detect changes –Natural sounds as data

Speech Natural language planning is complex; most speech output is from templates Concatenation: piece together prerecorded words or phrases Synthesis: rules map text to stream of phonemes Applications include: –Alerting when eyes are off screen –Interaction over telephone systems –Interfaces for the blind

A key design principle In any “perceptualization:” Find a mapping between domain and display elements and relationships that makes perceptually prominent those that should be conceptually prominent Apply this mapping in a consistent way

Examples Wacom tablet Speech I/O on Macintosh Photo visualization on PC