1. HCI
The Computer

2. Interacting with computers
To understand Human-Computer Interaction … need to understand computers!
what goes in and out of computer devices, paper, sensors, etc
what can the computer do? memory, processing, etc

3. HCI challenge Traditional I/O techniques may not be suitable
Sensory/motor requirements may not be present in the disabled user
New interaction methods and techniques are required

4. Von Neumann Architecture

5. Timeline Vacuum Tube Triode (1908)
Transistor 1947 (silicon, germanium)
Very Large Scale Integrated (VLSI)
Circuits; 1970s-- (> 1,000 transistors per chip)
Intel Itanium (Tukwila)
2008: 2 billion transistors

6. The Computer A computer system is made up of various elements
Each of these elements affects the interaction
input devices – text entry and pointing
output devices – screen (small&large), digital paper
virtual reality – special interaction and display devices
physical interaction – e.g. sound, haptic, bio-sensing
paper – as output (print) and input (scan)
memory and processing– memory, access, speed of processing, networks

7. How many computers … in your house? in your pockets? PC DVD, HiFi, TV
microwave, cooker, washing machine, fridge
Air Condition
security system
can you think of more?
in your pockets?
PDA
smartphone
digital camera
smart card
electronic car key
USB memory
try your pockets and bags

8. Richer interaction
sensors
and devices
everywhere

9. text entry devices keyboards (QWERTY et al.)
chord keyboards, phone pads
handwriting, speech

10. Input (1) Batch Large amounts Pre-prepared information
Standardised data loaded quickly using automated routines
Used a lot in the past (punch cards)

11. Keyboards Most common text input device
Allows rapid entry of text by experienced users
Keypress closes connection, causing a character code to be sent
Usually connected by cable, but can be wireless

12. Keyboards QWERTY Keyboard (1878)
the official standard of computer keyboards [ISO9995]
was designed for two finger typing
prevents stuck keys (on a typewriter)
the position of the keys the hands are held close together helping to cause Carpal Tunnel Syndrome
can be used without training
is cheap and easy to find
Any other keyboard?

13. Keyboards (ctd) - layout
Alphabetic layout
keys arranged in alphabetic order
not faster for trained typists
not faster for beginners either!
Dvorak layout
common letters under dominant fingers
biased towards right hand
common combinations of letters alternate between hands
10-15% improvement in speed and reduction in fatigue
But no significant adoption, when compared to QWERTY

14. Keyboards
Over the years, attempts were made to physically improve keyboards …
Tilting
Curving
Splitting

18. Keyboards (ctd) - special keyboards
Maltron left-handed keyboard
Virtual keyboard
Projection keyboard

19. Chord keyboards only a few keys - four or 5
letters typed as combination of keypresses
compact size – ideal for portable applications
short learning time – keypresses reflect letter shape
fast – once you have trained
BUT - social resistance, plus fatigue after extended use
NEW – niche market for some wearables

20. Keyboards Chord Keyboards Don’t use an alphabet Use 4/5 keys
Rely on combination of keys
Ideal for one hand or mobile devices
Can suffer fatigue after use
New application in mobile devices?

21. Phone keypad and T9 entry
use numeric keys with multiple presses
2 – a b c 6 - m n o
3 - d e f 7 - p q r s
4 - g h i 8 - t u v
5 - j k l 9 - w x y z
hello = [pause]555666
surprisingly fast!
T9 predictive entry
type as if single key for each letter
use dictionary to guess the right word
hello = … ‘hello’ not ‘gdjjm’
note 26 could mean ‘am’ or ‘an’; phone gives those options

22. Soft or on-screen keyboards
Keys are arranged in rows and columns
Operated by
eye tracking
scanning input methods
Text entry rate
0.5-5 wpm
6-8 wpm with rate enhancement techniques (prediction, ambiguity, abbreviation expansion)

23. Our Focus- Scanning Keyboards
Soft keyboards operated with scanning input methods Q W E R T Y U I O P A S D F G H J K L
Enter Z X C V B N M __
<--
Shift

24. Performance Models
The RG model by Rosen and Goodenough- Trepagnier (1981)
Based on three components
L -- average no of language units per word
A -- average no of motor acts required to input each language unit
T – average time required to carry out each motor act
TW = average time to compose an word = L*A*T

25. Input Devices

26. Skinput: Body as an Input Surface
CHI’2010
Use a tiny projector on body to show menus
Microphones to listen to taps on hand/arm
Signal processing and machine learning to differentiate positions

28. Handwriting recognition
Text can be input into the computer, using a pen and a digesting tablet
natural interaction
Technical problems:
capturing all useful information (path, pressure, etc) in a natural manner
segmenting joined up writing into individual letters
coping with different styles of handwriting
etc
Used in PDAs, tablet computers, smart white boards … … leave the keyboard on the desk!

29. Speech recognition Most successful when:
single user – initial training and learns peculiarities
limited vocabulary systems
Improving, but challenges such as:
external noise interfering
imprecision of pronunciation/articulation e.g. vagueness, pauses, clearing throat, repetition, etc
large vocabularies
different speakers

30. positioning, pointing and drawing
mouse, touchpad trackballs, joysticks etc. touch screens, tablets eyegaze, cursors

31. Mouse Since the mouse sits on a desk Disadvantage
Movement of mouse moves screen cursor
mouse movement in (x, z) plane …
Since the mouse sits on a desk
moving it about is easy
cursor is small and can thus be manipulated without obscuring the display
Disadvantage
hand-eye coordination problems for novice users

33. Mouse (ctd) Two methods for detecting motion Mechanical Optical
ball on underside of mouse turns/rotates as mouse is moved
can be used on almost any flat surface
Optical
light emitting diode on underside of mouse
detects fluctuating alterations in reflected light intensity to calculate relative motion
less susceptible to dust and dirt

34. Footmouse Motivation Relevant Challenges
foot pedals are used heavily in musical instruments (pianos, organs, drums) and in mechanical equipment (cars, cranes, sewing machines)
Relevant
hands-occupied situations
users with disabilities or with high-back or neck problems
Challenges
feet tend to be less precise than hands
practicality of having foot controls in the work environment (e.g. pedals under a desk may be operated accidentally, etc)

35. Touchpad a pointing device featuring a specialized surface that
can translate the motion and position of a user's fingers
to a relative position on screen
used mainly in laptop computers
good ‘acceleration’ settings important

36. Trackball and thumbwheels
ball is rotated inside static housing
like an upside down mouse!
separate buttons for picking/selection
very fast for gaming
used in some portable and notebook computers
Thumbwheels
two dials for X-Y cursor position
for fast scrolling – single dial on mouse

37. keyboard nipple Keyboard nipple or pointing stick
a pointing device typically mounted in a keyboard on laptops
pointing stick operates by sensing applied force
the velocity of the pointer depends on the applied force
force applied on the pointing stick is echoed on the screen by movements of the pointer

38. Joystick

39. Joystick Joystick pressure of stick translates to velocity of movement
buttons for selection e.g. on top or on front like a trigger
often used for computer games, aircraft controls and 3D navigation

40. Haptic devices - example
BMW iDrive
for selecting and controlling menus
one feels little bumps along the way as one maneuvers through options and menus
makes it easier to select options by feel
uses haptic technology from Immersion Corp

41. Idrive controller
The iDrive system gives the driver complete control over many of the vehicle’s functions while allowing him to concentrate on the road.

42. Touch screen

44. Touch-sensitive screen
Detect the presence of finger or stylus on the screen.
mobile devices e.g. smartphones and tablets
more natural gestures through multitouch (i.e. ability to recognize the presence of two or more points of contact with the screen surface, and therefore supporting more natural gestures such as pinching and reverse pinching)

45. Pointing Devices
Integrated gaze and face tracking system. A camera and IR light source tracks gaze by computing the angle between the corneal reflection and the centroid of the pupil.

46. Discrete positioning controls
in phones, TV controls etc
mini-joysticks (mobile phones)
central cursor pads & discrete left-right, up- down buttons (TV remote controls)
e.g. +/- for controlling volume & navigating channels
mainly for menu selection

47. Pointing Devices

48. Smart Systems

49. virtual reality
Virtual reality is an artificial environment that is created with software and presented to the user in such a way that the user suspends belief and accepts it as a real environment. On a computer, virtual reality is primarily experienced through two of the five senses: sight and sound.

50. Positioning and viewing in 3D
Positioning in 3D space
3D mouse: six-degrees of movement: x, y, z + roll, pitch, yaw
data glove: to detect finger position
VR helmets: to detect head motion and possibly eye gaze
whole body tracking: accelerometers, image/video processing, etc
Viewing in 3D
often use stereoscopic vision ( two 2D images are presented separately to the left and right eye of the viewer, which are then combined in the brain to give the perception of depth), but also possible to use real 3D display (displaying an image in three full dimensions)

51. Simulators and VR caves
scenes projected on walls
realistic environment
real controls
other people

52. bitmap screens (CRT & LCD) large & situated displays digital paper
display devices
bitmap screens (CRT & LCD)
large & situated displays digital paper

53. Display

54. Output Devices Permanent output Impermanent output Paper Microfiche
Film
Impermanent output
Screen
Sound
Tactile

55. Bitmap displays bitmap display or raster scan
the whole screen is scanned sequentially and horizontally
screen is vast number of coloured dots

56. Visual Output Cathode Ray Tube (CRT) Liquid Crystal Display (LCD)
Text based to high resolution
Produce minor health hazards
Since they are radiation emitting devices!
Liquid Crystal Display (LCD)
Alternative systems might not use a screen …
Lights
Gauges
Dials
Can you give examples?

57. Large and situated displays
Large displays
Used for meetings, lectures, etc
Where very large screen areas are required, several smaller screens, either LCD or CRT, can be placed together in a video wall where each screen displays a portion of the whole (or separate sub-image) and the result is an enormous image
Situated displays
As well as for lectures and meetings, display screens can be used in various public places to display information; they could even be interactive

58. paper: printing and scanning
print technology
fonts, page description, WYSIWYG
scanning, OCR

59. Permanent Output On paper using printers Impact Printers (Noisy)
Dot Matrix (Slow + Low Quality)
Daisy Wheel (Very Slow + 1 Font)
Chain and Band printers (Fast + Expensive)
Non-Impact Printers (Silent, High Quality, Fast)
Inkjet
Thermal (Expensive)
Laser (Cost effective + Colour expensive)

60. Permanent Output Printers Speed measured in Pages Per Minutes
Resolution measured in Dots Per Inch (dpi)
80 dpi (dot matrix) = Near Letter Quality
300 dpi – 1200 dpi (non-impact) = Near Book Quality
Store information in firmware such as
Typeface (Fonts)
Character set
Size
More complex printers use Page Description Language (PDL)
Communicates complex commands directly to printer
Used in pdf and postscript

61. Fonts and this is twenty-four point
Font – the particular style of text
Courier font
Helvetica font
Palatino font
Times Roman font
§´µº¿Â Ä¿~  (special symbol)
Size of a font
This is ten point Helvetica
This is twelve point
This is fourteen point
This is eighteen point
and this is twenty-four point

62. Fonts (ctd) Pitch Serif or Sans-serif
fixed-pitch – every character has the same width e.g. Courier
variable-pitched – some characters wider e.g. compare the “i” and the “m” in Times New Roman
Serif or Sans-serif
sans-serif – square-ended strokes
e.g. Helvetica
serif – with splayed ends (such as)
e.g. Times Roman

63. Readability of text lowercase UPPERCASE
All lowercase text considered to be easier to read than all uppercase
UPPERCASE
better for individual letters and non- words e.g. flight numbers: BA793 vs. ba793

64. Scanners Take paper and convert it into a digital document Useful in
desktop publishing for incorporating photographs and other images
document storage and retrieval systems, instead of physical paper storage

65. Optical character recognition
OCR converts bitmap back into text
different fonts
create problems for simple “template matching” algorithms
more complex systems segment text, decompose it into lines and arcs, and decipher characters that way
page format
columns, pictures, headers and footers

66. physical controls, sensors etc.
special displays and gauges
sound, touch, feel, smell
physical controls
environmental and bio-sensing

67. Sounds can take many forms used for beeps bongs (deep ringing sounds)
clonks (loud thudding sounds)
whistles
whirrs (like the sound of rapidly vibrating wings)
used for
error indication (e.g. incorrect command/input) or risky situations (e.g. when deleting files)
confirmation of actions e.g. keyclick
notification of events/status e.g. a new has arrived, download has been completed

68. multi-function control
Physical controls
specialist controls needed …
industrial controls, consumer products, etc.
easy-clean
smooth buttons
multi-function control
large buttons
clear dials
tiny buttons

69. Environment and bio-sensing
sensors all around us
car courtesy light
small switch on door
ultrasound detectors
security, washbasins
RFID security tags in shops
temperature, weight, location

70. MediaCup Han's purpose to drink coffee
cup has sensors
heat, movement, pressure
broadcasts state (IR)
used for awareness
user is moving, drinking, …
Han's purpose to drink coffee
incidentally colleagues are aware

71. OTHER ASPECTS

72. Computer Memory Short Term Long Term RAM Secondary Backing Storage
Hard disk
Disk drive
CD’s
Memory Sticks

73. Permanent Storage Cheaper Slower Permanent (Non-volatile)
Several types
Magnetic
Optical
Different capacities
Floppies 1.4 Mb
Memory Stick 64 Mb +
CD 640 Mb
DVD 4 Gb – 18 Gb
Blue Ray 15 Gb – 30 Gb
Harddisk 120 Gb + … Tera Bytes
Two writing types
Rewritable (Hard disk)
Write Once Read Many WORM (CD, punch cards)

74. What computers just cannot do.

75. M.C. Escher
Print Gallery

77. How do you represent music?
Score:
Audio samples
Spectrogram
Q: How do we represent music?
Show: score vs. audio file
Ask aloud “is score a faithful representation?”
Q: What is the relationship between the score representation and what you hear?
Explain spectrum -
Show adding waves in time
Adding waves in frequency
Show spectogram - 440, 880, both. (with sound)

78. Creating music: Synthesis
Examples:
Stevie Wonder
Tina Turner (FM)
Kanye West/ Daft Punk (vocoder)

79. Review Questions What are the most common interaction devices?
What are their limitations?
How can some of them be overcome by novel devices?
Suggest ideas for an interface which uses speech and sound effectively.
How can sound be used to convey information?

80. Review Questions
What are some of the factors you might consider when designing systems like a public access interactive kiosk?
Identify the strengths and weaknesses of multimedia and hypermedia systems.
Discuss how the speed of access to memory can affect the speed of an interactive computer system. Which other factors can affect such speed?

81. Revision