The Human Introduction, Modeling, Vision Human Computer Interaction, 2nd Ed. Dix, Finlay, Abowd, and Beale Chapter 1

Introduction Task System User About IT, HCI and humans … Interaction captured in figure below with “system, task, user” So far, have talked about “interface design” of system Guidelines, principles, evaluation, etc. Now will look at user Will consider “model” of human Models capture elements of whatever is being modeled that are relevant to some endeavor E.g., “cognitive engineering” Task “work on task” “commands” System User “gives” “performs” “feedback” and its representation (possibly manipulable)

Introduction On describing humans Psychology focuses on the individual To understate, a question of long standing … One of the big questions … Has and can be done in various ways and at different levels of analysis Moral, spiritual, …., psychological, … physiological, … chemical Reductionist Psychology focuses on the individual Many orientations through psychology’s short history Note: Clinical/personality orientations, e.g., Freud, Jung, have their own utility in explanation, but are rarely subject to scientific investigation Structural, turn of 20th century, relations among elements Gestalt, 1930’s and on, general, especially of perception, still useful Behaviorist, all S->R, which is fine for some things, but fall short for explanation of mental phenomena Information processing/Cognitive, current orientation Influences from Shannon’s ideas on information, shortcomings of behaviorism, successes in codifying information in computing “Human information processor”

Modeling Humans Any theory or model is an abstraction For HCI, goals are primarily in “Computer” and “Interaction” Utility of human model lies in how well it helps with interfaces Card, Moran, and Newell (1983) Model Human Processor “Classic” example of cognitive architecture with focus on humans interacting with computers Perceptual system Motor system Cognitive system Each has own processor and memory Principles of operation dictate system behavior under certain conditions A very simple model Dix et. al use similar information processing division of elements in chapter

Overview Dix et al. model: Idea tonight Information i/o … visual, auditory, haptic, movement Information stored in memory sensory, short-term, long-term Information processed and applied reasoning, problem solving, skill, error Idea tonight Give broad overview of elements of human Show that for some elements of user interface design, detailed knowledge is at least useful and perhaps critical For design - sensory, perceptual, and cognitive guidelines and principles

Some Terminology Sensation Perception Cognition Transduction of energy, etc. by sensory receptors E.g., light to neural impulses by retinal cells Forming a “mental image” or awareness and representation To “see a window” Top down and bottom up Demo, next slide Latin – cognitio – knowledge “Higher level” mental representations and procedures Process of thought

What’s below?

Demo

Model Human Processor + Attention A “useful” big picture - Card et al. ’83 … plus attention Senses/input ® f(attention, processing) ® motor/output Notion of “processors” Purely an engineering abstraction Detail next slide And in lectures!

Model Human Processor + Attention Sensory store Rapid decay “buffer” to hold sensory input for later processing Perceptual processor Recognizes symbols, phonemes Aided by LTM Cognitive processor Uses recognized symbols Makes comparisons and decisions Problem solving Interacts with LTM and WM Motor processor Input from cog. proc. for action Instructs muscles Feedback Results of muscles by senses Attention Allocation of resources

Model Human Processor - Original Card et al. ’83 An architecture with parameters for cognitive engineering … Will see visual image store, etc. tonight Memory properties Decay time: how long memory lasts Size: number of things stored Encoding: type of things stored

Model Human Processor - Original

Dix: Information Input and Output Again, human considered as information processor Input channels are the five senses With some more important than others For hci, vision primarily Output channels are human effectors E.g., limbs, fingers, head, vocal system

Vision Vision and visual perception studied across a range of disciplines Points tonight meant to highlight usefulness for HCI in knowing about vision For vision consider (Dix): Physical reception of stimulus Processing and interpretation of stimulus

The Eye - Physical Reception Mechanism for receiving light and transforming it into nerve transmissions A transducer: light -> nerve “firings” Light reflects from objects Some strikes retina Images are focused upside-down on retina Ganglion cells (brain!) detect pattern and movement As camera, has equivalent of lens + aperture (pupil) and film (retina) Note that image is upside down on retina! … perception

Environment: Visible Light Generally, the body’s sensory system is the way it is because it had survival value Led to success survival and reproduction Here, focus on human vision (because of computer), but all share basic notions Humans have receptors for (a small part of) the electromagnetic spectrum Have receptors sensitive to (fire when excited by) energy 400-700nm Snakes “see” infrared, some insects ultraviolet i.e., have receptors that fire What would life be like if humans could see other parts of electromagnetic spectrum???

Human Eye: Retinal Receptors Two types of (photo) receptors on retina: rods and cones Rods look like, well, rods … Will later look at color blindness … when cones fail Rods: spread all over the retinal surface (75 - 150 million) low resolution, no color vision, but very sensitive to low light (scotopic or dim-light vision) Cones: dense array around central portion of retina - fovea centralis (6 - 7 million) high-resolution, color vision, but require brighter light (photopic or bright-light vision)

Human Eye: Lens Eye has compound lens: cornea (power) and lens (adjust focal length) f = focal length of lens d = distance to object r = distance to image that is formed Flexibility of lens changes with age, approaching 0 at 60 years

Human Eye: Depth of Field and Focus Depth of focus – will see interesting affect … Distance over which objects are “in focus” without change in lens (focus) Note: Different colors have different depths of focus Varies with size of pupil Range of focus: Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large Rarely do computer systems model either depth of field or depth of focus

Human Eye: Depth of Field - Example Photographic Images: Depth of field longer with small aperture (f stop) Range of focus: Distance Near Far Depth of focus 50 cm 43 cm 60 cm 17 cm 1 m 75 cm 1.5 m 75 cm 2 m 1.2m 6.0m 1.8 m 3 m 1.5m Infinity Large

Human Eye: Chromatic Aberration

Human Eye: Chromatic Aberration Different wavelengths of light focus at different distances within eye Short-wavelength blue light refracted more than long-wavelength red light Focusing on a red patch, an adjacent blue patch will be significantly out of focus Strong illusory depth effects Human eye has no correction for chromatic aberration! Inadvisable: fine blue patterns in visualizations! Visual effects in soap bubbles, crystal sculptures, etc.

Using Physiology for Design Chromatic Aberration Different wavelengths focus differently Highly separated wavelengths (red & blue) can’t be focused simultaneously Guideline: Don’t use red-on-blue text It looks fuzzy and hurts to read

FYI: Human Eye: Receptors Lens focuses image on mosaic of photoreceptor cells lining retina From 1 – 100 receptors feed into 1 ganglion cell Fovea Small area in center of retina densely packed with cones Vision sharpest ½o - 2o of arc Receptor mosaic in fovea

Interpreting the Visual Signal Size and depth Visual angle indicates how much of view object occupies (relates to size and distance from eye) Visual acuity is ability to perceive detail (limited) familiar objects perceived as constant size (in spite of changes in visual angle when far away) cues like overlapping help perception of size and depth Brightness Human system not map directly to physical Color Different sensory elements responsible for perception of color Implications for color use, e.g., color blindness

Human Eye: Visual Angle Visual angle - angle subtended by object at eye of viewer In degrees, minutes, seconds of arc Thumbnail at arm’s length subtends ~1o of visual angle 1cm (2/5”) object at 57 cm (20”), monitor distance, ~1o of visual angle

FYI: Human Eye: Acuities - Grating Acuity Measurements of abilities to see detail Provide ultimate limits of information densities can perceive Resolve to about 1 min. of arc Roughly corresponds to with receptor spacing in fovea E.g., to see 2 lines as distinct blank space between should lie on receptor So, should be able to perceive lines separated by twice receptor spacing! Superacuities Resolution above what expected by receptor density due to integration of signals Grating Acuity - 1-2 minutes of arc 1 minute = 60 seconds - Ability to distinguish a pattern of bright and dark bars from a uniform gray background

FYI: Acuities: Point, Letter, Stereo, Vernier Point acuity 1 minute of arc Ability to resolve two distinct point targets Letter acuity 5 minute of arc Ability to resolve letters Snellen eye chart 20/20 means a 5-minute letter target can be seen 90% of time Stereo acuity 10 seconds of arc Ability to resolve objects in depth Measured as difference between 2 angles (a and b) for a just-detectable difference Vernier acuity Ability to see if two lines are collinear

FYI: Acuity Distribution and Visual Field Again, receptors densely packed at fovea Binocular overlap Region of visual field viewed by both eyes Only here (not in periphery) stereopsis Visual acuity non-uniformly distributed over visual field E.g., Can resolve only about 1/5 detail at 10o Next slide (tries) demonstrate “equi-resolvability” of characters as a function of distance from fovea r = f(dist. fovea) (stare at center and see smaller characters at center as well or better than those at periphery)

Human Eye: Resolution

FYI: Brightness, Luminance, Lightness subjective reaction to levels of light affected by luminance of object measured by just noticeable difference Ecologically, need to be able to manipulate objects in environment Information about quantity of light, of relatively little use Rather, what need to know about its use Human visual system evolved to extract surface properties Loose information about quantity and quality of light E.g., experience colored objects, not color light Color constancy Similarly, overall reflectance of a surface Lightness constancy

FYI: Luminance, Brightness, Lightness Amount of light (energy) coming from region of space, Measured as units energy / unit area E.g., foot-candles / square ft, candelas / square m Physical Brightness Perceived amount of light coming from a source Here, will refer to things perceived as self-luminous Lightness Perceived reflectance of a surface E.g., white surface is light, black surface is dark Again, Physical - Luminance Number of photons coming from a region of space Perceptual - Brightness Amount of light coming from a glowing source Reflectance of a surface, paint shade

FYI: Luminance Amount of light (energy) hitting the eye To take into account human observer: Weighted by the sensitivity of the photoreceptors to each wavelength Spectral sensitivity function: E.g., humans about 100 times less sensitive to light at 450nm than at 510nm Note, use of blue for detail, e.g., text, not seem good Compounded by chromatic aberration in which blue focuses at different point Later, will examine difference cone sensitivities

FYI: Brightness Perceived amount of light coming from a glowing (self-luminous) object E.g., instruments Perceived brightness very non-linear function of the amount of light Shine a light of some intensity on a surface, and ask an observer, “How bright?” Intensity = How bright is the point?” 1 1 4 2 16 4 Stevens power law Perceived sensation, S, is proportional to the stimulus intensity, I, raised to a power, n S = I n Here, Brightness = Luminancen With n = 0.333 for patches of light, 0.5 for points Applies only to lights in relative isolation in dark, so application more complicated Applies to many other perceptual channels Loudness (dB), smell, taste, heaviness, force, friction, touch, etc. Enables high sensitivity at low levels without saturation at high levels Just-noticeable difference depends on value

Dix: Interpreting the Signal - Color made up of hue, intensity, saturation cones sensitive to colour wavelengths blue acuity is lowest ~8% males and ~1% females color blind

Trichromacy Theory Recall, that there are 2 types of retinal receptors: Rods, low light, monochrome So overstimulated at all but low levels contribute little So only consider cones for color vision Cones, high light, color Not evenly distributed on retina Distribution of receptors across the retina, left eye shown; the cones are concentrated in the fovea, which is ringed by a dense concentration of rods http://www.handprint.com/HP/WCL/color1.html#oppmodel

Trichromacy Theory Cones (3 types) differentially sensitive to wavelengths “trichromacy” Each type cone has different peak sensitivity: S: 450 nm “blue” M: 540 nm “green” L: 580 nm “red” More later No accident 3 colors in monitor Color space: An arrangement of colors in a 3-dimensional space Monitor: R,G,B Primary paint colors: R,Y,B Printer: cyan, magenta, yellow There are many, each designed for different purposes Will consider several Can match all colors perceived with 3 colors Does not matter that spectral composition of that patch of light may be completely different But, chickens have 12 … Different gamut, more later Cone sensitivity functions Cone response space, defined by response of each of the three cone types. Becomes 2d with color deficiency

FYI: Color Space Ex.: RGB Color Cube Again, can specify color with 3 Will see other way RGB Color Cube Neutral Gradient Line Edge of Saturated Hues ppt example http://graphics.csail.mit.edu/classes/6.837/F01/Lecture02/ http://www.photo.net/photo/edscott/vis00020.htm

Cone Sensitivity Functions Cone receptors least sensitive to (least output for) to blue Relative sensitivity curves for the three types of cones, log vertical scale, cone spectral curves from Vos & Walraven, 1974 Relative sensitivity curves for the three types of cones, the Vos & Walraven curves on a normal vertical scale

Color Blindness ~9% male, and ~1% females have some form of color vision deficiency! Most common: Lack of long wave length sensitive receptors (red, protanopia) See figure at right bottome Lack of mid wave length receptors (green, deuteranopia) Results in inability to distinguish red and green E.g., cherries in 1st figure hard to see Trichromatic vs. dichromatic vision

Color Blindness Examples Normal: trichromatic No red, green, blue: dichromatic

Using Physiology for Design Fovea has few blue (S - short wavelength) cones Can’t resolve small blue features (unless they have high contrast with background) Lens and aqueous humor turn yellow with age Blue wavelengths are filtered out Lens weakens with age Blue is harder to focus Guideline: don’t use blue against dark backgrounds where small details matter (like text!)

FYI: Perceived Color Color perceived relative to context Are the “X”s in the figure below the same color? Easy implications for use in maps Contrast illusion An illusion is an extreme case Somewhat “surprising” because it leads to error Appears to be different color X!

FYI: Perceived Color With color of x touching …

FYI: Afterimage Occurs due to bleaching of photopigments (demo next slide) Implications for misperceiving (especially contiguous colors – and black and white) “I thought I saw …” To illustrate: Stare at + sign on left May see colors around circle Move gaze to right See yellow and desaturated red

Afterimage Example

Dix: Interpreting the signal (cont) The visual system compensates for: movement changes in luminance. Context is used to resolve ambiguity Optical illusions sometimes occur due to over compensation

FYI: Simultaneous Brightness Contrast Gray patch on a dark background looks lighter than the same patch on a light background Predicted by DOG model of concentric opponent receptive fields

FYI: Mach Bands Ernst Mach At point where uniform area meets a luminance ramp, bright band is perceived Said another way, appear where abrupt change in first derivative of brightness profile Simulated by DOG model Particularly a problem for uniformly shaded polygons in computer graphics Hence, various methods of smoothing are applied

Psychology and Design of Interactive Systems Some direct applications e.g. blue acuity is poor  blue should not be used for important detail However, correct application generally requires understanding of context in psychology, and an understanding of particular experimental conditions A lot of knowledge has been distilled in guidelines cognitive models experimental and analytic evaluation techniques

End ?

Optical Illusions the Ponzo illusion the Mueller-Lyer illusion

Optical Illusions

Preattentive Processing and Illusions What is wrong with this triangle? Impossible (or at least difficult) to build Cues for perception misleading Must rely on conscious (rational) processes intelligence to figure it out, Conscious/rational processes much slower

The Other Senses … But not smell and taste Hearing Touch Long latency, practical challenges Hearing Touch Kinesthetic – Movement Fitt’s law

Hearing Provides information about environment: distances, directions, objects etc. Physical apparatus: outer ear – protects inner and amplifies sound middle ear – transmits sound waves as vibrations to inner ear inner ear – chemical transmitters are released and cause impulses in auditory nerve Sound pitch – sound frequency loudness – amplitude timbre – type or quality

Hearing (cont) Humans can hear frequencies from 20Hz to 15kHz less accurate distinguishing high frequencies than low. Auditory system filters sounds can attend to sounds over background noise. for example, the cocktail party phenomenon.

Touch Provides important feedback about environment May be key sense for someone who is visually impaired Stimulus received via receptors in the skin thermoreceptors – heat and cold nociceptors – pain mechanoreceptors – pressure (some instant, some continuous) Some areas more sensitive than others, e.g., fingers Kinethesis - awareness of body position affects comfort and performance.

Movement Time taken to respond to stimulus: reaction time + movement time Movement time dependent on age, fitness etc. Reaction time - dependent on stimulus type: visual ~ 200ms auditory ~ 150 ms pain ~ 700ms Increasing reaction time decreases accuracy in the unskilled operator but not in the skilled operator.

Movement – Fitt’s Law Fitts' Law, 1954 Demo: http://www.tele-actor.net/fitts/

Movement – Fitt’s Law Fitts' Law, 1954 Mt = a + b log2(D/S + 1) Fundamental law of human sensory-motor system Practical application in interface design Describes the time taken to hit a screen target Time, Mt, to move hand to a target of size, S, at distance, D, away Mt = a + b log2(D/S + 1) where: a and b are empirically determined constants Mt is movement time D is Distance S is Size of target “index of difficulty”: log2(D/S + 1) Same performance at greater distance with greater size targets as large as possible, distances as small as possible

Fitt’s Law – Example Implication Hierarchical menus are hard to hit Especially when it takes two actions …

Psychology and Design of Interactive Systems Some direct applications e.g. blue acuity is poor  blue should not be used for important detail However, correct application generally requires understanding of context in psychology, and an understanding of particular experimental conditions A lot of knowledge has been distilled in guidelines cognitive models experimental and analytic evaluation techniques

End