1. Visual Perception and Cognition

2. Roadmap Visual Perception: how it works
Fun with Visual Perception: Party Tricks and other Amusements
Impact on Visualization: What do we know about visual perception that can inform our visualization practice?
Effects due to visual mechanism that we can use to our advantage
General Guidelines to follow (Bertin, XXX, )
Design Examples

3. Visual Perception What is visual perception?
The process of knowing or being aware of information through the eyes.
The process of acquiring, interpreting, selecting, and organizing sensory information.

4. Related Disciplines
Psychophysics: Applying methods of physics to measuring human perceptual systems.
How fast must light flicker until we perceive it as constant?
What change in brightness can we perceive?
Cognitive psychology: Understanding how people think. In this context, how it relates to perception.

5. Visual System Light path Retinal cells Cones Rods
Slide adapted from Stone & Zellweger
Light path
Cornea, pupil, lens, retina
Optic nerve, brain
Retinal cells
Rods and cones
Unevenly distributed
Cones
Three “color receptors”
Concentrated in fovea
Rods
Low-light receptor
Peripheral vision
Cones for color, rods for night vision
-- See other powerpoint slide for eye choices
Want better figure of eye.
From Gray’s Anatomy

6. Steven’s Power Laws p < 1 : underestimate p > 1 : overestimate
JT: cropped as image
See original below
p < 1 : underestimate
p > 1 : overestimate
Stevens, On the Theory of Scales of Measurement, Science 103:2684, 1946

7. Pictures vs. Eyes Pictures
Generally produced by purpose built cameras, or computer applications, or drawings.
For instance a camera has good optics, focus, white balance exposure controls. It captures a large image at constant high quality resolution (spatial, luminance, hue).
Eyes
Heuristics developed by evolution using inexpensive biophysical hardware to keep operator alive.
Human visual system is different in that it has relatively poor optics. It is constantly scanning, adjusting focus, white balance, exposure. It captures detail only in foveal area and very coarsely in the peripheral vision areas. We produce a 3D spatial reconstruction of our world based on our 2D snapshots.

8. One Simple Model of Perceptual Processing
Three stage process
Parallel extraction of low-level properties of scene
Pattern perception
Sequential goal-directed processing
Stage 1
Stage 2
Stage 3
Put in graphic from slide 7 (bottom) HERE
Early, parallel
detection of color, texture, shape, spatial attributes
Holding objects in working memory by
demands of active attention
Dividing visual field into regions
and simple
patterns

9. Stage 1 - Low-level, Parallel
Neurons in eye & brain responsible for different kinds of information (orientation, color, texture, movement, etc.)
Arrays of neurons work in parallel
Occurs “automatically”
Rapid
Information is transitory, briefly held in iconic store
Bottom-up data-driven model of processing
Often called “pre-attentive” processing

10. Pattern Perception Slow serial processing
Involves working and long-term memory
A combination of bottom-up feature processing and top-down attentional mechanisms
Different visual systems for object recognition and visually guided motion
Maybe make a tree here with the two subsystems rather than bullet points?

11. Stage 3 – Sequential Goal-Directed (Attentive)
• Splits into subsystems for object recognition and for interacting with environment • Increasing evidence supports independence of systems for symbolic object manipulation and for locomotion & action • First subsystem then interfaces to verbal linguistic portion of brain, second interfaces to motor systems that control muscle movements

12. Stage 3 – Attributes Top-down attention-driven model of processing
Slow serial processing
Involves working and long term memory
A few objects are constructed from the available patterns to provide answers to visual queries

13. Visual Working Memory different from verbal working memory
low capacity (3-5?)
locations egocentric
controlled by attention
time to change attention: 100 ms
time to get gist: 100 ms
not fed automatically to long term memory
JT: Moved here after Stage 3 discussion

14. Preattentive Processing
How does human visual system analyze images?
Some things seem to be done preattentively, without the need for focused attention
Generally take less than msecs (eye movements take 200 msecs)
Seems to be done in parallel by low-level vision system
An important contribution vision science makes to data visualization is that a limited set of visual properties can be detected very rapidly and accurately by the low-level visual system

15. How many 3’s
Watch this space and tell me how many 3’s you see… (1 second)

16. How many 3’s

17. How many 3’s
Answer? Ready to try again?

18. How many 3’s

19. How many 3’s
What about now? Do you have the correct answer? So, pop-outs (pre-attentive selection) helps us with this task. What else could help us with this task? Can you think of other types of tasks that pre-attentive recognition could help us with?

20. Fun with Visual Perception: Party Tricks and other Amusements
Our visual system because it is built from the ground up, and has evolved over millions of years, contains many “artifacts”.
These make for good party tricks
And they help inform us about ways in which we can take advantage of the vision system’s mechanics to help with visual understanding (especially quick recognition).
Also see website of optical illusions:

23. Stroop Effect

24. Stroop Effect
Read the words on the following slide as quickly as you can.
Ready?

25. OAK WATER FLOWER BOAT HOUSE TREE STONE PEANUT HORSE RADIO FOLDER
STREET
FLY
QUAKE
STORM
GRASS
ZIP
JT: Got coloring slide 25

26. Stroop Effect
OK. Let’s do it again. Read the words on the next slide as quickly as you can.
Ready?

27. RED BLACK YELLOW GREEN BLUE
JT: Got coloring from slide 25

28. Did you notice a difference?
Interference from engaging higher level processing. Actually you can probably do again at similar speed if you focus just on written word (left brain) and no connotations (right brain).
A Whole New Mind: Why Right-Brainers Will Rule the Future, Daniel Pink

29. Examples from Healey

30. Differing Amounts of “pop-out”
Task: for the next two slides, you will see two squares. Your job is to see how quickly you can pick out the square containing the red circle.

32. Make the rest of these like slide 42 in size, layout.

33. Ready for next one?

35. Good Job .
For the next slides your job is to pick out the two groupings. Each square contains two groups (which have common features). The two groups can be separated by a line splitting the square into two rectangles.
Raise your hand when you believe you know the answer.

36. Arrange this in nice layout, as big as possible.

42. So, you were likely best at this task when groups differed by both color and shape.
Still not bad when differed by just one attribute (color or shape—which was easier because it a better pre-attentive cue?).
Not so good when difference was mixture of color and shape (i.e. red squares and blue circles versus blue squares and red circles).

43. What you can’t see…
Can’t embed, click image to hyperlink to play online

44. Can’t embed, click image to hyperlink to play online

45. Best Resource Award!
A great resource for looking at many of these visual effects AND learning about the science behind them is Christopher Healey’s (NCSU) webpage on Perception in Visualization.
Let’s take a brief look. Be sure to try the java applets.

46. Other Good Resources
Michael Bach website

47. Impact on Visualization
We knows things from carefully controlled studies (not so many of these), and lots of practical empirical experience.
First, an example of controlled study to evaluate which visual elements were better for gauging magnitude (how much of something)

48. Controlled Study: Position vs Area/Angle
Tried to recreate in excel, but couldn’t get the pie chart to go in the right order. Easier to cut and copy from paint.
Figure 3. Graphs from position-angle experiment.

49. 5 ways to compare 2 magnitudes
Estimate the relative magnitude of the pair of bars marked with •
Cut and paste form paint
W.S. Cleveland R. McGill, Graphical perception,
JASA 39, pp , 1984

50. Results: Error in Judgements
Cut and paste from paint

51. How can we make use of this?
Best way is position or adjacent bar chart
How can you do this if you want to compare different values on same chart.
ADD EXAMPLE of dynamic stacked bar charts
Best practices with bar charts (we will cover later in charts module)

52. Potential Preattentive Features
hue
intensity
flicker
direction of motion
binocular lustre
stereoscopic depth
3-D depth cues
lighting direction
length
width
size
curvature
number
terminators
intersection
closure

53. Examples of pre-attentive tasks
Target detection
Is something there?
• Boundary detection
− Can the elements be grouped?
• Counting
− How many elements of a certain type are present?
Maybe add small visuals for each of these

54. Key Perceptual Properties
Texture 3D
Motion
Shape
Groupings
Spatial
Multiresolution (zoom)
Brightness/Luminance
Color
Maybe add small visuals here? Will look for examples

55. Texture
from: Jürgen Döllner
Pasted these in here
from: Colin Ware

56. Pre-Attentive Cues With Textures
• A visual texture represents that visual sensation that allows us to pre-attentively differentiate two adjacent, possibly structured parts in our visual field without eye movement
visual textures include micro-structures, patterns, profiles, etc.
to identify textures, an observations of about ms is sufficient (cognitively controlled processes require about ms)
Classification of textures is based on
coarseness, contrast, directionality (orientation), scale, line-likeness, regularity, roughness
Textures improve perception of position and orientation
Texture communicate information about the 3D structure regardless of their coloring
Slide
Slide adapted from Klaus Mueller/ Jürgen Döllner

57. Pre-Attentive Cues With Textures
Same surface with and without texture
Textures that do not include information are to be avoided in visualization
recall Tufte’s aesthetic principle that irrelevant decoration (= chart junk) should be avoided
Subtle textures for 3D visualizations, however, can be important elements of visual design
Slide
Slide adapted from Klaus Mueller/ Colin Ware

58. Texture Perception Textons
Objects
Textons
fundamental micro-structures in generic natural images
-basic elements in pre-attentive visual perception
Textons can be classified into three general categories:
1. elongated blobs (line segments, rectangles, ellipses) with specific properties such as hue, orientation, and width, at different level of scales
2. terminators (end of line segments)
3. crossings of line segments
Julesz believes that only a difference in textons or in their density can be detected pre-attentively
- no positional information about neighboring textons is available without focused attention
- pre-attentive processing occurs in parallel (fast!)
- focused attention occurs in serial (slower!)
Example: Look at two objects on right
although the two objects look very different in isolation
they are actually the same texton
Their textons
Gabor
Primitves
Slide adapted from Klaus Mueller/ Jürgen Döllner

59. Relation to Symbol and Texture Design
When designing textures to indicate different regions of a visualization, make sure that the textons are as different as possible
The same rules apply when designing symbol sets
Example: A tactical map may require the following symbols:
aircraft targets
tank targets
building targets
infantry position targets
Each of these target types can be classified as friendly or hostile
Targets exist whose presence is suspected but not confirmed (this uncertainty must be encoded)
Set of symbols designed to represent different classes of objects
- symbols should be as distinct as possible with respect to their pre- attentive processing
- recall: military reconnaissance must occur FAST!
Slide adapted from Klaus Mueller/ Jürgen Döllner

60. Textron use Textron on military type map Textron for fluid/air flows
ADD BETTER EXAMPLES!!!

61. Information Display in 3D: Depth Cues
3D display should provide depth cues
Linear perspective:
more distant objects become smaller in the image
can indicate focus, importance, or ordering
- elements of a uniform texture become smaller with distance
can give shape cues
Shadows:
show the relative height of objects above a surface
provide strong depth cues for objects in motion
can be semi-realistic and still work as a depth cue
Occlusion:
- very powerful depth cue
Slide adapted from Klaus Mueller/ Colin Ware

62. Information Display in 3D: Depth Cues
Shading:
- shape cues from shading (shape-from-shading)
diffuse
+ specular
+ shadows
shape from shading
(hole vs. hill)
specular can reveal fine detail
assume single light source
having more than one light
source can lead to confusion
Slide adapted from Klaus Mueller/ Colin Ware

63. Information Display in 3D: Depth Cues
Other depth cues:
depth of focus
motion parallax (structure from motion) --> how objects relate under motion (see next slide for examples)
steroscopic depth (binocular displays)
For fine-scale judgement, for example, threading a needle:
stereo is important, and shadows and occlusion
For large-scale judgement
linear perspective, motion parallax, and perspective are important
stereo is not so important
However, for information visualization
displays, one may exploit focus
to emphasize importance,
despite depth relationships
Slide adapted from Klaus Mueller/ Jürgen

64. Motion 3D Structure from Motion
Simple example of 3D structure from motion animation
Second example with user control
YouTube video embedded. Bach visual hyperlinked to site.
There is lots more to motion, but we won’t cover.

65. Shape, Symbol Symbols should be rapidly perceived and differentiated
Application for maps, military, etc.
Add small visuals here?

66. Uses of Symbols See Bertin’s work.
(Generally we won’t cover this as separate section due to time constraints)

67. Multiscale Resolution
Images at coarse scale can still be recognized. See Dali’s painting (different at different viewing distances).
Shark Picture. (look closely). From artist Chris Jordan (select running the numbers II, scroll down to hammerhead shark).

68. Luminance/Brightness
Measured amount of light coming from some place
Brightness
Perceived amount of light coming from source

69. Human Perception of Color

70. Perceptual Color Models
What is the gamut the human eye can see? CIE standard based on experimental measures
See next slide for power law scales

71. CIE updated: CIELAB (print) CIELUV (display)

72. Color Models RGB (based on technology model)
HSB (HVS) model (based on perceptual color model)
Hue - what people think of color
Saturation - intensity, whiteness
Brightness (Value) - light/dark

73. Contrast Important for fg-bg colors to differ in brightness
Hello, here is some text. Can you read what it says?
Hello, here is some text. Can you read what it says?
Hello, here is some text. Can you read what it says?
Hello, here is some text. Can you read what it says?
Hello, here is some text. Can you read what it says?
Hello, here is some text. Can you read what it says?

74. Color Background Contrast
Focus on the black circle for a few seconds, then switch to one of the white fields. For a moment, you will see a white circle (the complementary color to the black circle). This is due to the local change of the adaption level.
Another example (used in Florida Recount joke).
From: Colin Ware

75. Successive Contrast Surround matters, especially for colors
Afterburn effect:
focus on colored panels
then switch to white panel
do this for saturated and non-saturated backgrounds
Color contrast effects
use saturated colors sparingly, they may cause undesired effects
neutral borders can help
From Klaus Mueller

76. Chromatic Color is Irrelevant…
To perceiving object shapes
To perceiving layout of objects in space
To perceiving how objects are moving
Therefore, to much of modern life
Many people go much of their life without realizing they are color blind.

77. Color is Critical… To help us break camouflage
To judge the condition of objects (food)
To determine material types
Extremely useful for coding information
Some objects differ from their surroundings primarily by color
Food: Is this fruit ripe? Is this meat putrid? What kind of mushroom is this?
Material: What kind of stone is this (for making an axe or knife)?

78. Implications Color perception is relative
We are sensitive to small differences- hence need sixteen million colors when driving computer displays
Not sensitive to absolute values- hence we can only use less than 10 or 12 colors for coding
Brad’s view
Can use successfully for labeling small number of categorical data (12 or less).
Can in certain constrained cases be utilized as continuous scale (isoluminance example). But must be careful here; often not used properly in this case.

79. Spatial Map labeling example

80. Nautical Maps What are goals? Contrast these two:
What visualization choices were made?

81. Electronic Chart Example
Color in two ways:
Semantic Labels: Blue vs Brown is used to indicate region type (water, land)
Within these two colors (interval) shades of blue to indicate depth of water and shades on brown to indicate structures on land.

82. Applications Color interfaces Color coding/labeling (for nominal data)
Color sequences for ordinal data
Continuous Color for continuous data
Call attention to specific data
Increase appeal, memorability
Increase number of dimensions for encoding data

83. Color Categories Are there certain canonical colors?
Post & Greene ‘86 had people name different colors on a monitor
Pictured are ones with > 75% commonality
Black version: Colin Ware, Color Version: Post & Greene ‘86

84. Gender Differences for Color?
Number of colors men perceive vs women?
No difference (millions)
Number of colors used in describing objects? How fast one call recall name (label) of color?
Women are superior (more, faster) than men (Arthur, H., Johnson, G., & Young, A. (2007). Gender differences and color: Content and emotion of written descriptions. Social Behavior and Personality, 35(6), doi: /sbp ).

85. Color for Categories
Can different colors be used for categories of nominal variables?
Yes
Ware’s suggestion: 12 colors
red, green, yellow, blue, black, white, pink, cyan, gray, orange, brown, purple
Added Ware’s color palette here

86. Ordinal data Principles
Order: ordered values should be represented by perceptually-ordered colors
Separation: significantly different levels should be represented by distinguishable colors
Many good choices can be derived from “strips” through perceptually uniform color spaces (grey scale, heated object scale, etc).
Luminance: good for showing form
Many hues: useful for showing readable values

87. Interval Sequences: Contour Lines and Color
Both can indicate regions
Contour by showing the boundaries
Color to distinguish different objects/groups.
Choice of spacing
Regular intervals to enable interval comparison
Specific values to highlight regions (sea level)
Can attempt to use Uniform Color Space

88. Color for Sequences (continuous)
Can you order these (low->hi)

89. Possible Color Sequences
Gray scale
Full spectral scale
Single sequence
part spectral scale
Single sequence
single hue scale
Double-ended
multiple hue scale

90. Continuous Uses: Color Scales
Examples of two window and levels of same data (lung, soft tissue). Missing are psuedo, hue Color scales. Applied to medical images (from Hemminger Radiology work).

91. Color for Continuous Variables
Is done, but beware this is tricky, must be careful
For example see Ware, C., & Beatty, J. (1988). Using color dimensions to display data dimensions. Human Factors, 30(2),

92. Color as Scale + Luminance
Isoluminance Color Display (Hemminger).
Studies use of color vs luminance for showing two dimensions of finding (anatomical structure versus cancer signal) on map.
What’s obvious shortcoming with this visualization? (story)

93. Here’s VisCheck on my Isolum image

94. Colorblind and Color Scales
Vischeck by Bob Dougherty and Alex Wade. Allows you to simulate what a colorblind person would see when viewing your image or website.
Daltonize (part of vischeck) will correct (suggest) alternative color scheme so colorblind person can appreciate detail in image (switch Red/Green use to other hues).

95. Choosing Color Scales for Interfaces
Types of color scales
Continuous/Interval Data use continuous color scale (mono/divergent)
Categorical Data use labeling/qualitative color scale

96. Color Scales
Color Brewer Tool developed by Cynthia Brewer for GIS maps, is very helpful for picking safe continuous/discrete scales, and the number of colors in scales. She avoids uses saturated colors, and developed diverging color scales. Safe means OK for
Colorblind folks
print on different devices
photocopying

97. Color Schemes
Color Scheme Designer by Petr Stanicek. Allows you to easily visualize and modify/compare color scheme choices for several common (designer recommended) color palettes for interfaces
Mono
Complement
Triad
Analogic
Accented analogic

98. Generally don’t use complementary colors (disturbing to human visual system, hard to look at for a long time). Example: RG protein (microarray type colormaps, use high contrast (bad) and direct complementary colors (red/green) bad. Also use Blue/Yellow, but with similar problem. Theresa Maria Rhyne designed an alternative one. Solution was green/purple, which results in reasonable color blindness results for all three color blindness conditions (different but OK).
If you want contrast, use nearly complementary, or accented analogic.

99. Tough to get 10 different distinguishable color labeling palette, that are equally easily distinguished. This is why Brewer tool and others don’t go above this for categorical data. Some folks try to extend by varying something else (like value/luminance) but this complicates by suggesting categories which are not present, so probably not appropriate to the task.
A field guide to digital color by Maureen Stone. Great coverage, especially lots of information on color calibration for digital displays.
The interaction of Color: Josef Albers

100. Color Themes
Adobe’s Kuler tool. Social media tool allowing you to save and share your color theme schemes, and to search and see others as well.
Saves to your personal library (mykuler)
Explores/searches yours/worlds themes
Create theme from colormap or from your image

101. Color Scheme Takeaways
Consider CUT-DDV framework when choosing color scheme. Make conscious choice of type, number of levels, color, transparency, etc.
Use freely available tools to help design and test colors for your visualization; try several alternatives schemes
Check it for safe use

102. Bivariate Maps
Rows and columns: to preserve univariate information, display parameters should should be perceptually orthogonal (ideally)
Display map data with more than one variable per map location is challenging.
First do we have to use color, or contours to indicate regions (states, counties)?
What visual types do we have left to show the variables present at each map location?

103. Hue Saturation Bivariate Map
Tufte ‘83, pg. 153.

104. How easy to perceive separate channels of information?
Do we think this works?
How easy to perceive separate channels of information?
Population density
Change in population

106. Absolute number of disability, does this inform
Absolute number of disability, does this inform? If number is proportional to population is it interesting?
How well does the separate symbol (wheelchair) which varies in size to show quantity, work?
How well does the combination of this symbol (wheelchair) work in conjunction with quantitative variable for number of people with disability?

108. How effective is the color for showing distance to the Mississippi river?
Could we leave it out? Show river with more emphasis and depend on map information to visually show correlation to location of river?

109. Consider Data Nominal, ordinal, interval or ratio
Map or entity based coding
Amount of detail
Rate of change, i.e. is high spatial frequency present in a variable? If so may require different choice (think West Nile Virus example with widely varying rates all across the US).

110. Consider Audience (User)
Color deficient viewers?
Don’t depend on red-green differentiation
Use redundant scales
Application area conventions?
Use familiar scales (or at least know when you’re not)
Color associations with variables?
Use associated color
Color associations with data ranges?
Use red for bad range (in U.S.)
Use red for hot

111. Take home messages Use luminance for detail, shape and form
Use color for coding - few colors
Minimize contrast effects
Strong colors for small areas - contrast in luminance with background
Subtle colors can be used to segment large areas

112. More messages Color excellent for multi-dimensional data
Use additional tools to get quantities
Beware of artifacts due to color re-sampling
We need precise color for specification and standardization

114. What’s the best way to visualize this information?
Would map-based be better?
Can I easily pick out individual curves by color? (more than 10-12)
How about height/chart figure for values?
What about a data table?
What are the questions we’re trying to answer?

115. Influence on Visualization
Why we care
Exploit strengths, avoid weaknesses
Optimize, not interfere
Design criteria
Effectiveness
Expressiveness
No false messages

116. Design criteria: Effectiveness
Faster to interpret
More distinctions
Fewer errors 1 2 3 4 5 6 7
This?
Or this?

117. Sensory vs. Arbitrary Symbols
Understanding without training
Resistance to instructional bias
Sensory immediacy
Hard-wired and fast
Cross-cultural Validity
Arbitrary
Hard to learn
Easy to forget
Embedded in culture and applications

118. Which Properties are Appropriate for Which Information Types
Which Properties are Appropriate for Which Information Types? Final Take Aways

119. Channel Properties Luminance Channel Chromatic Channels Detail Form
Shading
Motion
Stereo
Surfaces of things
Labels
Categories,Berlin and Kay (about 6-10)
Red, green, yellow and blue are special (unique hues)

120. Rankings: Encoding quantitative data
Cleveland & McGill 1984, adapted from Spence 2006

121. Visual Property vs Data Type
Stephen Few’s Table:
Attribute
Quantitative
Qualitative
Line length
2-D position
Orientation
Line width
Size
Shape
Curvature
Added marks
Enclosure
Hue
Intensity

122. Interpretations of Visual Properties
Some properties can be discriminated more accurately but don’t have intrinsic meaning. (Senay & Ingatious 97, Kosslyn, others)
If there are intrinsic meanings use them (positive advantage and avoid confusion). And be aware when you use them that you are not sending conflicting messages.
Density (Greyscale)
Darker -> More
Size / Length / Area
Larger -> More
Position
Leftmost -> first, Topmost -> first (culture/language specific)
Hue
Culture specific meanings
Slope
no intrinsic meaning ???