2Roadmap Visual Perception: how it works Fun with Visual Perception: Party Tricks and other AmusementsImpact on Visualization: What do we know about visual perception that can inform our visualization practice?Effects due to visual mechanism that we can use to our advantageGeneral Guidelines to follow (Bertin, XXX, )Design Examples
3Visual Perception What is visual perception? The process of knowing or being aware of information through the eyes.The process of acquiring, interpreting, selecting, and organizing sensory information.
4Related DisciplinesPsychophysics: Applying methods of physics to measuring human perceptual systems.How fast must light flicker until we perceive it as constant?What change in brightness can we perceive?Cognitive psychology: Understanding how people think. In this context, how it relates to perception.
5Visual System Light path Retinal cells Cones Rods Slide adapted from Stone & ZellwegerLight pathCornea, pupil, lens, retinaOptic nerve, brainRetinal cellsRods and conesUnevenly distributedConesThree “color receptors”Concentrated in foveaRodsLow-light receptorPeripheral visionCones for color, rods for night vision-- See other powerpoint slide for eye choicesWant better figure of eye.From Gray’s Anatomy
6Steven’s Power Laws p < 1 : underestimate p > 1 : overestimate JT: cropped as imageSee original belowp < 1 : underestimatep > 1 : overestimateStevens, On the Theory of Scales of Measurement, Science 103:2684, 1946
7Pictures vs. Eyes Pictures Generally produced by purpose built cameras, or computer applications, or drawings.For instance a camera has good optics, focus, white balance exposure controls. It captures a large image at constant high quality resolution (spatial, luminance, hue).EyesHeuristics developed by evolution using inexpensive biophysical hardware to keep operator alive.Human visual system is different in that it has relatively poor optics. It is constantly scanning, adjusting focus, white balance, exposure. It captures detail only in foveal area and very coarsely in the peripheral vision areas. We produce a 3D spatial reconstruction of our world based on our 2D snapshots.
8One Simple Model of Perceptual Processing Three stage processParallel extraction of low-level properties of scenePattern perceptionSequential goal-directed processingStage 1Stage 2Stage 3Put in graphic from slide 7 (bottom) HEREEarly, paralleldetection of color, texture, shape, spatial attributesHolding objects in working memory bydemands of active attentionDividing visual field into regionsand simplepatterns
9Stage 1 - Low-level, Parallel Neurons in eye & brain responsible for different kinds of information (orientation, color, texture, movement, etc.)Arrays of neurons work in parallelOccurs “automatically”RapidInformation is transitory, briefly held in iconic storeBottom-up data-driven model of processingOften called “pre-attentive” processing
10Pattern Perception Slow serial processing Involves working and long-term memoryA combination of bottom-up feature processing and top-down attentional mechanismsDifferent visual systems for object recognition and visually guided motionMaybe make a tree here with the two subsystems rather than bullet points?
11Stage 3 – Sequential Goal-Directed (Attentive) • Splits into subsystems for object recognition and for interacting with environment • Increasing evidence supports independence of systems for symbolic object manipulation and for locomotion & action • First subsystem then interfaces to verbal linguistic portion of brain, second interfaces to motor systems that control muscle movements
12Stage 3 – Attributes Top-down attention-driven model of processing Slow serial processingInvolves working and long term memoryA few objects are constructed from the available patterns to provide answers to visual queries
13Visual Working Memory different from verbal working memory low capacity (3-5?)locations egocentriccontrolled by attentiontime to change attention: 100 mstime to get gist: 100 msnot fed automatically to long term memoryJT: Moved here after Stage 3 discussion
14Preattentive Processing How does human visual system analyze images?Some things seem to be done preattentively, without the need for focused attentionGenerally take less than msecs (eye movements take 200 msecs)Seems to be done in parallel by low-level vision systemAn important contribution vision science makes to data visualization is that a limited set of visual properties can be detected very rapidly and accurately by the low-level visual system
15How many 3’sWatch this space and tell me how many 3’s you see… (1 second)
19How many 3’sWhat about now? Do you have the correct answer? So, pop-outs (pre-attentive selection) helps us with this task. What else could help us with this task? Can you think of other types of tasks that pre-attentive recognition could help us with?
20Fun with Visual Perception: Party Tricks and other Amusements Our visual system because it is built from the ground up, and has evolved over millions of years, contains many “artifacts”.These make for good party tricksAnd they help inform us about ways in which we can take advantage of the vision system’s mechanics to help with visual understanding (especially quick recognition).Also see website of optical illusions:
24Stroop EffectRead the words on the following slide as quickly as you can.Ready?
25OAK WATER FLOWER BOAT HOUSE TREE STONE PEANUT HORSE RADIO FOLDER STREETFLYQUAKESTORMGRASSZIPJT: Got coloring slide 25
26Stroop EffectOK. Let’s do it again. Read the words on the next slide as quickly as you can.Ready?
27RED BLACK YELLOW GREEN BLUE JT: Got coloring from slide 25
28Did you notice a difference? Interference from engaging higher level processing. Actually you can probably do again at similar speed if you focus just on written word (left brain) and no connotations (right brain).A Whole New Mind: Why Right-Brainers Will Rule the Future, Daniel Pink
35Good Job .For the next slides your job is to pick out the two groupings. Each square contains two groups (which have common features). The two groups can be separated by a line splitting the square into two rectangles.Raise your hand when you believe you know the answer.
36Arrange this in nice layout, as big as possible.
42So, you were likely best at this task when groups differed by both color and shape. Still not bad when differed by just one attribute (color or shape—which was easier because it a better pre-attentive cue?).Not so good when difference was mixture of color and shape (i.e. red squares and blue circles versus blue squares and red circles).
43What you can’t see…Can’t embed, click image to hyperlink to play online
44Can’t embed, click image to hyperlink to play online
45Best Resource Award!A great resource for looking at many of these visual effects AND learning about the science behind them is Christopher Healey’s (NCSU) webpage on Perception in Visualization.Let’s take a brief look. Be sure to try the java applets.
47Impact on Visualization We knows things from carefully controlled studies (not so many of these), and lots of practical empirical experience.First, an example of controlled study to evaluate which visual elements were better for gauging magnitude (how much of something)
48Controlled Study: Position vs Area/Angle Tried to recreate in excel, but couldn’t get the pie chart to go in the right order. Easier to cut and copy from paint.Figure 3. Graphs from position-angle experiment.
495 ways to compare 2 magnitudes Estimate the relative magnitude of the pair of bars marked with •Cut and paste form paintW.S. Cleveland R. McGill, Graphical perception,JASA 39, pp , 1984
50Results: Error in Judgements Cut and paste from paint
51How can we make use of this? Best way is position or adjacent bar chartHow can you do this if you want to compare different values on same chart.ADD EXAMPLE of dynamic stacked bar chartsBest practices with bar charts (we will cover later in charts module)
52Potential Preattentive Features hueintensityflickerdirection of motionbinocular lustrestereoscopic depth3-D depth cueslighting directionlengthwidthsizecurvaturenumberterminatorsintersectionclosure
53Examples of pre-attentive tasks Target detectionIs something there?• Boundary detection− Can the elements be grouped?• Counting− How many elements of a certain type are present?Maybe add small visuals for each of these
54Key Perceptual Properties Texture3DMotionShapeGroupingsSpatialMultiresolution (zoom)Brightness/LuminanceColorMaybe add small visuals here? Will look for examples
55Texturefrom: Jürgen DöllnerPasted these in herefrom: Colin Ware
56Pre-Attentive Cues With Textures • A visual texture represents that visual sensation that allows us to pre-attentively differentiate two adjacent, possibly structured parts in our visual field without eye movementvisual textures include micro-structures, patterns, profiles, etc.to identify textures, an observations of about ms is sufficient (cognitively controlled processes require about ms)Classification of textures is based oncoarseness, contrast, directionality (orientation), scale, line-likeness, regularity, roughnessTextures improve perception of position and orientationTexture communicate information about the 3D structure regardless of their coloringSlideSlide adapted from Klaus Mueller/ Jürgen Döllner
57Pre-Attentive Cues With Textures Same surface with and without textureTextures that do not include information are to be avoided in visualizationrecall Tufte’s aesthetic principle that irrelevant decoration (= chart junk) should be avoidedSubtle textures for 3D visualizations, however, can be important elements of visual designSlide 23--Slide adapted from Klaus Mueller/ Colin Ware
58Texture Perception Textons ObjectsTextonsfundamental micro-structures in generic natural images-basic elements in pre-attentive visual perceptionTextons can be classified into three general categories:1. elongated blobs (line segments, rectangles, ellipses) with specific properties such as hue, orientation, and width, at different level of scales2. terminators (end of line segments)3. crossings of line segmentsJulesz believes that only a difference in textons or in their density can be detected pre-attentively- no positional information about neighboring textons is available without focused attention- pre-attentive processing occurs in parallel (fast!)- focused attention occurs in serial (slower!)Example: Look at two objects on rightalthough the two objects look very different in isolationthey are actually the same textonTheir textonsGaborPrimitvesSlide adapted from Klaus Mueller/ Jürgen Döllner
59Relation to Symbol and Texture Design When designing textures to indicate different regions of a visualization, make sure that the textons are as different as possibleThe same rules apply when designing symbol setsExample: A tactical map may require the following symbols:aircraft targetstank targetsbuilding targetsinfantry position targetsEach of these target types can be classified as friendly or hostileTargets exist whose presence is suspected but not confirmed (this uncertainty must be encoded)Set of symbols designed to represent different classes of objects- symbols should be as distinct as possible with respect to their pre- attentive processing- recall: military reconnaissance must occur FAST!Slide adapted from Klaus Mueller/ Jürgen Döllner
60Textron use Textron on military type map Textron for fluid/air flows ADD BETTER EXAMPLES!!!
61Information Display in 3D: Depth Cues 3D display should provide depth cuesLinear perspective:more distant objects become smaller in the imagecan indicate focus, importance, or ordering- elements of a uniform texture become smaller with distancecan give shape cuesShadows:show the relative height of objects above a surfaceprovide strong depth cues for objects in motioncan be semi-realistic and still work as a depth cueOcclusion:- very powerful depth cueSlide adapted from Klaus Mueller/ Colin Ware
62Information Display in 3D: Depth Cues Shading:- shape cues from shading (shape-from-shading)diffuse+ specular+ shadowsshape from shading(hole vs. hill)specular can reveal fine detailassume single light sourcehaving more than one lightsource can lead to confusionSlide adapted from Klaus Mueller/ Colin Ware
63Information Display in 3D: Depth Cues Other depth cues:depth of focusmotion parallax (structure from motion) --> how objects relate under motion (see next slide for examples)steroscopic depth (binocular displays)For fine-scale judgement, for example, threading a needle:stereo is important, and shadows and occlusionFor large-scale judgementlinear perspective, motion parallax, and perspective are importantstereo is not so importantHowever, for information visualizationdisplays, one may exploit focusto emphasize importance,despite depth relationshipsSlide adapted from Klaus Mueller/ Jürgen
64Motion 3D Structure from Motion Simple example of 3D structure from motion animationSecond example with user controlYouTube video embedded. Bach visual hyperlinked to site.There is lots more to motion, but we won’t cover.
65Shape, Symbol Symbols should be rapidly perceived and differentiated Application for maps, military, etc.Add small visuals here?
66Uses of Symbols See Bertin’s work. (Generally we won’t cover this as separate section due to time constraints)
67Multiscale Resolution Images at coarse scale can still be recognized. See Dali’s painting (different at different viewing distances).Shark Picture. (look closely). From artist Chris Jordan (select running the numbers II, scroll down to hammerhead shark).
68Luminance/Brightness Measured amount of light coming from some placeBrightnessPerceived amount of light coming from source
72Color Models RGB (based on technology model) HSB (HVS) model (based on perceptual color model)Hue - what people think of colorSaturation - intensity, whitenessBrightness (Value) - light/dark
73Contrast Important for fg-bg colors to differ in brightness Hello, here is some text. Can you read what it says?Hello, here is some text. Can you read what it says?Hello, here is some text. Can you read what it says?Hello, here is some text. Can you read what it says?Hello, here is some text. Can you read what it says?Hello, here is some text. Can you read what it says?
74Color Background Contrast Focus on the black circle for a few seconds, then switch to one of the white fields. For a moment, you will see a white circle (the complementary color to the black circle). This is due to the local change of the adaption level.Another example (used in Florida Recount joke).From: Colin Ware
75Successive Contrast Surround matters, especially for colors Afterburn effect:focus on colored panelsthen switch to white paneldo this for saturated and non-saturated backgroundsColor contrast effectsuse saturated colors sparingly, they may cause undesired effectsneutral borders can helpFrom Klaus Mueller
76Chromatic Color is Irrelevant… To perceiving object shapesTo perceiving layout of objects in spaceTo perceiving how objects are movingTherefore, to much of modern lifeMany people go much of their life without realizing they are color blind.
77Color is Critical… To help us break camouflage To judge the condition of objects (food)To determine material typesExtremely useful for coding informationSome objects differ from their surroundings primarily by colorFood: Is this fruit ripe? Is this meat putrid? What kind of mushroom is this?Material: What kind of stone is this (for making an axe or knife)?
78Implications Color perception is relative We are sensitive to small differences- hence need sixteen million colors when driving computer displaysNot sensitive to absolute values- hence we can only use less than 10 or 12 colors for codingBrad’s viewCan use successfully for labeling small number of categorical data (12 or less).Can in certain constrained cases be utilized as continuous scale (isoluminance example). But must be careful here; often not used properly in this case.
80Nautical Maps What are goals? Contrast these two: What visualization choices were made?
81Electronic Chart Example Color in two ways:Semantic Labels: Blue vs Brown is used to indicate region type (water, land)Within these two colors (interval) shades of blue to indicate depth of water and shades on brown to indicate structures on land.
82Applications Color interfaces Color coding/labeling (for nominal data) Color sequences for ordinal dataContinuous Color for continuous dataCall attention to specific dataIncrease appeal, memorabilityIncrease number of dimensions for encoding data
83Color Categories Are there certain canonical colors? Post & Greene ‘86 had people name different colors on a monitorPictured are ones with > 75% commonalityBlack version: Colin Ware, Color Version: Post & Greene ‘86
84Gender Differences for Color? Number of colors men perceive vs women?No difference (millions)Number of colors used in describing objects? How fast one call recall name (label) of color?Women are superior (more, faster) than men (Arthur, H., Johnson, G., & Young, A. (2007). Gender differences and color: Content and emotion of written descriptions. Social Behavior and Personality, 35(6), doi: /sbp ).
85Color for CategoriesCan different colors be used for categories of nominal variables?YesWare’s suggestion: 12 colorsred, green, yellow, blue, black, white, pink, cyan, gray, orange, brown, purpleAdded Ware’s color palette here
86Ordinal data Principles Order: ordered values should be represented by perceptually-ordered colorsSeparation: significantly different levels should be represented by distinguishable colorsMany good choices can be derived from “strips” through perceptually uniform color spaces (grey scale, heated object scale, etc).Luminance: good for showing formMany hues: useful for showing readable values
87Interval Sequences: Contour Lines and Color Both can indicate regionsContour by showing the boundariesColor to distinguish different objects/groups.Choice of spacingRegular intervals to enable interval comparisonSpecific values to highlight regions (sea level)Can attempt to use Uniform Color Space
88Color for Sequences (continuous) Can you order these (low->hi)
90Continuous Uses: Color Scales Examples of two window and levels of same data (lung, soft tissue). Missing are psuedo, hue Color scales. Applied to medical images (from Hemminger Radiology work).
91Color for Continuous Variables Is done, but beware this is tricky, must be carefulFor example see Ware, C., & Beatty, J. (1988). Using color dimensions to display data dimensions. Human Factors, 30(2),
92Color as Scale + Luminance Isoluminance Color Display (Hemminger).Studies use of color vs luminance for showing two dimensions of finding (anatomical structure versus cancer signal) on map.What’s obvious shortcoming with this visualization? (story)
94Colorblind and Color Scales Vischeck by Bob Dougherty and Alex Wade. Allows you to simulate what a colorblind person would see when viewing your image or website.Daltonize (part of vischeck) will correct (suggest) alternative color scheme so colorblind person can appreciate detail in image (switch Red/Green use to other hues).
95Choosing Color Scales for Interfaces Types of color scalesContinuous/Interval Data use continuous color scale (mono/divergent)Categorical Data use labeling/qualitative color scale
96Color ScalesColor Brewer Tool developed by Cynthia Brewer for GIS maps, is very helpful for picking safe continuous/discrete scales, and the number of colors in scales. She avoids uses saturated colors, and developed diverging color scales. Safe means OK forColorblind folksprint on different devicesphotocopying
97Color SchemesColor Scheme Designer by Petr Stanicek. Allows you to easily visualize and modify/compare color scheme choices for several common (designer recommended) color palettes for interfacesMonoComplementTriadAnalogicAccented analogic
98Generally don’t use complementary colors (disturbing to human visual system, hard to look at for a long time). Example: RG protein (microarray type colormaps, use high contrast (bad) and direct complementary colors (red/green) bad. Also use Blue/Yellow, but with similar problem. Theresa Maria Rhyne designed an alternative one. Solution was green/purple, which results in reasonable color blindness results for all three color blindness conditions (different but OK).If you want contrast, use nearly complementary, or accented analogic.
99Tough to get 10 different distinguishable color labeling palette, that are equally easily distinguished. This is why Brewer tool and others don’t go above this for categorical data. Some folks try to extend by varying something else (like value/luminance) but this complicates by suggesting categories which are not present, so probably not appropriate to the task.A field guide to digital color by Maureen Stone. Great coverage, especially lots of information on color calibration for digital displays.The interaction of Color: Josef Albers
100Color ThemesAdobe’s Kuler tool. Social media tool allowing you to save and share your color theme schemes, and to search and see others as well.Saves to your personal library (mykuler)Explores/searches yours/worlds themesCreate theme from colormap or from your image
101Color Scheme Takeaways Consider CUT-DDV framework when choosing color scheme. Make conscious choice of type, number of levels, color, transparency, etc.Use freely available tools to help design and test colors for your visualization; try several alternatives schemesCheck it for safe use
102Bivariate MapsRows and columns: to preserve univariate information, display parameters should should be perceptually orthogonal (ideally)Display map data with more than one variable per map location is challenging.First do we have to use color, or contours to indicate regions (states, counties)?What visual types do we have left to show the variables present at each map location?
106Absolute number of disability, does this inform Absolute number of disability, does this inform? If number is proportional to population is it interesting?How well does the separate symbol (wheelchair) which varies in size to show quantity, work?How well does the combination of this symbol (wheelchair) work in conjunction with quantitative variable for number of people with disability?
108How effective is the color for showing distance to the Mississippi river? Could we leave it out? Show river with more emphasis and depend on map information to visually show correlation to location of river?
109Consider Data Nominal, ordinal, interval or ratio Map or entity based codingAmount of detailRate of change, i.e. is high spatial frequency present in a variable? If so may require different choice (think West Nile Virus example with widely varying rates all across the US).
110Consider Audience (User) Color deficient viewers?Don’t depend on red-green differentiationUse redundant scalesApplication area conventions?Use familiar scales (or at least know when you’re not)Color associations with variables?Use associated colorColor associations with data ranges?Use red for bad range (in U.S.)Use red for hot
111Take home messages Use luminance for detail, shape and form Use color for coding - few colorsMinimize contrast effectsStrong colors for small areas - contrast in luminance with backgroundSubtle colors can be used to segment large areas
112More messages Color excellent for multi-dimensional data Use additional tools to get quantitiesBeware of artifacts due to color re-samplingWe need precise color for specification and standardization
114What’s the best way to visualize this information? Would map-based be better?Can I easily pick out individual curves by color? (more than 10-12)How about height/chart figure for values?What about a data table?What are the questions we’re trying to answer?
115Influence on Visualization Why we careExploit strengths, avoid weaknessesOptimize, not interfereDesign criteriaEffectivenessExpressivenessNo false messages
116Design criteria: Effectiveness Faster to interpretMore distinctionsFewer errors1234567This?Or this?
117Sensory vs. Arbitrary Symbols Understanding without trainingResistance to instructional biasSensory immediacyHard-wired and fastCross-cultural ValidityArbitraryHard to learnEasy to forgetEmbedded in culture and applications
118Which Properties are Appropriate for Which Information Types Which Properties are Appropriate for Which Information Types? Final Take Aways
119Channel Properties Luminance Channel Chromatic Channels Detail Form ShadingMotionStereoSurfaces of thingsLabelsCategories,Berlin and Kay (about 6-10)Red, green, yellow and blue are special (unique hues)
120Rankings: Encoding quantitative data Cleveland & McGill 1984, adapted from Spence 2006
121Visual Property vs Data Type Stephen Few’s Table:AttributeQuantitativeQualitativeLine length2-D positionOrientationLine widthSizeShapeCurvatureAdded marksEnclosureHueIntensity
122Interpretations of Visual Properties Some properties can be discriminated more accurately but don’t have intrinsic meaning. (Senay & Ingatious 97, Kosslyn, others)If there are intrinsic meanings use them (positive advantage and avoid confusion). And be aware when you use them that you are not sending conflicting messages.Density (Greyscale)Darker -> MoreSize / Length / AreaLarger -> MorePositionLeftmost -> first, Topmost -> first (culture/language specific)HueCulture specific meaningsSlopeno intrinsic meaning ???