1. The Perception of Correlation in Scatterplots Ronald A. Rensink Departments of Computer Science and Psychology University of British Columbia Vancouver, Canada (Work done with Gideon Baldridge)

2. 2 CPSC 444, 15 Apr 10 What’s the best way to visualize information? Look at scatterplots - perception of correlation - accuracy & precision Need a way to objectively evaluate performance

3. 3 CPSC 444, 15 Apr 10 Approach: Use techniques from vision science - decades of experience - basis of modern theories of perception

4. Vision Science: Threshold Techniques E.g., Relation between intensity and brightness Adjust intensity of test until its brightness just begins to differ from the standard… One light at fixed intensity (standard) Second light (test) at a different intensity StandardTest Just noticeable difference (jnd) - 75% correct 1 W above standard

5. A bit more rigorously… Forced-choice Technique Q: Which one has the greater intensity? Look at how accuracy of response varies with difference between test and standard

6. Psychometric function -describes accuracy (probability of correct) as a function of intensity of test light

7. For I = 10 W, jnd  I = 0.8W For a large range of intensities,  I / I = k = constant Weber’s Law For I = 30 W, jnd  I = 2.5W For I = 50 W, jnd  I = 4.0W  I / I =.08

8. Weber Assumption: Each  I / I corresponds to a unit of sensation (  S)   I / I = k =  S Integrating… S = log(I) Fechner’s Law S = Psychological quantity I = Physical quantity Each unit of S corresponds to an increase in I by factor of 10

9. 9 CPSC 444, 15 Apr 10 Approach: Apply this technique to correlation Forced-choice: Which one has the higher correlation ?

10. 10 CPSC 444, 15 Apr 10 1. Fix a particular base correlation (e.g., r = 0.6) 2. Find the jnd  r (75% correct level) for that correlation - keep showing pairs of scatterplots to observer - adjust differences until jnd is reached - criterion: level in three subwindows is the same Experimental Procedure - Precision

11. 11 CPSC 444, 15 Apr 10 Results  r = k(1/b - r) k: variability (= 0.24) b: offset (= 0.907)  u = ku  u u = k Weber’s Law (with u = 1/b - r) Only 2 parameters (k,b) to specify precision over all correlations

12. 12 CPSC 444, 15 Apr 10 Weber Assumption: Each  I / I corresponds to a unit of sensation (  S)   I / I = k =  S Integrating… S = log(I) Fechner’s Law Relating physical quantity I to psychological quantity S

13. 13 CPSC 444, 15 Apr 10 Assumption: Each  u / u corresponds to a unit of sensation (  g)   u / u = k =  g Integrating… g = log(u) Relating physical quantity r to psychological quantity g g(r) = log(1 - br) log(1 - b)

14. 14 CPSC 444, 15 Apr 10 Adjust test plot to be midway between reference plots Experimental Procedure - Accuracy Start with 0.0 & 1.0. Then 0.0 & 0.5 and 0.5 & 1.0. Then 0.0 & 0.25 and 0.25 & 0.5, etc, etc.

15. 15 CPSC 444, 15 Apr 10 Results

16. 16 CPSC 444, 15 Apr 10 Results g(r) = log(1 - br) log(1 - b) (b = 0.9) Fechner’s law (with u = 1-br)

17. 17 CPSC 444, 15 Apr 10 Conclusions Precision of correlation estimate given by linear function  r = k(1/b - r)0 < k, b < 1 Accuracy of correlation estimate given by log function  g(r) = log(1 - br) / log(1 - b) 0 < b < 1 These performance curves are completely described by k, b Thus, to evaluate a given scatterplot design: 1.Measure jnds at two base correlations 2.Calculate k and b 3.Plug into performance curves