1. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 1 Statistical Methods in Computer Science Descriptive Statistics Data 1: Frequency Distributions Ido Dagan

2. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 2 Concrete Theory: Relates Variables to Each Other Examples: Mathematically accurate Memory = 2*sizeof(input) + 3 Runtime = 500 + 30*sizeof(input) + 20 Asymptotically correct Memory = O(sizeof(input)) in worst case, Runtime = O(log (sizeof(input))) in best case Accuracy is proportional to run-time Qualitative User performance is increased with reduced cognitive load number of bugs discovered is monotonically decreasing, but positive, if the same programmer is used, otherwise, it increases

3. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 3 Behavior Parameters/Variables (typical of Computer Science) Hardware parameters CPU model and organization, cache organization, latencies in the system System parameters Memory availability, usage CPU running time (sometimes approximated by world-clock time) Communication bandwidth, usage Program characteristics requires floating-point, heavy disk usage, integer math, graphics large heap, large stack, uses non-local information,...

4. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 4 Scales of Measurements Nominal (also called categorical): No order, just labels e.g., “Algorithm Name” Ordinal (also called rank): Order, but not numerical Difference between ranks is not necessarily the same e.g., ranks in (hierarchical/military) organization Interval: Difference between values has same meaning everywhere e.g., temperature in Celsius (rise of 10 degrees is the same everywhere) But 100C is not twice as hot as 50C, and 0C is not lack of heat Ratio: Interval + Fixed zero point e.g., robot position, memory usage, run-time

5. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 5 Scale Hierarchy Nominal < Ordinal < Interval < Ratio Propositions that are true for some level, are true above it But not necessarily the other way around e.g., we can calculate the mean (average) value for numerical variables But not for nominal and ordinal e.g., we can calculate the most frequent value for all variables http://en.wikipedia.org/wiki/Levels_of_measurement “Numerical”

6. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 6 Variables Discrete: Can take on only certain values: symbols, exact numbers For ordinal, interval and ratio scales, this means there will be gaps e.g., User satisfaction surveys, memory usage Continuous: Can take on any value within its range: no gaps e.g., run-time, CPU temperature, robot velocity and position In practice: limited by measurement accuracy Up to researcher to determine needed accuracy

7. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 7 Data The collection of values that a variable X took during the measurement

8. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 8 Describing Data Our task: Describe the data we have collected Find ways to characterize it, represent it Find properties that are true of the data

9. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 9 Data Distribution The collection of data is called the sample distribution We will investigate distributions: Find values that “best” represent a distribution Measure their dispersion, range, shape Identify extraordinary values in a distribution Find visual representations for a distribution Remember hierarchy: Nominal < Ordinal < Interval < Ratio Think about how the following techniques apply

10. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 10 Frequency Distribution Examine the frequency of values f(x) = # of times variable took on value x.

11. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 11 Frequency Distribution Examine the frequency of values f(x) = # of times variable took on value x. ?

12. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 12 Frequency Distribution Examine the frequency of values f(x) = # of times variable took on value x. Convention (Ordinal/Numerical): Sort by value

13. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 13 Grouped Frequency Distributions In ordinal/numerical variables, possible to group values together Create Grouped Frequency Distributions

14. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 14 Grouped Frequency Distributions In ordinal/numerical variables, possible to group values together Create Grouped Frequency Distributions Warning: Loss of Information

15. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 15 Real and Apparent Limits Continuous values are more difficult to divide into intervals Score of 95 falls within 95-99, not within 90-94 But what about temperature of 94.87 ? 94 < 94.87 < 95 ! By convention, the real limits of a score are within ½ the measurement resolution If our resolution is 0.1, then limits are within 0.05 If our resolution 100, then limits are within 50 Note: we break convention only for exceptional cases e.g., age: “I am 35” is true of [35.0.. 36.0)

16. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 16 Real/Apparent Limits For example: Resolution of 0.01. Interval 95..99 really covers values 94.995 to 99.005 Apparent limits: 95..99 Real limits: 94.995 to 99.005 Resolution of 10: 740-800 really covers values 735 to 805.

17. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 17 Relative Frequency Distributions A frequency count can be misleading Algorithm X was fastest on 60,000 trials: Is this good? 100,000 people voted for candidate A: Is she the winner? Relative frequency distributions: translate f into percentage or ratio rel f (proportion) = f/N rel f (%) = 100 * f/N Warning: Can be misleading, if ignoring count magnitude 50% of all test cases succeeded (with only two cases…)

18. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 18 Relative Frequency Distributions f/N Example:

19. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 19 Cumulative Frequency Distribution For ordinal/numerical variables Where values are with respect to others: How many below or above Cumulative frequency distribution

20. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 20 Cumulative Frequency Distribution Based on the cumulative distribution, can answer question such as: What percentage of scores fall below 80? How many scores below 95?

21. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 21 Percentiles, Percentile Ranks (Value of) Percentile X: Value for which X percent of values are lower e.g. baby height We use P x to denote the Xth percentile, e.g., P 98 is in range 90-94. Percentile rank of value X: the percent of values that fall below X. e.g., percentile rank of the interval 65-69 is 12.

22. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 22 Computing Percentiles, P. Ranks How do we compute percentiles and percentile ranks from grouped data? What is the score which defines the top 20% of scores? Is it between 84 and 85?

23. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 23 Computing Percentiles We want to compute P 80. 80% of 50 cases = 40 cases. We look under the cum f heading. 32 of the 40 scores are less than 84.5 (real limit).

24. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 24 Computing Percentiles We want to compute P 80. 80% of 50 cases = 40 cases. We look under the cum f heading. 32 of the 40 scores are less than 84.5 (real limit).

25. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 25 Computing Percentiles We want to compute P 80. 80% of 50 cases = 40 cases. We look under the cum f heading. 32 of the 40 scores are less than 84.5 (real limit). We need 8 more.

26. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 26 The interval 85-89 contains 47-32 = 15 cases. real limit 84.5 These are spread over width of 5 (= 89.5- 84.5). Assume scores are evenly distributed within interval 8 more cases ==> 8/15 * 5 = 2.67 (linear interpolation) P 80 = 84.5 + 2.67 = 87.17 Computing Percentiles We want to compute P 80. 80% of 50 cases = 40 cases. We look under the cum f heading. 32 of the 40 scores are less than 84.5 (real limit). We need 8 more.

27. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 27 Computing Percentile Ranks We want to compute the percentile rank of 86 Lies in the interval 85-89, real limits 84.5 – 89.5. 86-84.5 = 1.5 score points. Width of interval = 5. Assuming uniform spread of scores in interval: 1.5/5 = 0.3 ==> 30% of scores in interval (0.3*15 = 4.5)

28. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 28 Computing Percentile Ranks We want to compute the percentile rank of 86 Lies in the interval 85-89, real limits 84.5 – 89.5. 86-84.5 = 1.5 score points. Width of interval = 5. 1.5/5 = 0.3 ==> 30% of scores in interval (0.3*15 = 4.5) So we have 32 scores up to 84.5 4.5 scores from 84.5 to 86. Total: 4.5 + 32 = 36.5 scores. 36.5/50 = 73%. This is the percentile rank of 86.

29. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 29 Frequency Distributions and Scales

30. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 30 Displaying Frequency Distributions: Nominal Data

31. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 31 Displaying Frequency Distributions: Ordinal/Numerical Data Histogram

32. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 32 Displaying Frequency Distributions: Ordinal/Numerical Data Histogram: Different Grouping

33. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 33 Lying with Visuals

34. Statistical Methods in Computer Science © 2006-now Gal Kaminka / Ido Dagan 34 Characteristics of Distributions Shape, Central Tendency, Variability Different Central Tendency Different Variability