1. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 1 CS1512 Foundations of Computing Science 2 Lecture 23 Probability and statistics (4) © J R W Hunter, 2006

2. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 2 a priori and Experimental Probabilities Ranges and Odds In both cases the minimum probability is 0 a priori – event cannot occur a blue ball is drawn throwing 7 on an ordinary dice experimental – event has not occurred In both cases the maximum probability is 1 a priori – event must occur sun will rise tomorrow (knowledge of physics) experimental – the event always has occurred the sun has risen every day in my experience For two events, express as odds: odds on a red ball are 3/10 to 7/10 i.e. 3 to 7

3. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 3 Question I think theres a better than evens chance that English Jim will win the 2:50 at Exeter today. How likely is it to snow tomorrow? What kind of probability are we talking about here? a priori? experimental? Neither future event so no experimental evidence (and only happens once) reasoning from first principles – what principles? Subjective best guess most valuable when the guesser is an expert

4. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 4 Probability of two independent events Experiment trial 1: pick a ball – event E 1 = the ball is red replace the ball in the bag trial 2: pick a ball – event E 2 = the ball is red What is the probability of the event E : E = picking two red balls on successive trials (with replacement). P( E ) = P( E 1 and E 2 ) or P( E 1 E 2 ) If the two trials are independent (the outcome of the first trial does not affect the outcome of the second) then: P( E 1 and E 2 ) = P( E 1 )* P( E 2 ) - the multiplication law

5. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 5 Probability of two red balls R R R W W W W W W W R Y Y Y N N N N N N N W N N N N N N N N N N 9 favourable outcomes out of 100 possibilities all equally likely probability of two reds = 9/100 ( = 3/10 * 3/10)

6. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 6 Probability of two non-independent events Experiment trial 1: pick a ball – event E 1 = the ball is red do not replace the ball in the bag trial 2: pick a ball – event E 2 = the ball is red What is the probability of the event E : E = picking two red balls on successive trials (without replacement). P( E 1 and E 2 ) = P( E 1 )* P( E 2 | E 1 ) the general form of the multiplication law

7. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 7 Probability of two red balls R R W W W W W W W R Y Y N N N N N N N W N N N N N N N N N first draw second draw – assuming red chosen first time 6 favourable outcomes out of 90 possibilities all equally likely probability of two reds = 6/90 ( = 3/10 * 2/9)

8. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 8 Conditional probabilities P( E 2 | E 1 ) = the probability that E 2 will happen given that E 1 has already happened. If E 2 and E 1 are (conditionally, statistically) independent then P( E 2 | E 1 ) = P( E 2 ) because the occurrence of E 1 has no effect on the occurrence of E 2 soP( E 1 and E 2 ) = P( E 1 )* P( E 2 | E 1 ) = P( E 1 )* P( E 2 ) Be careful about assuming independence.

9. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 9 (Revd. Thomas) Bayes Theorem P( E 1 and E 2 ) = P( E 1 )* P( E 2 | E 1 ) order of E 1 and E 2 is not important (probability of two red balls without replacement) P( E 2 and E 1 ) = P( E 2 )* P( E 1 | E 2 ) P( E 2 )* P( E 1 | E 2 ) = P( E 1 )* P( E 2 | E 1 ) P( E 1 )* P( E 2 | E 1 ) P( E 1 | E 2 ) = P( E 2 )

10. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 10 Bayes Theorem Why bother? Suppose E 1 is a particular disease (say measles) Suppose E 2 is a particular disease (say spots) P( E 1 ) is the probability of a given patient having measles (before you see them) P( E 2 ) is the probability of a given patient having spots (for whatever reason) P( E 2 | E 1 ) is the probability that someone who has measles has spots Bayes theorem allows us to calculate the probability: P( E 1 | E 2 ) = P (patient has measles given that the patient has spots)

11. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 11 Probability of two mutually exclusive events Experiment bag contains 3 red balls, 7 white balls and 5 blue balls trial: pick a ball event E 1 = the ball is red event E 2 = the ball is blue What is the probability of the event E = picking either a red ball or a blue ball P( E ) = P( E 1 or E 2 ) or P( E 1 E 2 ) If the two events are mutually exclusive (they cant both happen in the same trial) then: P( E 1 or E 2 ) = P( E 1 ) + P( E 2 ) - the addition law

12. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 12 Additive probabilities 1 2 3 4 5 6 1 2 3 4 5 6 7 2 3 4 5 6 7 8 3 4 5 6 7 8 9 4 5 6 7 8 9 10 5 6 7 8 9 10 11 6 7 8 9 10 11 12 9 outcomes correspond to event E = E 1 or E 2 36 possible outcomes probability = 9/36 (= ¼ ) = 5/36 + 4/36 = P(8) + P(9) Probability of throwing 8 ( E 1 ) or 9 ( E 2 ) with two dice:

13. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 13 Probability of two non-mutually exclusive events 1 2 3 4 5 6 1 1,1 1,2 1,3 1,4 1,5 1,6 2 2,1 2,2 2,3 2,4 2,5 2,6 3 3,1 3,2 3,3 3,4 3,5 3,6 4 4,1 4,2 4,3 4,4 4,5 4,6 5 5,1 5,2 5,3 5,4 5,5 5,6 6 6,1 6,2 6,3 6,4 6,5 6,6 11 outcomes correspond to event E E = E 1 or E 2 or [implicitly] ( E 1 and E 2 ) 36 possible outcomes probability = 11/36 = 6/36 + 6/36 – 1/36 E = Probability of throwing at least one 5 with two dice E1 = 5 on first dice; E2 = 5 on second dice

14. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 14 Discrete random variables Variable an attribute or property of an element which has a value Discrete variable a variable where the possible values are countable Discrete random variable a discrete variable where the values taken are associated with a probability Discrete probability distribution formed by the possible values and their probabilities P ( X = ) =...

15. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 15 Example – fair coin Let variable ( X ) be number of heads in one trial (toss). Values are 0 and 1 Probability distribution: Value of X ( i ) 0 1 P ( X = i ) 0.5 0.5

16. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 16 Example – total of two dice Let variable ( X ) be total when two dice are tossed. Values are 2, 3, 4,... 11, 12 Probability distribution: P ( X = 2 ) = 1/36 P ( X = 3 ) = 2/36... 1 2 3 4 5 6 1 2 3 4 5 6 7 2 3 4 5 6 7 8 3 4 5 6 7 8 9 4 5 6 7 8 9 10 5 6 7 8 9 10 11 6 7 8 9 10 11 12

17. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 17 Bernouilli probability distribution Bernoulli trial: one in which there are only two possible and exclusive outcomes; call the outcomes success and failure; consider the variable X which is the number of successes in one trial Probability distribution: Value of X ( i ) 0 1 P ( X = i ) (1-p) p

18. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 18 Binomial distribution Carry out n Bernoulli trials; in each trial the probability of success is p X is the number of successes in n trials? Possible values are: 0, 1, 2,... i,... n -1, n We state (without proving) that the distribution of X is: P(X=r) = n C r p r (1-p) n-r n! n C r = n! = n (n-1) (n-2)... 2 1 r! (n-r)!

19. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 19 Example of binomial distribution Probability of r heads in n trials ( n = 20) P(X = r)

20. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 20 Example of binomial distribution Probability of r heads in n trials ( n = 100) P(X = r)

21. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 21 Probability of being in a given range Probability of r heads in n trials ( n = 100) P(0 X < 5) P(5 X < 10)... Moving towards the notion of a continuous distribution

22. CS1512 www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/ 22 Probability of being in a given range P(55 X < 65) = P(55 X < 60) + P(60 X < 65) (mutually exclusive events) Add the heights of the columns