PROBABILITY AND BAYES THEOREM 1

2 POPULATION SAMPLE PROBABILITY STATISTICAL INFERENCE

3 PROBABILITY: A numerical value expressing the degree of uncertainty regarding the occurrence of an event. A measure of uncertainty. STATISTICAL INFERENCE: The science of drawing inferences about the population based only on a part of the population, sample.

4 PROBABILITY CLASSICAL INTERPRETATION If a random experiment is repeated an infinite number of times, the relative frequency for any given outcome is the probability of this outcome. Probability of an event: Relative frequency of the occurrence of the event in the long run. – Example: Probability of observing a head in a fair coin toss is 0.5 (if coin is tossed long enough). SUBJECTIVE INTERPRETATION The assignment of probabilities to event of interest is subjective – Example: I am guessing there is 50% chance of raining today.

5 PROBABILITY Random experiment – a random experiment is a process or course of action, whose outcome is uncertain. Examples ExperimentOutcomes Flip a coinHeads and Tails Record a statistics test marksNumbers between 0 and 100 Measure the time to assembleNumbers from zero and above a computer

6 Performing the same random experiment repeatedly, may result in different outcomes, therefore, the best we can do is consider the probability of occurrence of a certain outcome. To determine the probabilities, first we need to define and list the possible outcomes PROBABILITY

7 Determining the outcomes. – Build an exhaustive list of all possible outcomes. – Make sure the listed outcomes are mutually exclusive. The set of all possible outcomes of an experiment is called a sample space and denoted by S. Sample Space

8 Countable Uncountable (Continuous ) Finite number of elements Infinite number of elements

9 EXAMPLES Countable sample space examples: – Tossing a coin experiment S : {Head, Tail} – Rolling a dice experiment S : {1, 2, 3, 4, 5, 6} – Determination of the sex of a newborn child S : {girl, boy} Uncountable sample space examples: – Life time of a light bulb S : [0, ∞) – Closing daily prices of a stock S : [0, ∞)

10 EXAMPLES Examine 3 fuses in sequence and note the results of each experiment, then an outcome for the entire experiment is any sequence of N’s (non-defectives) and D’s (defectives) of length 3. Hence, the sample space is S : { NNN, NND, NDN, DNN, NDD, DND, DDN, DDD}

11 – Given a sample space S ={O 1,O 2,…,O k }, the following characteristics for the probability P(O i ) of the simple event O i must hold: – Probability of an event: The probability P(A), of event A is the sum of the probabilities assigned to the simple events contained in A. Assigning Probabilities

12 Assigning Probabilities P(A) is the proportion of times the event A is observed.

13 Intersection The intersection of event A and B is the event that occurs when both A and B occur. The intersection of events A and B is denoted by (A and B) or A  B. The joint probability of A and B is the probability of the intersection of A and B, which is denoted by P(A and B) or P(A  B).

14 Union The union event of A and B is the event that occurs when either A or B or both occur. At least one of the events occur. It is denoted “A or B” OR A  B

15 Complement Rule The complement of event A (denoted by A C ) is the event that occurs when event A does not occur. The probability of the complement event is calculated by P(A C ) = 1 - P(A) A and A C consist of all the simple events in the sample space. Therefore, P(A) + P(A C ) = 1

16 MUTUALLY EXCLUSIVE EVENTS Two events A and B are said to be mutually exclusive or disjoint, if A and B have no common outcomes. That is, A and B =  (empty set) The events A 1,A 2,… are pairwise mutually exclusive (disjoint), if A i  A j =  for all i  j.

17 EXAMPLE The number of spots turning up when a six- sided dice is tossed is observed. Consider the following events. A: The number observed is at most 2. B: The number observed is an even number. C: The number 4 turns up.

18 VENN DIAGRAM A graphical representation of the sample space. 2 S A B C ABAB A B 2 ABAB A B 2 2 A  C =  A and C are mutually exclusive

19 AXIOMS OF PROBABILTY (KOLMOGOROV AXIOMS) Given a sample space S, the probability function is a function P that satisfies 1) For any event A, 0  P(A)  1. 2) P(S) = 1. 3) If A 1, A 2,… are pairwise disjoint, then

20 THE CALCULUS OF PROBABILITIES If P is a probability function and A is any set, then a. P(  )=0 b. P(A)  1 c. P(A C )=1  P(A)

21 THE CALCULUS OF PROBABILITIES If P is a probability function and A and B any sets, then a.P(B  A C ) = P(B)  P(A  B) b.If A  B, then P(A)  P(B) c. P(A  B)  P(A)+P(B)  1 (Bonferroni Inequality) d. (Boole’s Inequality)

22 EQUALLY LIKELY OUTCOMES The same probability is assigned to each simple event in the sample space, S. Suppose that S={s 1,…,s N } is a finite sample space. If all the outcomes are equally likely, then P({s i })=1/N for every outcome s i.

23 For any two events A and B P(A  B) = P(A) + P(B) - P(A  B) Addition Rule

24 ODDS The odds of an event A is defined by It tells us how much more likely to see the occurrence of event A. P(A)=3/4  P(A C )=1/4  P(A)/P(A C ) = 3. That is, the odds is 3. It is 3 times more likely that A occurs as it is that it does not.

CONDITIONAL PROBABILITY (Marginal) Probability: P(A): How likely is it that an event A will occur when an experiment is performed? Conditional Probability: P(A|B): How will the probability of event A be affected by the knowledge of the occurrence or nonoccurrence of event B? If two events are independent, then P(A|B)=P(A) 25

CONDITIONAL PROBABILITY 26

Example Roll two dice S=all possible pairs ={(1,1),(1,2),…,(6,6)} Let A=first roll is 1; B=sum is 7; C=sum is 8 P(A|B)=?; P(A|C)=? Solution: P(A|B)=P(A and B)/P(B) P(B)=P({1,6} or {2,5} or {3,4} or {4,3} or {5,2} or {6,1}) = 6/36=1/6 P(A|B)= P({1,6})/(1/6)=1/6 =P(A) A and B are independent 27

Example P(A|C)=P(A and C)/P(C)=P(Ø)/P(C)=0 A and C are disjoint Out of curiosity: P(C)=P({2,6} or {3,5} or {4,4} or {5,3} or {6,2}) = 5/36

BAYES THEOREM Suppose you have P(B|A), but need P(A|B). Can be generalized to more than two events. 29

Example Let: – D: Event that person has the disease; – T: Event that medical test results positive Given: – Previous research shows that 0.3 % of all Turkish population carries this disease; i.e., P(D)= 0.3 % = – Probability of observing a positive test result for someone with the disease is 95%; i.e., P(T|D)=0.95 – Probability of observing a positive test result for someone without the disease is 4%; i.e. P(T| )= 0.04 Find: probability of a randomly chosen person having the disease given that the test result is positive. 30

Example Solution: Need P(D|T). Use Bayes Thm. P(D|T)=P(T|D)*P(D)/P(T) P(T)=P(D and T)+P( and T) = 0.95* *0.997 = P(D|T) =0.95*0.003 / = 6.67 % Test is not very reliable! 31