1. CHAPTER 3 Probability Theory 3.1 - Basic Definitions and Properties 3.2 - Conditional Probability and Independence 3.3 - Bayes’ Formula 3.4 - Applications (biomedical)

2. POPULATIONP(POPULATION) = 1 2 A= “Lung Cancer” P(A) corresponds to the ratio of the probability of A, relative to the entire population. A = lung cancer (sub-)population B = “Smoker” B = smoking (sub-)population A ∩ B Probability of lung cancer Probability of lung cancer and smoker Informal Description… P(A ⋂ B) = the probability that both events occur simultaneously in the popul.

3. That is, 3 A= “Lung Cancer” B = “Smoker” A ∩ B P(A | B) corresponds to the ratio of the probability of A ∩ B, relative to the probability of B. A = lung cancer (sub-)population P(A) corresponds to the ratio of the probability of A, relative to the entire population. B = smoking (sub-)population Probability of lung cancer Probability of lung cancer, given smoker CONDITIONAL PROBABILITY Informal Description… Probability of lung cancer and smoker P(A ⋂ B) = the probability that both events occur simultaneously in the popul.

4. Probability of “Primary Color,” given “Hot Color” = ? EECEC F0.300.150.45 FCFC 0.300.250.55 0.600.401.0 OutcomeProbability Red0.10 Orange0.15 Yellow0.20 Green0.25 Blue0.30 1.00 POPULATION E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Probability Table Venn Diagram 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow

5. 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow Probability of “Primary Color,” given “Hot Color” = ? EECEC F0.300.150.45 FCFC 0.300.250.55 0.600.401.0 E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Probability Table Venn Diagram 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow OutcomeProbability Red0.10 Orange0.15 Yellow0.20 Green0.25 Blue0.30 1.00 POPULATION P(E | F) 0.667 Conditional Probability = P(F | E)

6. EECEC F0.300.150.45 FCFC 0.300.250.55 0.600.401.0 E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Probability Table Venn Diagram 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow Probability of “Primary Color,” given “Hot Color” = ? P(E | F) 0.667 OutcomeProbability Red0.10 Orange0.15 Yellow0.20 Green0.25 Blue0.30 1.00 POPULATION Conditional Probability P(F | E) 0.5 =

7. EECEC F0.300.150.45 FCFC 0.300.250.55 0.600.401.0 E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Probability Table Venn Diagram 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow Probability of “Primary Color,” given “Hot Color” = ? P(E | F)P(E C | F) 0.667 = 1 – 0.667 = 0.333 OutcomeProbability Red0.10 Orange0.15 Yellow0.20 Green0.25 Blue0.30 1.00 POPULATION Conditional Probability P(F | E) 0.5

8. EECEC F0.300.150.45 FCFC 0.300.250.55 0.600.401.0 E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Probability Table Venn Diagram 0.15 0.30 0.25 0.30 E F Blue Green Orange Red Yellow Probability of “Primary Color,” given “Hot Color” = ? P(E | F)P(E C | F) P(E | F C ) 0.667 = 1 – 0.667 = 0.333 OutcomeProbability Red0.10 Orange0.15 Yellow0.20 Green0.25 Blue0.30 1.00 POPULATION Conditional Probability P(F | E) 0.5 0.545 Red Yellow

9. 9 Example:

10. WomenMen Fractures 952 3431295 No Fractures 129314172710 224517604005 WomenFractures n = 4005 9521293343 1417 10 P(Fracture, given Woman) ≈ P(Fracture, given Man) ≈ P(Man, given Fracture) ≈ P(Woman, given Fracture) = 952 / 2245 = 0.424 343 / 1760 = 0.195 343 / 1295 = 0.265 1 – 343 / 1295 = 952 / 1295 = 0.735 P(Fracture) ≈1295 / 4005 = 0.323 P(Fracture and Woman) ≈ 952 / 4005 = 0.238

11. WomenFractures n = 4005 9521293343 1417 11 P(Fracture, given Woman) ≈ P(Fracture, given Man) ≈ P(Man, given Fracture) ≈ P(Woman, given Fracture) = 952 / 2245 = 0.424 343 / 1760 = 0.195 343 / 1295 = 0.265 1 – 343 / 1295 = 952 / 1295 = 0.735 P(Fracture) ≈1295 / 4005 = 0.323 P(Fracture and Woman) ≈ 952 / 4005 = 0.238 “Osteoporosis-related fractures are more than twice as likely to occur among women than men.” “A person who suffers an osteoporosis- related fracture is almost three times more likely to be a woman than a man.”

12. WomenFractures n = 4005 9521293343 1417 P(Fracture, given Woman) ≈ P(Fracture, given Man) ≈ P(Man, given Fracture) ≈ P(Woman, given Fracture) = 952 / 2245 = 0.424 343 / 1760 = 0.195 343 / 1295 = 0.265 1 – 343 / 1295 = 952 / 1295 = 0.735 P(Fracture) ≈1295 / 4005 = 0.323 P(Fracture and Woman) ≈ 952 / 4005 = 0.238 “Osteoporosis-related fractures are more than twice as likely to occur among women than men.” “A person who suffers an osteoporosis- related fracture is almost three times more likely to be a woman than a man.” ? ? ? ?

13. Def: The conditional probability of event A, given event B, is denoted by P(A|B), and calculated via the formula Def: The conditional probability of event A, given event B, is denoted by P(A|B), and calculated via the formula 13 Thus, for any two events A and B, it follows that P(A ⋂ B) = P(A | B) × P(B). B occurs with prob P(B) Given that B occurs, A occurs with prob P(A | B) Both A and B occur, with prob P(A ⋂ B) Example: Randomly select two cards with replacement from a fair deck. P(Both Aces) = ? Example: P(Live to 75) × P(Live to 80 | Live to 75) = P(Live to 80) P(Ace 1 ) = 4/52P(Ace 2 | Ace 1 ) = 4/52 P(Ace 1 ∩ Ace 2 ) = (4/52) 2 P(Ace 2 | Ace 1 ) = 3/51 P(Ace 1 ∩ Ace 2 ) = (4/52)(3/51) Exercises: P(Neither is an Ace) = ? P(Exactly one is an Ace) = ? P(At least one is an Ace) = ? Exercises: P(Neither is an Ace) = ? P(Exactly one is an Ace) = ? P(At least one is an Ace) = ? AB Example: Randomly select two cards without replacement from a fair deck. P(Both Aces) = ?

14. 14 Def: The conditional probability of event A, given event B, is denoted by P(A|B), and calculated via the formula Def: The conditional probability of event A, given event B, is denoted by P(A|B), and calculated via the formula Thus, for any two events A and B, it follows that P(A ⋂ B) = P(A | B) × P(B). B occurs with prob P(B) Given that B occurs, A occurs with prob P(A | B) Both A and B occur, with prob P(A ⋂ B) Example: P(Live to 75) × P(Live to 80 | Live to 75) = P(Live to 80) Tree Diagrams P(B)P(B) P(Bc)P(Bc) P(A | B) P(A c | B) P(A | B c ) P(A c | B c ) P(A ⋂ B)P(A ⋂ B) P(Ac ⋂ B)P(Ac ⋂ B) P(A ⋂ Bc)P(A ⋂ Bc) P(Ac ⋂ Bc)P(Ac ⋂ Bc) EventAAcAc B P(A ⋂ B)P(A ⋂ B)P(Ac ⋂ B)P(Ac ⋂ B) BcBc P(A ⋂ Bc)P(A ⋂ Bc)P(Ac ⋂ Bc)P(Ac ⋂ Bc) A B A ⋂ BA ⋂ B A ⋂ BcA ⋂ Bc Ac ⋂ BAc ⋂ B Ac ⋂ BcAc ⋂ Bc Multiply together “branch probabilities” to obtain “intersection probabilities” AB

15. 15 Example: Bob must take two trains to his home in Manhattan after work: the A and the B, in either order. At 5:00 PM… The A train arrives first with probability 0.65, and takes 30 mins to reach its last stop at Times Square. The B train arrives first with probability 0.35, and takes 30 mins to reach its last stop at Grand Central Station. At Times Square, Bob exits, and catches the second train. The A arrives first with probability 0.4, then travels to Brooklyn. The B train arrives first with probability 0.6, and takes 30 minutes to reach a station near his home. At Grand Central Station, the A train arrives first with probability 0.8, and takes 30 minutes to reach a station near his home. The B train arrives first with probability 0.2, then travels to Queens. With what probability will Bob be exiting the subway at 6:00 PM?

16. 16 Example: 5:00 5:30 6:00 0.65 0.35 0.4 0.6 0.8 0.2 MULTIPLY: 0.26 0.39 0.28 0.07 ADD: 0.67 Bob must take two trains to his home in Manhattan after work: the A and the B, in either order. At 5:00 PM… The A train arrives first with probability 0.65, and takes 30 mins to reach its last stop at Times Square. The B train arrives first with probability 0.35, and takes 30 mins to reach its last stop at Grand Central Station. At Times Square, Bob exits, and catches the second train. The A arrives first with probability 0.4, then travels to Brooklyn. The B train arrives first with probability 0.6, and takes 30 minutes to reach a station near his home. At Grand Central Station, the A train arrives first with probability 0.8, and takes 30 minutes to reach a station near his home. The B train arrives first with probability 0.2, then travels to Queens. With what probability will Bob be exiting the subway at 6:00 PM?

17. 17 Example:

18. 18 Let events C, D, and E be defined as: E = Active vitamin E C = Active vitamin C D = Disease (Total Cancer) TreatmentD +D –Totals Placebo E and C4793653 E (+ placebo C)4913659 C (+ placebo E)4803673 Active E and C4933656 Totals194314641 D – 3174 3168 3193 3163 12698 E D C 493 491 480 479 3163 31683193 3174 P(C) ≈ 7329 / 14641 = 0.5 P(D, given E) ≈ 984 / 7315 = 0.135 P(D) ≈ 1943 / 14641 = 0.133 P(D, given C) ≈ 973 / 7329 = 0.133 These study results suggest that D is statistically independent of both C and E, i.e., no association exists. P(E) ≈ 7315 / 14641 = 0.5 “balanced”

19. 19 POPULATION

20. OutcomeProbability Red0.10 Orange0.18 Yellow0.17 Green0.22 Blue0.33 1.00 POPULATION EECEC F0.270.180.45 FCFC 0.330.220.55 0.600.401.0 Probability Table Venn Diagram 0.18 0.27 0.22 0.33 E F Blue Green Orange Red Yellow

21. EECEC F0.270.180.45 FCFC 0.330.220.55 0.600.401.0 E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 Probability Table Venn Diagram 0.18 0.27 0.22 0.33 E F Blue Green Orange Red Yellow OutcomeProbability Red0.10 Orange0.18 Yellow0.17 Green0.22 Blue0.33 1.00 POPULATION Conditional Probability P(E | F) P(F | E) 0.60 = P(E) 0.45 = P(F) P(F) = 0.45

22. EECEC F0.270.180.45 FCFC 0.330.220.55 0.600.401.0 Probability Table Venn Diagram 0.18 0.27 0.22 0.33 E F Blue Green Orange Red Yellow OutcomeProbability Red0.10 Orange0.18 Yellow0.17 Green0.22 Blue0.33 1.00 POPULATION Conditional Probability P(E | F) = P(E) P(F | E) = P(F) E = “Primary Color”= {Red, Yellow, Blue} F = “Hot Color”= {Red, Orange, Yellow} P(E) = 0.60 P(F) = 0.45 Events E and F are “statistically independent” “Primary colors” comprise 60% of the “hot colors,” and 60% of the general population. “Hot colors” comprise 45% of the “primary colors,” and 45% of the general population.

23. EECEC F0.270.180.45 F C 0.330.220.55 0.600.401.0 OutcomeProbability Red0.10 Orange0.18 Yellow0.17 Green0.22 Blue0.33 1.00 B occurs with prob P(B) Given that B occurs, A occurs with prob P(A | B) Both A and B occur, with prob P(A ⋂ B) 23 Def: Two events A and B are said to be statistically independent if P(A | B) = P(A), Neither event provides any information about the other. Is 0.27 = 0.60 × 0.45? P(E ⋂ F) = P(E) P(F)? YES! which is equivalent to P(A ⋂ B) = P(A | B) × P(B). If either of these two conditions fails, then A and B are statistically dependent. P(A)P(A) Example: Are events A = “Ace” and B = “Black” statistically independent? P(A) = 4/52 = 1/13, P(B) = 26/52 = 1/2, P(A ⋂ B) = 2/52 = 1/26 YES! P(A)P(A) E = “Primary Color” = {Red, Yellow, Blue} F = “Hot Color” = {Red, Orange, Yellow} Events E and F are “statistically independent”  = P(E) P(F) = Example:

24. 24 Example: According to the American Red Cross, US pop is distributed as shown. Rh Factor Blood Type +– Row marginals: O.384.077.461 A.323.065.388 B.094.017.111 AB.032.007.039 Column marginals:.833.166.999 Def: Two events A and B are said to be statistically independent if P(A | B) = P(A), Neither event provides any information about the other. Are “Type O” and “Rh+” statistically independent? = P(O) = P(Rh+) Is.384 =.461 ×.833? P(O ⋂ Rh+) =.384 YES! which is equivalent to P(A ⋂ B) = P(A | B) × P(B). If either of these two conditions fails, then A and B are statistically dependent. P(A)P(A)

25.  A and B are statistically independent if: P(A | B) = P(A) IMPORTANT FORMULAS  P(A c ) = 1 – P(A)  P(A ⋃ B) = P(A) + P(B) – P(A ⋂ B) 25 = 0 if A and B are disjoint  P(A ⋂ B) = P(A | B) P(B) P(A ⋂ B) = P(A) P(B)  DeMorgan’s Laws (A ⋃ B) c = A c ⋂ B c (A ⋂ B) c = A c ⋃ B c AB  Distributive Laws A ⋂ (B ⋃ C) = (A ⋂ B) ⋃ (A ⋂ C) A ⋃ (B ⋂ C) = (A ⋃ B) ⋂ (A ⋃ C) Others…

26. Example: In a population of individuals:  60% of adults are male P(B | A) = 0.6  40% of males are adults P(A | B) = 0.4  30% are men P(A ⋂ B) = 0.3 What percentage are adults? 26 A = Adult B = Male What percentage are males? Are “adult” and “male” statistically independent in this population? 0.3 Men BoysWomen Girls

27. Example: In a population of individuals:  60% of adults are male P(B | A) = 0.6  40% of males are adults P(A | B) = 0.4  30% are men P(A ⋂ B) = 0.3 ⟹ P(B ⋂ A) = 0.6 P(A) 0.3 P(A) = 0.3 / 0.6 What percentage are adults? 27 A = Adult B = Male What percentage are males? Are “adult” and “male” statistically independent in this population? 0.3 ⟹ P(A ⋂ B) = 0.4 P(B) 0.3 P(B) = 0.3 / 0.4 0.20.45 AdultChild Male0.300.450.75 Female0.200.050.25 0.50 1.00 0.05 P(A | B) = P(A)? OR P(B | A) = P(B)? OR P(A ⋂ B) = P(A) P(B)? P(A) = 0.3 / 0.6 = 0.5, or 50% 0.5 – 0.3 = … P(B) = 0.3 / 0.4 = 0.75, or 75% 0.75 – 0.3 = … Men BoysWomen Girls

28. Example: In a population of individuals:  60% of adults are male P(B | A) = 0.6  40% of males are adults P(A | B) = 0.4  30% are men P(A ⋂ B) = 0.3 ⟹ P(B ⋂ A) = 0.6 P(A) 0.3 P(A) = 0.3 / 0.6 What percentage are adults? 28 A = Adult B = Male What percentage are males? Are “adult” and “male” statistically independent in this population? 0.3 ⟹ P(A ⋂ B) = 0.4 P(B) 0.3 P(B) = 0.3 / 0.4 0.20.45 AdultChild Male0.300.450.75 Female0.200.050.25 0.50 1.00 0.05 P(A | B) = P(A)? OR P(B | A) = P(B)? OR P(A ⋂ B) = P(A) P(B)? NO 0.4 ≠ 0.5 0.6 ≠ 0.75 P(A) = 0.3 / 0.6 = 0.5, or 50% P(B) = 0.3 / 0.4 = 0.75, or 75% 0.3 ≠ (0.5)(0.75) Men BoysWomen Girls

29. 29 A = Adult B = Male 0.30.20.45 AdultChild Male0.300.450.75 Female0.200.050.25 0.50 1.00 0.05 P(A | B) = 0.4 What percentage of males are boys? What percentage of females are women? What percentage of children are girls? P(A C | B) = 0.6 60% P(A C | B) = 1 – P(A | B) Men BoysWomen Girls = 1 – 0.4 = 0.6 - OR - P(A | B C ) =0.8 80% P(B C | A C ) = 0.1 10%

30. P(A ⋂ B) = 0.3, i.e., 30% P(B) = 0.75, i.e., 75% P(A) = 0.5, i.e., 50%  30% are men Example: In a population of individuals:  60% of adults are male P(B | A) = 0.6 ⟹  40% of males are adults P(A | B) = 0.4 What percentage are adults? 30 A = Adult B = Male What percentage are males? Men BoysWomen Girls  5% are girls ⟹ P(A) = 0.95 / 1.9 P(B ⋂ A) = 0.6 P(A) ⟹ P(A ⋂ B) = 0.4 P(B) 0.05 ⟹ 95% are not girls P(A ⋃ B) = 0.95 P(A ⋃ B) = P(A) + P(B) − P (A ⋂ B) 0.95 0.4 P(B) = 0.6 P(A) P(B) = 1.5 P(A) What percentage are men? 0.95 = P(A) + 1.5 P(A) − 0.6 P(A) 0.95 = 1.9 P(A) 0.30.20.45