Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 1www.eclecticon.info Conditional Probability and Bayes’ Theorem Consider two events A and B. Each can either occur or not occur. Event A not occurring is denoted by A’ and B not occurring is denoted by B’. We can construct TWO tree diagrams to map out the possible permutations of outcomes. The probability of A occurring given B has occurred is P(A|B). Events A and B occurring is P(A,B). Note the order does not matter for the latter. B B’ B A A’ A A B B’ Now P(A,B) = P(B,A), hence This is called Bayes’ Theorem, and allows P(B|A) to be computed from P(A|B), P(A) and P(B)

Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 2www.eclecticon.info Conditional probability example P(A|B) = 1/3, P(A) = 1/4 and P(B) = 1/5 : Find all the other probabilities in both tree diagrams corresponding to events A and B. B B’ B A A’ A A B B’

Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 3www.eclecticon.info Reality (or model thereof). We could work this out from historical statistics Probabilities we are really interested in Obviously these two views are equivalent, hence This is Bayes’ Theorem Consider a medical test (Pass, Y or Fail, N) for a disease. The probability of passing (or indeed failing) is conditional upon whether the patient actually has the disease (true, T) or not (false, F). Unfortunately the latter is what we want to infer from the test, not the other way round. Tests are never perfect, and there will be four possible outcomes: The middle two options are obviously undesirable, and often ignored by medical practitioners. Thomas Bayes Now note: Possible outcomeConditional probability Pass test given person actually has the diseaseP(Y|T) = t Fail test given person actually has the disease (“false negative”)P(N|T) = 1-t Pass test given person doesn’t actually have the disease (“false positive”)P(Y|F) = q Fail test given person doesn’t actually have the diseaseP(N|F) = 1-q Bayes’ Theorem

Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 4www.eclecticon.info Probability of having the disease given a positive test result is: i.e. because of the low probability of actually having the disease in the first place the overall probability of someone having the disease given a positive test result is actually very low! To give a better than 95% accurate result, our test accuracy t must be t = 0.999/(0.001/ ) = 99.8% accurate. Example: Testing a member of the public at random for a disease. The chance of having disease is p = 1/1000 The disease test is 95% accurate. i.e. t = 1 - q = 0.95, and symmetric! (i.e. false positive is equally unlikely as a false negative) Note if q = 1 - t Example from The Signal and the Noise by Nate Silver p247 PRIOR PROBABILITY Initial estimate of how likely it is that terrorists would crash planes into Manhattan skyscrapers p0.005% A NEW EVENT OCCURS: FIRST PLANE HITS WORLD TRADE CENTER Probability of plane hitting if terrorists are attacking Manhattan skyscrapers t100% Probability of plane hitting if terrorists are not attacking Manhattan skyscrapers (i.e. an accident) q0.008% POSTERIOR PROBABILITY Revised estimate of probability of terror attack, given first plane hitting World Trade Center 38% But then probability of terror attack, given second plane hitting the World Trade Center is 99.99% since we re-do the analysis but set p = 38%

PRIOR PROBABILITY Probability of Colonel Mustard being of murderous intentp5% CONDITIONAL PROBABILITIES: LIKELIHOODS Probability of Professor Plum meeting his doom given Colonel Mustard is a potential murderer t50% Probability of Professor Plum dying given Colonel Mustard is not feeling particularly murderous. i.e. he dies via natural causes, or someone else kills him.... q1% POSTERIOR PROBABILITY Probability of Colonel Mustard being the murderer of Professor Plum, given Professor Plum is observed to be dead P? Cluedo example Plum dies (Y) Plum lives (N) Plum dies (Y) C. Mustard is a murderer C. Mustard not a killer Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 5www.eclecticon.info

Pass test (Y) Fail test (N) Pass test (Y) Hypothesis true (T) Hypothesis not true (F) Mathematics topic handout: Conditional probability & Bayes Theorem Dr Andrew French. PAGE 6www.eclecticon.info