Lecture 4A: Probability Theory Review Advanced Artificial Intelligence

Outline Axioms of Probability Product and chain rules Bayes Theorem Random variables PDFs and CDFs Expected value and variance

Introduction Sample space - set of all possible outcomes of a random experiment – Dice roll: {1, 2, 3, 4, 5, 6} – Coin toss: {Tails, Heads} Event space - subsets of elements in a sample space – Dice roll: {1, 2, 3} or {2, 4, 6} – Coin toss: {Tails}

examples Coin flip – P(H) – P(T) – P(H,H,H) – P(x1=x2=x3=x4) – P({x1,x2,x3,x4} contains more than 3 heads)

Set operations

Conditional Probability

examples Coin flip – P(x1=H)=1/2 – P(x2=H|x1=H)=0.9 – P(x2=T|x1=T)=0.8 – P(x2=H)=?

Conditional Probability

P(A, B)0.005 P(B)0.02 P(A|B)0.25

Quiz P(D1=sunny)=0.9 P(D2=sunny|D1=sunny)=0.8 P(D2=rainy|D1=sunny)=? P(D2=sunny|D1=rainy)=0.6 P(D2=rainy|D1=rainy)=? P(D2=sunny)=? P(D3=sunny)=?

Joint Probability Multiple events: cancer, test result 13 Has cancer?Test positive?P(C,TP) yes yesno0.002 noyes0.196 no 0.784

Joint Probability The problem with joint distributions It takes 2 D -1 numbers to specify them! 14

Conditional Probability Describes the cancer test: Put this together with: Prior probability 15

Has cancer?Test positive?P(TP, C) yes no yes no Has cancer?Test positive?P(TP, C) yes yesno0.002 noyes0.196 no Conditional Probability We have: We can now calculate joint probabilities 16

Conditional Probability “Diagnostic” question: How likely do is cancer given a positive test? 17 Has cancer?Test positive?P(TP, C) yes yesno0.002 noyes0.196 no 0.784

Bayes Theorem

Posterior Probability Likelihood Normalizing Constant Prior Probability

Bayes Theorem

Random Variables

Cumulative Distribution Functions

Probability Density Functions

f(X) X

Probability Density Functions f(X) X

Probability Density Functions f(x) x F(x) 1 x

Probability Density Functions f(x) x F(x) 1 x

Expectation

Variance

Gaussian Distributions