Advanced Artificial Intelligence Lecture 2A: Probability Theory Review

Outline Axioms of Probability Product and chain rules Bayes Theorem Random variables PDFs and CDFs Expected value and variance

Introduction Sample space - set of all possible outcomes of a random experiment Dice roll: {1, 2, 3, 4, 5, 6} Coin toss: {Tails, Heads} Event space - subsets of elements in a sample space Dice roll: {1, 2, 3} or {2, 4, 6} Coin toss: {Tails}

 

examples Coin flip P(H) P(T) P(H,H,H) P(x1=x2=x3=x4) P({x1,x2,x3,x4} contains more than 3 heads)

Set operations  

Conditional Probability  

Conditional Probability  

examples Coin flip P(x1=H)=1/2 P(x2=H|x1=H)=0.9 P(x2=T|x1=T)=0.8

Conditional Probability  

Conditional Probability   P(A, B) 0.005 P(B) 0.02 P(A|B) 0.25  

Quiz P(D1=sunny)=0.9 P(D2=sunny|D1=sunny)=0.8 P(D2=rainy|D1=sunny)=? P(D2=sunny|D1=rainy)=0.6 P(D2=rainy|D1=rainy)=? P(D2=sunny)=? P(D3=sunny)=? 0.2,0.4,0.78,0.756

Joint Probability Multiple events: cancer, test result Has cancer? Test positive? P(C,TP) yes 0.018 no 0.002 0.196 0.784

Joint Probability The problem with joint distributions It takes 2D-1 numbers to specify them!

Conditional Probability Describes the cancer test: Put this together with: Prior probability

Conditional Probability We have: We can now calculate joint probabilities Has cancer? Test positive? P(TP, C) yes 0.018 no 0.002 0.196 0.784 Has cancer? Test positive? P(TP, C) yes no

Conditional Probability “Diagnostic” question: How likely do is cancer given a positive test? Has cancer? Test positive? P(TP, C) yes 0.018 no 0.002 0.196 0.784

Bayes Theorem  

Posterior Probability Bayes Theorem   Posterior Probability A in unobserved, but B is observed Likelihood Prior Probability Normalizing Constant

Bayes Theorem   A in unobserved, but B is observed

Random Variables  

Cumulative Distribution Functions   F(x) is monotonically non-decreasing

Probability Density Functions   PDF is also called probability mass function when applied to discrete random variables

Probability Density Functions   PDF is also called probability mass function when applied to discrete random variables

Probability Density Functions   PDF is also called probability mass function when applied to discrete random variables

Probability Density Functions   f(X)   X   PDF is also called probability mass function when applied to discrete random variables

Probability Density Functions   f(X)   X   PDF is also called probability mass function when applied to discrete random variables

Probability Density Functions   f(x)   x   PDF is also called probability mass function when applied to discrete random variables F(x) 1 x  

Probability Density Functions   f(x)   x   PDF is also called probability mass function when applied to discrete random variables F(x) 1 x  

Expectation   PDF is also called probability mass function when applied to discrete random variables

Expectation   PDF is also called probability mass function when applied to discrete random variables

Variance  

Gaussian Distributions