1. Probability theory 2010 Conditional distributions  Conditional probability:  Conditional probability mass function: Discrete case  Conditional probability mass function: Continuous case

2. Probability theory 2010 Conditional probability mass functions - examples  Throwing two dice  Let Z 1 = the number on the first die  Let Z 2 = the number on the second die  Set Y = Z 1 and X = Z 1 +Z 2  Radioactive decay  Let X = the number of atoms decaying within 1 unit of time  Let Y = the time of the first decay

3. Probability theory 2010 Using conditional probability mass functions to compute joint and marginal densities  Discrete case  Continuous case

4. Probability theory 2010 Using conditional probability mass functions to compute marginal densities - Gibb’s sampler  Suppose that for two random variables X and Y we know  Then  Moreover, the solution to this fixed-point equation can be obtained by successively sampling

5. Probability theory 2010 Conditional expectation  Discrete case  Continuous case  Notation

6. Probability theory 2010 Conditional expectation - rules

7. Probability theory 2010 Calculation of expected values through conditioning  Discrete case  Continuous case  General formula

8. Probability theory 2010 Calculation of expected values through conditioning - example  Primary and secondary events  Let N denote the number of primary events  Let X 1, X 2, … denote the number of secondary events for each primary event  Set Y = X 1 + X 2 + … + X N  Assume that X 1, X 2, … are i.i.d. and independent of N

9. Probability theory 2010 Calculation of variances through conditioning Variation in the expected value of Y induced by variation in X Average remaining variation in Y after X has been fixed

10. Probability theory 2010 Variance decomposition in linear regression

11. Probability theory 2010 Proof of the variance decomposition We shall prove that It can easily be seen that

12. Probability theory 2010 Regression and prediction Regression function: Theorem: The regression function is the best predictor of Y based on X Proof:

13. Probability theory 2010 Best linear predictor Theorem: The best linear predictor of Y based on X is Proof: Differentiate with respect to the parameters of the linear predictor. Ordinary linear regression

14. Probability theory 2010 Expected quadratic prediction error of the best linear predictor Theorem: Proof: ……. Ordinary linear regression

15. Probability theory 2010 Martingales The sequence X 1, X 2,… is called a martingale if Example 1: Partial sums of independent variables with mean zero Example 2: Gambler’s fortune if he doubles the stake as long as he loses and leaves as soon as he wins

16. Probability theory 2010 Exercises: Chapter II 2.8, 2.11, 2.23, 2.35, 2.37 Use conditional distributions/probabilities to explain why the envelop-rejection method works