Pairs of Random Variables Random Process

Introduction  In this lecture you will study:  Joint pmf, cdf, and pdf  Joint moments  The degree of “correlation” between two random variables  Conditional probabilities of a pair of random variables

Two Random Variables  The mapping is written as to each outcome is S

Example 1

Example 2

Two Random Variables  The events evolving a pair of random variables (X, Y) can be represented by regions in the plane

Two Random Variables  To determine the probability that the pair is in some region B in the plane, we have  Thus, the probability is  The joint pmf, cdf, and pdf provide approaches to specifying the probability law that governs the behavior of the pair (X, Y)  Firstly, we have to determine what we call product form where A k is one-dimensional event

Two Random Variables  The probability of product-form events is  Some two-dimensional product-form events are shown below

Pairs of Discrete Random Variables  Let the vector random variable assume values from some countable set  The joint pmf of X specifies the probabilities of event  The values of the pmf on the set S X,Y provide

Pairs of Discrete Random Variables

 The probability of any event B is the sum of the pmf over the outcomes in B  When the event B is the entire sample space S X,Y, we have

Marginal Probability Mass Function  The joint pmf provides the information about the joint behavior of X and Y  The marginal probability mass function shows the random variables in isolation similarly

Example 3

The Joint Cdf of X and Y  The joint cumulative distribution function of X and Y is defined as the probability of the event  The properties are

The Joint Cdf of X and Y

Example 4

The Joint Pdf of Two Continuous Random Variables  Generally, the probability of events in any shape can be approximated by rctangles of infinitesimal width that leads to integral operation  Random variables X and Y are jointly continuous if the probability of events involving (X, Y) can be expressed as an integral of probability density function  The joint probability density function is given by

The Joint Pdf of Two Continuous Random Variables

 The joint cdf can be obtained by using this equation  It follows  The probability of rectangular region is obtained by letting

The Joint Pdf of Two Continuous Random Variables  We can, then, prove that the probability of an infinitesimal rectangle is  The marginal pdf’s can be obtained by

The Joint Pdf of Two Continuous Random Variables

Example 5

Example 6

Independence of Two Random Variables  X and Y are independent random variable if any event A 1 defined in terms of X is independent of any event A 2 defined in terms of Y  If X and Y are independent discrete random variables, then the joint pmf is equal to the product of the marginal pmf’s

Independence of Two Random Variables  If the joint pmf of X and Y equals the product of the marginal pmf’s, then X and Y are independent  Discrete random variables X and Y are independent iff the joint pmf is equal to the product of the marginal pmf’s for all x j, y k

Independence of Two Random Variables  In general, the random variables X and Y are independent iff their joint cdf is equal to the product of its marginal cdf’s  In continuous case, X and Y are independent iff their joint pdf’s is equal to the product of the marginal pdf’s

Joint Moments and Expected Values  The expected value of is given by  Sum of random variable

Joint Moments and Expected Values  In general, the expected value of a sum of n random variables is equal to the sum of the expected values  Suppose that, we can get

Joint Moments and Expected Values  The jk-th joint moment of X and Y is given by  When j = 1 and k = 1, we can say that as correlation of X and Y  And when E[XY] = 0, then we say that X and Y are orthogonal

Conditional Probability Case 1: X is a Discrete Random Variable  For X and Y discrete random variables, the conditional pmf of Y given X = x is given by  The probability of an event A given X = x k is found by using  If X and Y are independent, we have

Conditional Probability  The joint pmf can be expressed as the product of a conditional pmf and marginal pmf  The probability that Y is in A can be given by

Conditional Probability  Example:

Conditional Probability  Suppose Y is a continuous random variable, the conditional cdf of Y given X = x k is  We, therefore, can get the conditional pdf of Y given X = x k  If X and Y are independent, then  The probability of event A given X = x k is obtained by

Conditional Probability  Example: binary communications system

Conditional Probability Case 2: X is a continuous random variable  If X is a continuous random variable then P[X = x] = 0  If X and Y have a joint pdf that is continuous and nonzero over some region of the plane, we have conditional cdf of Y given X = x

Conditional Probability  The conditional pdf of Y given X = x is  The probability of event A given X = x is obtained by  If X and Y are independent, then and  The probability Y in A is

Conditional Probability  Example

Conditional Expectation  The conditional expectation of Y given X = x is given by  When X and Y are both discrete random variables

Conditional Expectation  In particular we have where

Pairs of Jointly Gaussian Random Variables  The random variables X and Y are said to be jointly Gaussian if their joint pdf has form

Lab assignment  In group of 2 (for international class: do it personally), refer to Garcia’s book, example 5.49, page 285  Run the program in MATLAB and analyze the result  Your analysis should contain:  The purpose of the program  Line by line explanation of the program (do not copy from the book, remember NO PLAGIARISM is allowed)  The explanation of Fig and 5.29  The relationship between the purpose of the program and the content of chaper 5 (i.e. It answers the question: why do we study Gaussian distribution in this chapter?)  Try using different parameter’s values, such as 100 observation, observation, etc and analyze it  Due date: next week

Regular Class: NEXT WEEK: QUIZ 1 Material: Chapter 1 to 5, Garcia’s book Duration: max 1 hour