Download presentation

Presentation is loading. Please wait.

Published byLinda Long Modified about 1 year ago

1
Random Variable A random variable X is a function that assign a real number, X(ζ), to each outcome ζ in the sample space of a random experiment. Domain of the random variable -- S Range of the random variable -- Sx Example 1: Suppose that a coin is tossed 3 times and the sequence of heads and tails is noted. Sample space S={HHH,HHT,HTH,HTT,THH,THT,TTH, TTT} X :number of heads in three coin tosses. ζ : HHH HHT HTH THH HTT THT TTH TTT X(ζ): Sx={0,1,2,3}

2
**Probability of random variable**

Example 2: The event {X=k} ={k heads in three coin tosses} occurs when the outcome of the coin tossing experiment contains k heads. P[X=0]=P[{TTT}]=1/8 P[X=1]=P[{HTH}]+P[{THT}]+P[{TTH}]=3/8 P[X=2]=P[{HHT}]+P[{HTH}]+P[{THH}]=3/8 P[X=3]=P[{HHH}]=1/8 Conclusion: B⊂SX A={ζ: X(ζ) in B} P[B]=P[A]=P[ζ: X(ζ) in B]. Event A and B are referred to as equivalent events. All numerical events of practical interest involves {X=x} or {X in I}

3
**Events Defined by Random Variable**

If X is a r.v. and x is a fixed real number, we can define the event (X=x) as (X=x)={ζ: X(ζ)=x)} (x1<X≤x2)={ζ: x1<X(ζ)≤x2} These events have probabilities that are denoted by P[X=x]=P{ζ: X(ζ}=x} P[x1<X≤x2]=P{ζ: x1<X(ζ)≤x2}

4
**Distribution Function**

The cumulative distribution function (cdf) of a random variable X is defined as the probability of events {X ≤ x}: Fx(x)=P[X ≤ x] for -∞< x ≤ +∞ In terms of underlying sample space, the cdf is the probability of the event {ζ: X(ζ)≤x}. Properties:

5
**A typical example of cdf**

Tossing a coin 3 times and counting the number of heads x X≤x FX(x) -1 1 2 3 4 Ø {TTT} {TTT,TTH,THT,HTT} {TTT,TTH,THT,HTT,HHT,HTH,THH} S 1/8 4/8 7/8

6
**Two types of random variables**

A discrete random variable has a countable number of possible values. X: number of heads when trying 5 tossing of coins. The values are countable A continuous random variable takes all values in an interval of numbers. X: the time it takes for a bulb to burn out. The values are not countable.

7
**Example of cdf for discrete random variables**

Consider the r.v. X defined in example 2.

8
**Discrete Random Variable And Probability Mass Function**

Let X be a r.v. with cdf FX(x). If FX(x) changes value only in jumps and is constant between jumps, i.e. FX(x) is a staircase function, then X is called a discrete random variable. Suppose xi < xj if i<j. P(X=xi)=P(X≤xi) - P(X≤xj)= FX(xi) - FX(xi-1) Let px(x)=P(X=x) The function px(x) is called the probability mass function (pmf) of the discrete r.v. X. Properties of px(x):

9
**Example of pmf for discrete r.v.**

Consider the r.v. X defined in example 2.

10
**Continuous Random variable and Probability Density function**

Let X be a r.v. with cdf FX(x) . If FX(x) is continuous and also has a derivative dFX(x) /dx which exist everywhere except at possibly a finite number of points and is piecewise continuous, then X is called a continuous random variable. Let The function fX(x) is called the probability density function (pdf) of the continuous r.v. X . fX(x) is piecewise continuous. Properties:

11
**Conditional distribution**

Conditional probability of an event A given event B is defined as Conditional cdf FX(x|B) of a r.v. X given event B is defined as If X is discrete, then the conditional pmf pX(x|B) is defined by If X is continuous r.v., then the conditional pdf fX(x|B) is defined by

12
**Mean and variance Mean:**

The mean (or expected value) of a r.v. X, denoted by μX or E(X), is defined by Moment: The nth moment of a r.v. X is defined by Variance: The variance of a r.v. X, denoted by σX2 or Var(X), is defined by

13
**Expectation of a Function of a Random variable**

Given a r.v. X and its probability distribution (pmf in the discrete case and pdf in the continuous case), how to calculate the expected value of some function of X, E(g(X))? Proposition: (a) If X is a discrete r.v. with pmf pX(x), then for any real-valued function g, (b) If X is a continuous r.v. with pdf fX(x), then for any real-valued function g,

14
Limit Theorem Markov's Inequality: If X is a r.v. that takes only nonnegative values, then for any value a>0, Chebyshev's Inequality: If X is a random variable with mean μ and variance σ2, then for any value k>0

15
**Application of Limit theorem**

Suppose we know that the number of items produced in a factory during a week is a random variable with mean 500. (a) What can be said about the probability that this week's production will be at least 1000? (b) If the variance of a week's production is known to equal 100, then what can be said about the probability that this week's production will be between 400 and 600? Solution: Let X be number of item that will be produced in a week. (a) By Markov's inequality, P{X≥1000}≤E[X]/1000=0.5 (b) By Chebyshev's inequality, P{|X-500|≥100}≤ σ2/(100)2=0.01 P {|X-500|<100}≥1-0.01=0.99.

16
**Some Special Distribution**

Bernoulli Distribution Binomial Distribution Poisson Distribution Uniform Distribution Exponential Distribution Normal (or Gaussian) Distribution Conditional Distribution ……

17
**Bernoulli Random Variable**

An experiment with outcome as either a "success" or as a "failure" is performed. Let X=1 if the outcome is a "success" and X=0 if it is a "failure". If the pmf is given as following, such experiments are called Bernoulli trials, X is said to be a Bernoulli random variable. Note: 0 ≤ p ≤ 1 Example: Tossing coin once. The head and tail are equally likely to occur, thus p=0.5. pX(1)=P(H)=0.5, pX(1)=P(T)=0.5.

18
**Binomial Random Variable**

Suppose n independent Bernoulli trails, each of which results in a "success" with probability p and in a "failure with probability 1-p, are to be performed. Let X represent the number of success that occur in the n trials, then X is said to be a binomial random variable with parameters (n,p). Example: Toss a coin 3 times, X=number of heads. p=0.5

19
**Geometric Random Variable**

Suppose the independent trials, each having probability p of being a success, are performed until a success occurs. Let X be the number of trails required until the first success occurs, then X is said to be a geometric random variable with parameter p. Example: Consider an experiment of rolling a fair die. The average number of rolls required in order to obtain a 6:

20
**Poisson Random Variable**

A r.v. X is called a Poisson random variable with parameter λ(>0) if its pmf is given by An important property of the Poisson r.v. is that it may be used to approximate a binomial r.v. when the binomial parameter n is large and p is small. Let λ=np

21
**Uniform Random Variable**

A uniform r.v.X is often used when we have no prior knowledge of the actual pdf and all continuous values in some range seem equally likely.

22
**Exponential Random Variable**

The most interesting property of the exponential r.v. is "memoryless". X can be the lifetime of a component.

23
**Gaussian (Normal) Random Variable**

An important fact about normal r.v. is that if X is normally distributed with parameter μ and σ2, then Y=aX+b is normally distributed with paramter a μ+b and (a2 σ2); Application: central limit theorem-- the sum of large number of independent r.v.'s,under certain conditions can be approximated b a normal r.v. denoted by N(μ;σ2)

24
**The Moment Generating Function**

The important property: All of the moment of X can be obtained by successively differentiation.

25
**Application of Moment Generating Function**

The Binomial Distribution (n,p)

26
Entropy Entropy is a measure of the uncertainty in a random experiment. Let X be a discrete r.v. with SX={x1,x2, …,xk} and pmf pk=P[X=xk]. Let Ak denote the event {X=xk}. Intuitive facts: the uncertainty of Ak is low if pk is close to one, and it is high if pk is close to zero. Measure of uncertainty:

27
**Entropy of a random variable**

The entropy of a r.v. X is defined as the expected value of the uncertainty of its outcomes: The entropy is in units of ''bits'' when the logarithm is base 2 Independent fair coin flips have an entropy of 1 bit per flip. A source that always generates a long string of A's has an entropy of 0, since the next character will always be an 'A'.

28
**Entropy of Binary Random Variable**

Suppose r.v. X with Sx={0,1}, p=P[X=0]=1-P[X=1]. (Flipping a coin). The HX=h(p) is symmetric about p=0.5 and achieves its maximum at p=0.5; The uncertainty of event (X=0) and (X=1) vary together in complementary manner. The highest average uncertainty occurs when p(0)=p(1)=0.5;

29
**Reduction of Entropy Through Partial Information**

Entropy quantifies uncertainty by the amount of information required to specify the outcome of a random experiment. Example: If r.v. X equally likely takes on the values from set {000,001,010,…,111} (Flipping coins 3 times), given the event A={X begins with a 1}={100,101,110,111}, what is the change of entropy of r.v.X ?

30
Thanks! Question?

31
**Extending discrete entropy to the continuous case: differential entropy**

Quantization method: Let X be a continuous r.v. that takes on values in the interval [a b]. Divide [a b] into a large number K of subintervals of length ∆. Let Q(X) be the midpoint of the subinterval that contains X. Find the entropy of Q. Let xk be the midpoint of the kth subinterval, then P[Q= xk]=P[X is in kth subinterval]=P[xk-∆/2<X< xk+∆/2]≈ fX(xk) ∆ Trade off: ∆→0, HQ→∞ Differential Entropy is defined as

32
**The Method of Maximum Entropy**

The maximum entropy method is a procedure for estimating the pmf or pdf of a random variable when only partial information about X, in the form of expected values of functions of X, is available. Discrete case: X being a r.v. with Sx={x1,x2,…,xk} and unknown pmf px(xk). Given the expected value of some function g(X) of X:

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google