ECE 5345 Sums of RV’s and Long-Term Averages The Law of Large Numbers

Slides:

Advertisements

Similar presentations

Copyright R.J. Marks II EE 505 Sums of RV’s and Long-Term Averages (The Central Limit Theorem)

Advertisements

Random Variable A random variable X is a function that assign a real number, X(ζ), to each outcome ζ in the sample space of a random experiment. Domain.

ELEC 303 – Random Signals Lecture 18 – Statistics, Confidence Intervals Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 10, 2009.

STAT 270 What’s going to be on the quiz and/or the final exam?

Review of Basic Probability and Statistics

Chapter 1 Probability Theory (i) : One Random Variable

Copyright Robert J. Marks II ECE 5345 Random Processes - Example Random Processes.

Descriptive statistics Experiment  Data  Sample Statistics Sample mean Sample variance Normalize sample variance by N-1 Standard deviation goes as square-root.

1 Chap 5 Sums of Random Variables and Long-Term Averages Many problems involve the counting of number of occurrences of events, computation of arithmetic.

SUMS OF RANDOM VARIABLES Changfei Chen. Sums of Random Variables Let be a sequence of random variables, and let be their sum:

Estimation of parameters. Maximum likelihood What has happened was most likely.

Determining Sample Sizes For Estimating . Situation We want to estimate the average life of certain batteries –We know it costs about $15 to test each.

A random variable that has the following pmf is said to be a binomial random variable with parameters n, p The Binomial random variable.

Class notes for ISE 201 San Jose State University

1 Confidence Intervals for Means. 2 When the sample size n< 30 case1-1. the underlying distribution is normal with known variance case1-2. the underlying.

Chapter 21 Random Variables Discrete: Bernoulli, Binomial, Geometric, Poisson Continuous: Uniform, Exponential, Gamma, Normal Expectation & Variance, Joint.

Standard error of estimate & Confidence interval.

1/2555 สมศักดิ์ ศิวดำรงพงศ์

1 As we have seen in section 4 conditional probability density functions are useful to update the information about an event based on the knowledge about.

Tch-prob1 Chap 3. Random Variables The outcome of a random experiment need not be a number. However, we are usually interested in some measurement or numeric.

Statistics for Engineer Week II and Week III: Random Variables and Probability Distribution.

Week11 Parameter, Statistic and Random Samples A parameter is a number that describes the population. It is a fixed number, but in practice we do not know.

1 6. Mean, Variance, Moments and Characteristic Functions For a r.v X, its p.d.f represents complete information about it, and for any Borel set B on the.

PROBABILITY AND STATISTICS FOR ENGINEERING Hossein Sameti Department of Computer Engineering Sharif University of Technology Mean, Variance, Moments and.

7 sum of RVs. 7-1: variance of Z Find the variance of Z = X+Y by using Var(X), Var(Y), and Cov(X,Y)

EE 5345 Multiple Random Variables

The final exam solutions. Part I, #1, Central limit theorem Let X1,X2, …, Xn be a sequence of i.i.d. random variables each having mean μ and variance.

Confidence Interval & Unbiased Estimator Review and Foreword.

IE 300, Fall 2012 Richard Sowers IESE. 8/30/2012 Goals: Rules of Probability Counting Equally likely Some examples.

Chapter 4-5 DeGroot & Schervish. Conditional Expectation/Mean Let X and Y be random variables such that the mean of Y exists and is finite. The conditional.

Bayesian Prior and Posterior Study Guide for ES205 Yu-Chi Ho Jonathan T. Lee Nov. 24, 2000.

Week 121 Law of Large Numbers Toss a coin n times. Suppose X i ’s are Bernoulli random variables with p = ½ and E(X i ) = ½. The proportion of heads is.

1 Probability and Statistical Inference (9th Edition) Chapter 5 (Part 2/2) Distributions of Functions of Random Variables November 25, 2015.

Probability and Moment Approximations using Limit Theorems.

Lecture 15 Parameter Estimation Using Sample Mean Last Time Sums of R. V.s Moment Generating Functions MGF of the Sum of Indep. R.Vs Sample Mean (7.1)

Sums of Random Variables and Long-Term Averages Sums of R.V. ‘s S n = X 1 + X X n of course.

Basic statistics Usman Roshan.

Introduction to Probability - III John Rundle Econophysics PHYS 250

Inference about the slope parameter and correlation

Renewal Theory Definitions, Limit Theorems, Renewal Reward Processes, Alternating Renewal Processes, Age and Excess Life Distributions, Inspection Paradox.

Ch3.5 Hypergeometric Distribution

3. Random Variables (Fig.3.1)

Math 4030 – 4a More Discrete Distributions

Probability Distributions: a review

ASV Chapters 1 - Sample Spaces and Probabilities

Chapter 4 Continuous Random Variables and Probability Distributions

12. Principles of Parameter Estimation

Basic statistics Usman Roshan.

Appendix A: Probability Theory

The Bernoulli distribution

Multinomial Distribution

5.6 The Central Limit Theorem

Using the Tables for the standard normal distribution

Chernoff bounds The Chernoff bound for a random variable X is

The University of Texas at Dallas

ECE 7251: Signal Detection and Estimation

Determining Which Method to use

9. Limit Theorems.

11. Conditional Density Functions and Conditional Expected Values

8. One Function of Two Random Variables

ECE 5345 Sums of RV’s and Long-Term Averages Confidence Intervals

Bernoulli Trials Two Possible Outcomes Trials are independent.

Determination of Sample Size

11. Conditional Density Functions and Conditional Expected Values

copyright Robert J. Marks II

12. Principles of Parameter Estimation

Further Topics on Random Variables: 1

Berlin Chen Department of Computer Science & Information Engineering

8. One Function of Two Random Variables

Continuous Distributions

Presentation transcript:

ECE 5345 Sums of RV’s and Long-Term Averages The Law of Large Numbers

Sum of RV’s Sum Mean of the sum If i.i.d.

Sum of RV’s Variance

Sum of RV’s Variance: Special case: independent RV’s The Kronecker delta is Thus…

Sum of RV’s For i.i.d. RV’s, if The mean is… The variance is…

Sum of RV’s: The Characteristic Function For i.i.d. RV’s, if Then… And… Note: can find the same result for mean and variance by differentiation and substitution.

Recall: for i.i.d. RV’s Then… S is Gaussian if the Xk’s are Gaussian. S is Poisson if the Xk’s are Poisson. S is Binomial if the Xk’s are Binomial. S is Gamma if the Xk’s are Gamma. S is Cauchy if the Xk’s are Cauchy. S is Negative Binomial if the Xk’s are Negative Binomial

The Average (Sample Mean) Then…

The Average If samples are i.i.d… Thus…

The Average of i.i.d. RV’s The average… If samples are i.i.d… And…

Mean and Variance of the Average of i.i.d. RV’s Thus… Or… As it should be!

Mean and Variance of the Average of i.i.d. RV’s Thus… Or… Makes Sense!

Mean and Variance of the Average of i.i.d. RV’s A for bigger n pdf’s… A X x X

Apply Chebyshev to Average of i.i.d. RV’s Since… And… Thus…

The Weak Law of Large Numbers This gives, in the limit, the “weak law of large numbers” The average, in this sense, approaches the mean.

The Strong Law of Large Numbers This is a stronger statement:

Sloppy Confidence Intervals on Bernoulli Trials Unknown p parameter in Bernoulli trial. E(X)=E(A)=p, var(X)=p(1-p) ¼ The weak law of large numbers gives:

Sloppy Confidence Intervals on Bernoulli Trials (cont) Assume we want to be 95% or more sure that the result is within 1% of being correct. How many samples do we need to take to assure this? Set Solving gives n=50,000. Chebyshev gives loose bounds. There are better ways to do this.

Sample Variance The average (or sample mean) is used to estimate the variance. The sample variance is used to estimate the variance. Note: A and V can be shown to be independent RV’s.