Dr. Ahmed Abdelwahab Introduction for EE420

Probability Theory Probability theory is rooted in phenomena that can be modeled by an experiment with an outcome that is subject to chance. Moreover, if the experiment is repeated, the outcome can differ because of the influence of an underlying random phenomenon or chance mechanism. Such an experiment is referred to as a random experiment. The following three features describe the random experiment: 1. The experiment is repeatable under identical conditions. 2. On any trial of the experiment, the outcome is unpredictable. 3. For a large number of trials of the experiment, the outcomes exhibit statistical regularity; that is, a definite average pattern of outcomes is observed if the experiment is repeated a large number of times.

Let event A denote one of the possible outcomes of a random experiment. For example, in the coin-tossing experiment, event A may represent "heads." Suppose that in n trials of the experiment, event A occurs N(A) times. We may then assign the ratio N(A)/n to the event A. This ratio is called the relative frequency of the event A. Clearly, the relative frequency is a nonnegative real number less than or equal to one. That is to say, and the probability of event A is defined as

The total of all possible outcomes of the random experiment is called the sample space, which is denoted by S. An event corresponds to either a single sample point or a set of sample points. In particular, the entire sample space S is called the sure event; the null set ϕ is called the null or impossible event; and a single sample point is called an elementary event.

A probability system consists of the following three aspects: 1. A sample space S of elementary events (outcomes). 2. A class ξ of events that are subsets of S. 3. A probability measure P(.) assigned to each event A in the class ξ, which has the following properties: (i) P(S) = 1 (ii)0 ≤ P(A) ≤ 1 (iii) If A + B is the union of two mutually exclusive events in the class ξ, then P(A + B) = P(A) + P(B) Properties (i), (ii), and (iii) are known as the axioms of probability. Axioms (i), (ii), and (iii) constitute an implicit definition of probability.

Joint Probability P(not a) = 1 − P(a). Union of a & b = P(a or b) = P(a+b) = P(a)+P(b) − P(a and b). We will often denote P(a and b) as the intersection (joint probability) of a & b by P(a,b). i.e. P(a,b) = P(a)+P(b) − P(a+b). If P(a,b) = 0, we say a and b are mutually exclusive.

Conditional probability P(a|b) is the probability of a, given that we know b P(a|b) is called Conditional probability of a given b. The joint probability of both a and b is given by: P(a, b) = P(a|b) P(b) and since P(a,b) = P(b,a), we have Bayes’Theorem : P(a,b) = P(a|b)P(b) = P(b,a) = P(b|a)P(a)

Statistical Independence If two events a and b are such that P(a|b) = P(a), then, the events a and b are statistically independent (SI). Note that from Bayes’Theorem, also, P(b|a) = P(b), therefore, P(a,b) = P(a|b)P(b) = P(a)P(b). This last equation is often taken as the definition of statistical independence.

Statistics of Continuous Random Variable

Gaussian Distribution The random variable Y has a Gaussian distribution if its probability density function has the form: where µ Y is the mean and σ Y 2 is the variance of the random variable Y. Normalized Gaussian distribution has µ Y =0 and σ Y 2 =1 as shown in the figure.