Permutations & Combinations and Distributions

Permutations & Combinations and Distributions
Krishna.V.Palem Kenneth and Audrey Kennedy Professor of Computing Department of Computer Science, Rice University

Take Home II - Generalizing the sum of expectations result (hint)
2. Prove that the expectation of sum of n random variables is equal to the sum of expectation of the n random variables. Let x1, x2, x3…. xn be n random variables Let z = x1 + x2 + x3…. + xn To prove

Hint for the proof Use the result E(X+Y)=E(X)+E(Y) to generalize for n random variables Consider E(X1 + X2 + X3…. + Xn ) Let X2 + X3…. + Xn = Y1 Then E(X1 + Y1) = E(X1) + E(Y1) Now consider X3…. + Xn = Y2 and repeat the same procedure

Contents Permutations and Combinations
Calculating probabilities using combinations Distribution Proof of Law of Large Numbers Binomial Distribution Normal Distribution

Permutations vs. Combinations
Both are ways to count the possibilities The difference between them is whether order matters or not Consider a 5-card hand: A♦, 5♥, 7♣, 10♠, K♠ Is that the same hand as: K♠, 10♠, 7♣, 5♥, A♦ Does the order the cards are handed out matter? If yes, then we are dealing with permutations If no, then we are dealing with combinations

Permutations A permutation is an ordered arrangement of the elements of some set S Let S = {a, b, c} c, b, a is a permutation of S b, c, a is a different permutation of S An r-permutation is an ordered arrangement of r elements of the set A♦, 5♥, 7♣, 10♠, K♠ is a 5-permutation of the set of cards The notation for the number of r-permutations: P(n,r) For example, poker hand is one of P(52,5) permutations

Permutations Number of poker hands (5 cards):
r-permutation notation: P(n,r) The poker hand is one of P(52,5) permutations

Deriving the formula of Permutations
There are n ways to choose the first element n-1 ways to choose the second n-2 ways to choose the third … n-r+1 ways to choose the rth element By the product rule, that gives us: P(n,r) = n(n-1)(n-2)…(n-r+1)

Combinations What if order doesn’t matter?
In poker, the following two hands are equivalent: A♦, 5♥, 7♣, 10♠, K♠ K♠, 10♠, 7♣, 5♥, A♦ The number of r-combinations of a set with n elements, where n is non-negative and 0≤r≤n is:

Deriving the formula for Combinations
Let C(n,r) be the number of ways to generate unordered combinations The number of ordered combinations (i.e. r-permutations) is P(n,r) The number of ways to order a single one of those r-permutations P(r,r) The total number of unordered combinations is the total number of ordered combinations (i.e. r-permutations) divided by the number of ways to order each combination Thus, C(n,r) = P(n,r)/P(r,r) (1)

Deriving the formula for Combinations
But from the derivation of permutation formula, we know that Hence, substituting n=r, we get Replacing (2) and (3) in (1), we get (since, 0! = 1) (2) (3)

In-class Exercise - 1 Card Terminology: face value – same number cards (2-10, J, Q, K, A) has 4 cards of same face value suite – set of cards with same symbol four suites – diamond, heart, spade, clubs each suite has 13 cards Q) In a standard deck of cards, compute the number of ways you can deal each of the following five-card hands in poker. 1. Total number of different possible hands (five cards in a hand) 2. Number of distinct Flush (all 5 cards have the same suite) 3. Number of distinct Four of a kind (4 same face value cards) A) 1. C (52,5) 2. C (13,5) * C (4,1) 3. C (13,1) * C (48,1)

In-Class Exercise -2 Q) Now, compute the probability of getting a flush in a five- card poker game? A) Number of favorable events = C (13,5) * C (4,1) Total no. of events = C (52,5) Hence, Probability = C (13,5) * C (4, 1)/ C ( 52,5) Probability of outcome No. of favorable events Total no. of events

In-Class Exercise - 3 Consider an example: In an experiment of 20 coin tosses, we want to calculate the probability of heads falling exactly 5 times. How do we do this? Solution: Probability of heads in 1 coin toss = ½ Probability of heads falling in 5 of the coin tosses = ½* ½* ½* ½* ½ = (1/2)5 (Method of intersection of events) Probability of heads not falling in 1 coin toss = ½ Probability of heads not falling in the rest (20-5=15) coin tosses = (1/2)15

Use of Combinations to Calculate Probabilities
Hence, the probability of getting exactly 5 heads out of 20 tosses = (1/2)5 *(1/2)15 =(1/2)20 Q) Did we account for which of the coin tosses had an event HEAD? A) No Q) How do we account for it? Permutations or Combinations? A) Combinations, as the order of selection is not important Is this correct?

Q) How do we select 5 tosses out of 20 tosses with heads outcome using combinations? Let us make a table of all possible outcomes of 20 coins which have 5 HEADs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 H T … We can see that this table can be generated by choosing 5 places out of the 20 places where H can occur. Thus the total number of such combinations would be C(20,5)

Hence, the probability of getting exactly 5 heads out of 20 coin tosses is given by = C (20,5) * (1/2)20 How do we generalize this method of computing probabilities? Consider an example: In an experiment of 20 “biased” coin tosses, we want to calculate the probability of heads falling exactly 5 times. How do we do this? Given probability of HEAD = p Question

Consider an example: In an experiment of 20 “biased” coin tosses, we want to calculate the probability of heads falling exactly 5 times. How do we do this? Assume that the probability of HEAD = p Solution: Probability of heads in 1 coin toss = p Probability of heads falling in 5 of the coin tosses = p*p*p*p*p = (p)5 Probability of heads not falling in 1 coin toss = 1-p Probability of heads not falling in the rest (20-5=15) coin tosses = (1- p)15

If we generalize the number of trials and the number of HEADs or successes also we obtain Assume that in n trails of an event we want to compute the probability P of getting k successes when the probability of success in each trial is p We denote this by the following expression P(number of heads=k) = C(n,k) * pk * (1-p)n-k Binomial Distribution

Schedule for the next 2 weeks
5 Oct – Tutorial Session II Covers Expectations, Permutations & Combinations, Basic Distributions 7 Oct – Mini Project 1 15 % of Final Grade Can do it as a take home if the time provided in the class is not sufficient 12 Oct – Fall Break Holiday 14 Oct – Project Proposal report due and in class discussion on the proposals

Project Discussion

Calculating probabilities using combinations Distribution Binomial Distribution Normal Distribution

What is a distribution? Distribution Consider the following experiment
Define a variable x which takes as many values as the number of events Event Event 1 Event 2 ..Etc. Probabilities p1 p2 ..Etc. Event X Event 1 1 Event 2 2 ..Etc. Therefore using the probabilities of the events, we can define a function which relates the variable x and the probabilities of the events Distribution Probability (Event i) Where i={1,2,…} Here ‘x’ is called a random variable.

probability of the random variable taking a particular value
What is a distribution? A distribution is a function defined on the random variable that gives the value of the probability of the random variable taking a particular value The probability distribution describes the range of possible values that a random variable can attain and the probability that the value of the random variable is within any (measurable) subset of that range. Examples of a Distribution i p(i) 1 1/6 2 3 4 5 6 Uniform Distribution Binomial Distribution Guassian Distribution

Example of a Distribution
Suppose you flip a coin two times. This experiment can have four possible outcomes: HH, HT, TH, and TT. Now, let the variable random X represent the number of Heads that result from this experiment. X can take on the values 0, 1, or 2. The table, equation and graph below, which associate each outcome with its probability, are all representations of probability distribution for above example. Distribution Table Distribution Equation Distribution graph

Video on Terms in Distributions

Variance Variance of a random variable or probability distribution is a measure of statistical dispersion, averaging the squared distance of its possible values from the expected value (mean). If random variable X has expected value (mean) μ = E(X), then the variance Var(X) of X is given by: Variance

Standard Deviation Standard deviation is the positive square root of the variance. It is given by: Low standard deviation indicates that the data points tend to be very close to the same value (the mean), while high standard deviation indicates that the data are “spread out” over a large range of values A plot of a normal distribution (or bell curve). Each colored band has a width of one standard deviation.

Useful derivation for Variance
In probability theory, the computational formula for the variance Var(X) of a random variable X is the formula Derivation (from definition) (expansion of expectation formula)

Law of Large Numbers The law of large numbers (LLN) describes the long-term stability of the mean of a random variable. Given a random variable with a finite expected value, if its values are repeatedly sampled, as the number of these observations increases, their mean will tend to approach and stay close to the expected value for example, consider the coin toss experiment. The frequency of heads (or tails) will increasingly approach 50% over a large number of trials. Mathematically, it can be represented as, if Mean is , then

Proof of Law of Large Numbers
First, let us derive the Chebyshev Inequality which simplifies the derivation of law of large numbers Chebyshev Inequality: Let X be a discrete random variable with expected value µ= E(X), and let > 0 be any positive real number Let m(x) denote the distribution function of X. Then the probability that X differs from µ by at least is given by Proof of Chebyshev Inequality

We know that, But, V(X) is clearly at least as large as Replacing (x- µ)2 with , to get a lower bound, Hence, we get

Let X1, X2, , Xn be an independent trials process, with finite expected value µ = E(Xj) and finite variance = V (Xj ). Let Xn be the mean of X1,X2,… Xn. Hence, Equivalently, But from Chebyshev’s inequality, we have

Replacing X with Xn, we get Hence, we get As n approaches infinity, the expression approaches 1. Hence, we have obtained,

Binomial Distribution
Binomial distribution is the discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p It can be applied in a wide variety of practical situations for k = 0,1,2,3…. n, where is called the ‘Binomial Coefficient’

Binomial distribution is a very interesting distribution in the sense that it can be applied in a wide variety of practical situations. An example, Assume 5% of a very large population to be green-eyed. You pick 40 people randomly. The number of green-eyed people you pick is a random variable X which follows a binomial distribution with n = 40 and p = Let us see how this distribution varies with different values of n and p with respect to X.

For the previous example, this graph shows the variation in probability Notice how it peaks in the middle and dies away at the ends probability(p) X=number of green eyed people Another elementary example of a binomial distribution is: Roll a standard die ten times and count the number of sixes. Denote the number of sixes by the random variable X The distribution of this random number X is a binomial distribution with n = 10 and p = 1/6. Can you plot this distribution and see how it varies with X

In-Class Exercise Let us try out an example of a binomial distribution: Consider a standard die roll for 20 times Q) Denote the number of times the outcome of the roll an even number by a random variable X. Compute the probability distribution of X = 8 for this event. Q) Denote the number of times the outcome of the roll is ‘6’ by the random variable Y. Compute the probability distribution of Y equal to 4 for this event. Q) Denote the number of times the outcome of the roll is ‘2’ by the random variable Z. Compute the probability distribution of Z less than or equal to 4 for this event. Use Binomial Distribution to solve these questions.

Attributes of Binomial Distribution
If X ~ B(n, p) (that is, X is a binomially distributed random variable with total ‘n’ events and probability of success ‘p’ in each event), Expected value or mean of X is Variance of X is Standard deviation of X is

Video on Binomial Distribution : A Summary

Derivation of Variance of Binomial Distribution
We have seen that variance is equal to In using this formula we see that we now also need the expected value of X 2: We can use our experience gained before in deriving the mean. We know how to process one factor of k. This gets us as far as

(again, with m = n − 1 and s = k − 1). We split the sum into two separate sums and we recognize each one The first sum is identical in form to the one we calculated in the Mean (above). It sums to mp. The second sum is unity. Using this result in the expression for the variance, along with the Mean (E(X) = np), we get

Deriving the Expectation of Binomial Distribution
If X ~ B(n, p) (that is, X is a binomially distributed random variable with total ‘n’ events and probability of success ‘p’ in each event), then the expected value of X is We apply the definition of the expected value of a discrete random variable to the binomial distribution The first term in the summation (for k=0) equals to 0 and can be removed. In the rest of the summation, we expand the C(n,k) term,

Deriving the Expectation of Binomial Distribution
Since n and k are independent of the sum, we get Assume, m = n − 1 and s = k − 1. Limits are changed accordingly where x=1-p, y=p, m=n & s=k Hence, as (x+y) = ((1-p)+p) = 1, we get This is similar to the expansion of a binomial theorem

We have seen that variance is equal to We now compute the value of E(X2): Use a similar approach as in the derivation of the mean to expand C(n,k) assume m = n − 1 and s = k − 1

We split the sum into two separate sums The first sum is identical in form to the one we calculated in the Mean (above). It sums to mp. The second sum is unity (binomial theorem). Hence, we get

In-Class Exercise Let us continue the previous example of the binomial distribution: Consider a standard die roll for 100 times instead of 20 times Q) Denote the number of times the outcome of the roll is ‘2’ by the random variable X. Compute the probability distribution of X greater than or equal to 60 for this event. What if we consider the die roll a million times and need to compute the probability that X is greater than or equal to 100,000 for this event? Difficult Impossible!

How to Compute Distributions for Large ‘N’?
Abraham de Moivre noted that the shape of the binomial distribution approached a very smooth curve when the number of events increased he considered a coin toss experiment De Moivre tried to find a mathematical expression for this curve to find the probabilities involving large number of events more easily. led to the discovery of the Normal curve

Example by De Moivre Coin Toss Experiment
Random variable X = Number of heads Number of events ‘N’ increases Can be approximated as a curve

Video on Galton Board Game
Demonstrates how Binomial distribution gives rise to a Normal/Gaussian distribution as number of trials/events tends to infinity

Calculating probabilities using combinations Distribution Binomial Distribution Normal Distribution

Video on Normal Distribution
First 2 mins only

Normal Distribution To indicate that a real-valued random variable X is normally distributed with mean μ and variance σ2 ≥ 0, we write The normal distribution is defined by the following equation: All normal distributions are symmetric and have bell-shaped density curves with a single peak. Note: Normal distribution is a continuous probability distribution while Binomial distribution is a discrete probability distribution

In-Class Exercise Let us try out an example of a normal distribution:
Consider a coin toss experiment for 1000 tosses Q) Denote the number of times the outcome of the toss is heads by a random variable X. Compute the probability distribution of X occurring at most 600 times. A) A) Since, the original event is a binomial distribution and we use normal distribution to approximate it, we can use µ=np & = np(1-p). Hence, x<=600; µ = 1000*1/2 = 500 and = 1000*1/2*(1-1/2) =250 Substituting this in the normal distribution equation, we get Calculating, we get Probability of x<=600 = How would you use Binomial Distribution to solve this question? Difficult How would you use Normal Distribution to solve this question? Source of calculation:

Examples of Few Applications of Normal Distribution
Approximately normal distributions occur in many situations In counting problems Binomial random variables, associated with yes/no questions; Poisson random variables, associated with rare events; In physiological measurements of biological specimens: logarithm of measures of size of living tissue (length, height, weight); length of inert appendages (hair, claws, nails, teeth) of biological specimens, in the direction of growth Measurement errors Financial variables Light intensity intensity of laser light is normally distributed;

Normal Distribution To indicate that a real-valued random variable X is normally distributed with mean μ and variance σ2 ≥ 0, we write The normal distribution is defined by the following equation: All normal distributions are symmetric and have bell-shaped density curves with a single peak. Note: Normal distribution is a continuous probability distribution while Binomial distribution is a discrete probability distribution

In-Class Exercise Let us try out the previously stated “nearly impossible” problem using a normal distribution: Consider a coin toss experiment for 1,000,000 tosses Q) Denote the number of times the outcome of the toss is heads by a random variable X. Compute the probability distribution of X occurring at most 100,000 times. A) How would you use Binomial Distribution to solve this question? Difficult How would you use Normal Distribution to solve this question?

In-Class Exercise Since, the original event is a binomial distribution and we can use normal distribution to approximate it. We know that µ=np & = np(1-p). Hence, x<=100000; µ = 1,000,000*1/2 = 500,000 and = 1,000,000*1/2*(1-1/2) =250,000 Substituting this in the normal distribution equation, we get Calculating the integral with limits from 0 to 100,000; we get Probability of x<=100,000 = Source of calculation:

Examples of Few Applications of Normal Distribution
Approximately normal distributions occur in many situations In counting problems Binomial random variables, associated with yes/no questions; Poisson random variables, associated with rare events; In sports statistical analyses: calculating mean physical attributes like heights, weights etc and their standard deviations estimating the probabilities of winning the games Measurement errors Financial variables Light intensity intensity of laser light is normally distributed;

Example Application of Bayes Theorem

Permutations & Combinations and Distributions

Similar presentations

Presentation on theme: "Permutations & Combinations and Distributions"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Permutations & Combinations and Distributions

Similar presentations

Presentation on theme: "Permutations & Combinations and Distributions"— Presentation transcript:

Similar presentations

About project

Feedback