STATISTIC & INFORMATION THEORY (CSNB134) MODULE 4 PRINCIPLES OF PROBABILITY

Points to Ponder Activities pertaining computer information systems are basically surrounded by uncertainty. For example, the chances that our hard disk will crash within 2 years, the chances that data will get corrupted over a transmission channel, the accuracy of an error detection mechanism etc. – we can never be certain of these events. However, it is important that we are able to induce these measurements as such we could take safeguards in protecting our data and our IT infrastructure. Hence :- the applications of probability

Probability Probability allows us to use sample data to conclude about the population. Probability distribution models some random variables of the population. Probability can be calculated given data obtained from observations. An experiment is the process by which an observation (or measurement) is obtained. An event is an outcome of an experiment. ExperimentEvents Record an ageA: person is 30 years old B: person is older than 65 Toss a dieA: observe an odd number B: observe a number greater than 2

Overview of Probability ExperimentEvents Alien encounterA: An alien came up to my front door P(A) = 0????

Sample Space. An event that cannot be decomposed is called a simple event. Denoted by E with a subscript. Each simple event will be assigned a probability, measuring “how often” it occurs., The set of all simple events of an experiment is called the sample space, S. E.g. Toss a die S={E 1, E 2, E 3, E 4, E 5, E 6 } : Events: A: observe an odd number {E 1,E 3,E 5 } B: observe a number greater than 2 {E 3,E 4,E 5,E 6 } C: observe a 6 {E 6 } D: observe a 3 {E 3 }

Mutual Exclusive Two events are mutually exclusive if, when one event occurs, the other cannot, and vice versa.  E.g. Experiment: Toss a die :  Events: A: observe an odd number {E 1,E 3,E 5 } B: observe a number greater than 2 {E 3,E 4,E 5,E 6 } C: observe a 6 {E 6 } D: observe a 3 {E 3 }  Mutual Exclusive? (1) A & B – NO (2) C & D – YES (3) A & C – YES (4) B & D - NO

The Probability of an Event The probability of an event A measures “how often” we think A will occur. We write P(A). Suppose that an experiment is performed n times. The relative frequency for an event A is If we let n get infinitely large,

The Probability of an Event (cont.) P(A) must be between 0 and 1. If event A can never occur, P(A) = 0. If event A always occurs when the experiment is performed, P(A) =1. The sum of the probabilities for all simple events in S equals 1.  The probability of an event A is found by adding the probabilities of all the simple events in A. E.g. Experiment: Toss a die EventsProbability A: observe an odd numberP(A) = 3/6 = 1/2 B: observe a number greater than 2P(B) = 4/6 = 2/3 C: observe a 6P(C) = 1/6

Exercise 1 Toss a fair coin twice. What is the probability of observing at least one head? H H 1st Coin 2nd Coin E i P(E i ) H H T T T T H H T T HH HT TH TT 1/4 P(at least 1 head) = P(E 1 ) + P(E 2 ) + P(E 3 ) = 1/4 + 1/4 + 1/4 = 3/4 P(at least 1 head) = P(E 1 ) + P(E 2 ) + P(E 3 ) = 1/4 + 1/4 + 1/4 = 3/4

Exercise 2 A bowl contains three M&Ms ®, one red, one blue and one green. A child selects two M&Ms at random. What is the probability that at least one is red? 1st M&M 2nd M&M E i P(E i ) RB RG BR BG 1/6 P(at least 1 red) = P(RB) + P(RG) +P(BR) + P(GR) = 1/6 + 1/6 + 1/6 + 1/6 = 4/6 = 2/3 P(at least 1 red) = P(RB) + P(RG) +P(BR) + P(GR) = 1/6 + 1/6 + 1/6 + 1/6 = 4/6 = 2/3 m m m m m m m m m GB GR

Counting Rules Fortunately, there are several counting rules that can be applied in order to calculate the probability of events of an experiment. - the mn rules - permutations - combinations

The mn Rule Example: Toss two coins. The total number of simple events is: 2 X 2 = 4 If an experiment is performed in two stages, with m ways to accomplish the first stage and n ways to accomplish the second stage, then there are mn ways to accomplish the experiment. This rule is easily extended to k stages, with the number of ways equal to n1 n2 n3 … nk

Exercise 3 Example: Toss three coins. The total number of simple events is : Example: Two M&Ms are drawn from a dish containing two red and two blue candies. The total number of simple events is: Example: Toss two dice. The total number of simple events is: 6 X 6 = 36 4 X 3 = 12 2 X 2 X 2 = 8

Exercise 4 Example: In data networking, a packet can take 2 routes from Router A to Router B, 3 routes from Router B to router C and 3 routes from Router C to Router D respectively. How many possible A to D routes are available? If one link between Router B and C is down, how many possible A to D routes are available?: 2 X 3 X 3 = 18 2 X 2 X 3 = 12

Permutations The number of ways you can arrange n distinct objects, taking them r at a time is Example: How many 3-digit lock combinations can we make from the numbers 1, 2, 3, and 4? The order of the choice is important!

Exercise 5 Example: A lock consists of five parts and can be assembled in any order. A quality control engineer wants to test each order for efficiency of assembly. How many orders are there? The order of the choice is important!

Combinations The number of distinct combinations of n distinct objects that can be formed, taking them r at a time is Example: Three members of a 5-person committee must be chosen to form a subcommittee. How many different subcommittees could be formed? The order of the choice is not important!

Exercise 6 A box contains six M&Ms ®, four red and two green. A child selects two M&Ms at random. What is the probability that exactly one is red? The order of the choice is not important! 4  2 =8 ways to choose 1 red and 1 green M&M. P( exactly one red) = 8/15

In a Nutshell……… How to differentiate between Permutation & Combination? The ‘MADU 3’ Scenario E.g. Abu is given a chance to choose 3 women as his wives amongst five pretty women Aminah, Salmah, Eton, Esah and Joyah. Case 1: What is the probability that Eton, Esah and Joyah are chosen as the first, second and third wife respectively? Order is important!! First wife must be Eton!! Use Permutations!! Case 2: What is the probability that Eton, Esah and Joyah are chosen as the wives? Order is not important!! Anyone can be the first wife. Use Combinations!!

Something to ponder…. Practice makes PURRRfect!! U may do exercises from any text books on Probability & Statistics; amongst those that I am using:- Mendenhall, Beaver & Beaver, ‘Introduction to Probability & Statistics’, 12 th Edition, Thomson Brooks

STATISTIC & INFORMATION THEORY (CSNB134) PRINCIPLES OF PROBABILITY --END--