1. STT 315 This lecture note is based on Chapter 3
Acknowledgement: Author is thankful to Dr. Ashok Sinha, Dr. Jennifer Kaplan and Dr. Parthanil Roy for allowing him to use/edit some of their slides.

2. Some terminologies
Probability is used to infer probable feature(s) of sample taken from a population.
An experiment is an action that leads to one of several possible outcomes.
An outcome of an experiment is also called a sample point.
The collection of all possible outcomes (i.e. sample points) of an experiment is known as sample space [will be mostly denoted by S].
An event is a subset of the sample space with which we can attach a probability.
We say “the event A occurred” if an outcome from A resulted in the experiment.

3. Examples
Suppose we toss a coin and observe the resulting face at the top.
Experiment: tossing the coin.
Sample points: Head (H) and Tail (T).
Sample space: S = {H, T}.
Events: {H}, {T}, {H,T} and Φ (the empty set).
Suppose we throw a die and observe the resulting face at the top.
Experiment: throwing the die.
Sample points: 1, 2, 3, 4, 5 and 6.
Sample space: S = {1, 2, 3, 4, 5, 6}.
Events: any subset of S, e.g. {1}, or {2,4,6}, {3, 4, 5, 6} etc.

4. Modeling Probability
Suppose you have a fair/unbiased coin (i.e., a coin with equal chance of getting a head or a tail).
Question: If you toss a fair coin, what is the chance of getting a head?
Answer: 50% = ½.

5. Modeling Probability
Suppose you have a fair/unbiased die (i.e., a die with equal chance of getting all the six numbers 1-6).
Question: If you toss a fair die, what is the chance of getting a 5?
Answer: 1/6.

6. Modeling Probability
Suppose you have a fair/unbiased die (i.e., a die with equal chance of getting all the six numbers 1-6).
Question: If you toss a fair die, what is the chance of getting an even outcome (2 or 4 or 6)?
Answer: 3/6 = ½ = 50%.

7. In General
We typically denote events (whose probabilities are calculated) by capital letters A, B, C etc. Probability of an event A,
Number of outcomes in A
P(A) =
Total number of possible outcomes
provided all the outcomes are equally likely.

8. Example Suppose you toss a fair coin twice.
What is the probability that the two outcomes are different?
All possible outcomes = {HH, HT, TH, TT}.
Since the coin is fair, all the 4 outcomes are equally likely.
Let A denote the event that the two outcomes are different. Then A = {HT, TH}.
P(A)= 2/4 = ½ .

9. Example Suppose you toss a fair coin twice.
What is the probability that at least one head occurred?
All possible outcomes = {HH, HT, TH, TT}.
Since the coin is fair, all the 4 outcomes are equally likely.
Let A denote the event that at least one had occurred. Then A = {HH, HT, TH}.
P(A)= ¾.

10. Rules of Probability Basic rules:
Probability of any event is a number between 0 and 1, i.e. for any event A, 0≤𝑃 𝐴 ≤1.
P 𝑆 =1.
P 𝐴 𝑐 =𝑃 𝑛𝑜𝑡 𝐴 =1−𝑃 𝐴 .
P 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵 ,
i.e., 𝑃 𝐴 𝑜𝑟 𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴 𝑎𝑛𝑑 𝐵 .

11. Complement of a set
The complement of an event A (denoted by Ac) is the event that A does not occur.
Ac means “not A”.
P 𝐴 𝑐 =𝑃 𝑛𝑜𝑡 𝐴 =1−𝑃 𝐴 .

12. An Example
Suppose that the probability that the traffic light is green on a crossing is 35%. What is the chance that the light is not green?
P(not green) = 1-P(green) = = 0.65.

13. Intersection of sets
“A and B” denotes the event that A and B both occur, denoted by 𝐴∩𝐵.
It consists of all outcomes that belong to both A and B.
𝑨∩𝑩

14. Union of sets
“A or B” means either A occurs or B occurs or both occur (i.e., “or” means and/or). It is denoted by 𝐴∪𝐵, and is called “A union B”.
In this picture,
“𝐴∪𝐵” is the complete colored region.
(𝐴∪𝐵)

15. Example of 𝐴∩𝐵 and 𝐴∪𝐵 Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = second outcome is head = {HH, TH}
and B = first outcome is tail = {TH, TT}.
Then 𝐴∩𝐵 = {TH} = the first outcome is tail and the second outcome is head,
and 𝐴∪𝐵 = {HH, TH, TT}.

16. How to find P 𝐴∪𝐵 ?
P 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵 .

17. Example of A∪𝐵
Suppose I pick a student at random from this class. Let A be the event that the chosen student is a vegetarian and B be the event that he/she is a junior.
Suppose further that 6% of the students present in this class are vegetarians and 30% are juniors and only 2% are both junior and vegetarian.
What is the chance that selected student is either a vegetarian or a junior? P 𝐴∪𝐵 =?

18. Example of A∪𝐵 P(A or B) = P(A) + P(B) - P(A and B)
Suppose I pick a student at random from this class. Let A be the event that the chosen student is a vegetarian and B be the event that he/she is a junior.
Suppose further that 6% of the students present in this class are vegetarians and 30% are juniors and only 2% are both junior and vegetarian.
What is the chance that selected student is either a vegetarian or a junior?
P(A or B) = P(A) + P(B) - P(A and B)
= = 0.34.

19. Another Example
A survey found out that 56% of MSU students live in a campus dorm, 62% participate in a campus meal program and 42% do both.
What is the probability that a randomly selected MSU student either lives on campus dorm or has a campus meal plan?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(either lives on campus or has a campus meal plan)
= P(L or M) = P(L) + P(M) - P(L and M)
= = 0.76.

20. Another Example
A survey found out that 56% of MSU students live in a campus dorm, 62% participate in a campus meal program and 42% do both.
What is the probability that a randomly selected MSU student lives on campus but does not have a campus meal plan?
Let L = {student lives on campus} and
M = {student has a campus meal plan}.
P(a student lives on campus but does not have a campus meal plan)
= P[L and (not M)] = ?

21. Solve Using a Picture
P[L and (not M)] = P(L) - P(L and M) = = 0.14.

22. Similarly …
P[M and (not L)] = P(M)-P(L and M) = = Therefore, there is a 20% chance that a randomly selected MSU student lives off campus but participates in a campus meal program.

23. Another Example
A survey found out that 56% of MSU students live in a campus dorm, 62% participate in a campus meal program and 42% do both.
What is the probability that a randomly selected MSU student neither lives on campus nor participates in a campus meal plan?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(neither lives on campus nor participates in a campus meal plan)
= P(neither L nor M)= ?

24. Once again, use a picture 
P(neither L nor M) = P[not(L or M)] = 1 - P(L or M) = 1 - [P(L) + P(M) - P(L and M)] = 1 - ( ) = 0.24.

25. Another Example
A survey found out that 56% of MSU students live in a campus dorm, 62% participate in a campus meal program and 42% do both.
What is the probability that a randomly selected MSU student either lives on campus or participates in a campus meal plan but not both?
Let L = {student lives on campus}, and
M = {student has a campus meal plan}.
P(either lives on campus or participates in a campus meal plan but not both)
= P(exactly one of L and M) = ?

26. Draw Picture Again!
P(exactly one of L and M) = P[L and (not M)] + P[M and (not L)] = = 0.34.

27. Disjoint Events
Two events A and B are called disjoint or mutually exclusive if they can never occur together, i.e. A∩𝐵= ∅ (empty set). If A and B are disjoint then, 𝑃 𝐴∩𝐵 =0.

28. Example of Disjoint Events
Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = both outcomes are head = {HH},
and B = first outcome is tail = {TH, TT}.
Then A and B are disjoint because they can never occur together.

29. Independence

30. Independent Events 𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .
Two events A and B are called independent if
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .
Roughly speaking, this means that the occurrence (or non-occurrence) of A has nothing to do with the occurrence (or non-occurrence) of B.

31. Example of Independent Events
Suppose you toss a fair coin twice.
Here S = {HH, HT, TH, TT}.
Define A = second outcome is head = {HH, TH},
and B = first outcome is tail = {TH, TT}.
Then A and B = {TH} and P(A and B)= ¼ while P(A) = P(B) = 2/4 = ½ and therefore
P(A and B) = ¼ = ½ × ½ =P(A)P(B), which means A and B are independent.

32. Disjoint and Independent
Suppose A and B are two events.
If A and B are disjoint then
𝑃 𝐴∩𝐵 =0.
On the other hand, A and B are independent if
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 .

33. Disjoint and Independent
If two events A and B are disjoint then can they be independent as well?
Ans: Only if either P(A)=0, or P(B)=0, or both.
This is because if A and B are disjoint and independent then
0=𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 ,
and hence either P(A)=0, or P(B)=0, or both.

34. Disjoint and Independent: An Example
Suppose a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word.
Suppose A is the event that the word “cupcakes” is chosen and B is the event that the letter “v” is selected.
Then A and B are
independent,
disjoint.
neither.

35. Disjoint and Independent: Another Example
Suppose a fair coin is tossed twice.
Suppose A = head occurs in the first toss
and
B = head occurs in the second toss.
Then A and B are
independent,
disjoint.
neither.

36. Application of Independence
If it is known that two events A and B are independent only then we can calculate P(A and B) as follows: 𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵 . Sometimes the independence of A and B is assumed implicitly although it is not explicitly stated.

37. Example
If the probability that one will encounter a red light in a traffic signal is 0.61 then find the probability that you will find the light to be red on both Monday and Tuesday.
Here “red on Monday” and “red on Tuesday” are implicitly assumed to be independent.
Therefore, P(red on Monday and red on Tuesday)
=P(red on Monday) × P(red on Tuesday)
=(0.61) × (0.61) =

38. P(A ∩ B ∩ C ∩ D) = P(A)P(B)P(C)P(D).
More on Independence
More than two events can also be independent. For example if 4 events A, B, C, D are independent then
P(A ∩ B ∩ C ∩ D) = P(A)P(B)P(C)P(D).
Just as in the case of two events, the independence of 4 events may not always be stated explicitly.

39. Harder Example
If the probability that one will encounter a red light in a traffic signal is 0.61 then find the probability that you will have to stop in a red light at least once during the work week?
P(at least one red in 5 days of a week)
=1 – P(no red in 5 days of a week)
=1 – P[(no red on Monday) and (no red on Tuesday) and (no red on Wednesday) and (no red on Thursday) and (no red on Friday)]
=1 – (0.39) × (0.39) × (0.39) × (0.39) × (0.39)
=0.991.

40. Example
Suppose a letter is chosen at random from the sentence “Dave loves cupcakes”. What is the probability that the letter “v’’ will be selected? There are total 17 letters and two of them are “v”. So P(“v” is selected) = 2∕17 =

41. Some Formulas Given two events A and B.
Probability at least one of A or B occurs =P(A U B)
Probability that both A “and” B occurs =P(A “and” B)
Probability Exactly one of A or B occurs
=P(A “ and” not B) + P( not A “and” B)
Probability of Neither of A nor B =P( not A “and” not B)=1-P(AUB)
In general,
Probability of at least 1 of A and B =1-P( None of A “and” B)

42. Example
A consumer organization estimates that over a 1-year period 19% of the cars will need to be repaired once, 12% will need repairs twice, 5% will require three of more repairs.
Suppose two cars are chosen at random. What is the probability that:
Neither will need repair ?
Both will need repair ?
At least one car will need repair ?
Exactly 1 need repair ?

43. Conditional Probability
And
Bayes’ Rule

44. Conditional Probability
Consider the below example:
Example 1: What is probability that you get a “5” in single throw of a die.
Example 2: If I tell you, that the number is odd, what is probability that the number is “5” ?
Why these two probabilities are different ? Since the later, we have more information and we condition the known information to get the outcome. It maximizes the event outcome.
Definition:
Probability of An event “A” given event “B” is given by
P(A/B)=P(A and B) / P(B)
Another example : What is probability that tomorrow’s whether is rainy ? What is tomorrow’s weather if if I tell you today’s weather sunny ?

45. Example
Toss a fair coin thrice. A = at least two heads = {HHH,HHT,HTH,THH}, B = all three are heads = {HHH}. Hence, P(A) = 4 8 = 1 2 , and P(B) = What is P(B|A)? What is P(A|B)? Here 𝐴∩𝐵={𝐻𝐻𝐻}. So 𝑃 𝐵 𝐴 = 𝑃(𝐴∩𝐵) 𝑃(𝐴) = = 1 4 and 𝑃 𝐴 𝐵 = 𝑃(𝐴∩𝐵) 𝑃(𝐵) = =1.

46. Multiplication rule
From the definition of conditional probability that P(A ∩ B) = P(B) P(A|B) = P(A) P(B|A). For events A, B and C we have P(A ∩ B ∩ C) = P(A) P(B|A) P(C| A∩B). It can be extended similarly to larger number of events.

47. Example
Suppose a letter is chosen at random from the sentence “Dave loves cupcakes”. What is the probability that the letter “v’’ will be selected? There are total 17 letters and two of them are “v”. So P(“v” is selected) = 2∕17 =

48. Example
Suppose first a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word.
What is the probability that the letter “v’’ will be selected?
Here it depends on which word is selected first.
P(“v” | “Dave”)
= P(“v” is selected, under the condition that “Dave” is selected first) = ¼ = 0.25.
P(“v” | “Dave”) is the conditional probability of selecting “v” given that “Dave” is selected first.
P(“v” | “loves”)
= P(“v” is selected, under the condition that “loves” is selected first) = 1∕5 = 0.2.
P(“v” | “cupcakes”) = 0.

49. Example
Suppose first a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word. What is the probability that the letter “v’’ will be selected? So we found that the chance of selecting “v” is NOT independent of the words selected first. In this case, P(“Dave” and “v” are selected) = P(“Dave” is selected first) P(“v” | “Dave”) = ⅓ × ¼ = 1∕12. Similarly, P(“loves” and “v” are selected) = ⅓ × 1∕5 = 1∕15, and P(“cupcakes” and “v” are selected) = ⅓ × 0 = 0. So P(“v” is selected) = 1∕12 + 1∕ = The computation is easily understood with the following tree diagram.

50. Example P(“v” is selected) = ⅓ × ¼ + ⅓ × 1∕5 + ⅓ × 0 = 0.15.
Suppose a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word. What is the probability that the letter “v’’ will be selected? ⅓ ⅓ ⅓
Dave
loves
cupcakes
1/5
4/5 1 v
not “v” v
not “v” v
not “v”
P(“v” is selected) = ⅓ × ¼ + ⅓ × 1∕5 + ⅓ × 0 = 0.15.

51. Example
Suppose first a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word. If the selected letter is “v”, what is the conditional probability that the word “Dave” was selected in the first place? 𝑃 𝐷𝑎𝑣𝑒 𝑣 = 𝑃(Dave 𝑎𝑛𝑑 v) 𝑃(𝑣) = 𝑃 𝐷𝑎𝑣𝑒 𝑃(𝑣|𝐷𝑎𝑣𝑒) 0.15 = 1 3 × =0.556.

52. Example
The probability that an adult American man has high blood pressure and/or cholesterol level are shown in table:
Blood Pressure
High Ok
Cholesterol Level High
Ok ?
Find the missing probability.
What is the probability that a randomly selected man has both conditions ?
What is the probability that he has high blood pressure ?
What is the probability that a person has high blood pressure given that he has high cholesterol ?
What is the probability that a man has high cholesterol level if it’s known that he has high blood pressure ?

53. Example
The probability that an adult American man has high blood pressure and/or cholesterol level are shown in table:
Blood Pressure
High Ok
Cholesterol Level High
Ok ?
Find the missing probability
What is the probability that a randomly selected man has both conditions ? 0.11
What is the probability that he has high blood pressure ? 0.27
What is the probability that a person has high blood pressure given that he has high cholesterol ?
What is the probability that a man has high cholesterol level if it’s known that he has high blood pressure

54. Bayes’ rule Suppose E1, …, Ek are events such that
𝐸 1 ∪ 𝐸 2 ∪…∪ 𝐸 𝑘 =𝑆 (exhaustive),
𝐸 𝑖 ∩ 𝐸 𝑗 =∅ for any 𝑖≠𝑗, (mutually exclusive).
Then for any event A with 𝑃 𝐴 >0,
𝑃 𝐸 𝑗 𝐴 = 𝑃( 𝐸 𝑗 ∩𝐴) 𝑃(𝐴) = 𝑃 𝐸 𝑗 𝑃 𝐴|𝐸 𝑗 𝑃 𝐸 1 𝑃 𝐴|𝐸 1 +…+𝑃 𝐸 𝑘 𝑃 𝐴|𝐸 𝑘 .
Remark: Many a problem can be solved using tree diagram only, without taking refuge to this complicated formula.

55. Example
Suppose first a word is chosen at random from the sentence “Dave loves cupcakes” and then a letter is selected at random from the chosen word. If the selected letter is “v”, what is the conditional probability that the word “Dave” was selected in the first place? 𝑃 𝐷𝑎𝑣𝑒 𝑣 = 𝑃(Dave∩v) 𝑃(𝑣) = 𝑃 𝐷𝑎𝑣𝑒 𝑃(𝑣|𝐷𝑎𝑣𝑒) 0.15 = 1 3 × =0.556.