1. Virtual University of Pakistan
Lecture No. 20 of the course on Statistics and Probability by
Miss Saleha Naghmi Habibullah

2. IN THE LAST LECTURE, YOU LEARNT
Relative Frequency Definition of Probability
Axiomatic Definition of Probability
Laws of Probability
Rule of Complementation
Addition Theorem

3. TOPICS FOR TODAY Application of Addition Theorem
Conditional Probability
Multiplication Theorem

4. First of all, let us consider in some detail the Addition Law or the General Addition Theorem of Probability:

5. P(AB) = P(A) + P(B) – P(AB)
ADDITION LAW
If A and B are any two events defined in a sample space S, then
P(AB) = P(A) + P(B) – P(AB)

6. In words, this law may be stated as follows:

7. “If two events A and B are not mutually exclusive, then the probability that at least one of them occurs, is given by the sum of the separate probabilities of events A and B minus the probability of the joint event A  B.”

8. Example:
If one card is selected at random from a deck of 52 playing cards, what is the probability that the card is a club or a face card or both?

9. Let A represent the event that the card selected is a club, B, the event that the card selected is a face card, and A  B, the event that the card selected is both a club and a face card. Then we need P(A  B).

10. Now P(A) = 13/52, as there are 13 clubs,
P(B) = 12/52, as there are 12 faces cards,

11. and P(A  B) = 3/52, since 3 of clubs are also face cards.
Therefore the desired probability is

12. P(A B) = P(A) + P(B) – P(A  B)
= 13/ /52 - 3/52
= 22/52.

13. COROLLARY-1
If A and B are mutually exclusive events, then
P(AB) = P(A) + P(B)
(Since A  B is an impossible event, hence P(AB) = 0.)

14. EXAMPLE
Suppose that we toss a pair of dice, and we are interested in the event that we get a total of 5 or a total of 11.
What is the probability of this event?

15. SOLUTION
In this context, the first thing to note is that ‘getting a total of 5’ and ‘getting a total of 11’ are mutually exclusive events. Hence, we should apply the special case of the addition theorem.

16. If we denote ‘getting a total of 5’ by A, and ‘getting a total of 11’ by B, then
P(A) = 4/36 (since there are four outcomes favourable to the occurrence of a total of 5),
and P(B) = 2/36 (since there are two outcomes favourable to the occurrence of a total of 11).

17. Hence:

18. The probability that we get a total of 5 or a total of 11 is given by
P(AB) = P(A) + P(B) = 4/36 + 2/36 = 6/36 = 16.67%.

19. COROLLARY-2
If A1, A2, …, Ak are k mutually exclusive events, then the probability that one of them occurs, is the sum of the probabilities of the separate events, i.e.
P(A1,  A2  …  Ak) = P(A1) + P(A2)+ … + P(Ak).

20. Let us now consider an interesting example to illustrate the way in which probability problems can be solved:

21. EXAMPLE
Three horses A, B and C are in a race; A is twice as likely to win as B and B is twice as likely to win as C. What is the probability that A or B wins?

22. Evidently, the events mentioned in this problem are not equally likely.
Let P(C) = p
Then P(B) = 2p as B is twice as likely to win as C.
Similarly
P(A) = 2P(B) = 2(2p) = 4p

23. In this problem, we assume that no two of the horses A, B and C cannot win the race together (i.e. the race cannot end in a draw).
Hence, the events A, B and C are mutually exclusive.

24. Since A, B and C are mutually exclusive and collectively exhaustive, therefore the sum of their probabilities must be equal to 1.

25. Thus
p + 2p + 4p = 1 or p = 1/7
 P(C) = 1/7,
P(B) = 2(1/7) = 2/7,
and P(A) = 4(1/7) = 4/7.

26. Hence
P(AB) = P(A) + P(B)
= 4/7+ 2/7
= 6/7.

27. Having discussed the addition theorem in some detail, we would now like to discuss the Multiplication Theorem.
But, before we are in a position to take up the multiplication theorem, we need to consider the concept of CONDITIONAL PROBABILITY.

28. CONDITIONAL PROBABILITY
The sample space for an experiment must often be changed when some additional information pertaining to the outcome of the experiment is received.

29. The effect of such information is to REDUCE the sample space by excluding some outcomes as being impossible which BEFORE receiving the information were believed possible.

30. The probabilities associated with such a reduced sample space are called conditional probabilities.
The following example illustrates the concept of conditional probability:

31. EXAMPLE
Suppose that we toss a fair die. Then the sample space of this experiment is S = {1,2,3,4,5,6}.
Suppose we wish to know the probability of the outcome that the die shows 6 (say event A).
Also, suppose that, before seeing the outcome, we are told that the die shows an EVEN number of dots (say event B).

32. Then the information that the die shows an even number excludes the outcomes 1, 3 and 5, and thereby reduces the original sample space to a sample space that consists of three outcomes 2, 4 and 6,
i.e. the reduced sample space is B = {2, 4, 6}.

33. (The sample space is reduced.) 
(The sample space is reduced.)

34. Then, the desired probability in the reduced sample space B is 1/3 (since each outcome in the reduced sample space is EQUALLY LIKELY).

35. This probability 1/3 is called the conditional probability of the event A because it is computed under the CONDITION that the die has shown an even number of dots.

36. In other words,
P(die shows 6/die shows even numbers)
= 1/3,

37. (where the vertical line is read as given that, and the information following the vertical line describes the conditioning event).

38. Sometimes, it is not very convenient to compute a conditional probability by first determining the number of sample points that belong to the reduced sample space.
In such a situation, we can utilize the following alternative method of computing a conditional probability:

39. CONDITIONAL PROBABILITY
If A and B are two events in a sample space S and if P(B) is not equal to zero, then the conditional probability of the event A given that event B has occurred, written as P(A/B), is defined by
where P(B) > 0.
(If P(B) = 0, the conditional probability P(A/B) remains undefined.)

40. Similarly
where P(A) > 0.

41. It should be noted that P(A/B) SATISFIES all the basic axioms of probability, namely:

42. i) 0 < P(A/B) < 1.
ii)
iii) P(A1A2/B) = P(A1/B) + P(A2/B) (provided that the events A1 and A2 are mutually exclusive).

43. Example A man tosses two fair dice. What is the
conditional probability that the sum of the two
dice will be 7 given that
The sum is greater than 6
The two dice had the same outcome

44. Solution The sample space S for this experiment
consists of the following 36 equally likely
outcomes:

45. Let A = {sum is 7}
Then
The total no. ordered pairs in this set is 6.
Hence, we have:
P(A) = 6/36

46. Let B = {sum greater than 6}
Therefore

47. The total no. ordered pairs in this set is 21.
Hence, we have:
P(B) = 21/36
Let
C = {the two dice have the same outcome}
Therefore,
The total no. ordered pairs in this set is 6.
P(C) = 6/36

48. The conditional probabilities P(A/B) and P(A/C) are given by the formulae:

49. In order to compute P(A/B), we first need to find
, and, for this purpose, we first need to
determine the number of ordered pairs that belong to
this event:
The total no. ordered pairs in this set is 6.
Hence, we have:

50. Hence, using the definition of conditional
probability, we get

51. In order to compute P(A/C), we first need to find
, and, for this purpose, we first need to
determine the number of ordered pairs that belong to
this event:
This is a null set because, obviously, none of the six ordered
pairs
is such whose sum is 7.

52. Since is a null set (an empty set), hence
the total number of ordered pair in this set is 0.
Hence,

54. i) 0 < P(A/B) < 1.
ii)
iii) P(A1A2/B) = P(A1/B) + P(A2/B) (provided that the events A1 and A2 are mutually exclusive).

55. Let us now apply this concept to a real-world example:

56. EXAMPLE-2
At a certain elementary school in a Western country, the school-record of the past ten years shows that 75% of the students come from a two-parent home, and that 20% of the students are low-achievers and belong to two-parent homes.

57. What is the probability that such a randomly selected student will be a low achiever GIVEN THAT he or she comes from a two-parent home?

58. SOLUTION
Let A denote a low achiever and B a student from a two-parent home.
Applying the relative frequency definition of probability, we have
P(B) = 0.75 and P(A  B) = 0.20.
Thus, we obtain

59. And now, let us consider the Multiplication Theorem of probability:

60. It is interesting to note that the multiplication theorem is obtained very conveniently from the formula of conditional probability:

62. And if we interchange the roles of A and B, we obtain:
P(A  B) = P(A) P(B/A),
provided P(A) > 0.

63. If A and B are any two events defined in a sample space S, then
MULTIPLICATION LAW
If A and B are any two events defined in a sample space S, then
P(A  B)
= P(A) P(B/A), provided P(A) > 0,
= P(B) P(A/B), provided P(B) > 0.
(The second form is easily obtained by interchanging A and B.)

64. This is called the GENERAL rule of multiplication of probabilities.
It can be stated as follows:

65. P(A  B) = P(A) P(B/A), MULTIPLICATION LAW
“The probability that two events A and B will both occur is equal to the probability that one of the events will occur multiplied by the conditional probability that the other event will occur given that the first event has already occurred.”
P(A  B) = P(A) P(B/A),

66. Let us apply the concept of multiplication theorem to an example:

67. EXAMPLE
A box contains 15 items, 4 of which are defective and 11 is good. Two items are selected. What is the probability that the first is good and the second defective?

68. Let A represent the event that the first item selected is good, and B, the event that the second items is defective.
Then we need to calculate the probability of the JOINT event A  B by the rule P(A  B) = P(A)P(B/A).

69. We have:
Since all the items are equally likely to be chosen, hence P(A) = 11/15.

70. Given the event A has occurred, there remain 14 items of which 4 are defective.
Therefore the probability of selecting a defective item after a good item has been selected is 4/14 i.e.
P(B/A) = 4/14.

71. Hence
P(A  B) = P(A)P(B/A)
= 11/15  4/14
= 44/210
= 0.16.

72. Example Suppose a card is drawn at random from a
deck of ordinary playing cards.
What is the probability that it is a diamond, a
face card or a king?

73. Solution Let A = the card drawn is a diamond
B = the card drawn is a face card, and
C = the card drawn is a king.

74. Since the events A, B, C are not mutually
exclusive, hence the required probability is
given by:
P(A U B U C) = P(A) + P(B) + P(C)
– P(A∩B) – P(A∩B)
– P(A∩B) + P(A∩B∩C)

75. Now, in this example, we have:
P(A) = 13/52
P(B) = 12/52
P(C) = 4/52

76. Also:
P(A  B) = P(A)P(B/A)
= 13/52 × 3/13 = 3/52
(This is so, because, once a card of diamond
has been drawn, the conditional probability
of obtaining a face card given that the card
is a diamond is 3/13, the reason being that
there are only 3 face cards among the 13
cards of diamonds.)

77. Similarly:
P(B  C) = P(B)P(C/B)
= 12/52 × 4/12 = 4/52
The conditional probability of obtaining a
king given that the card is a face card is 4/12,
the reason being that there are only four kings
among the 12 face cards.

78. And
P(A  C) = P(A)P(C/A)
= 13/52 × 1/13 = 1/52
The conditional probability of obtaining a
king given that the card is a diamond is 1/13,
the reason being that there is only 1 king
among the 13 diamond cards.

79. Also
P(A∩B∩C) = P(A)P(B/A)P(C/A∩B)
This is the extension of the general multiplication
theorem to the case of two events, and it states that
the probability that A occurs and B occurs, and C
occurs multiplied by the probability that B occurs
given that A has already occurred multiplied by the
probability that C occurs given that both A and B
have already occurred.

80. In this example:
P(A∩B∩C) = P(A)P(B/A)P(C/A∩B)
= 13/52 × 3/13 × 1/3
= 1/52
Hence, substituting all the values, we obtain
P(A U B U C) = 13/ /52 + 4/52
– 3/52 – 4/52 – 1/52 + 1/52
= 22/52 = 0.42

81. Hence, the probability that the card that is
drawn is a diamond, a face card or a king (or
any two of them simultaneously, or all three
of them simultaneously) is 42%

82. In this lecture, the concepts of the Addition Theorem and the Multiplication Theorem of probability have been discussed in some detail.
In order to differentiate between the situation where the addition theorem is applicable and the situation where the multiplication theorem is applicable, the main point to keep in mind is that whenever we wish to compute the probability that either A occurs or B occurs, we should think of the Addition Theorem, where as, whenever we wish to compute the probability that both A and B occur, we should think of the Multiplication Theorem.

83. IN TODAY’S LECTURE, YOU LEARNT
Application of Addition Theorem
Conditional Probability
Multiplication Theorem

84. IN THE NEXT LECTURE,
YOU WILL LEARN
Independent and Dependent Events
Multiplication Theorem of Probability for Independent Events
Marginal Probability
Bayes’ Theorem