Chapter 1 Probability Spaces 主講人 : 虞台文

Content Sample Spaces and Events Event Operations Probability Spaces Conditional Probabilities Independence of Events Reliabilities Bayes’ Rule

Chapter 1 Probability Spaces Sample Spaces and Events

Definitions  Sample Spaces and Events The set of all possible outcomes of a random experiment is called the sample space, denoted by , of that experiment. An element    is called a sample point. A subset A   is called an event.

Example 1 The set of all possible outcomes of a random experiment is called the sample space, denoted by , of that experiment. An element    is called a sample point. A subset A   is called an event. The set of all possible outcomes of a random experiment is called the sample space, denoted by , of that experiment. An element    is called a sample point. A subset A   is called an event. Tossing a die.  sample space of tossing a die  an event whose face number is less than 4  an event whose face number is lager than 3  an event whose face number is odd  an event whose face number is even

Notations  A  A    a sample space  an event       an sample point the probability for the occurrence of event A

Example 2 1. Suppose that the die in Example 1 is fair. Find the probability of each event. 2. Suppose that the die is unfair, and with Find the probability of each event.

Example 2 1. Suppose that the die in Example 1 is fair. Find the probability of each event. 2. Suppose that the die is unfair, and with Find the probability of each event.

Example 2 1. Suppose that the die in Example 1 is fair. Find the probability of each event. 2. Suppose that the die is unfair, and with Find the probability of each event.

Example 2 1. Suppose that the die in Example 1 is fair. Find the probability of each event. 2. Suppose that the die is unfair, and with Find the probability of each event.

Example 3  Bulb Life (Months)  sample space of a bulb life  the life of a bulb is not less than 10 months  the life of a bulb is not less than 10 but less than 20 months  the life of a bulb is less than 10 months  the life of a bulb is larger than 15 but not larger than 30 months

Example 3  Bulb Life (Months)  sample space of a bulb life  the life of a bulb is not less than 10 months  the life of a bulb is not less than 10 but less than 20 months  the life of a bulb is less than 10 months  the life of a bulb is larger than 15 but not larger than 30 months

Example 3  Bulb Life (Months)

Example 4 Tossing three balanced coins 1. Write the sample space of this experiment. 2. Write the event A to denote that at least two coins land heads. 3. P(A) =?

Example 4 Tossing three balanced coins 1. Write the sample space of this experiment. 2. Write the event A to denote that at least two coins land heads. 3. P(A) =?

Example 4 Tossing three balanced coins 1. Write the sample space of this experiment. 2. Write the event A to denote that at least two coins land heads. 3. P(A) =? The method to define a sample space is not unique, e.g.,

Chapter 1 Probability Spaces Events Operations

Event Operations Intersection  And Union  Or Complement  Not

Intersection (  )  And Venn Diagram  AB ABAB

Example 5 1. A 1  A 3 = {1, 3} 2. A 2  A 4 = {4, 6} 3. A 1  A 2 =  The face number is less than 4 and odd. The face number is larger than 3 and even. A null event.

Example 5 1. A 1  A 3 = {1, 3} 2. A 2  A 4 = {4, 6} 3. A 1  A 2 =  The face number is less than 4 and odd. The face number is larger than 3 and even. A null event. Remark: Two events A and B are said to be mutually exclusive if A ∩ B = .

Example 5 1. A 1  A 3 = {1, 3} 2. A 2  A 4 = {4, 6} 3. A 1  A 2 =  Assume the die is fair.

Example 6  Bulb Life (Months) A null event.

Example 6  Bulb Life (Months) The bulb life is lager than 15 but less than 20 months

Example 6  Bulb Life (Months)

Union (  )  Or Venn Diagram  AB ABAB

Example 7 1. A 1  A 3 = {1, 2, 3, 5} 2. A 2  A 4 = {2, 4, 5, 6} 3. A 1  A 2 =  1 The face number is less than 4 or odd. The face number is larger than 3 or even. A universal event.

Example 8  Bulb Life (Months) A universal event.

Example 8  Bulb Life (Months) The bulb life is not less than 10 but not large than 30 months

Example 8  Bulb Life (Months)

 Complement  Not Venn Diagram A AcAc

Example 9-1 The face number is not less than 4.

Example 9-2  Bulb Life (Months)

Laws of Event Algebra 1. Associative laws 2. Commutative laws 3. Distributive laws

Laws of Event Algebra 4. Identity laws 5. Complementation laws 6. Idempotent laws

Laws of Event Algebra 7. Domination laws 8. Absorption laws 9. De Morgan’s laws

More on De Morgan’s laws

Chapter 1 Probability Spaces Probability Spaces

 -Field A nonempty collection of subsets A is called  -field of a set  provided that the following two properties hold: 1. A 2.

Example 10 Let  = {1, 2, 3, 4, 5, 6}. 1. Let A 1 = {A 0 = , A 1 = {1, 2, 3}, A 2 = {4, 5, 6}, A 3 =  }. Whether or not A 1 forms a  -field of  ? 2. Let A 2 = {A 1 = {1, 6}, A 2 = {2, 5}, A 3 = {3, 4}}. Add minimum number of subsets of  into A 2 such that A 2 becomes a -field of .

The Axioms of Probability A probability measure P on a  -field of subset A of a set  is a real-valued function having domain A satisfying the following properties: 1. P(  ) = 1 ; 2. P(A)  0 for all A  A ; 3. If A i  A, i=1,2,... are mutually disjoint then  AA1A1 A2A2 A3A3 A4A4

Definition  Probability Space A probability measure P on a  -field of subset A of a set  is a real-valued function having domain A satisfying the following properties: 1. P(  ) = 1 ; 2. P(A)  0 for all A  A ; 3. If A i  A, i=1,2,... are mutually disjoint then A probability space, denoted by ( , A, P), is a set , a  -field of subsets A, and a probability measure P defined on A.

Example 11  = {1, 2, 3, 4, 5, 6}. A = {A 0 = , A 1 = {1, 2, 3}, A 2 = {4, 5, 6}, A 3 =  } P (A 0 ) = 0, P (A 1 ) = 1/2, P (A 2 ) = 1/2, P (A 3 ) = 1 ( , A, P)  A probability space?

Example 11  = {1, 2, 3, 4, 5, 6}. A = {A 0 = , A 1 = {1, 2, 3}, A 2 = {4, 5, 6}, A 3 =  } P (A 0 ) = 0, P (A 1 ) = 1/3, P (A 2 ) = 2/3, P (A 3 ) = 1 ( , A, P)  A probability space?

Example 11  = {1, 2, 3, 4, 5, 6}. A = {A 0 = , A 1 = {1, 2, 3}, A 2 = {4, 5, 6}, A 3 =  } P (A 0 ) = 1/3, P (A 1 ) = 1/3, P (A 2 ) = 1/3, P (A 3 ) = 1 ( , A, P)  A probability space?

Example 11  = {1, 2, 3, 4, 5, 6}. A = {A 0 = , A 1 = {1, 2, 3}, A 2 = {4, 5, 6}, A 3 =  } P (A 0 ) = 0, P (A 1 ) = 1/3, P (A 2 ) = 1/3, P (A 3 ) = 2/3 ( , A, P)  A probability space?

Theorem 1-1 For any event A, P(A c ) = 1  P(A). Pf) Facts:

Theorem 1-2 For any two events A and B, P(A  B) = P(A) + P(B)  P(A  B) Pf) A B

More on Theorem 1-2 A B C

Theorem 1-3 See the text for the proof.

Theorem 1-3 S1S1 S2S2 S3S3 SnSn

Example 12 Let A, B be two events of a sample space  with P(A) = 1/2, P(B) = 1/2, P(A  B) = 2/3 1. Find the probabilities of the following events: (1) P(A c ) (2) P(B c ) (3) P(A  B) (4) P(A c  B) (5) P(A c  B) (6) P(A  B c ) (7) P(A c  B c ) (8) P(A c  B c ). 2. Are A, B mutually exclusive?  AB 1/3 1/6

Chapter 1 Probability Spaces Conditional Probabilities

 Definition Let A, B be two events such that P(A) > 0. Then the conditional probability of B given A, written P(B|A), is defined by A B If P(A) = 0, then P(B|A) is undefined.

Example 13 Toss three balanced coins. Let A denote the event that two coins land heads, B denote that the first coin lands heads. Find P(A), P(B), P(B|A), and P(A|B).

By Product

Chapter 1 Probability Spaces Independence of Events

Definition Two events A and B are independent if and only if A B

Example 14 Toss one balanced coin three times. Let A denote the event that the first two tosses land heads, B denote that exactly one toss lands heads, and C denote that the third toss lands tails. 1. Are A, B independent? 2. Are A, C independent? 3. Are B, C independent? Toss one balanced coin three times. Let A denote the event that the first two tosses land heads, B denote that exactly one toss lands heads, and C denote that the third toss lands tails. 1. Are A, B independent? 2. Are A, C independent? 3. Are B, C independent?

Example 14 Toss one balanced coin three times. Let A denote the event that the first two tosses land heads, B denote that exactly one toss lands heads, and C denote that the third toss lands tails. 1. Are A, B independent? 2. Are A, C independent? 3. Are B, C independent? Toss one balanced coin three times. Let A denote the event that the first two tosses land heads, B denote that exactly one toss lands heads, and C denote that the third toss lands tails. 1. Are A, B independent? 2. Are A, C independent? 3. Are B, C independent?   

Theorem 1-4 A B AcAc B A BcBc AcAc BcBc   

Theorem 1-5 A B  P(B|A)=P(B) Pf)

Definition n events A 1, A 2,..., A n are said to be mutually independent if and only if for any different k = 2,..., n events satisfy

Example 15 Given the following Venn Diagram, are events A, B, C mutually independent? P(A) = 0.6, P(B) = 0.8, P(C) = 0.5 P(A  B  C) = 0.24 = P(A)P(B)P(C) P(A  B) = 0.48 = P(A)P(B) P(A  C) = 0.30 = P(A)P(C) P(B  C) = 0.38  P(B)P(C) 

Remarks 1. P(A 1  A 2  · · ·  A n ) = P(A 1 )P(A 2 ) · · · P(A n ) does not imply that A 1, A 2,..., A n are pairwise independent (Example 15). 2. A 1, A 2,..., A n are pairwise independent does not imply that they are mutually independent.

Example 16 Toss two dice. Let A = “The 1st die is 1,2, or 3” B = “The 2nd die is 4,5, or 6” C = “The sum of two dice is 7” Show that A, B, C are pairwise independent but not mutually independent. Toss two dice. Let A = “The 1st die is 1,2, or 3” B = “The 2nd die is 4,5, or 6” C = “The sum of two dice is 7” Show that A, B, C are pairwise independent but not mutually independent.

Chapter 1 Probability Spaces Reliabilities

Reliability of a component  Let R i denote the probability of a component in a system which is functioning properly (event A i ). We will assume that the failure events of components in a system are mutually independent. Reliabilities of systems  1. series systems — The entire system will fail if any one of its component fails. 2. parallel systems — The entire system will fail only if all its component fail.

Reliability of a Series System ( R ss ) C1C1 C1C1 C2C2 C2C2 CnCn CnCn AB

Reliability of a Parallel System ( R ps ) C1C1 C1C1 C2C2 C2C2 CnCn CnCn A B......

Example 17 Consider the following system. C1C1 C1C1 C2C2 C2C2 C3C3 C3C3 AB Let R 1 = R 2 = R 3 = Find the reliability of the system.

Example 17

Chapter 1 Probability Spaces Bayes’ Rule

A Story  The umbrella must have been made in Taiwan

Event Space  A Partition of Sample Space  日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5  a partition of 

Event Interests Us ( A )  日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5  a partition of  A

Preliminaries  日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5 A Prior Probabilities: Likelihoods:

Law of Total Probability 日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5 A Prior Probabilities: Likelihoods:

Law of Total Probability 日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5 A Prior Probabilities: Likelihoods:

Goal: Posterior Probabilities 日 德 法 義 台 B1B1 B2B2 B3B3 B4B4 B5B5 A Prior Probabilities: Likelihoods: Goal:

Goal: Posterior Probabilities Prior Probabilities: Likelihoods: Goal:

Bayes’ Rule Given

Example 18 Suppose that the population of a certain city is 40% male and 60% female. Suppose also that 50% of the males and 30% of the females smoke. Find the probability that a smoker is male. M : A selected person is male W : A selected person is female S : A selected person who smokes Define We are given  M W S NS S

Example 19 Consider a binary communication channel. Owing to noise, error may occur. For a given channel, assume a probability of 0.94 that a transmitted 0 is correctly received as a 0 and a probability of 0.91 that a transmitted 1 is received as a 1. Further assumed a probability of 0.45 of transmitting of a 0. Determine 1. Probability that a 1 is received. 2. Probability that a 0 is received. 3. Probability that a 1 was transmitted, given that a 1 was received. 4. Probability that a 0 was transmitted, given that a 0 was received. 5. Probability of an error. Consider a binary communication channel. Owing to noise, error may occur. For a given channel, assume a probability of 0.94 that a transmitted 0 is correctly received as a 0 and a probability of 0.91 that a transmitted 1 is received as a 1. Further assumed a probability of 0.45 of transmitting of a 0. Determine 1. Probability that a 1 is received. 2. Probability that a 0 is received. 3. Probability that a 1 was transmitted, given that a 1 was received. 4. Probability that a 0 was transmitted, given that a 0 was received. 5. Probability of an error.

Example 19 Consider a binary communication channel. Owing to noise, error may occur. For a given channel, assume a probability of 0.94 that a transmitted 0 is correctly received as a 0 and a probability of 0.91 that a transmitted 1 is received as a 1. Further assumed a probability of 0.45 of transmitting of a 0. Determine 1. Probability that a 1 is received. 2. Probability that a 0 is received. 3. Probability that a 1 was transmitted, given that a 1 was received. 4. Probability that a 0 was transmitted, given that a 0 was received. 5. Probability of an error. Consider a binary communication channel. Owing to noise, error may occur. For a given channel, assume a probability of 0.94 that a transmitted 0 is correctly received as a 0 and a probability of 0.91 that a transmitted 1 is received as a 1. Further assumed a probability of 0.45 of transmitting of a 0. Determine 1. Probability that a 1 is received. 2. Probability that a 0 is received. 3. Probability that a 1 was transmitted, given that a 1 was received. 4. Probability that a 0 was transmitted, given that a 0 was received. 5. Probability of an error T 0 : A 0 is transmitted T 1 : A 1 is transmitted R 0 : A 0 is received R 1 : A 1 is received Define T0T0 R0R0 T1T1 R1R1

Example T0T0 R0R0 T1T1 R1R1

Example 20 There are n boxes each contains a white and b black balls. Now randomly select one ball from the 1 st box to put it into the 2 nd box, then randomly select one ball from the 2 nd box to put it into the 3 rd box,... This procedure is continued. Let W i and B i denote the events that the chosen ball from the ith box being white and black, respectively. Determine 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? There are n boxes each contains a white and b black balls. Now randomly select one ball from the 1 st box to put it into the 2 nd box, then randomly select one ball from the 2 nd box to put it into the 3 rd box,... This procedure is continued. Let W i and B i denote the events that the chosen ball from the ith box being white and black, respectively. Determine 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?

Example 20 There are n boxes each contains a white and b black balls. Now randomly select one ball from the 1 st box to put it into the 2 nd box, then randomly select one ball from the 2 nd box to put it into the 3 rd box,... This procedure is continued. Let W i and B i denote the events that the chosen ball from the ith box being white and black, respectively. Determine 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? There are n boxes each contains a white and b black balls. Now randomly select one ball from the 1 st box to put it into the 2 nd box, then randomly select one ball from the 2 nd box to put it into the 3 rd box,... This procedure is continued. Let W i and B i denote the events that the chosen ball from the ith box being white and black, respectively. Determine 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? a b a b a b a b a b 123 n1n1 n...

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? Define p i =P(W i ).

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?......

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?......

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?......

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?  

Example P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ? 1. P(W n ) = ?; 2. P(W n |W 1 ) = ?; 3. lim n → ∞ P(W n |W 1 ) = ?