Introduction to probability BSAD 30 Dave Novak Source: Anderson et al., 2013 Quantitative Methods for Business 12 th edition – some slides are directly from J. Loucks © 2013 Cengage Learning

Overview Experiments and the Sample Space Assigning Probabilities to Experimental Outcomes Events and Their Probabilities Some Basic Relationships of Probability Bayes’ Theorem Simpson’s Paradox

Uncertainty Managers often base their decisions on an analysis of uncertainties such as the following: What are the chances that sales will decrease if we increase prices? What is the likelihood a new assembly method will increase productivity? What are the odds that a new investment will be profitable?

Probability Probability is a numerical measure of the likelihood that an event will occur Probability values are always assigned on a scale from 0 to 1 You can think of probability in terms of percentage A probability near zero indicates an event is quite unlikely to occur A probability near one indicates an event is almost certain to occur

Probability as a numerical measure of likelihood Increasing Likelihood of Occurrence Probability: The event is very unlikely to occur The event is very unlikely to occur The occurrence of the event is just as likely as just as likely as it is unlikely The occurrence of the event is just as likely as just as likely as it is unlikely The event is almost certain to occur The event is almost certain to occur

Statistical experiments A statistical experiment differs somewhat from an experiment in the physical sciences In statistical experiments, probability determines outcomes Even though the experiment is repeated in exactly the same way, an entirely different outcome may occur For this reason, statistical experiments are often called random experiments

An experiment and its sample space An experiment is any process that generates well-defined outcomes Flipping a coin 10 times The sample space for an experiment is the set of all experimental outcomes The exact H or T results from all 10 times An experimental outcome is also called a sample point The result of a particular coin flip

An experiment and its sample space Experiment Toss a coin Inspection a part Conduct a sales call Roll a die Play a football game Sample Space Head, tail Defective, non-defective Purchase, no purchase 1, 2, 3, 4, 5, 6 Win, lose, tie

Assigning probabilities 1) Probability assigned to each experimental outcome must be between 0 and 1 inclusive 0 < P(E i ) < 1 for all i where: E i is the ith experimental outcome and P(E i ) is its probability

Assigning probabilities 2) The sum of the probabilities for all experimental outcomes must be equal to 1 P(E 1 ) + P(E 2 ) P(E n ) = 1 where: n is the number of experimental outcomes

Assigning probabilities If we throw two dice together, the possible outcomes are: 2, 3, 4, … 12 However, each outcome is not equally likely What is the probability that each outcome will occur?

Assigning probabilities Total of DiceSpecific Outcomes on Pairs of DiceProbability Event Occurs 2D1=1 + D2=1 (1+1)1/36 = 3% 3D1=1 + D2=2 (1+2), D1=2 + D2=1 (2+1)2/36 = 1/18 = 6% 41+3, 2+2, 3+13/36 = 1/12 = 8% 51+4, 2+3, 3+2, 4+14/36 = 1/9 = 11% 61+5, 2+4, 3+3, 4+2, 5+15/36 = 14% 71+6, 2+5, 3+4, 4+3, 5+2, 6+16/36 = 1/6 = 17% 82+6, 3+5, 4+4, 5+3, 6+25/36 = 14% 93+6, 4+5, 5+4, 6+34/36 = 1/9 = 11% 104+6, 5+5, 6+43/36 = 1/12 = 8% 115+6, 6+52/36 = 1/18 = 6% /36 = 3%

Assigning probabilities P(E 1 ) + P(E 2 ) P(E n ) = 1

Assigning probabilities Three ways of assigning probabilities 1) Classical method Assume equally likely outcomes 2) Relative frequency method Assign probabilities based on experimentation or historical data 3) Subjective method Assign probabilities based on judgment

Classical method Rolling a die If an experiment has n possible outcomes (where n=6), the classical method would assign a probability of 1/n to each outcome Sample Space: S = {1, 2, 3, 4, 5, 6} Probabilities: Each sample point has a 1/6 (or 0.166) chance of occurring

Relative frequency method Lucas Tool Rental would like to assign probabilities to the number of car polishers it rents each day. Office records show the following frequencies of daily rentals for the last 40 days Number of Polishers Rented Number of Days

Relative frequency method Each probability assignment is given by dividing the frequency (number of days) by the total frequency (total number of days) 4/404/40 Probability Number of Polishers Rented Number of Days /4010/40

Subjective method When economic conditions and a company’s circumstances change rapidly it might be inappropriate to assign probabilities based solely on historical data We can use any data available as well as our experience and intuition, but ultimately a probability value should express our degree of belief that the experimental outcome will occur What is a potentially serious drawback associated with this approach?

Subjective method Consider the case in which a couple just made an offer to purchase a house. Two outcomes are possible: One believes the probability their offer will be accepted is 0.8; thus, P(E 1 ) = 0.8 and P(E 2 ) = 0.2. Two, believes the probability that their offer will be accepted is 0.6; hence, P(E 1 ) = 0.6 and P(E 2 ) = 0.4. Person two’s probability estimate for E 1 reflects a greater pessimism that their offer will be accepted E 1 = their offer is accepted E 2 = their offer is rejected E 1 = their offer is accepted E 2 = their offer is rejected

Events and their probabilities An event is a collection of sample points The probability of any event is equal to the sum of the probabilities of the sample points in the event If we can identify all the sample points of an experiment and assign a probability to each sample point, we can compute the probability of the event

Events and their probabilities Rolling a die Event E = Probability of getting an even number when rolling a die E = {2, 4, 6} P(E) = P(2) + P(4) + P(6) P(E) = P(2) + P(4) + P(6) = 1/6 + 1/6 + 1/6 = 3/6 =.5

Four relationships of probability These relationships can be used to compute the probability of an event, without knowing all the sample point probabilities 1) Compliment of an event 2) Addition law 3) Conditional probabilities 4) Multiplication law

Example You invest in two stocks: Markley Oil and Collins Mining, and determine that the possible outcomes of these investments three months from now are: Investment Gain or Loss Investment Gain or Loss in 3 Months (in $000) in 3 Months (in $000) Markley Oil Collins Mining  20 8  2

Example Assume an analyst makes the following probability estimates Exper. Outcome Net Gain or Loss Probability (10, 8) (10,  2) (5, 8) (5,  2) (0, 8) (0,  2) (  20, 8) (  20,  2) $18,000 Gain $18,000 Gain $8,000 Gain $8,000 Gain $13,000 Gain $13,000 Gain $3,000 Gain $3,000 Gain $8,000 Gain $8,000 Gain $2,000 Loss $2,000 Loss $12,000 Loss $12,000 Loss $22,000 Loss $22,000 Loss

Example Viewed as a tree diagram Gain 5 Gain 8 Gain 10 Gain 8 Lose 20 Lose 2 Even Markley Oil (Stage 1) Collins Mining (Stage 2) ExperimentalOutcomes (10, 8) Gain $18,000 (10, -2) Gain $8,000 (5, 8) Gain $13,000 (5, -2) Gain $3,000 (0, 8) Gain $8,000 (0, -2) Lose $2,000 (-20, 8) Lose $12,000 (-20, -2) Lose $22,000

Example Compute the probability that your investment in Markley Oil will be profitable

Example Compute the probability that your investment in Collins Mining will be profitable

Complement of an event The complement of event A is defined to be the event consisting of all sample points that are not in A Event A AcAcAcAc Sample Space S Sample Space S VennDiagramVennDiagram

Union of two events The union of events A and B is the event containing all sample points that are in A or B or both The union of events A and B is denoted by A  B Sample Space S Sample Space S Event A Event B

Example Compute the probability of the union of two events

Example Compute the probability of the union of two events

Intersection of two events The intersection of events A and B is the set of all sample points that are in both A and B The intersection of events A and B is denoted by A  Sample Space S Sample Space S Event A Event B Intersection of A and B

Example Compute the probability of the intersection of two events

Addition law The addition law provides a way to compute the probability of event A, or B, or both A and B occurring The addition law is written as: P(A  B) = P(A) + P(B)  P(A  B 

Example Demonstrate the addition law

Mutually exclusive events Two events are said to be mutually exclusive if the events have no sample points in common Two events are mutually exclusive if, when one event occurs, the other cannot occur Two events are mutually exclusive if, when one event occurs, the other cannot occur Sample Space S Sample Space S Event A Event B

Mutually exclusive events If events A and B are mutually exclusive, P(A  B  = 0 In this case, the addition law is: P(A  B) = P(A) + P(B) There is no need to include “  P(A  B  ” There is no need to include “  P(A  B  ”

Conditional probability The probability of an event occurring given that another event has already occurred is called a conditional probability A conditional probability is computed as: The conditional probability of A given B is denoted by P(A|B) The conditional probability of A given B is denoted by P(A|B)

Example Calculate a conditional probability

Example Calculate a conditional probability

Example Calculate a conditional probability

Multiplication law The multiplication law provides a way to compute the probability of the intersection of two events The multiplication law is written as: P(A  B) = P(B) P(A|B)

Example Demonstrate the multiplication law

Joint and marginal conditional probabilities A joint probability gives the probability of an intersection of two events A marginal probability gives the probability of each single event separately These probabilities are often shown in a joint probability table

Joint and marginal conditional probabilities Collins Mining Profitable (C) Not Profitable (C c ) Markley Oil Profitable (M) Not Profitable (M c ) Total Total Joint Probabilities (appear in the body of the table) Joint Probabilities (appear in the body of the table) Marginal Probabilities (appear in the margins of the table) Marginal Probabilities (appear in the margins of the table)

Example Where are the values in the table coming from?

Independent events If the probability of event A is not changed by the existence of event B, we would say that events A and B are independent Two events are independent if: P(A|B) = P(A) P(B|A) = P(B) or

Multiplication law for independent events The multiplication law can be used to test whether or not two events are independent of one another The multiplication law is written as: P(A  B) = P(A) P(B)

Example Use the multiplication law to test the independence of two events

Mutual exclusive ≠ independence Two events with nonzero probabilities cannot be both mutually exclusive and independent If one mutually exclusive event is known to occur, the other cannot occur; thus, the probability of the other event occurring is reduced to zero (and they are therefore dependent) Two events that are not mutually exclusive, might or might not be independent

Bayes’ Theorem When discussing conditional probabilities, it may be possible to revise certain probabilities as new information arises Initial probabilities are referred to as prior probabilities When initial probabilities are revised using new information, these new probabilities are referred to as posterior probabilities

Bayes’ Theorem Bayes’ theorem provides the means for revising the prior probabilities We will not be working through examples of Bayes’ theorem here, but Business Analytics and Finance concentration students would be well served to do so! New Information New Information Application of Bayes’ Theorem Application of Bayes’ Theorem Posterior Probabilities Posterior Probabilities Prior Probabilities Prior Probabilities

Simpson’s paradox A paradox that occurs in probability where a trend that is observed in multiple groups of data disappears or is reversed when those groups are combined Be careful when aggregating data Look for meaningful splits in the data, and consider groups or classes that might have different behavior. If you do not do this, you might obtain odd or incorrect correlations Different sample sizes, biased samples, etc.

Simpson’s paradox example In 1973, the University of California, Berkeley was sued for discrimination against women who had applied for admission to graduate school Admission data for the fall of 1973 showed that men applying were more likely than women to be admitted, and the difference was so large that it was unlikely to be due to chance Source: ApplicantsAdmitted Men % Women %

Simpson’s paradox example When examining the data from individual departments, it appeared that no department was significantly biased against women. In fact, most departments had a "small but statistically significant bias in favor of women Source: Department MenWomen ApplicantsAdmittedApplicantsAdmitted A82562%10882% B56063%2568% C32537%59334% D41733%37535% E19128%39324% F3736%3417%

Simpson’s paradox example What was happening? Women tended to apply to more competitive departments with low rates of admission even among qualified applicants, whereas men tended to apply to less- competitive departments with high rates of admission among the qualified applicants The data from specific departments constitute a proper defense against charges of discrimination Source:

Summary Experiments and the Sample Space Assigning Probabilities to Experimental Outcomes Classical Relative frequency Subjective Events and Their Probabilities

Summary Some Basic Relationships of Probability 1) Compliment of an event 2) Addition law 3) Conditional probabilities 4) Multiplication law There are a number of terms to know! Bayes’ theorem – what is it? Simpson’s paradox – what is it?