# Risk Management Part 2 – Variability and Risk: Twin Sons of Different Mothers by Donald E. Shannon, PMP, CFCM, CPCM, DML The Contract Coach, Albuquerque,

## Presentation on theme: "Risk Management Part 2 – Variability and Risk: Twin Sons of Different Mothers by Donald E. Shannon, PMP, CFCM, CPCM, DML The Contract Coach, Albuquerque,"— Presentation transcript:

Risk Management Part 2 – Variability and Risk: Twin Sons of Different Mothers by Donald E. Shannon, PMP, CFCM, CPCM, DML The Contract Coach, Albuquerque, NM

Disclaimer Information in this presentation makes reference to various software products. This should not be interpreted as a recommendation or endorsement by any of the sponsors of any one product. Individuals should conduct appropriate research to identify a product that best meets their specific needs. Where appropriate, credit has been given to the software OEM especially where screen shots of their products have been used. A listing of the products commonly used to accomplish the simulation and scheduling functions described herein is included at the end of this presentation

Where Are We Going? Quantitative (numerical or statistical) analysis Quantitative (numerical or statistical) analysis What is variability What is variability The nature of random variables The nature of random variables How they are depicted How they are depicted Confidence intervals Confidence intervals Variability (uncertainty) in program / project management Variability (uncertainty) in program / project management Why estimates never seem to be right Why estimates never seem to be right Adding risk to uncertainty = real world variability Adding risk to uncertainty = real world variability Adding risk and uncertainty to our cost/schedule model Adding risk and uncertainty to our cost/schedule model

Variability Why things are never exactly alike…. Trivia questions: 1.Who is the driver in the above? 2.What year was the photo taken and how can you tell? 3.On average how long does a F-1 pit stop take in 2014

Trivia questions: 1.Who is the driver in the above? – Kimi Raikkonen 2.What year was the photo taken and how can you tell? – 2008. Because Kimi’s car is #1 and he won the F1 championship in 2007 giving him that number for the following year. 3.On average how long does a F1 pit stop take in 2014? – 3 Seconds

A 3-Second Pit Stop? Let’s Fact Check That Claim Is the average pit stop time actually 3.0 seconds? Is the average pit stop time actually 3.0 seconds? Data from 13 pit stops during the most recent F-1 race Data from 13 pit stops during the most recent F-1 race Most consistent team was Williams with 3 stops each of which was 3.0 seconds Most consistent team was Williams with 3 stops each of which was 3.0 seconds Fastest team was Red Bull with an average stop of 2.675 seconds Fastest team was Red Bull with an average stop of 2.675 seconds Further analysis would likely show that the average time of 2.8 in this sample was within the expected range and there is not enough evidence to reject the supposed average of 3.0 seconds Further analysis would likely show that the average time of 2.8 in this sample was within the expected range and there is not enough evidence to reject the supposed average of 3.0 seconds DriverTeamStop 1 Stop 2Average VETRB3.12.32.7 RICRB2.92.42.65 GROLOT2.9- MASWIL3.0- BOTWIL3.0 HAMMER3.84.34.05 ROSMER2.83.02.9 Average2.682.983.04 Pit Stop Times 2014 F-1 Grand Prix of Spain

Variability (Uncertainty) The output of any task when repeated will vary to some degree from its predecessors or successors The output of any task when repeated will vary to some degree from its predecessors or successors Size, weight, volume etc Size, weight, volume etc Performance time Performance time Your morning commute is a good example Your morning commute is a good example Depending on road conditions and traffic it may vary significantly from day-to-day Depending on road conditions and traffic it may vary significantly from day-to-day Even when conditions are perfect the time still varies by a few seconds or minutes Even when conditions are perfect the time still varies by a few seconds or minutes

Random Variables We call these probabilistic events because the answer can not be described by a single value. We call these probabilistic events because the answer can not be described by a single value. The value of these events (time, cost, etc.) is variable. The value of these events (time, cost, etc.) is variable. So how do we describe variability So how do we describe variability We establish ranges of possible values We establish ranges of possible values We define measures of central tendency (average, mean, median, mode) We define measures of central tendency (average, mean, median, mode) We identify how the values are spread out (variance) We identify how the values are spread out (variance) We quantify how far possible values are displaced from the center (standard deviation or z- score) We quantify how far possible values are displaced from the center (standard deviation or z- score)

On Closer Examination Reviewing some terms you probably know The pit stop range (low to high) was 2 Seconds (2.3 – 4.3) The pit stop range (low to high) was 2 Seconds (2.3 – 4.3) 3.0 was the most common (mode) time recorded 3.0 was the most common (mode) time recorded 3.0 was the middle value (median) 3.0 was the middle value (median) 3.04 was the arithmetic average (also called mean or μ) of all the times recorded 3.04 was the arithmetic average (also called mean or μ) of all the times recorded The standard deviation (σ) was.54 The standard deviation (σ) was.54 All data lies within +/- 3 σ All data lies within +/- 3 σ μ 1 σ 2 σ 3 σ -1 σ

Random Variables Random variables are either discrete (whole numbers 1,2,3) or continuous Random variables are either discrete (whole numbers 1,2,3) or continuous There are separate rules for displaying the data for each There are separate rules for displaying the data for each Discrete RVs are typically shown in histograms (bar charts) Discrete RVs are typically shown in histograms (bar charts) Continuous RV are typically shown in probability distribution functions (PDF) or cumulative distribution functions (CDF) Continuous RV are typically shown in probability distribution functions (PDF) or cumulative distribution functions (CDF)

Histograms Data may be either continuous or discrete Data may be either continuous or discrete Discrete data is grouped into classes or “bins” Discrete data is grouped into classes or “bins” Bin values displayed on x-axis Bin values displayed on x-axis Count or number in class displayed on y-axis Count or number in class displayed on y-axis A smoothed line enclosing the tips of each column may provide insight as to the underlying distribution (Normal, Lognormal, Beta, etc. A smoothed line enclosing the tips of each column may provide insight as to the underlying distribution (Normal, Lognormal, Beta, etc.

Probability Distribution Function The total area under the curve is always equal to 1 (100%) The total area under the curve is always equal to 1 (100%) The tallest point of the curve is the mode (most often occurring value) The tallest point of the curve is the mode (most often occurring value) In symmetric distributions the mean and median are co-located with the mode In symmetric distributions the mean and median are co-located with the mode If the distribution is not symmetric we say it is ‘skewed’ with the direction of the skew (left or right) being where the preponderance of the values reside. If the distribution is not symmetric we say it is ‘skewed’ with the direction of the skew (left or right) being where the preponderance of the values reside. The shape of the distribution may give a clue to the underlying distribution but beware – sometimes the shape is misleading The shape of the distribution may give a clue to the underlying distribution but beware – sometimes the shape is misleading Values on the x-axis may be actual or z (standard deviations). If z scores those to the left of the mean are negative and those to the right are positive. Values on the x-axis may be actual or z (standard deviations). If z scores those to the left of the mean are negative and those to the right are positive.

Cumulative Distribution Function Values start at zero Values start at zero Values end at 1 (or 100 percent) Values end at 1 (or 100 percent) Value on the y axis (vertical) is at cumulative probability for the value on the x axis i.e., the sum of the probabilities from zero to the selected x value Value on the y axis (vertical) is at cumulative probability for the value on the x axis i.e., the sum of the probabilities from zero to the selected x value The shape of the curve is an indication of the shape of the probability distribution function (PDF). The shape of the curve is an indication of the shape of the probability distribution function (PDF).

Confidence Intervals What is the likelihood of an event happening? What is the likelihood of an event happening? Single point estimate of continuous variable is undefined Single point estimate of continuous variable is undefined Therefore we phrase the question as: Therefore we phrase the question as: Probability that x will exceed some value Probability that x will exceed some value Probability that x is less than some value Probability that x is less than some value Probability that x is between a and b. Probability that x is between a and b.

Using a PDF to Answer Questions Consider the normal distribution Consider the normal distribution Average IQ is 100 Average IQ is 100 Standard deviation is 10 points Standard deviation is 10 points 68% lie between 90 – 110 68% lie between 90 – 110 95.4% lie between 80 – 120 95.4% lie between 80 – 120 99.74 lie between 70 – 130 99.74 lie between 70 – 130 PDFs help us answer questions such as: PDFs help us answer questions such as: What is the probability of someone having an IQ between 110 and 120? (13.6%) What is the probability of someone having an IQ between 110 and 120? (13.6%) What is the probability of an IQ of 125? (Trick question. If IQ is a continuous random variable the probability of one precise value is undefined) What is the probability of an IQ of 125? (Trick question. If IQ is a continuous random variable the probability of one precise value is undefined)

Uncertainty in Program Management Why things always take longer and cost more than you planned ….

Murphy’s Law Often stated as: “Anything that can go wrong, will go wrong” Often stated as: “Anything that can go wrong, will go wrong” References to the principle date to at least 1877 References to the principle date to at least 1877 Famous corollaries include: Famous corollaries include: If anything just cannot go wrong, it will anyway If anything just cannot go wrong, it will anyway … Usually in the worst possible (or most inconvenient) time in the worst possible way. … Usually in the worst possible (or most inconvenient) time in the worst possible way. Things will be lost or damaged in inverse relationship to their value or need. Things will be lost or damaged in inverse relationship to their value or need. But the real culprit in all of this is nature’s uncertainty. But the real culprit in all of this is nature’s uncertainty.

Uncertainty Uncertainty is an admission that the outcome of any event is a random variable Uncertainty is an admission that the outcome of any event is a random variable Therefore the data describing that event (cost, performance time etc.) will be variable Therefore the data describing that event (cost, performance time etc.) will be variable Forecasting the outcome of an uncertain event can only be stated probabilistically Forecasting the outcome of an uncertain event can only be stated probabilistically Probabilistic outcomes typically involve Probabilistic outcomes typically involve A percent likelihood the event will occur A percent likelihood the event will occur Sometimes an indication of underlying distribution (uniform, binomial, normal, etc,) Sometimes an indication of underlying distribution (uniform, binomial, normal, etc,) When we plan for uncertain events we typically plan for the “most likely” outcome When we plan for uncertain events we typically plan for the “most likely” outcome “On average” it costs “x” or takes “y” days “On average” it costs “x” or takes “y” days Sometimes we do better Sometimes we do better Sometimes we do worse Sometimes we do worse

Please Fasten Your Seatbelts Statistics Airline Flight 101 Now Departing We can’t talk about uncertainty and quantitative risk management without delving into statistics. We can’t talk about uncertainty and quantitative risk management without delving into statistics. Keep in mind that the goal is not to teach you the math but the underlying concepts and terminology Keep in mind that the goal is not to teach you the math but the underlying concepts and terminology We’ll let the computer do the math – all we care about is the output We’ll let the computer do the math – all we care about is the output But to understand the output you have to know the concepts. But to understand the output you have to know the concepts.

Uncertainty in Program Management The time to perform a task (e.g., a pit stop) is a random variable The time to perform a task (e.g., a pit stop) is a random variable Because of this nature, when we express a task duration as a specific value (single point estimate) we ignore uncertainty Because of this nature, when we express a task duration as a specific value (single point estimate) we ignore uncertainty Single point estimates are typically “most likely” values Single point estimates are typically “most likely” values Better estimates are possible if we describe 3 points Better estimates are possible if we describe 3 points Minimum (Best Case) Minimum (Best Case) Maximum (Worst Case) Maximum (Worst Case) Most Likely Most Likely The three point estimate is especially useful if dealing with ‘expert opinion” The three point estimate is especially useful if dealing with ‘expert opinion” 15 Minimum (a)Maximum (b) Most Likely (m) 6 9 Task Performance Time

Triangular Distribution Triangle distribution is a continuous frequency distribution often used to model random variables (cost or performance time) in program management Triangle distribution is a continuous frequency distribution often used to model random variables (cost or performance time) in program management Fast Fast Easy to use Easy to use Provides reasonable accuracy Provides reasonable accuracy Tends to be slightly optimistic i.e., values returned tend to be a little less than what ends up being the case Tends to be slightly optimistic i.e., values returned tend to be a little less than what ends up being the case Formula Formula Mean = a+m+b ÷ 3 = 30 ÷3 = 10 Mean = a+m+b ÷ 3 = 30 ÷3 = 10 Mode = m = 9 Mode = m = 9 STDEV = σ=√(a 2 +m 2 +c 2 -am-ab-mc) 18 STDEV = σ=√(a 2 +m 2 +c 2 -am-ab-mc) 18 Minimum (a)Maximum (b) Most Likely (m) 6 9 15 μ =10 σ=1.58 Note: Don’t worry about doing the number crunching – that’s why we have software to do this for us!

Triangular Distribution Probability of a value of x is given by the area of interest. Probability of a value of x is given by the area of interest. Example. What is the probability that x is less than 7.25? Example. What is the probability that x is less than 7.25? f(x) = 2(x-a) (b-a)(m-a) f(x) = 2(x-a) (b-a)(m-a) f(x) = 2 x (7.25-6) (15-6)(9-3) f(x) = 2 x (7.25-6) (15-6)(9-3) f(x) = 2.5 / 54 =.0462963 f(x) = 2.5 / 54 =.0462963 p(x) = ½ *.0462963 * 1.25 =.029 or 2.9% p(x) = ½ *.0462963 * 1.25 =.029 or 2.9% Minimum (a)Maximum (b) Most Likely (m) 6 9 15 μ =10 σ=1.58 7.25 Note: Don’t worry about doing the number crunching – that’s why we have software to do this for us!

Special Use Of the Triangular Distribution Some scheduling applications use a “weighted” version of the triangle distribution called the “PERT” distribution Some scheduling applications use a “weighted” version of the triangle distribution called the “PERT” distribution PERT (Program Evaluation and Review Technique) adds (statistical) emphasis to the “Most Likely” value and weigh it 4 times as likely as the Minimum or Maximum PERT (Program Evaluation and Review Technique) adds (statistical) emphasis to the “Most Likely” value and weigh it 4 times as likely as the Minimum or Maximum Average (mean) is then = a+4m+b / 6 Average (mean) is then = a+4m+b / 6 Criticism is that PERT tends to be optimistic Criticism is that PERT tends to be optimistic Second criticism is that PERT ignores likelihood of events outside 3 points (closed interval) Second criticism is that PERT ignores likelihood of events outside 3 points (closed interval) Minimum (a)Maximum (b) Most Likely (m) 6 9 15 μ =9.5 The PERT Triangular Distribution σ=1.50

Other Distributions Used In Program Management Beta Distribution is a typically mound shaped continuous distribution Beta Distribution is a typically mound shaped continuous distribution Very flexible and can take on a number of shapes depending on the parameters used Very flexible and can take on a number of shapes depending on the parameters used May be either closed form (upper example) or open form (lower example) May be either closed form (upper example) or open form (lower example) Is less optimistic than triangular Is less optimistic than triangular

Other Distributions Used In Program Management Lognormal Distribution is a typically mound shaped continuous distribution Lognormal Distribution is a typically mound shaped continuous distribution Very flexible and can take on a number of shapes depending on the parameters used Very flexible and can take on a number of shapes depending on the parameters used Open ended toward the upper side allowing for extreme values Open ended toward the upper side allowing for extreme values Is the resultant distribution when two triangular distributions are combined. Is the resultant distribution when two triangular distributions are combined.

Why Estimates of Cost or Schedule End Up Being Wrong Murphy was an optimist ……

Leading Factors Optimism bias Optimism bias Everything takes longer than planned Everything takes longer than planned Everything costs more than you thought Everything costs more than you thought Improper estimates Improper estimates Corrupt data Corrupt data Failure to properly consider risk Failure to properly consider risk

Optimism Bias Values predicted are “better” or “rosier” than real world experience would show Values predicted are “better” or “rosier” than real world experience would show Sources of Optimism Bias Sources of Optimism Bias Expert Opinion on average tends to be as much as 25% optimistic for commonly performed tasks 1 Expert Opinion on average tends to be as much as 25% optimistic for commonly performed tasks 1 Use of the triangle distribution when another (e.g., Beta) is more appropriate Use of the triangle distribution when another (e.g., Beta) is more appropriate The sequence of collecting estimates for Minimum, Maximum and Most likely can introduce bias 2 The sequence of collecting estimates for Minimum, Maximum and Most likely can introduce bias 2 1 Bias in Memory Predicts Bias in Estimation of Future Task Durations, Roy 2007 2 Herding Cats: Why 3 Point Estimates Create False Optimism

Improper Estimates Estimates are commonly built on mathematical models Estimates are commonly built on mathematical models Models must be appropriate for the application selected Models must be appropriate for the application selected Models must be used within the range they were designed to predict Models must be used within the range they were designed to predict Another estimating technique is to compare something new to something already done Another estimating technique is to compare something new to something already done Effort is often scaled by some percentage Effort is often scaled by some percentage Only as good a technique as the data on which it is based and the similarity of the two tasks/projects Only as good a technique as the data on which it is based and the similarity of the two tasks/projects

Risk Risk adds an additional element to uncertainty Risk adds an additional element to uncertainty Broadens the range of possible values – usually to: Broadens the range of possible values – usually to: Increase costs Increase costs Extend performance time Extend performance time Risk is an “event” such that it either takes place or does not take place Risk is an “event” such that it either takes place or does not take place Risk events have 3 parameters Risk events have 3 parameters Polarity ( +/- ) Polarity ( +/- ) Likelihood – their chance of occurring Likelihood – their chance of occurring Impact – the cost, delay, or opportunity associated with the risk Impact – the cost, delay, or opportunity associated with the risk

Risk as a Variable Risks are a two-step process Risks are a two-step process Step 1 … Does risk occur Step 1 … Does risk occur Step 2 … What is the impact Step 2 … What is the impact Risk impacts are often modeled using the triangular distribution Risk impacts are often modeled using the triangular distribution Minimum impact (cost, delay, or both) Minimum impact (cost, delay, or both) Most likely impact Most likely impact Worst case impact Worst case impact Data comes from risk register as was completed in Part 1 Data comes from risk register as was completed in Part 1 Minimum (a)Maximum (b) Most Likely (m) 1 3 6 Delay associated with risk

When Worlds Collide What happens if we combine a variable (cost or schedule) and risk ….

How Risk Changes an Event If the risk event does not happen, no change to event If the risk event does not happen, no change to event If the risk event happens then the task performance time will be the sum of two probabilities If the risk event happens then the task performance time will be the sum of two probabilities Task Performance time (no risk) Task Performance time (no risk) Risk Impact Risk Impact Summing probability distributions is a bit tricky Summing probability distributions is a bit tricky Minimum (a)Maximum (b) Most Likely (m) 1 3 6 Delay associated with risk 15 Minimum (a)Maximum (b) Most Likely (m) 6 9 Task Performance Time +

Combining Probability Distributions

Using Method of Moments to Combine PDF’s Method of Moments Technique Method of Moments Technique Analytical technique Analytical technique Used to calculate the “moments” of the combined distribution Used to calculate the “moments” of the combined distribution The resultant distribution from adding two triangular distributions is a lognormal distribution. 1 The resultant distribution from adding two triangular distributions is a lognormal distribution. 1 The Moments of that are: The Moments of that are: Mean = μ = μ 1 + μ 2 … μ n Mean = μ = μ 1 + μ 2 … μ n Variance = σ 2 = it depends 2 Variance = σ 2 = it depends 2 Skewedness 1 = ϑ = Skewedness 1 = ϑ = Kurtosis = κ = 12/5 = 2.387 Kurtosis = κ = 12/5 = 2.387 The math needed to calculate these is outside the scope of this presentation The math needed to calculate these is outside the scope of this presentation 1.Analytic Method for Cost and Schedule Risk Analysis, Raymond P. Covert, NASA, 5 April, 2013, pp 34 - 37 2.Calculating variance for the sum of two distributions is complicated when the two distributions are correlated. Formula shown is for correlated data

Using Simulation to Combine PDF’s Monte Carlo Technique Monte Carlo Technique Generate random number between 0 and 1 Generate random number between 0 and 1 Convert random number to duration based on triangular distribution 6,9,15 Convert random number to duration based on triangular distribution 6,9,15 Generate second random number between 0 and 1 Generate second random number between 0 and 1 Convert second random number to duration based on triangular distribution 1,3,6 Convert second random number to duration based on triangular distribution 1,3,6 Add duration 1 to duration 2 Add duration 1 to duration 2 Record value Record value Repeat numerous times Repeat numerous times Compute statistics such as mean etc. from collected data Compute statistics such as mean etc. from collected data

Why Do I Favor Simulation? Mostly because it provides equivalent results without having to be a math major. Mostly because it provides equivalent results without having to be a math major. Praised for Praised for Ability to provide statistics of a simulated CDF or PDF formed by complex modeling of random variables Ability to provide statistics of a simulated CDF or PDF formed by complex modeling of random variables Ease of use Ease of use Criticized for: Criticized for: Non-uniform sampling Non-uniform sampling Unable to correlate two distributions using Pearson product – moment correlation coefficients Unable to correlate two distributions using Pearson product – moment correlation coefficients Does not provide reasonable results with small number of trials Does not provide reasonable results with small number of trials Simulation output of 3,460 iterations © Intavar Institute Risky Project 6.0

Going Forward Adding risk and variability to our cost and schedule estimates ….

Time to Build Our Model Our model of the effort (contract, project, etc) must include: Our model of the effort (contract, project, etc) must include: Tasks to be performed Tasks to be performed Resources to be used in accomplishing those tasks Resources to be used in accomplishing those tasks Labor Labor Materials Materials Other (Travel, subcontracts, etc.) Other (Travel, subcontracts, etc.) Cost data for the each of the resources and tasks Cost data for the each of the resources and tasks Schedule data Schedule data Risk Data Risk Data

Tools to be Used Excel or Numbers spreadsheet Excel or Numbers spreadsheet Project, Primavera, FastTrack, or other scheduling application Project, Primavera, FastTrack, or other scheduling application Simulation program such as: Simulation program such as: @Risk @Risk Risky Project Risky Project Primavera Primavera Full Monty Full Monty

What We Will Do Create Work Breakdown Structure Create Work Breakdown Structure Expand (decompose) WBS into various tasks Expand (decompose) WBS into various tasks Identify resources for each task Identify resources for each task Create risk register Create risk register Map risks to each task Map risks to each task Determine cost baseline with and without risk Determine cost baseline with and without risk Create probabilistic estimates for cost at completion and completion date. Create probabilistic estimates for cost at completion and completion date.

Summary The language of quantitative analysis is statistics – learn the language Schedule (task) duration and cost are random variables Nothing is certain until it occurs Risk is another random variable that modifies cost, schedule, or both You don’t need a PhD to do the math … let the software do that while you interpret the results

Produced by: The Contract Coach The Contract Coach 5338 La Colonia Dr NW Albuquerque, NM 87120 (505) 259-8485 http://www.contract-coach.com

Monte Carlo Simulation Software MS Excel Add-in MS Excel Add-in @Risk @Risk Oracle Crystal Ball Oracle Crystal Ball Risk Solver Risk Solver General Purpose Products General Purpose Products Analytica Analytica GoldSim GoldSim Reno Reno Oracle Crystal Ball Oracle Crystal Ball SPSS SPSS MS Project Add-in MS Project Add-in Full Monty Full Monty Risky Project (also supports Primavera P6) Risky Project (also supports Primavera P6) Enterprise Systems Enterprise Systems Risk+ (Deltek) Risk+ (Deltek) Primavera EPPM & Primavera Risk (Oracle) Primavera EPPM & Primavera Risk (Oracle)

Download ppt "Risk Management Part 2 – Variability and Risk: Twin Sons of Different Mothers by Donald E. Shannon, PMP, CFCM, CPCM, DML The Contract Coach, Albuquerque,"

Similar presentations