Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Risk Management. 2 Road Ahead Risk Management Process Cost and Schedule Risk Estimating Likelihood Mitigation Utility and Consequences Sensitivity Analysis.

Similar presentations

Presentation on theme: "1 Risk Management. 2 Road Ahead Risk Management Process Cost and Schedule Risk Estimating Likelihood Mitigation Utility and Consequences Sensitivity Analysis."— Presentation transcript:

1 1 Risk Management

2 2 Road Ahead Risk Management Process Cost and Schedule Risk Estimating Likelihood Mitigation Utility and Consequences Sensitivity Analysis

3 3 Reference Material Risk Management Guide for DOD Acquisition, 6 th Ed, Ver 1.0, August %206Ed%20Aug06.pdf. %206Ed%20Aug06.pdf INCOSE Systems Engineering Handbook, Ver 3, INCOSE-TP , June, 2006, Chapter 7.3

4 4 Why Do We Care? Every change from a current state to a future state occurs as a result of a process. The future behavior of any process is affected by uncertainties. Management of the process requires consideration of these uncertainties to minimize their influence on the desired final outcome of the process.

5 5 Elementary Process Transformation Input Output UnknownsUncertainty

6 6 Risk vs. Issues Risk refers to a future uncertain event. If an event has occurred it is no longer uncertain. It becomes an issue or a problem. Management of either requires expenditure of resources.

7 7 Risk Definitions Statistics: P[Undesirable Event] Risk Management: P[Undesirable Event] Plus the Consequences of the Event INCOSE considers both Undesirable and Desirable events

8 8 Consequences Can Impact: Technical Performance –Key Performance Parameters –Operational capability –Supportability Schedule Cost The above are not mutually independent

9 9 Risk Categories Technical Risk – A technical requirement may not be satisfied during the life cycle Cost Risk – Available budget may be exceeded Schedule Risk – May fail to reach Scheduled Milestones Programmatic Risk – Events beyond the control of the Program Manager

10 10 A Risk Reporting Matrix Likelihood Level Consequence Level Low Risk Moderate Risk High Risk

11 11 Possible Risk Likelihood Criteria LevelLikelihoodP[Occur] 1Not Likelyp< Low Likelihood0.10<=p<0.30 3Likely0.30<=p<0.70 4Highly Likely0.70<=p<0.90 5Near Certainty0.90<=p

12 12 Possible Consequence Level Criteria LevelTechnical Performance ScheduleCost 1None to Minimal 2Can be tolerated Little program impact Slip< ? moC< 1% of Budget 3Moderate, Limited program impact Slip? Mo 1%<=C<5% of Budget 4Significant degradation, May jeopardize program Critical path affected 5%<=C<10% of Budget 5Severe degradation, Will jeopardize program success Cannot meet key program milestones 10% of Budget <= C

13 13 Risk Criteria Depend on Program Risk LevelCriteria 1 P< P<0.02 3P<0.05 4P<0.25 5P>0.25

14 14 Risk Reporting Illustration Likelihood Level Consequence Level Risk Title (Category) Cause Mitigation Approach

15 15 Risk Management A process to minimize the adverse effects of uncertain future events on the achievement of end state objectives. Basic Tasks: –Identify and characterize process properties. –Decide on a course of action to minimize adverse affects of events on program objectives. –Implement and control the course of action.

16 16 Risk Management Functions Planning Resourcing Staffing Controlling

17 17 Planning Define the strategy and process to be used Establish a Risk Management Plan (RMP) –Tasks –Schedules –Reviews –Reporting Define the resources required –People –Funds –Space and support resources

18 18 An Example RMP Format Summary 1. Introduction 2. Program Summary 3. Risk Management Strategy and Process 4. Responsible/Executing Organization 5. Risk Management Process and Procedures 6. Risk Identification 7. Risk Analysis 8. Risk Mitigation Planning 9. Risk Mitigation Implementation 10. Risk Tracking

19 19 Events May Cause RMP Update A change in acquisition strategy, Preparation for a milestone decision, Results and findings from event–based technical reviews, An update of other program plans, Preparation for a Program Objective Memorandum submission, or A change in support strategy.

20 20 Identify Uncertain Events Estimate P[Occur] Estimate Consequences Formulate Alternative Courses of Action Evaluate Alternatives Choose ApproachExecute and Track Working? OR Continue or Stop Yes No Risk Management Process Planning

21 21 Risk (Uncertain Event) Identification What can go wrong? If EVENT happens then CONSEQUENCE results Find everywhere Mr. Murphy can rear his ugly head!

22 22 What to Look At Current and proposed staffing, process, design, suppliers, concept of operation, resources, interfaces, interactions, etc. Test results and failures (especially readiness results) Potential Shortfalls Trends External Influences (programmatic, political)

23 23 Potential Root Causes of Risk Events Need (Threat) Requirements Technical Baseline Test & Evaluation Modeling & Simulation Technology Logistics Production/Facilities Concurrency Industrial Capability Cost Management Schedule External Factors Budget Earned Value Realism

24 24 Possible Risk Management Actions Avoid –Redesign –Change requirements Accept Control –Expand resources –Reduce likelihood and/or consequences Transfer –By mutual agreement to party more qualified to mitigate

25 25 Interactions and Consequences Schedule affects cost C = A +BT Budget constrains cost Forces B vs T tradeoff Schedule constraint affects cost Causes B increase Technology maturity and workforce skills affect all terms

26 26 Example:Program to modify an existing guided submunition Current Characteristics Weight20 kg Length50 cm Diameter15 cm Required Characteristics Weight15 kg Length40 cm Diameter10 cm

27 27 Example Schedule

28 28 Uncertain Events? If Then

29 29 Example Guided Antitank Submunition Effectiveness: –Dictates minimum number/carrier Carrier Vehicle –Constrains length, diameter, and weight

30 30 Technical Performance Vector ElementRequiredCurrent Length (L)40 cm50 cm Diameter (D)10 cm15 cm Weight (W)15 kg20 kg

31 31 Actions to Meet Requirements ElementDiscrepancy Cause Corrective Action Length Use of discrete components Special chips DiameterSeeker antennaNew detector Weight Discrete components, battery Reduce power requirements, use integrated circuits

32 32 Activities Required to Reduce Length IDDescriptionTime (Mo) Preceded byExpected Length A1Analysis & Design A2Breadboard Fabrication2A150 A3Test1A250 A4Modify Breadboard0.5A350 A5Retest0.5A443 A6Admin Lead Time1.5A1--- A7Develop Chips3A5, A6, C543 A8Test Chips1A740

33 33 A1A2A3A4A5 A6 A7A From C Length Reduction Program

34 34

35 35 Activities to Reduce Diameter IDDescriptionTime (Mo Preceded ByExpected Diameter B1Trade Offs B2Subcontractor Lead Time B3Prototype design & Fab5B1, B213 B4Prototype Test1B313 B5Redesign3B4, C513 B6Retest0.5B510

36 36 B1 B2 B3B4B5B From C5 Diameter Reduction Program

37 37

38 38 Activities Required to Reduce Weight IDDescriptionTime (Mo)Preceded ByExpected Weight C1Trade Offs C2Administrative Lead Time1C1, A1, B1, B220 C3Design1C220 C4Refine & Deliver1C3, A3, B420 C5Integrate & Test1.5C417 C6Administrative Lead Time1C517 C7Delivery2C617 C8Integrate & Test2C715

39 39 C1 C2 C3 C4C5C6C C Weight Reduction Program From A1 From A3 From B1From B2From B4

40 40

41 41 C1 C2 C3 C4C5C6C C B1 B2 B3B4B5B D2D4 D A1A2A3A4A5 A6 A7A D1 D3 D Development Program

42 42

43 43

44 44

45 45 Effect of Variability on Schedule For the length reduction program: –Average time to achieve 43 cm = 5 months –95% confidence band: 4.49 –5.51 months Assumes Normal Distribution of activity time and 10% coefficient of variation Activity time distributions are usually triangular (a,a,c) –Moves mean and right tail to the right

46 46 Time Tech Parameter 95 % Confidence Regions Parameter – Time Relationship

47 47 Cost C = A + B*T C = Activity cost A = Fixed Cost B = Expenditure Rate T = Elapsed Time

48 48

49 49 Schedule and Cost Risk

50 50 Summary All activities affecting the desired end result and their interactions must be considered. Network representation takes care of this Activities must be considered at a low enough level to permit reasonable accurate time estimates

51 51 Summary (contd) Cost for development programs is a function of time There is variability in everything Variability can cause the critical path to change Plan for the occurrence of bad outcomes

52 52 Decision Environments Certainty Uncertainty Risk

53 53 Probability Refresher

54 54 Classical Interpretation N possibilities –Equally likely –One must occur –S of N possibilities = event success P[success] =S/N

55 55 Frequency Interpretation P[event] = proportion of the time event occurs over the long run Not very practical for situations that result in only one trial

56 56 Probability Axioms Consider sample space S for events A, B, C … in S. 0 P[A] 1 for all A in S P[S] = 1 If A and B are mutually exclusive, P[A B] = P[A or B or both] = P[A] + P[B]

57 57 Theorem If A is an event in finite S, and E i, i = 1, 2, …n are events comprising A, then

58 58 Theorem If A and B are two events in S then where is the common part of A and B

59 59 Total Probability If B i, i = 1, 2, …n, are mutually exclusive events, then

60 60 Conditional Probability

61 61 Bayes Theorem If B i, i = 1, 2, …n, are mutually exclusive events, then P[B i ] are prior, or a priori probabilities and must be determined prior to some experiment that results in event A based on the nature of the problem, data, experience, or subjectively based on experience.

62 62 Expected Value Discrete Distribution Continuous Distribution

63 63 Subjective Probability Consider the following Events –Flip a coin and let it hit the floor. Before looking at the coin, what is the probability it is heads? –What is the probability that the coin flip before the 2009 Auburn/Alabama football game resulted in tails? Both events have occurred.The outcome is certain. The only uncertainty is in your mind; i.e., in your degree of belief

64 64 Subjective Probability Problems Same phrase has different connotations with different people Interpretation is context dependent People are uncomfortable doing it There is no correct value

65 65 Example You and a friend disagree on P[Your team will win its next game]. Discussion of factors, home advantage, injuries, weather, etc., does not result in agreement. How do you resolve the disagreement?

66 66 Assessing Subjective Probabilities Direct Inquiry –Just make an estimate based on knowledge and perception –May not be able to come up with a value –May not have much confidence in the result Analysis of bets Comparison of Lotteries

67 67 Analysis of Bets Compare two bets Look for indifference point Example –LA Lakers vs Boston Celtics –What do you think is P[Lakers Win] Bets –Win X$ if LA wins, lose Y$ if LA loses –Lose X$ if LA wins, win Y$ if LA loses

68 68 Analysis of Bets (Contd) Bet for LA Bet Against LA LA wins LA loses LA wins X -Y -X Y Procedure: Change X and Y until willing to take either bet.

69 69 Analysis of Bets (Contd) Expected Value At indifference EV should be the same

70 70 Example For LA Against LA For LA Against LA Indifferent Round 1 Round 2 Round n......

71 71 Reference Lottery Lottery 1 –Win A if LA wins –Win B if LA loses Lottery 2 (Reference) –Win A with probability p –Win B with probability 1-p

72 72 Reference Lottery (Contd) Lottery 1 Lottery 2 Reference LA wins LA loses 1-p p A B A B Procedure: Change p until willing to take either bet. Outcome of lottery 2 is determined by a random process. Choose p, generate a random variate, x, from U(0,1) distribution. If x p then win A; otherwize, win B

73 73 Example Round 1 Round 2 Round 3 Prefer lottery 2 Prefer lottery 1 } Prefer lottery 2 Prefer lottery 1 } } Home in slowly. Check for consistency with probability axioms and theorems.

74 74 Continuous Probability Distributions Strategies –Direct assessment of p –Fractile assessment of X Probability Distribution Function (Cumulative probability)

75 75 Direct Assessment Estimate range of values for x, x min, x max Pick value x i : x min < x i < x max Estimate Repeat for a number of points (for 3 points pick mid range, then mid range of the two segments unless distribution is strongly skewed) Use Reference lottery approach to find indifference

76 76 Reference Lottery Lottery 1 Lottery 2 Reference xx i 1-F(x) F(x) $1000 $0 $1000 $0

77 77 Example Estimate time to get home from work Min 20 minF(t)= F(t)= F(t)= F(t)=0.85 Max 40F(t)=0.95

78 78 Fractile Method Pick range values for x: x min, x max Pick a number of fractiles –F(x)=0.05 Min –F(x)=0.95 Max –F(x)=0.50 –F(x)=0.25 –F(x) =0.75 Note order Use Reference lottery to find indifference

79 79 Reference Lottery Lottery 1 Lottery 2 Reference xx i 1-p p $1000 $0 $1000 $0

80 80 Heuristics and Biases Representativeness Availability Anchoring and adjusting Motivational bias

81 81 Representativeness Judge that something or someone belongs to a particular category –Stereotyping –Insensitive to base rates and prior probabilities –Unreliable information –Failure to account for inherent uncertainty Misunderstanding random processes –Regression to the mean –Extreme outcomes likely to be followed by one closer to the mean

82 82 Availability Judge P[event] according to ease of recalling similar events Influenced by unbalanced reporting Illusory correlation – pair of events perceived as happening together frequently

83 83 Anchoring and Adjusting Choose initial point then adjust about it Affects continuous distributions more that discrete probabilities If initially estimate median will tend to underestimate extremes so distribution is too narrow Estimate extremes first- the worst extreme before the other

84 84 Motivational Bias Incentives often exist that motivate person to report forecasts or probabilities that do not reflect their true beliefs –Salesman forecasting sales –Weather forecasters forecasting rain –Program managers predicting lower cost, shorter schedule, and low risk

85 85 Decomposition Break problem down to make probability assessment easier –Permits using people with subject matter knowledge –More likely to get a realistic estimate Look at fault tree analysis and hazard analysis Use laws of probability to reconstruct the problem Very important concept for planning and constructing mitigation efforts

86 86 Coherence All assessed probabilities MUST obey the laws of probability

87 87 Using Test Results Requirements are often placed on characteristics that are stochastic –Reliability –Accuracy Can use probability to estimate probability of not satisfying the requirement if test data exist Example: Impact accuracy of a projectile specified by the standard deviation of miss distance

88 88 Example Requirement: Data yield sample estimate S, sample size n where Recall Then Look up alpha in a table or use the chidist(chi,dof) function in Excel to find Alpha=0.067 Alpha is the risk probability

89 89

90 90 Value of Information Information reduces uncertainty –Toss a pair of dice and do not look at result –Estimate the probability that it is 7 –If told it is not 5, does this change your estimate? Information has a cost and a worth Should never pay more than it is worth

91 91 Example * A new manufacturing process has been developed. It will cost $0.5M to implement. It will save $0.5M over investment if implemented by your current vender. There are two other vendors willing to implement the process, but both require some relief from EPA regulations in order to provide savings, which could be least double those offered by the current vendor. No relief offers savings, but not as great as keeping the current vendor. It is known the EPA is re-examining these regulations and could either provide relief, make no change, or increase the requirements. The current vendor is not impacted regardless of any changes. An increase will cause a loss by both of the other vendors which you must reimburse. *Based on an example in Making Hard Decisions, An introduction to Decision Analysis; PWS Kent, 1991; Robert T. Clemen

92 92 Payoffs EPA Action Vendor Relief P= 0.5 Same P= 0.3 Increase P=0.2 V1$0.5 V2$1.0$0.2-$0.1 V3$1.5$0.1-$1.0

93 93 Decision Tree Structure V1 V2 V3 Relief Same Increase $0.5M $1.0M $0.2M -$0.1M $1.5M $0.1M -$1.0M (0.5M) ($0.54M) ($0.58M) (xxx)=EMV

94 94 Dilema Highest expected saving is obtained by choosing Vendor 3 This choice also involves the highest potential loss –Risk probability is 0.2 –Risk consequence is -$1.0 M –High Yellow How to mitigate?

95 95

96 96 Possible Actions Obtain Information –Re-estimate probabilities –Hire knowledgeable consultant Knows for sure Probably knows Questions: –What is information worth? –How reliable is the consultants result?

97 97 Perfect Information (xxx)=EMV $0.5 $1.0 $1.5 $0.5 $0.2 $0.1 $0.5 -$0.1 -$1.0 Relief 0.5 Same 0.3 Increase 0.2 Perfect Information No Information ($($ ($0.58M From previous analysis) ($1.0M) EMVPI = $1.0M - $0.58M = $0.42M V1 V2 V3

98 98 Imperfect Information Relief Same Increase Relief Same Increase Relief Same Increase Relief Same Increase Same Increase Relief Same Increase No Consultant Hire Consultant Relief Same Increase V3 V2 V1 $1.5M $0.1M -$1.0M $1.0M $0.2M -$0.1M $1.0M $0.2M -$0.1M $1.0M $0.2M -$0.1M $0.5M ($0.58M) (xxx)=EMV ? Relief

99 99 Problem Need to find conditional probabilities for chance events Solution –Total probability law –Bayes theorem

100 100 Solution Total Probability Where: R=Relief S=Same I=Increase R denotes predict Relief, etc.

101 101 Bayes Theorem

102 102

103 103 Imperfect Information Relief Same Increase Relief Same Increase Relief Same Increase Relief Same Increase Same Increase Relief Same Increase No Consultant Hire Consultant Relief Same Increase V3 V2 V1 $1.5M $0.1M -$1.0M $1.0M $0.2M -$0.1M $1.0M $0.2M -$0.1M $1.0M $0.2M -$0.1M $0.5M ($0.58M) (xxx)=EMV Relief ($1.164M) ($0.835M) ($0.187M) ($0.293M) -($0.188) ($0.219M) ($0.822M) EMVI=$0.822M-$0.580M=$0.242M

104 104 Impact on Risk Probability of $1M loss reduced from 0.2 to Risk changed from high yellow to low level Can pay consultant up to $0.242M

105 105 Problem Using EMV may lead to solutions that may not be intuitively appealing Example: –A1Win $30, p=0.5EMV = $14.50 Lose $1, p=0.5 –A2Win $2000, p=0.5EMV = $500 Lose $1000, p=0.5 Choose A2 based on EMV What about Risk and Consequence ? –Would probably rather have A1

106 106 Reason EMV is valid for the long run; i.e., multiple occurrences of the chance event This is a one time event Ignores the range of possible outcomes Play 10 times –Max loss A1 = $10 –Max loss A2 = $20000

107 107 Solution Find a transformation of consequence into a utility measure, U Must accommodate attitude toward the risk/consequence combination –Risk averse –Risk taking –Risk neutral

108 108 Utility Functions Tabular Math functions Graph

109 109 Risk Attitudes

110 110 Scenario Forced gamble –Win $500, p=0.5 –Lose $500, p=0.5 Would you pay $x to get out of this gamble? –If so, you are risk averse Examples –Insurance –Ransom

111 111 Comment Not everyone is risk averse Many people are risk seeking over some range and risk averse over others Depends on wealth level (range of consequences) Risk attitudes are important in analyzing mitigation alternatives

112 112 Mixed Risk Attitude

113 113 Transformation Utility function transforms wealth (consequence) into a measure that accounts for risk attitude Certainty equivalent (CE) –Gamble: Win $2000, p=0.5EMV=$990 Lose $20, p=0.5 –Offered $300 for the gamble. Ask $301, wont accept $299 Then CE=$300 for this gamble If CE ~ EMV then –U(CE)=EMV($) –Can substitute U(x) for $x EMV – CE = Risk Premium

114 114 Utility Function Assessment CE Method 1.Fix min and max values of wealth 2.Set U(min)=0, U(max)=1 3.Structure a lottery min max CE Find CE 1. Then Example:

115 115 CE Method (Contd) 5. Pick another range, say CE 1 – max 6. Structure a new lottery CE 1 max CE Set range 0 –CE 1 8. Repeat 6. To obtain U(CE 3 ) = Stop or further subdivide the intervals 10. Either draw the curve or fit a function

116 116 Probability Equivalent (PE) Method 1.Fix max and min of wealth 2.Set U(min) = 0, U(max) = 1 3.Pick CE: min < CE < max 4.Structure lottery min max CE 1-p p 5. Find p: CE ~ p(min) + (1-p)(max) Then: 6. Repeat steps 3. – 5. For other values of CE

117 117 Risk Tolerance Consider the utility function R determines shape Larger R -> Flatter function Smaller R -> more concave (risk averse) Hence, R depends on risk attitude

118 118 Determination of R Consider a lottery Y Y/ A1 A2 Find largest value of Y for which A1 > A2 R = Y To find CE, Find E(U) for the decision. Solve For x = CE

119 119 Caveats Utilities do not add U(A+B) = U(A)+U(B) Utilities do not express strength of preference. –They only provide a numerical scale for ordering preferences Utility functions are not the same person to person –They are subjective and express personal preferences

120 120 Example Revisited V1 V2 V3 Relief Same Increase $0.5M $1.0M $0.2M -$0.1M $1.5M $0.1M -$1.0M (0.5M) ($0.54M) ($0.58M) $U 1.5M M M M M M M0.00

121 121 Utility Axioms Ordering and transitivity Consider events A 1, A 2, A 3. Then A 1 > A 2, A 2 > A 1, or A 1 ~ A 2 If A 1 > A 2 and A 2 > A 3, then A 1 > A 3 Reduction of compound uncertain events A DM is indifferent between compound uncertain events (a complicated mix of gambles and lotteries) and a simple uncertain event as determined using standard probability manipulations.

122 122 Example A1 A2 E1 E2 E3 E4 E5 E6 E7 E8 E

123 123 Example Continued A1 A2 E1 E4 E5 E6 E7 E8 E =0.5* =0.33* =0.33* =0.33* =0.5* =0.5*0.5

124 124 Example Concluded A1 A = = =

125 125 Axioms Continued Continuity A DM is indifferent between outcome A and an uncertain event with outcomes A1 and A2 where A1>A>A2. Hence we can construct a reference gamble with p(A1) and (1-p)(A2) such that the DM is indifferent between A and the gamble Substitutability A DM is indifferent between an uncertain event A and one found by substituting for A an equivalent uncertain event (gamble can be substituted for CE)

126 126 Axioms Continued Monotonicity Given two reference gambles having the same possible outcomes, a DM will prefer the one with higher probability of winning the preferred outcome. Invariance Only outcome payoffs and probabilities are needed to determine DMs preferences Boundedness No outcomes are infinitely bad or infinitely bad

127 127 Comments There are some controversies and paradoxes regarding some of the axioms If you accept them then –There exist U1, U2, …, Un (utilities) with associated payoffs such that the overall preference for uncertain events A and B can be determined by E(U) –You should be using E(U) to make decisions (rational behavior)

128 128 Sensitivity Analysis To manage risk we need both probabilities and consequences Probabilities, utilities, and maybe consequences are likely subjective estimates Need to find out how much change will impact the decision

129 129 Example Problem Again V1 V2 V3 Relief Same Increase $0.5M $1.0M $0.2M -$0.1M $1.5M $0.1M -$1.0M (0.5M) ($0.54M) ($0.58M) $U 1.5M M M M M M M

130 130 Change Utility for $1.0M

131 131 Change in Probabilities

132 132 Summary Risk management process Cost and schedule risk –Models –Monte carlo simulation Estimation of likelihoods Analysis of mitigation alternatives Value of information Estimation of utility metrics EMV vs Expected utility Sensitivity analysis

Download ppt "1 Risk Management. 2 Road Ahead Risk Management Process Cost and Schedule Risk Estimating Likelihood Mitigation Utility and Consequences Sensitivity Analysis."

Similar presentations

Ads by Google