November 8, 2007 Conference Brief RISC-IQ: Risk Integrated w / Schedule & Cost – Intelligent Quantification Applications for Systems Engineers This document is confidential and is intended solely for the use and information of the client to whom it is addressed. International Council on System Engineering

1 Booz Allen Proprietary The need for Risk Management to be Integrated with Systems Engineering is well understood  In a recent Department of Defense briefing titled “Technical Excellence through Systems Engineering”, it was determined that there was a lack of understanding in programs on systems engineering. One of the reasons sighted was: –Cost and schedule estimation and risk management processes inconsistently aligned with systems engineering processes  26% of surveyed systems engineers listed “Integration with other program management” as a way to drive technical rigor back into programs. This included: –Linkage with acquisition strategy, IMP, IMS, logistics, testing, and risk management  As a result, a recommended strategy for driving technical rigor back into programs was to have risk management integrated, effective, and well resourced

2 International Council on System EngineeringBooz Allen Proprietary So what?  The question isn’t “Should we do risk management?”  The question is, as a Systems Engineer: –What do I do with the risk program I have? –Is the risk program providing a good return for the expense? –What, if anything, are the risk program outputs telling me? –Can I use the risk program to communicate more effectively?  The rest of this discussion supposes a mature risk program exists: –Roles and Responsibilities are defined (i.e. a Risk Mgt plan exists) –Training has been provided –Resources have been provided (tools, staff) –Risk Management products are reviewed at regular intervals in formal reviews (Risk Review Boards)

3 International Council on System EngineeringBooz Allen Proprietary RISC-IQ was developed to help extract actionable information from traditionally stove-piped data streams RISC-IQ enables critical decision making Risk Exposure? Impact Relationships? Goals Too Risky? Which Design? More Reserves? Major Drivers? Adequately Mitigated?

4 International Council on System EngineeringBooz Allen Proprietary RISC-IQ works by establishing a structured, repeatable process that integrates the potential impacts of risk on program baselines Risk Integrated with Schedule and Cost – Intelligent Quantification Objective: Assess completeness of baseline documentation and understand existing processes. Objective: Compare program risk profile at mitigation completion to determine outstanding risk exposure. Objective: Provide insight into where risks affect the program and uncover their true impacts. Mitigation Strategy Revision Risk Analysis Risk Analysis Schedule Development Schedule Development Cost Estimating Cost Estimating Program Risk Baseline Risk-WBS Mapping Risk Impact Analysis Integrated Master Schedule Most Likely Cost Estimate Pre-Mitigation Cost Estimate Economic Effectiveness Analysis Baseline Product Development Pre-Mitigation Schedule Estimate Mitigation Plan Development Mitigation Resource Analysis Economic Effectiveness Assessment Post-Mitigation Cost Estimate Initial Risk Accounting Schedule Factor Analysis Cost Factor Analysis Post-Mitigation Schedule Estimate

5 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

6 International Council on System EngineeringBooz Allen Proprietary RISC-IQ uses explicit, quantified statements for risk probability and impact to determine risk scores Probability 5Very High 4High 3Medium 2Low 1Very Low Impact 5Very High 4High 3Medium 2Low 1Very Low Probability 5>70% % % % 1<10% Cost Impact 1>70% % of Project Budget % of Project Budget % of Project Budget 5<10% of Project Budget  The right-hand set of tables: –Communicates the leadership’s sensitivity to risk –Reduces personal bias –Creates a foundation for quantitative analysis VS

7 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

8 International Council on System EngineeringBooz Allen Proprietary RISC-IQ can be used to help select and support multiple types of development cycle  Waterfall Development Model & Derivatives  Assists with estimates of overall cost and schedule estimates  Maps risk mitigation activities to development phases  Quantified risk score can be used to establish threshold for entering the next development phase  Spiral Type Develop Model  Enables capability vs. risk trade study for design features  Economic analysis of risk help apportion development resources to highest need areas  Tracking of risk profile useful in determining if the current spiral met its goals

9 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

10 International Council on System EngineeringBooz Allen Proprietary RISC-IQ is useful for evaluating and communicating the impacts of design selection on risk exposure  As part of design trade activities, RISC-IQ provides input to the engineering processes by analyzing design options based on risk profiles  The risk team helps determine the risk variability across the options and the dependency of any one risk to key features/designs within an option  RISC-IQ applies a “Dependency” framework for this activity. –Design Independent Risks- –Design Independent Risks- Risks that remain relatively constant when key aspects of the system are altered –Design Dependent Risks- –Design Dependent Risks- Risks that vary significantly based on changes to the system design Design Independent Risk Example Option 1Option 2Option Risk Score Design Dependent Risk Example Option 1Option 2Option Risk Score Little to no change in the risk score as the design or option changes Significant change in the risk score as the design or option changes

11 International Council on System EngineeringBooz Allen Proprietary RISC-IQ provides a simple analytical framework for quickly highlight what risks drive any given design option  Dependency analysis is performed for every risk to determine whether the risk applies to a given option and the strength of dependency by design parameter  Shown below is a sanitized example from a recent client engagement Results Sample

12 International Council on System EngineeringBooz Allen Proprietary The RISC-IQ design dependency approach establishes a method to evaluating requirements and overall risk tolerance  Using the quantitative risk scores we can compare a single risk area across platforms, or the variability of a single risk within a given platform Risk Level Weight Risk Satellite A option Satellite B option Satellite D option Satellite C option Medium Low High Critical Risk Level Communications Risk Option 1 Option 2 Option 4 Option 3 Medium Low High Critical (Placing the options in order of complexity, i.e. performance or technology, allows the program to understand the ‘risk curve’ as the options become more aggressive or challenging in their profile.)

13 International Council on System EngineeringBooz Allen Proprietary Once the analytical framework and quantitative scoring is established, risk can then be treated as a design variable “Risk As An Independent Variable”  Risk can be treated as an independent variable and applied in a similar fashion to cost in CAIV methodologies; “Risk As An Independent Variable” Risk Level Technology Design Variable/Option (1) Risk curves are generated for designs or aspects of each design using dependency model Performance Metric Technology Design Variable/Option (2) Performance Analysis working group determines relationship between the design variable and system performance Variable Risk Metric Variable Effectiveness Metric ƒ (Variable) = XX Effectiveness Performance Metric (3) Utility Analysis working group performs effectiveness analysis to show relationship to performance metric Risk Sensitivity Performance Sensitivity Effectiveness Analysis

14 International Council on System EngineeringBooz Allen Proprietary RISC-IQ then uses this analytical framework show when additional performance isn’t worth the additional risk  By combining these curves, it’s possible to identifying the point of diminishing risk tolerance for any design variable or option –Point of Diminishing Risk Tolerance- “The point at which the increase in additional design capability, or the selection of a proposed option, reduces effectiveness by increasing risk to the program.” Risk Tolerance Design Variable/Option Effectiveness Risk = Point of Diminishing Risk Tolerance

15 International Council on System EngineeringBooz Allen Proprietary RISC-IQ takes the results of the design trades and evaluates them by source category and the work to satisfy the requirements WBS # WBS Element Comms Concurrency Cost Design Developer Logistics M&S Mgmt Performance Production Requirements Schedule T&E Technology Threat Total 1.0Prog Mgmt Network Mission Ground GSE Program Management Subcontractor TT&C Communications 5.0Launch Space PMO 7.0Space SEIT Bus Payload Spacecraft AI&T Total A review by WBS elements shows which areas of the program need additional risk identification. Critical Risk Categories with Few Risks Identified A review by categories show holes in existing risk areas that should be investigated further.  The following is an example from a commercial satellite project analysis

16 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

17 International Council on System EngineeringBooz Allen Proprietary RISC-IQ uses a convention of aligning risks to the Work Breakdown Structure (WBS) to help define resource requirements  Aligning risks to the program WBS serves as the basis for allocating cost and schedule impacts throughout the program  During the risk analysis phase, WBS elements are correlated to each other and risks mapped to WBS elements Mapping provides an easily understood foundation that is repeatable, traceable, and defensible

18 International Council on System EngineeringBooz Allen Proprietary  After mapping, any WBS element may have multiple risks that affect it  The combined affect of the risks on the cost element is related to the type of the risk, its likelihood and severity, and the cost of mitigation actions that could avert the risk  Using probabilistic models, the combined affect is determined and applied to the base cost estimate RISC-IQ uses this WBS mapping to show the impacts of risk on the original estimate WBS Baseline Estimate Risk-Adjusted Estimate $1,179,000 50%

19 International Council on System EngineeringBooz Allen Proprietary This activity helps justify mitigation requests and can highlight when mitigation is no longer effective Represents the initial baseline cost distribution without any included risk effects Represents the cost distribution with potential risk effects BEFORE any mitigation Represents the cost distribution AFTER successful mitigation, PLUS the required mitigation investment. As a result of a $500,000 investment in risk mitigation, the program avoided a potential increase of $645,000. $645,000 Mitigation Investment= $500,000 RISC-IQ then applies the WBS mapping and pre- and post- mitigation scores to guide mitigation activity  The final determination of cost considers the remaining risk exposure AFTER mitigation, plus the mitigation investment required

20 International Council on System EngineeringBooz Allen Proprietary  Relationships between risks and schedule elements for critical path determination must account for: –Predecessor-Successor logic –Parent-Child –Parallel-Effort –Conditional Dependencies The same process can be performed on any unit of measure. Risk Adjusted Schedule is particularly complex Risk Adjusted Delivery LikelihoodSchedule Sensitivity By Task

21 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

22 International Council on System EngineeringBooz Allen Proprietary RISC-IQ can help determine if the engineering activity is proceeding as planned  As a “snapshot” of project health, RISC-IQ can reveal whether or not the engineering activity is likely to deliver the expected results  In the example below, a RISC-IQ analysis demonstrated a previously unknown risk exposure of $148M and potential project cost growth of 25%  Incorporating RISC-IQ throughout a project’s lifecycle results in better initial planning and helps answer questions like: –Are prototype or testing activities reducing risk as planned? –Have risks been reduced sufficiently to pass entry / exit criteria for established technical reviews? –Are new risks been uncovered faster than they are being closed? –What are the impacts of “mid-stream” requirements changes?

23 International Conference on System EngineeringBooz Allen Proprietary Agenda  Project / Program Initiation  Design Cycle Selection  Engineering Trades  Defining Resources  Measuring Execution  Managing Groups of Projects

24 International Council on System EngineeringBooz Allen Proprietary RISC-IQ practices for a single project are extensible to groups of projects or systems of systems the must interact to deliver capability… Low Med High Critical Average Composite Risk Ranking by Program Average Risk Score  By standardizing risk analysis across all multiple programs, we provide insight into high risk programs and overall trends

25 International Council on System EngineeringBooz Allen Proprietary … and program-by-program risk exposure analysis provides comparative insights what programs can be improved Program Cost (in thousands) $39,100 Initial Program Estimate Revised Estimate Budget at Risk Mitigation Investment Reduced Cost Exposure Analysis will lead to a cumulative assessment of the total risk exposure, the investment allocated to reduce it, and the resultant investment benefit.

26 International Council on System EngineeringBooz Allen Proprietary Added Duration Contract Length Total Risk Exposure In a matrixed organization where Systems Engineering is a shared resource, using RISC-IQ to align risk-adjusted program schedules helps prioritize the timing of resources and risk reduction Post-Mitigation Delivery Pre-Mitigation Delivery Contract Delivery Program 1 Program 4 Program 2 Program 3 Program N T0T0 T3T3 T 18 T6T6 T9T9 T 12 T 15 T2T2 T5T5 T 20 T8T8 T 11 T 14 T 17 T1T1 T4T4 T 19 T7T7 T 10 T 13 T 16 The overlap of additional program duration requires allocation of program and/or enterprise resources Understanding most probable program delivery allows S&IS to: Risk exposure is the amount of schedule slip reduced through successful risk mitigation  Inform clients of most probable schedule  Plan for resources in advance  Address contract issues  Evaluate effectiveness of proposed mitigation  Understand when enterprise reserves will be needed

27 International Council on System EngineeringBooz Allen Proprietary In summary, risk management provides the Systems Engineer with quite a bit of “so what” Concept Development DesignPlanning & ExecutionEnterprise RISC-IQ can help established risk tolerance and incorporate risk as variable benefit trades RISC-IQ provides unique insight into project cost and schedule realism, reserve requirements and health during execution RISC-IQ highlights what specific requirements or technologies are project drivers RISC-IQ at the enterprise level highlights overall reserves needs, resource shortfalls due to slips, and supports benchmarking

28 International Council on System EngineeringBooz Allen Proprietary For more information on how risk management can be applied to your specific challenges, please contact: Tweed Ross Colorado Springs, CO Robert Makar San Diego, CA Michael Lopez Los Angeles, CA