Presentation is loading. Please wait.

Presentation is loading. Please wait.

Peter Lierni, PMP, CISA 09 November 2007

Similar presentations


Presentation on theme: "Peter Lierni, PMP, CISA 09 November 2007"— Presentation transcript:

1 Peter Lierni, PMP, CISA 09 November 2007
Insights from Assessing the Risk Management Programs of Major Defense Acquisition Programs Peter Lierni, PMP, CISA 09 November 2007 Lierni © 2007

2 Contributors to Risk Management Body of Knowledge
International Council on Systems Engineering (INCOSE) Defense Acquisition University (DAU) Project Management Institute (PMI) Software Engineering Institute (SEI) Information System Audit and Control Association (ISACA) et al Lierni © 2007

3 Risk, Issue, and Opportunity Management
Risks — future uncertainties that could have an adverse impact to cost, performance, schedule, etc. “Apply to both products and processes” Issues — “realized risks” that are now problems and impacting things such as cost, performance, schedule, etc. “Issues could generate other risks” Opportunities — future uncertainties that if realized could improve cost, performance, schedule, etc. Distinguish between risk, issue, and opportunity management Lierni © 2007

4 Questions a Mature Risk Management Program Should Be Able to Answer
What are the program’s risks? (e.g., technical, schedule, cost, etc.) What is the impact of these risks on program efforts? Have risk mitigation action officers been assigned? Have risk mitigation action due dates been assigned? How far has the program come with respect to accomplishment of planned mitigation actions? How far has the program come with respect to meeting program objectives? How does the program stand with respect to comparable efforts? To what extent has experience been incorporated into the program? Lierni © 2007

5 Essential Elements of A Mature Risk Management Program
Thorough risk assessment as part of Business Case Analysis (BCA) that at minimum addresses cost, schedule, and performance Early and incremental and iterative use of Modeling and Simulation (M&S) and testing throughout the Systems Development Lifecycle (SDLC) to investigate various design options and system level requirement Acquisition Strategy that addresses all major known program risks “Include risk mitigation strategies and assumptions, as well as rationales for assumptions” Experience from similar programs sought and incorporated to enable risk reduction before initiating program planning Risk management well integrated with program’s SE approach Technical baseline of system being acquired continually used to assess technical risk Lierni © 2007

6 Essential Elements of A Mature Risk Management Program (Continued)
Key subcontractors/suppliers integrated into risk management planning, execution, monitoring, and control activities “Most effectively accomplished through Integrated Product Team (IPT)” Well-chartered and executed decision-making bodies [e.g., Configuration Control Board (CCB), IPTs, Risk Review Board (RRB)] Joint Government/Contractor collaborative relationship and feedback mechanism in existence between all SE activities and risk management “Risk data should be visible, accessible, and understandable to both” Risk Watch List and Program-level Risk Cube Program/technical reviews provide input into risk management process and vice versa Evident how the Government/Contractor PMs use program/technical reviews to assess risk Lierni © 2007

7 Essential Elements of a Mature Risk Management Program (Cont.)
Trade studies that consider risks associated with alternatives Continuity in explanation of how risk is addressed in key program documentation [e.g., Acquisition Strategy Report (ASR), Request for Proposal (RFP), Program Management Plan (PMP), Systems Engineering Plan (SEP), Risk Management Program Plan, Test Evaluation Strategy (TES), etc.] “Best efforts should be made to ensure documents reflect most current risk status and critical mitigating actions of the overall program” Web-enabled Risk Management Information System (RMIS) to ensure awareness and accessibility of risk information (horizontally & vertically) “Information in RMIS should be current, complete, clear, and cogent to best enable decision-making bodies. Particularly important with regards to traceability, monitoring, and control of mitigation plans” Best enabled by a core set of risk management principles and effective leadership Lierni © 2007

8 Use Holistic Risk Management Approach to Navigate the Program
Change Management — ability to accommodate change that reduces adverse impact “Risk management should be viewed as a component of change management” Communication Effectiveness — ability to reduce the likelihood of misunderstandings among parties involved in business dealings Risk Assessment — ability to set or determine risk amount and its potential impact Risk Mitigation — ability to reduce adverse impacts of assessed risks Performance Management — ability to manage change (and risk) quantitatively Knowledge Management — ability to accumulate and apply knowledge for organizational benefit and growth Lierni © 2007

9 Risk Management Is Inherently A Natural Part of Change Management
Change is constant with all programs Change results from known things “as planned” Change results from eventual unknowns “as unplanned” Change management is the ability to accommodate change that reduces adverse impact Lierni © 2007

10 Look and Risk & Change Three Ways
Change occurs to address risk (i.e., to eliminate something negative) or to realize something positive Risk analysis should be performed on change being considered “Technology X vs. Technology Y” Risk mitigation plans may have to be implemented to address risks that could prevent the change from successfully being implemented Lierni © 2007

11 Mechanisms to Accomplish Effective Risk Management
Effective communication is essential to managing risk Risk management can best be accomplished through the use of CCB and IPT processes “These forums when properly implemented provide a proven means for reducing the likelihood of miscommunication” Lierni © 2007

12 Integrating Risk Management with Program Controls
Objective of a well-managed risk management program is to provide a tool for balancing cost, schedule, and performance goals within program funding “Especially on programs with designs that approach or exceed state-of-the- art or have tightly constrained or optimistic cost, schedule, and performance goals” Often there is lack of a linkage amongst the following processes such that they are used as effective tools to enable risk reduction: Work Breakdown Structure (WBS) Integrated Master Plan (IMP)/Integrated Master Schedule (IMS) Earned Value Management (EVM) Performance Measures Risk Mitigation/Issue Recovery Plans Knowledge Management (KM) Most effective when working together vs. alone Lierni © 2007

13 Rationale for Integrating Risk Management with Program Controls
Increased vigilance Better communication Increased responsiveness Enhanced internal programmatic controls Improved application of Management Reserve (MR) Increased learning Better technical planning All of these attributes when evident are enablers of program risk reduction Lierni © 2007

14 WBS Technical risk management should be based on individual product or specific critical processes (e.g., design, development, and test) affecting individual WBS elements Risk assessments and mitigation activities should be conducted on individual WBS elements “Emphasize technology, product/process maturity or perceived quality and deviations from the cost & schedule baseline” IPTs should carefully review those sections of WBS that they are responsible for to identify, assess, and track technical risks IPTs should primarily look for impact on cost and schedule, and the resulting effect on the overall product Identified WBS-derived risks and associated mitigation plans should have related WBS element number for the risk specified Implementing a risk mitigation plan is an important reason for a scope change and should be reflected in WBS “Updates should be done in a timely manner and reflected in updates to IMS” Lierni © 2007

15 IMP/IMS Employ as tool for planning, executing, and tracking risk mitigation efforts “Conduct Schedule Risk Assessments” Ensure significant risks identified by Government in RFP are addressed in Contractor IMS in response to Government Use IMP/IMS to enable risk management “Ensure staff responsible for IMP/IMS process work with IPTs to regularly identify moderate-to-high risk tasks to ensure that specific risk reduction (handling) activities are properly reflected in IMS” Have Program Manager (PM) regularly assess the status of risk management activities based on inclusion of risk mitigation activities in IMP/IMS Ensure risk mitigation activities in IMP/IMS are flagged so that they are easy to call out from the other tasks in IMP/IMS Lierni © 2007

16 IMP/IMS (Continued) Do not manage IMS at exclusion of risk management
“An IMS summarized at too high a level often results in masking critical elements of the plan necessary to execute the program and fails to show the risk management approaches being used” Review IMS for completeness and consistency with program staff responsible for IMS “Work together to evaluate duration and logical relationships to ensure they will accomplish the desired risk mitigation” Lierni © 2007

17 EVM Ensure specific risk-handling actions are reflected in detailed work packages as part of performance baseline Have IPTs monitor effectiveness of risk-handling actions by providing periodic comparisons of actual work accomplished in terms of cost and schedule with the work planned and budgeted Analyze cost/schedule variances in work packages containing risk-handling actions to isolate root causes and gain insights into need to modify actions Understanding root causes of cost/schedule variances in work packages containing risk-handling actions allows opportunity to improve technical planning Lierni © 2007

18 Enable Vigilance Lack of Measures-driven Approach to Risk Management
Monitor areas of known risk (e.g., product, process, people, etc.) Provides early detection of new risks before irrevocable impacts on cost/schedule occur Use to assess effectiveness of risk-handling actions Lierni © 2007

19 Think SIPOC Program staff responsible for individual processes should employ the notion of Supplier, Input, Process, Output , Customer (SIPOC) from Six Sigma (∂) “Enables better communication and collaboration with other program staff responsible for individual processes” Effective choreography should exist amongst all program staff responsible for individual processes for integrated risk management to truly be realized Measures should be developed with participation of Government/ Contractor stakeholders so that they: Answer stakeholder’s question(s) Focus on the key thing(s) necessary to answer the question Reflect stakeholder’s vocabulary Are weighted according to what stakeholder believes is important Lierni © 2007

20 Bad vs. Good Risk Management and Funding Availability to Mitigate Risk
Lack of a measures-driven approach to program risk management causes .... Bad Risk Management Less program funds available to mitigate any possible risk associated with unknown/unknown risks Greater portion of program funds expended mitigating risk associated with known and known/unknown risks As compared to a measures-driven approach to program risk management which causes .... Good Risk Management Less program funds expended mitigating risk associated with known and known/unknown risks Greater portion of program funds available to mitigate any possible risk associated with unknown/unknown risks Lierni © 2007

21 Evidence of Proper Risk Mitigation Planning
Mitigation Plans task-oriented with realistic and achievable actions Planned and actual start/completion dates Action officer accountable for over all status of mitigation plan Individual(s) and/or organization(s) assigned responsibility to implement and report status of assigned tasks Requisite resources (i.e., personnel, capital equipment, facilities, procured items) identified Included in IMS, particularly risks with initial status of “red” Funded with MR employed as necessary Mapped to WBS to at least tier-three level “Enables better EVM” Quantifiable and/or tangible measures of success for closure criteria Lierni © 2007

22 Evidence of Proper Risk Mitigation Planning (Continued)
Likelihood of risk realization as near-term, mid-term, or far-term event Logical explanation of reduction to probability and/or impact Off-ramp (contingency plan) enabled by trigger point(s), particularly for “red” and higher-rated “yellow” risks being tracked “Contingency plans should be developed and implemented in same manner specified for risk mitigation plans” Minutes to date available that reflect mitigation plan reporting status and outcomes Risk mitigation plans with “risk burn-down” graphs Lierni © 2007 Lierni © 2007

23 Prevent Organizational Knowledge from Being Lost
Systematically secure knowledge gained from outcomes of risk mitigation “Certain knowledge and useful experiences otherwise gained could be lost to the detriment of future technical planning and improving risk reduction on the program, as well as the portfolio of programs within the organization” Have PM continually seek out and capture lessons learned, particularly as root cause analysis is performed throughout the program Use “knowledge gained over time” to improve processes, as well as entry and exit criteria of program/technical readiness reviews Lessons Learned “1) Repeatable 2) Traceable 3) Assignable 4) Measurable 5) Provides Benefit” Knowledge  Certainty  Control  Risk Lierni © 2007

24 Risk Program Health Metrics
Employ metrics to assess overall risk management program effectiveness. Emphasize: Trends Planning Accountability Communications effectiveness Without the last three items, risk is less likely to be mitigated and issues recovered from Lierni © 2007

25 Example Risk Program Health Metrics
# of Total Current Period vs. Total Prior Period Open Risks by Status (e.g., Red/Yellow/Green) by IPT # Total Risks by IPT # of Total Risks by Status # of Total Risks by IPT by Status # of Total Current Period vs. Total Prior Period Open Risk Aging (e.g., 1-30 days/31-60 days/61-90 days/ 91+ days) by IPT # of Total Risks by Age # of Total Risks by Age over Time for Current Period vs. Prior Period # of Total Risks by Age by IPT # of Total Open Risks by Specific Mitigation Plan Action Officer # of Mitigation Plan Action Items (i.e., Tasks) Open/Closed vs. Total across All Plans # of Mitigation Plan Action Items Open/Closed vs. Total across All Plans by IPT Lierni © 2007

26 Example Risk Program Health Metrics (Continued)
# of Total Mitigation Plans Assigned and Not Developed # of Total Mitigation Plans Assigned by IPT and Not Developed # of Total Mitigation Plans Assigned by Current Status and Not Developed # of Total Mitigation Plans Assigned by IPT by Current Status and Not Developed # of Mitigation Plans developed and Not Reflected in IMS # of Mitigation Plans Developed and Unfunded # of Mitigation Plans Developed without Resources Identified for All Tasks # of Mitigation Plans Developed without Due Dates Identified for All Tasks # of Mitigation Plans Developed without Tasks Currently Reported on at latest IPT/RRB meeting (e.g., Risk Open/Closed/other Update) Metrics best presented graphically! Lierni © 2007 Lierni © 2007

27 Summary Contact Information: Lierni © 2007


Download ppt "Peter Lierni, PMP, CISA 09 November 2007"

Similar presentations


Ads by Google