1. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER ASN (RDA) Chief Systems Engineer ASN (RDA) Chief Systems Engineer Naval Probability of Program Success (PoPS) Executive Brief NDIA / ICPM Naval Probability of Program Success (PoPS) Executive Brief NDIA / ICPM 1 June 2009 Mr. Carl Siel ASN(RDA) Chief Systems Engineer carl.siel@navy.mil

2. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Agenda Background DON Acquisition Governance Core and Naval PoPS PoPS v1.0 Structure Products and Tools Guidebook, Handbooks, Visuals, Templates How used Individual Program Portfolio of Programs Identification of Systemic Issues Gate Review Path Forward

3. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER DoN Acquisition Governance The Secretary of the Navy o With CNO, CMC, ASNRDA, SYSCOM Commanders o Comprehensive review of the Acquisition process o Challenges in Program Planning and Execution. Enhance the Acquisition Governance process o Inject Early Senior Leadership o Continuous Engagement and Transparency Increase discipline during each phase of Program Maturity Codified by SECNAVNOTE on 26 February 2008 Incorporated into SECNAVINST 5000.2D “Two Pass / Six Gate”

4. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER DoN Acquisition Governance First Pass - Requirements Establishment Second Pass - Acquisition Execution System Design Specification Capability and Performance Expectations Gates - Reviews to Assess Readiness to Proceed Program Health Assessment at each Gate

5. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Gate Review Core and PoPS “Core” = Detailed information germane to the Gate Decision “PoPS” = Holistic view of overall program health and readiness to proceed

6. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER PoPS Background 19 Jan 2008 PDASN Memo - Interim use of Air Force “PoPS Spreadsheet Operations Guide” to Assess Program Health During Gate Reviews 6 October 2008 PDASN Memo Naval PoPS Guidance Directed the use of the methods and tools described in the Naval PoPS Guidebook and supplemental Handbooks Required for all DON Gate Reviews (ACATs I & II) Shall serve as the standard DON method of representing the health of all ACAT Programs and any other programs subject to the DON Acquisition process Consistent means to assess program health Used any time program health is discussed

7. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Agenda Background DON Acquisition Governance Core and Naval PoPS PoPS v1.0 Structure Products and Tools Guidebook, Handbooks, Visuals, Templates How used Individual Program Portfolio of Programs Identification of Systemic Issues Gate Review Path Forward

8. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Leveraging Air Force PoPS 8 Relocated Removed Added DON Interim PoPS 5 Factors 22 Metrics Subjective Criteria Naval PoPS 1.0 4 Factors 17 Metrics More Objective Criteria

9. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Naval PoPS v1 9 *Notional representation of Criteria. Criteria are Gate- and Metric-specific. The number of Criteria will vary. Program Health 4 Factors 17 Metrics Criteria*

10. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Naval PoPS Criteria (Gate 2 Example) Metric Factor Criteria 10

11. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Naval PoPS Process – R/Y/G Thresholds 11  Criteria Color Thresholds Green = 100% of max score Yellow = 66% of max score Red = 33% of max score  Metric, Factor and Program Health Color Thresholds Green = 80-100% of max score Yellow = 60-<80% of max score Red = <60% of max score Example of Criteria Color Thresholds Example of Green Threshold for “Parameter Status” Metric G Y R G Y R

12. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Naval PoPS Scoring Gate 2 Example 17 Metrics 4 Factors 4 pts each2774.52.514.5 0.50.80.2 0.250.66 0.14 1.50.52 Criteria G = 100% Y = 66% R = 33% G=80-100% Y=60-<80% R=<60% G=80-100% Y=60-<80% R=<60% 77.49/100 G=80-100% Y=60-<80% R=<60% 26.21/4021.47/2323.85/295.96/8 17.28/2 4 2/26.93/1414/147.47/92.48/52.66/39/9.83/11.55/2 2.15/33.46/4.86/1 2.49/32.15/31.32/2 12 Gate 2

13. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Factor and Metric Level Maximum Scores

14. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER “Killer Blow” Metrics 14 A Killer Blow is interpreted as a negative situation exists for the program of such significance that it must be remedied..

15. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER 15 Naval PoPS from Tool Gate 2 Review without “Killer Blows”

16. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER 16 Naval PoPS from Tool Gate 2 Review with “Killer Blows”

17. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Naval PoPS Products Guidebook, Handbook, Visuals and Templates Naval PoPS Guidebook oOverview information on Naval PoPS methodology and implementation Naval PoPS Criteria Handbook and Spreadsheets oComplete listing of Program Health Criteria per Gate, per Metric oInstructions on completing Program Health Assessments in Criteria Spreadsheets Naval PoPS Visuals Handbook, Spreadsheets, and Templates oComplete set of Program Health Visual Templates for each Gate Review oInstructions on creating Visual graphs/charts using Visuals Spreadsheets 17

18. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER How we’re using PoPS  Individual Program Reviews o Identify matters that affect Program Health  PEO Portfolio Reviews o Health of all ACATS within a PEO  Senior Leadership assistance  Identify Systemic Issues o Action Item Tracking by PoPS Metrics o “Advocates” by PoPS Metrics  Establish action teams to address issues with Department implications

19. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER ‘Portfolio of Portfolios’ Approach Gate 6 Review Sub-Portfolio Gate 6 Review Sub-Portfolio Gate 6 Review Sub-Portfolio Program A (ACAT _) Program B (ACAT _) Program C (ACAT _) Each program within a sub portfolio will complete a PoPS Gate 6 Program Health Assessment. Gate 6 Review Non-Active Programs Portfolio Program XX (ACAT _) Program X (ACAT _) Program Y (ACAT _) Gate 6 Review Sub-Portfolio PEO / Team Portfolio Health Assessment Pre-Gate 6 Review ACAT Programs PEO / Team Portfolio Health Assessment Post-Gate 6 Review In-Service Programs PEO / Team Portfolio Health Assessment Gate 6 Sufficiency Review ACAT Programs Program X1 (ACAT _) Program Y1 (ACAT _) Program A1 (ACAT _) Program B1 (ACAT _) Program C1 (ACAT _) Program X2 (ACAT _) Program Y2 (ACAT _) Program X3 (ACAT _) Program Y3 (ACAT _)

20. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER PEO / Team Overall Portfolio Health

21. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Systemic Issue Advocates  Acquisition Management– Principal Civilian DASN (PCD)  Manning – PCD  Contract Execution DASN A&LM  Industry / Company Assessment – DASN AIR and SHIPS/IWS)  Sustainment – DASN A&LM  Interdependencies – DASN C4I  Budget – DASN M&B and EXW  Cost Estimating – ASN FMB  CONOPS – OPNAV / HQMC  Government – DASN C4I  Program Advocacy – DASN C4I  Scope Evolution – DASN M&B with Product DASNs  Parameter Status – RDA CHSENG  T&E – DON T&E Executive  Technical Maturity – RDA CHSENG  Naval POPS – RDA CHSENG  NOTPOPS Other – RDA CHSENG  NOTPOPS Life Cycle Costs – DASN A&LM

22. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Agenda Background DON Acquisition Governance Core and Naval PoPS PoPS v1.0 Structure Products and Tools Guidebook, Handbooks, Visuals, Templates How used Individual Program Portfolio of Programs Identification of Systemic Issues Gate Review Path Forward

23. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER 23 Gate Review Update New DoDI 5000.02 (Dec 2008) 2 Post-IBR Post-CDR CPD 1 34 5 6 6 6 6  Reassess Core Entrance / Exit and Program Health Criteria  Functional Sub Groups addressing emergent initiatives  For each, determine needed revisions to o Entrance and Exit Criteria o Core Templates o Program Health Criteria and Templates  Align Gates to the new DODI 5000.02 o Risk Management o Configuration Steering Board o T&E o Open Architecture o Technology Protection o Sustainment o TOC o Cost Estimating

24. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Gate Review Core and Program Health The Gate Review process is being used and updated to: o Inject Senior Leadership early o Ensure continuous engagement and transparency o Increase discipline during each phase of Program Maturity

25. RDA CHSENG, April 2009 DA HIEF YSTEMS NGINEER Questions?