Copyright © 2008 Industrial Committee on Program Management. All rights reserved. Predictive Measures of Program Performance June 9, 2008 Industrial Committee on Program Management (ICPM)

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 2 Members BAE SystemsSusan Dong Battelle Bill Altman Bell Helicopter Bob Kenney Boeing Steve Goo BoeingRandy Steeno Honeywell Keith Munson Honeywell Tracie Thompson Lockheed Martin Peter Wynne Lockheed MartinSteve Stern MCR Neil Albert Northrop Grumman Marilyn McAlice Northrop Grumman Kevin Carpenter Raytheon Skip Burns Rockwell Collins Ron Hornish SI International Bill Chadick ArmyCraig Tallman NAVAIRDave Burgess NAVAIRTed Rogers USAFLtCol Fred Gregory

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 3 Agenda Recap from last NDIA Meeting Common Definitions Plan and Next Steps Help Needed

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 4 Recap from NDIA Meeting

METRIC DEFINITION Metric Name: Metric Backup Cost Performance Index (CPI) Purpose of Chart: Compare the value of work performed to the actual cost of that work to determine how efficiently work was performed – “bang for the buck” as one element to assess program progress Deployment Criteria / Linkage : NDIA Industry Standard Metric. Links to SPI, CPI-TCPI Source of Data: Program Management, Program Control Finance Benchmark/Comparative Data: None at this time Applicable Life Cycle Phase: All (Development, Production, Sustainment) Predictive / Leading Qualities: When trended, indicates patterns that can be used to project an outcome based on time phased actual performance Warning Signs & Actions to Take: When rolling average less than 0.95, identify root cause and take corrective action Metric Definition: This metric is the ratio of budgeted cost of work performed (BCWP) divided by actual cost of work performed (ACWP). BCWP / ACWP = % Comp * total BCWS / ACWP [(% Comp * total budget hours) / actual hours] Rating =1.00: On-cost; CPI 1.00: Positive cost Blue: CPI > 1.00; Green: 0.95 < CPI < 1.00; Yellow: 0.90 < CPI < 0.95; Red: CPI < 0.90 Creation Date: 05/16/08 Revision Date: 06/05/08

METRIC DEFINITION Metric Name: Metric Backup Schedule Performance Index (SPI) Purpose of Chart: Compare the value of work performed to the value of work scheduled Deployment Criteria / Linkage : NDIA Industry Standard Metric. Links to CPI Source of Data: Program Management, Program Control Finance Benchmark/Comparative Data: None at this time Applicable Life Cycle Phase: All (Development, Production, Sustainment) Usage Assumption: SPI should be used in conjunction with detailed schedule milestone metrics (late start, late finish, etc.) Predictive / Leading Qualities: When trended, indicates patterns that can be used to project an outcome based on time phased actual performance Warning Signs & Actions to Take: When rolling average less than 0.95, identify root cause and take corrective action Metric Definition: This metric is the ratio of budgeted cost of work performed (BCWP) divided by budgeted cost of work scheduled (BCWS). BCWP / BCWS = % Comp * total BCWS / BCWS [(% Comp * total budget hours) / budget hours] Rating =1.00: On-schedule; SPI 1.00: Ahead of schedule Blue: SPI > 1.00; Green: 0.95 < SPI < 1.00; Yellow: 0.90 < SPI < 0.95; Red: SPI < 0.90 Note: SPI does not guarantee that the work completed is the same work that was planned and should be used with detailed milestone metrics. Creation Date: 05/16/08 Revision Date: 06/05/08

METRIC DEFINITION Metric Name: Metric Backup Estimate at Completion (EAC) Purpose of Chart: Measure the estimate of total cost for authorized work (EAC) including actual costs plus estimated costs to complete Deployment Criteria / Linkage: NDIA Industry Standard Metric. Links to SPI, CPI, Average Cost, Burn Rate Source of Data: Finance Benchmark/Comparative Data: None at this time Applicable Life Cycle Phase: All (Development, Production, Sustainment) Usage Assumption: EAC should be used in conjunction with other metrics: 1) schedule completion, 2) performance to date, 3) remaining work and its anticipated performance, 4) rates, 5) outstanding commitments, 6) approved and/or pending scope changes, 7) funding constraints, 8) subcontractor EACs, and 9) program risks and opportunities Predictive / Leading Qualities: When trended, provides a comparative analysis (objective indicator) of projected outcomes based upon actual performance Warning Signs & Actions to Take: If Program EAC is less than CPI-derived EAC, understand the differences; if not rationalized, take management reserve against EAC to align with statistical EAC. Metric Definition: Management's assessment of probable outcomes based on current relevant information (i.e. performance to date), realistic plans and assumptions, cost quantification of risks and opportunities (e.g. cost to mitigate risks and opportunities with cost reductions), comparison/usage of available management reserve, sound management judgment, and the proper inclusion of financial assessments and assumed liabilities. Compares Program “Grass Roots” EAC with forecasted EAC calculated based upon past performance. Key Attributes/Thresholds: IPT EAC CUM CPI EAC 6-month EAC Creation Date: 05/16/08 Revision Date: 06/05/08 NORTHROP GRUMMAN PRIVATE/PROPRIETARY–LEVEL 1 NORTHROP GRUMMAN PRIVATE/PROPRIETARY–LEVEL 1 Stat EAC vs. EAC Jul-05Aug-05Sep-05Oct-05Nov-05Dec-05 $ in Millions IPT EAC Cum CPI EAC 6 Mos EAC

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 8 Plan & Next Steps Determine Metrics Determine Metrics Define Relevant Predictive Metrics Define Relevant Predictive Metrics Determine Alarms, Warning Signs & Actions to Take Determine Alarms, Warning Signs & Actions to Take Coordinate Metrics with Working Group Coordinate Metrics with Working Group Finalize Metric Definitions Finalize Metric Definitions Propose Metric Definitions to NDIA Propose Metric Definitions to NDIA 4/30 5/16 5/23 6/6 6/9 3/17 Estimate at Completion (EAC) Cost Performance Index (CPI) Schedule Performance Index (SPI) Approved? Analyze Aggregate Relationship(s) Analyze Aggregate Relationship(s) Select Next Metrics Yes No

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 9 Help Needed None

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 10 BACKUP

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 11 Charter & Objective – Program Measures & EVM Charter Facilitate the use of predictive measures including earned value to ensure program success Meeting frequency Period of performance Objective Communicate a set of predictive measures that will help contractors and their government counterparts predict program performance (early enough to make corrective action effective) and understand root causes of performance –Predictive measures that cover the program’s lifecycle from pre- award through contract close-out –Predictive measures that can be tailored to the contract characteristic, contract type, and phase of the program Recommend joint next steps

Copyright © 2008 Industrial Committee on Program Management. All rights reserved. 12 Commonly Measured Predictive Metrics Metric CategoryExamples EVM Cost PerformanceCPI vs TCPI Estimate at Completion (EAC)IEAC vs Program EAC Staffing (Critical Skills)Open Requisition/Aging, Fill Rate Risks and OpportunitiesRisk Burndown & Opportunity Capture integrated with MR; MR Burndown, %ETC, Risk Register Schedule (IMS) Quality & PerformanceSPI, BEI, CPLI, Float, Late Starts/Finishes Requirements Definition & StabilityAdded, Changed, Deleted Requirements, Volatility, Validation, TBD Burndown Technical Performance MeasuresSLAs, TPP, KPP Contract HealthUCAs, Volume, Traffic, Requests, Funding Profile Supply Chain PerformanceCost, Schedule, Quality, Delivery Performance, Process Compliance 30/60/90 Day Look AheadSignificant Upcoming Milestones/Events Comparison Top Issues & Action ItemsCorrective / Preventive, Latency, Product QualityDefect Rate (Actual vs Predicted), Timeliness of delivery Program Financial SummaryBillings, Expenditure Profile, Funding ResourcesInfrastructure, Availability of PP&E, GFE Availability & Quality Customer SatisfactionCPAR, BPAR, Award Fee, Survey Productivity VariancePlan vs Actual Component per Unit